diff --git a/docs/databases/index.md b/docs/databases/index.md index ec4e5ea4b..2bac5e867 100644 --- a/docs/databases/index.md +++ b/docs/databases/index.md @@ -20,6 +20,8 @@ SQLSpec supports synchronous and asynchronous **database adapters** spanning emb | [`oracledb`](oracledb.md) | `sqlspec[oracledb]` | Sync + Async | `:1` / `:name` | `ConnectionPool` / `AsyncConnectionPool` + DRCP | PyCapsule Batches | Oracle AQ (`DBMS_AQ`) + Table Queue | Sessions + Memory (Oracle 26ai `VECTOR` HNSW, Oracle Text, RRF) | | [`mssql_python`](mssql.md) | `sqlspec[mssql-python]` | Sync | `?` (`qmark`) | `MssqlPythonConnectionPool` | Native `BulkCopy` | Table Queue | Sessions + Memory (`LIKE`) | | [`pymssql`](mssql.md) | `sqlspec[pymssql]` | Sync | `%(name)s` / `%s` | `PymssqlConnectionPool` | Row | Table Queue | Sessions + Memory (`LIKE`) | +| [`sybase`](sybase.md) | `sqlspec[sybase]` | Sync | `?` (`qmark`) / `%(name)s` | `SybaseConnectionPool` | Row / Stream | Table Queue | Sessions + Memory (`LIKE`) | +| [`sqlanywhere`](sqlanywhere.md) | `sqlspec[sqlanywhere]` | Sync | `?` (`qmark`) | `SqlanywhereConnectionPool` | Row / Stream | Table Queue | Sessions + Memory (`LIKE`) | | [`db2`](db2.md) | `sqlspec[db2]` | Sync + Async | `?` (`qmark`) | `Db2SyncConnectionPool` / `Db2AsyncConnectionPool` | Schema-Guided Row | Table Queue (`SKIP LOCKED DATA`) | Sessions + Memory (`POSSTR`) | | [`duckdb`](duckdb.md) | `sqlspec[duckdb]` | Sync | `?` (`qmark`) | `DuckDBConnectionPool` | Native Zero-Copy | Table Queue | Sessions + Memory (`FTS`, `ARRAY` Cosine) | | [`bigquery`](bigquery.md) | `sqlspec[bigquery]` | Sync | `@name` (`named`) | Stateless HTTP/gRPC Client | Storage Read / Write API | Table Queue | Sessions + Memory (`SEARCH` / `LIKE`) | @@ -35,6 +37,8 @@ SQLSpec supports synchronous and asynchronous **database adapters** spanning emb - **[MySQL and MariaDB](mysql.md)**: Side-by-side setup for `asyncmy`, `pymysql`, `aiomysql`, and `mysql-connector-python`, TLS verification, and `LOAD DATA LOCAL INFILE` bulk loading. - **[Oracle Database](oracledb.md)**: Thin and Thick modes, mTLS/PKCS#12 wallets, TNS topologies, Oracle 26ai `VECTOR` + Hybrid RRF search, PyCapsule Arrow streaming, and `DBMS_CLOUD.COPY_DATA`. - **[Microsoft SQL Server](mssql.md)**: Microsoft's official `mssql-python` driver with streaming Arrow `BulkCopy` alongside FreeTDS `pymssql`. +- **[SAP Adaptive Server Enterprise](sybase.md)**: Sybase ASE 16.0+ via FreeTDS TDS 5.0 (`pymssql`), transactional DDL behavior, and catalog introspection. +- **[SAP SQL Anywhere](sqlanywhere.md)**: Embedded and client-server SQL Anywhere 12+ via `sqlanydb`, recursive dependency analysis, and automatic rollback on context exit. - **[IBM Db2](db2.md)**: Sync and async DRDA pooling (`Db2SyncConfig` / `Db2AsyncConfig`), savepoint cursor retention, and lowercase column normalization. - **[DuckDB](duckdb.md)**: In-process OLAP queries over Parquet/S3, hybrid native/Arrow materialization, and JSON statement profiling. - **[Google Cloud BigQuery](bigquery.md)**: Interactive `query_and_wait` execution, Storage Read/Write APIs, reservation governance, and multi-statement script jobs. diff --git a/docs/databases/sqlanywhere.md b/docs/databases/sqlanywhere.md new file mode 100644 index 000000000..9c2393dbc --- /dev/null +++ b/docs/databases/sqlanywhere.md @@ -0,0 +1,53 @@ +# SAP SQL Anywhere + +`sqlspec.adapters.sqlanywhere` provides synchronous database connectivity for SAP SQL Anywhere 12+ through the official `sqlanydb` driver (`>= 1.0.14`), offering connection pooling (`SqlanywhereConfig`), custom SQLGlot `sqlanywhere` dialect compilation, recursive foreign-key dependency analysis, and Litestar, Events, and Google ADK session/memory store extensions. + +## Install and When to Choose + +```bash +uv add "sqlspec[sqlanywhere]" +``` + +- Choose **`SqlanywhereConfig`** for embedded, edge, and client-server applications interacting with SAP SQL Anywhere database engines. +- `sqlanydb` wraps SAP's native client C library (`dbcapi`) via `ctypes`. Ensure `libdbcapi_r.so` (Linux), `libdbcapi_r.dylib` (macOS), or `dbcapi.dll` (Windows) is installed and available in the system library path. + +## Minimal Connection and Pool Setup + +```python +from sqlspec import SQLSpec +from sqlspec.adapters.sqlanywhere import SqlanywhereConfig + +spec = SQLSpec() +config = spec.add_config( + SqlanywhereConfig( + connection_config={ + "uid": "dba", + "pwd": "sql", + "eng": "demo_server", + "dbn": "demo", + "host": "localhost:2638", + } + ) +) +``` + +## Key Configuration Parameters + +| Setting | Location | Description | +| --- | --- | --- | +| `uid` / `user` | `connection_config` | Database user name (default DBA administrative principal). | +| `pwd` / `password` | `connection_config` | Database user password. | +| `eng` / `server` | `connection_config` | Running SQL Anywhere database engine name. | +| `dbn` / `database` | `connection_config` | Target database catalog name. | +| `host` | `connection_config` | Network host and port string (`host:port`, default port 2638). | +| `dbf` | `connection_config` | Direct database file path for auto-starting engines. | +| `pool_recycle_seconds` | `connection_config` | Maximum connection age in seconds before recycling. | +| `health_check_interval` | `connection_config` | Idle duration in seconds before running ping probes. | +| `json_serializer` / `json_deserializer` | `driver_features` | Custom JSON serialization/deserialization callables. | + +## Dialect and Catalog Notes + +- **Automatic Rollback on Context Exit**: SQL Anywhere connections maintain an implicit transaction upon statement execution. `SqlanywhereSessionContext` automatically rolls back uncommitted changes if a session exits with open work or an error, ensuring clean connection return to the pool. +- **Pagination**: Supports standard suffix `SELECT ... LIMIT n OFFSET m` as well as SQL Anywhere `TOP n START AT (m + 1)` syntax. +- **Identity Resync**: Fixture seeding supports primary key identity resetting via `CALL sa_reset_identity('', '', )`. +- **System Catalogs**: Schema discovery queries `SYS.SYSTAB`, `SYS.SYSTABCOL`, `SYS.SYSIDX`, `SYS.SYSIDXCOL`, `SYS.SYSFKEY`, and `SYS.SYSDOMAIN`. diff --git a/docs/databases/sybase.md b/docs/databases/sybase.md new file mode 100644 index 000000000..23d9b8838 --- /dev/null +++ b/docs/databases/sybase.md @@ -0,0 +1,82 @@ +# SAP Adaptive Server Enterprise (Sybase ASE) + +`sqlspec.adapters.sybase` provides first-class support for SAP Adaptive Server Enterprise (SAP ASE) 16.0+ via `pymssql` (`>= 2.4.4`) over FreeTDS TDS 5.0 wire protocol, featuring connection pooling (`SybaseConfig`), custom SQLGlot `sybase` dialect compilation, system catalog introspection, and Litestar, Events, and Google ADK session/memory store extensions. + +## Install and When to Choose + +```bash +uv add "sqlspec[sybase]" +``` + +- Choose **`SybaseConfig`** for transactional OLTP applications communicating with SAP ASE database instances. +- Prebuilt `pymssql` wheels ship statically linked FreeTDS with native TDS 5.0 wire support across Linux, macOS, and Windows. +- Note: Neither Apache Arrow ADBC nor GizmoData provides native ADBC drivers for SAP ASE, so SQLSpec communicates with ASE directly via FreeTDS TDS 5.0 DB-API connections. + +## Connection Setup and Protocol Negotiation + +Sybase ASE requires TDS 5.0 protocol negotiation. FreeTDS negotiates TDS 5.0 either when an environment variable `TDSVER=5.0` is set before process start, or when a FreeTDS server section alias is defined in `freetds.conf`: + +=== "Environment Variable (`TDSVER=5.0`)" + + ```bash + export TDSVER=5.0 + ``` + + ```python + from sqlspec import SQLSpec + from sqlspec.adapters.sybase import SybaseConfig + + spec = SQLSpec() + config = spec.add_config( + SybaseConfig( + connection_config={ + "server": "ase.example.internal", + "port": 5000, + "user": "sa", + "password": "secret_password", + "database": "app_db", + "autocommit": True, + } + ) + ) + ``` + +=== "FreeTDS Server Alias (`server_alias`)" + + ```python + from sqlspec import SQLSpec + from sqlspec.adapters.sybase import SybaseConfig + + spec = SQLSpec() + config = spec.add_config( + SybaseConfig( + connection_config={ + "server_alias": "SYBASE_PROD", + "user": "sa", + "password": "secret_password", + "database": "app_db", + } + ) + ) + ``` + +## Key Configuration Parameters + +| Setting | Location | Description | +| --- | --- | --- | +| `server` / `host` | `connection_config` | Database host name or IP address (default port `5000`). | +| `server_alias` / `freetds_server` | `connection_config` | Section name in `freetds.conf` configured with `tds version = 5.0`. | +| `database` / `db` | `connection_config` | Target database catalog name. | +| `user` / `username` | `connection_config` | Authentication login principal. | +| `password` | `connection_config` | Authentication password credentials. | +| `autocommit` | `connection_config` | Defaults to `True` to allow DDL execution outside implicit transactions. | +| `conn_properties` | `connection_config` | Sybase session initialization commands (defaults to quoted identifier and ANSINULL options). | +| `pool_recycle_seconds` | `connection_config` | Maximum connection age in seconds before recycling (defaults to 24 hours). | +| `health_check_interval` | `connection_config` | Minimum idle time in seconds before running liveness probes. | +| `json_serializer` / `json_deserializer` | `driver_features` | Custom JSON text serialization/deserialization callables. | + +## Dialect and Catalog Notes + +- **Autocommit and DDL in Transactions**: In Sybase ASE, statements like `CREATE TABLE` and `CREATE INDEX` raise error 2762 inside multi-statement transactions unless database option `ddl in tran` is enabled. `SybaseConfig` defaults `autocommit=True` and begins explicit transactions on demand via `driver.begin()`. +- **Pagination**: Supports both unparenthesized `SELECT TOP n ...` and ANSI SQL pagination `ROWS LIMIT n OFFSET m`. +- **System Catalogs**: Reflection queries introspect `dbo.sysobjects`, `dbo.syscolumns`, `dbo.sysindexes`, `dbo.sysreferences`, and `dbo.systypes`. diff --git a/pyproject.toml b/pyproject.toml index 7737cd7f2..0ad4f0788 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -73,7 +73,9 @@ pymssql = ["pymssql"] pymysql = ["pymysql"] sanic = ["sanic"] spanner = ["google-cloud-spanner>=3.64.0"] +sqlanywhere = ["sqlanydb>=1.0.14"] starlette = ["starlette"] +sybase = ["pymssql"] uuid = ["uuid-utils>=0.17.0"] [dependency-groups] @@ -156,6 +158,8 @@ pg_textsearch = "sqlspec.dialects.postgres:PGTextSearch" pgvector = "sqlspec.dialects.postgres:PGVector" spangres = "sqlspec.dialects.spanner:Spangres" spanner = "sqlspec.dialects.spanner:Spanner" +sqlanywhere = "sqlspec.dialects.sybase:SQLAnywhere" +sybase = "sqlspec.dialects.sybase:Sybase" [tool.uv] exclude-newer = "7 days" @@ -260,6 +264,9 @@ include = [ "sqlspec/dialects/db2/_generators.py", # Db2 custom dialect render hooks "sqlspec/dialects/db2/_parsers.py", # Db2 custom dialect parser hooks "sqlspec/dialects/db2/_transforms.py", # Db2 custom dialect transform helpers + "sqlspec/dialects/sybase/_generators.py", # Sybase and SQL Anywhere custom dialect render hooks + "sqlspec/dialects/sybase/_parsers.py", # Sybase and SQL Anywhere custom dialect parser hooks + "sqlspec/dialects/sybase/_transforms.py", # Sybase and SQL Anywhere custom dialect transform helpers "sqlspec/storage/registry.py", # Safe storage registry/runtime routing "sqlspec/storage/errors.py", # Safe storage error normalization "sqlspec/storage/_paths.py", # Pure storage path handling @@ -484,6 +491,8 @@ markers = [ "mysql_connector: marks tests using mysql-connector", "pymysql: marks tests using pymysql", "psqlpy: marks tests using psqlpy", + "sqlanywhere: marks tests using the sqlanywhere adapter", + "sybase: marks tests using the sybase adapter", ] python_files = ["test_*.py"] testpaths = ["tests"] @@ -539,6 +548,8 @@ module = [ "ibm_db.*", "ibm_db_dbi", "ibm_db_dbi.*", + "sqlanydb", + "sqlanydb.*", "sqlglot", "sqlglot.*", ] diff --git a/sqlspec/adapters/sqlanywhere/__init__.py b/sqlspec/adapters/sqlanywhere/__init__.py new file mode 100644 index 000000000..cdc65e606 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/__init__.py @@ -0,0 +1,40 @@ +"""SAP SQL Anywhere adapter for SQLSpec.""" + +from sqlspec.adapters.sqlanywhere._typing import SqlanywhereConnection, SqlanywhereCursor +from sqlspec.adapters.sqlanywhere.config import ( + SqlanywhereConfig, + SqlanywhereConnectionParams, + SqlanywhereDriverFeatures, + SqlanywherePoolParams, +) +from sqlspec.adapters.sqlanywhere.core import ( + DEFAULT_SQLANYWHERE_PORT, + build_connection_config, + create_mapped_exception, + default_statement_config, + driver_profile, +) +from sqlspec.adapters.sqlanywhere.data_dictionary import SqlanywhereDataDictionary, SqlanywhereSyncDataDictionary +from sqlspec.adapters.sqlanywhere.driver import SqlanywhereDriver, SqlanywhereExceptionHandler +from sqlspec.adapters.sqlanywhere.migrations import SqlanywhereSyncMigrationTracker +from sqlspec.adapters.sqlanywhere.pool import SqlanywhereConnectionPool + +__all__ = ( + "DEFAULT_SQLANYWHERE_PORT", + "SqlanywhereConfig", + "SqlanywhereConnection", + "SqlanywhereConnectionParams", + "SqlanywhereConnectionPool", + "SqlanywhereCursor", + "SqlanywhereDataDictionary", + "SqlanywhereDriver", + "SqlanywhereDriverFeatures", + "SqlanywhereExceptionHandler", + "SqlanywherePoolParams", + "SqlanywhereSyncDataDictionary", + "SqlanywhereSyncMigrationTracker", + "build_connection_config", + "create_mapped_exception", + "default_statement_config", + "driver_profile", +) diff --git a/sqlspec/adapters/sqlanywhere/_typing.py b/sqlspec/adapters/sqlanywhere/_typing.py new file mode 100644 index 000000000..d9420b415 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/_typing.py @@ -0,0 +1,123 @@ +"""SAP SQL Anywhere adapter type definitions. + +This module contains type aliases and classes that are excluded from mypyc +compilation to avoid ABI boundary issues. +""" + +import contextlib +from typing import TYPE_CHECKING, Any + +from mypy_extensions import mypyc_attr + +from sqlspec.utils.module_loader import import_optional_attr + +SQLANYDB_MODULE: Any = None +_DRIVER_CLS: Any = None + + +class SqlanywhereError(Exception): + """SAP SQL Anywhere driver exception base.""" + + +if TYPE_CHECKING: + from collections.abc import Callable + from types import TracebackType + from typing import TypeAlias + + from sqlspec.adapters.sqlanywhere.driver import SqlanywhereDriver + from sqlspec.core import StatementConfig + + SqlanywhereConnection: TypeAlias = Any + SqlanywhereRawCursor: TypeAlias = Any + SqlanywhereQueryParams: TypeAlias = Any + +if not TYPE_CHECKING: + SqlanywhereConnection = Any + SqlanywhereRawCursor = Any + SqlanywhereQueryParams = Any + +__all__ = ( + "SQLANYDB_MODULE", + "SqlanywhereConnection", + "SqlanywhereCursor", + "SqlanywhereError", + "SqlanywhereQueryParams", + "SqlanywhereRawCursor", + "SqlanywhereSessionContext", +) + + +class SqlanywhereCursor: + """Context manager for SQL Anywhere sqlanydb cursor operations.""" + + __slots__ = ("connection", "cursor") + + def __init__(self, connection: "SqlanywhereConnection") -> None: + self.connection = connection + self.cursor: SqlanywhereRawCursor | None = None + + def __enter__(self) -> "SqlanywhereRawCursor": + self.cursor = self.connection.cursor() + assert self.cursor is not None + return self.cursor + + def __exit__(self, *_: Any) -> None: + if self.cursor is not None and hasattr(self.cursor, "close"): + with contextlib.suppress(Exception): + self.cursor.close() + + +@mypyc_attr(allow_interpreted_subclasses=True, native_class=False) +class SqlanywhereSessionContext: + """Sync context manager for SAP SQL Anywhere sessions.""" + + __slots__ = ( + "_acquire_connection", + "_connection", + "_driver", + "_driver_features", + "_prepare_driver", + "_release_connection", + "_statement_config", + ) + + def __init__( + self, + acquire_connection: "Callable[[], Any]", + release_connection: "Callable[..., Any]", + statement_config: "StatementConfig", + driver_features: "dict[str, Any]", + prepare_driver: "Callable[[SqlanywhereDriver], SqlanywhereDriver]", + ) -> None: + self._acquire_connection = acquire_connection + self._release_connection = release_connection + self._statement_config = statement_config + self._driver_features = driver_features + self._prepare_driver = prepare_driver + self._connection: Any = None + self._driver: SqlanywhereDriver | None = None + + def __enter__(self) -> "SqlanywhereDriver": + global _DRIVER_CLS + if _DRIVER_CLS is None: + _DRIVER_CLS = import_optional_attr("sqlspec.adapters.sqlanywhere.driver", "SqlanywhereDriver") + self._connection = self._acquire_connection() + self._driver = _DRIVER_CLS( + connection=self._connection, statement_config=self._statement_config, driver_features=self._driver_features + ) + assert self._driver is not None + return self._prepare_driver(self._driver) + + def __exit__( + self, exc_type: "type[BaseException] | None", exc_val: "BaseException | None", exc_tb: "TracebackType | None" + ) -> "bool | None": + if self._connection is None: + return None + try: + if self._driver is not None: + self._driver._release_open_work() + finally: + self._release_connection(self._connection) + self._connection = None + self._driver = None + return None diff --git a/sqlspec/adapters/sqlanywhere/adk/__init__.py b/sqlspec/adapters/sqlanywhere/adk/__init__.py new file mode 100644 index 000000000..5b4598be7 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/adk/__init__.py @@ -0,0 +1,5 @@ +"""SAP SQL Anywhere ADK store module.""" + +from sqlspec.adapters.sqlanywhere.adk.store import SqlanywhereADKConfig, SqlanywhereADKMemoryStore, SqlanywhereADKStore + +__all__ = ("SqlanywhereADKConfig", "SqlanywhereADKMemoryStore", "SqlanywhereADKStore") diff --git a/sqlspec/adapters/sqlanywhere/adk/store.py b/sqlspec/adapters/sqlanywhere/adk/store.py new file mode 100644 index 000000000..6b94477ae --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/adk/store.py @@ -0,0 +1,875 @@ +"""SAP SQL Anywhere ADK stores for Google Agent Development Kit session storage.""" + +from datetime import datetime +from typing import TYPE_CHECKING, Any, ClassVar, Final, Literal, cast + +from sqlspec.adapters.sqlanywhere._typing import SqlanywhereCursor, SqlanywhereError +from sqlspec.adapters.sqlanywhere.core import ( + extract_sqlstate_and_sqlcode, + quote_double_identifier, + resolve_rowcount, + split_schema_and_table, +) +from sqlspec.config import ADKConfig +from sqlspec.extensions.adk import BaseSyncADKStore, StoredEvent, StoredSession, normalize_session_list_options +from sqlspec.extensions.adk.memory.store import BaseSyncADKMemoryStore +from sqlspec.utils.serializers import from_json, to_json + +if TYPE_CHECKING: + from collections.abc import Sequence + from datetime import timedelta + + from sqlspec.adapters.sqlanywhere.config import SqlanywhereConfig + from sqlspec.extensions.adk import SessionOrderBy + from sqlspec.extensions.adk.memory._types import StoredMemory + +__all__ = ("SqlanywhereADKConfig", "SqlanywhereADKMemoryStore", "SqlanywhereADKStore") + +SQLANYWHERE_TABLE_NOT_FOUND_SQLCODE: Final[int] = -141 +JSON_COLUMN_TYPE: Final[str] = "LONG VARCHAR" + + +class SqlanywhereADKConfig(ADKConfig): + """SAP SQL Anywhere ADK extension settings.""" + + +class SqlanywhereADKStore(BaseSyncADKStore["SqlanywhereConfig"]): + """Synchronous SAP SQL Anywhere ADK session/event store.""" + + connector_name: ClassVar[str] = "sqlanywhere" + __slots__ = () + + def __init__(self, config: "SqlanywhereConfig") -> None: + super().__init__(config) + + def create_tables(self) -> None: + """Create ADK tables and indexes using SQL Anywhere ``IF NOT EXISTS`` DDL.""" + if not self.create_schema_enabled: + self.reconcile_schema() + return + + with self._config.provide_session() as driver: + driver.execute_script(self._sessions_table_ddl()) + driver.execute_script(self._events_table_ddl()) + driver.execute_script(self._app_states_table_ddl()) + driver.execute_script(self._user_states_table_ddl()) + driver.execute_script(self._metadata_table_ddl()) + for index_name, index_table, columns in self._index_specs(): + driver.execute(_create_index_sql(index_table, index_name, columns)) + driver.commit() + self.reconcile_schema(assume_existing=True) + + def create_session( + self, session_id: str, app_name: str, user_id: str, state: "dict[str, Any]", owner_id: "Any | None" = None + ) -> StoredSession: + """Create a new ADK session.""" + owner_column = f", {quote_double_identifier(self._owner_id_column_name)}" if self._owner_id_column_name else "" + owner_param = ", ?" if self._owner_id_column_name else "" + insert_sql = f""" + INSERT INTO {_table_ref(self._session_table)} ( + id, app_name, user_id{owner_column}, state, create_time, update_time + ) + VALUES (?, ?, ?{owner_param}, ?, CURRENT UTC TIMESTAMP, CURRENT UTC TIMESTAMP) + """ + select_sql = f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = ? AND user_id = ? AND id = ? + """ + params: tuple[Any, ...] + if self._owner_id_column_name: + params = (session_id, app_name, user_id, owner_id, to_json(state)) + else: + params = (session_id, app_name, user_id, to_json(state)) + self._execute(insert_sql, params, commit=True) + row = self._execute_fetchone(select_sql, (app_name, user_id, session_id)) + if row is None: + msg = "Failed to fetch created session" + raise RuntimeError(msg) + return _session_record_from_row(row) + + def get_session( + self, app_name: str, user_id: str, session_id: str, *, renew_for: "int | timedelta | None" = None + ) -> "StoredSession | None": + """Return a scoped session or ``None`` if absent.""" + try: + if renew_for is not None and self._calculate_expires_at(renew_for) is not None: + self._execute( + f""" + UPDATE {_table_ref(self._session_table)} + SET update_time = CURRENT UTC TIMESTAMP + WHERE app_name = ? AND user_id = ? AND id = ? + """, + (app_name, user_id, session_id), + commit=True, + ) + row = self._execute_fetchone( + f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = ? AND user_id = ? AND id = ? + """, + (app_name, user_id, session_id), + ) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return None + raise + return _session_record_from_row(row) if row is not None else None + + def update_session_state(self, app_name: str, user_id: str, session_id: str, state: "dict[str, Any]") -> None: + """Replace a session's durable state.""" + self._execute( + f""" + UPDATE {_table_ref(self._session_table)} + SET state = ?, update_time = CURRENT UTC TIMESTAMP + WHERE app_name = ? AND user_id = ? AND id = ? + """, + (to_json(state), app_name, user_id, session_id), + commit=True, + ) + + def list_sessions( + self, + app_name: str, + user_id: "str | None" = None, + *, + order_by: "SessionOrderBy" = "update_time", + descending: bool = True, + limit: "int | None" = None, + offset: "int | None" = None, + ) -> "list[StoredSession]": + """List ADK sessions for an application, optionally scoped to a user.""" + column, direction, page_limit, page_offset = normalize_session_list_options(order_by, descending, limit, offset) + if page_limit == 0: + return [] + + sql, params = _session_list_query( + self._session_table, app_name, user_id, column, direction, page_limit, page_offset + ) + try: + rows = self._execute_fetchall(sql, params) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return [] + raise + return [_session_record_from_row(row) for row in rows] + + def delete_session(self, app_name: str, user_id: str, session_id: str) -> None: + """Delete a session and its associated events.""" + self._execute( + f"DELETE FROM {_table_ref(self._events_table)} WHERE app_name = ? AND user_id = ? AND session_id = ?", + (app_name, user_id, session_id), + commit=True, + ) + self._execute( + f"DELETE FROM {_table_ref(self._session_table)} WHERE app_name = ? AND user_id = ? AND id = ?", + (app_name, user_id, session_id), + commit=True, + ) + + def append_event(self, event_record: StoredEvent) -> None: + """Append an event to a session.""" + self._execute(_insert_event_sql(self._events_table), _event_insert_params(event_record), commit=True) + + def append_event_and_update_state( + self, + event_record: StoredEvent, + app_name: str, + user_id: str, + session_id: str, + state: "dict[str, Any]", + *, + app_state: "dict[str, Any] | None" = None, + user_state: "dict[str, Any] | None" = None, + ) -> StoredSession: + """Atomically append an event and update durable session/scoped state.""" + update_sql = f""" + UPDATE {_table_ref(self._session_table)} + SET state = ?, update_time = CURRENT UTC TIMESTAMP + WHERE app_name = ? AND user_id = ? AND id = ? + """ + select_sql = f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = ? AND user_id = ? AND id = ? + """ + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + try: + cursor.execute(update_sql, (to_json(state), app_name, user_id, session_id)) + cursor.execute(select_sql, (app_name, user_id, session_id)) + row = cursor.fetchone() + if row is None: + _raise_session_not_found(session_id) + cursor.execute(_insert_event_sql(self._events_table), _event_insert_params(event_record)) + if app_state is not None: + app_json = to_json(app_state) + cursor.execute(self._upsert_app_state_sql(), (app_name, app_json)) + if user_state is not None: + user_json = to_json(user_state) + cursor.execute(self._upsert_user_state_sql(), (app_name, user_id, user_json)) + except Exception: + conn.rollback() + raise + conn.commit() + return _session_record_from_row(row) + + def get_events( + self, + app_name: str, + user_id: str, + session_id: str, + after_timestamp: "datetime | None" = None, + limit: "int | None" = None, + ) -> "list[StoredEvent]": + """Return events for a scoped session ordered by event timestamp.""" + if limit == 0: + return [] + sql, params = self._events_query(app_name, user_id, session_id, after_timestamp, limit) + try: + rows = self._execute_fetchall(sql, params) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return [] + raise + return [_event_record_from_row(row) for row in rows] + + def delete_expired_events(self, before: datetime, app_name: "str | None" = None) -> int: + """Delete events older than ``before``.""" + sql = f"DELETE FROM {_table_ref(self._events_table)} WHERE timestamp < ?" + params: list[Any] = [before] + if app_name is not None: + sql += " AND app_name = ?" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return 0 + raise + + def delete_idle_sessions(self, updated_before: datetime, app_name: "str | None" = None) -> int: + """Delete sessions whose update_time is older than ``updated_before``.""" + sql = f"DELETE FROM {_table_ref(self._session_table)} WHERE update_time < ?" + params: list[Any] = [updated_before] + if app_name is not None: + sql += " AND app_name = ?" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return 0 + raise + + def delete_idle_user_states(self, updated_before: datetime, app_name: "str | None" = None) -> int: + """Delete user state rows whose update_time is older than ``updated_before``.""" + sql = f"DELETE FROM {_table_ref(self._user_state_table)} WHERE update_time < ?" + params: list[Any] = [updated_before] + if app_name is not None: + sql += " AND app_name = ?" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return 0 + raise + + def get_app_state(self, app_name: str) -> "dict[str, Any] | None": + """Return app-scoped state.""" + try: + row = self._execute_fetchone( + f"SELECT TOP 1 state FROM {_table_ref(self._app_state_table)} WHERE app_name = ?", (app_name,) + ) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return None + raise + return _json_dict(_row_value(row, "state", 0)) if row is not None else None + + def get_user_state(self, app_name: str, user_id: str) -> "dict[str, Any] | None": + """Return user-scoped state.""" + try: + row = self._execute_fetchone( + f""" + SELECT TOP 1 state + FROM {_table_ref(self._user_state_table)} + WHERE app_name = ? AND user_id = ? + """, + (app_name, user_id), + ) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return None + raise + return _json_dict(_row_value(row, "state", 0)) if row is not None else None + + def upsert_app_state(self, app_name: str, state: "dict[str, Any]") -> None: + """Insert or replace app-scoped state.""" + encoded = to_json(state) + self._execute(self._upsert_app_state_sql(), (app_name, encoded), commit=True) + + def upsert_user_state(self, app_name: str, user_id: str, state: "dict[str, Any]") -> None: + """Insert or replace user-scoped state.""" + encoded = to_json(state) + self._execute(self._upsert_user_state_sql(), (app_name, user_id, encoded), commit=True) + + def get_metadata(self, key: str) -> "str | None": + """Return an ADK metadata value.""" + try: + row = self._execute_fetchone( + f'SELECT TOP 1 value FROM {_table_ref(self._metadata_table)} WHERE "key" = ?', (key,) + ) + except SqlanywhereError as exc: + if _is_sqlanywhere_table_missing(exc): + return None + raise + return str(_row_value(row, "value", 0)) if row is not None else None + + def set_metadata(self, key: str, value: str) -> None: + """Set an ADK metadata value.""" + self._execute(_upsert_metadata_sql(self._metadata_table), (key, value), commit=True) + + def _index_specs(self) -> "list[tuple[str, str, str]]": + """Return ``(index_name, table, columns)`` specs for session and event indexes.""" + return [*_sessions_index_specs(self._session_table), *_events_index_specs(self._events_table)] + + def _sessions_table_ddl(self) -> str: + """Return SQL Anywhere DDL for the ADK session table.""" + return _sessions_table_ddl(self._session_table, JSON_COLUMN_TYPE, self._owner_id_column_ddl) + + def _events_table_ddl(self) -> str: + """Return SQL Anywhere DDL for the ADK event table.""" + return _events_table_ddl(self._events_table, self._session_table, JSON_COLUMN_TYPE) + + def _app_states_table_ddl(self) -> str: + """Return SQL Anywhere DDL for the app-scoped state table.""" + return _app_states_table_ddl(self._app_state_table, JSON_COLUMN_TYPE) + + def _user_states_table_ddl(self) -> str: + """Return SQL Anywhere DDL for the user-scoped state table.""" + return _user_states_table_ddl(self._user_state_table, JSON_COLUMN_TYPE) + + def _metadata_table_ddl(self) -> str: + """Return SQL Anywhere DDL for the ADK metadata table.""" + return _metadata_table_ddl(self._metadata_table) + + def _drop_app_states_table_sql(self) -> str: + return f"DROP TABLE IF EXISTS {_table_ref(self._app_state_table)}" + + def _drop_user_states_table_sql(self) -> str: + return f"DROP TABLE IF EXISTS {_table_ref(self._user_state_table)}" + + def _drop_metadata_table_sql(self) -> str: + return f"DROP TABLE IF EXISTS {_table_ref(self._metadata_table)}" + + def _drop_tables_sql(self) -> "list[str]": + return [ + self._drop_metadata_table_sql(), + self._drop_user_states_table_sql(), + self._drop_app_states_table_sql(), + f"DROP TABLE IF EXISTS {_table_ref(self._events_table)}", + f"DROP TABLE IF EXISTS {_table_ref(self._session_table)}", + ] + + def _upsert_app_state_sql(self) -> str: + return _upsert_state_sql(self._app_state_table, ("app_name",)) + + def _upsert_user_state_sql(self) -> str: + return _upsert_state_sql(self._user_state_table, ("app_name", "user_id")) + + def _events_query( + self, + app_name: str, + user_id: str, + session_id: str, + after_timestamp: "datetime | None" = None, + limit: "int | None" = None, + ) -> "tuple[str, tuple[Any, ...]]": + return _events_query(self._events_table, app_name, user_id, session_id, after_timestamp, limit) + + def _execute_fetchone(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> "Any | None": + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, params) + row = cursor.fetchone() + if commit: + conn.commit() + return row + + def _execute_fetchall(self, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, params) + return list(cursor.fetchall()) + + def _execute(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> int: + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, params) + rowcount = resolve_rowcount(cursor) + if commit: + conn.commit() + return rowcount + + +class SqlanywhereADKMemoryStore(BaseSyncADKMemoryStore["SqlanywhereConfig"]): + """SAP SQL Anywhere ADK memory store using sqlanydb.""" + + __slots__ = () + + def __init__(self, config: "SqlanywhereConfig") -> None: + super().__init__(config) + + def create_tables(self) -> None: + """Create the memory table and indexes using SQL Anywhere ``IF NOT EXISTS`` DDL.""" + if not self.create_schema_enabled: + self.reconcile_schema() + return + + if not self._enabled: + return + with self._config.provide_session() as driver: + driver.execute_script(self._memory_table_ddl()) + for index_name, index_table, columns in self._memory_index_specs(): + driver.execute(_create_index_sql(index_table, index_name, columns)) + driver.commit() + self.reconcile_schema(assume_existing=True) + + def insert_memory_entries(self, entries: "list[StoredMemory]", owner_id: "object | None" = None) -> int: + """Bulk insert memory entries with event-id deduplication.""" + if not self._enabled: + msg = "ADK memory store is disabled" + raise RuntimeError(msg) + if not entries: + return 0 + + owner_column = f", {quote_double_identifier(self._owner_id_column_name)}" if self._owner_id_column_name else "" + owner_value = ", ?" if self._owner_id_column_name else "" + sql = f""" + INSERT INTO {_table_ref(self._memory_table)} ( + id, session_id, app_name, user_id, scope, event_id, author, timestamp, + content_json, content_text, metadata_json{owner_column} + ) + SELECT ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?{owner_value} + WHERE NOT EXISTS ( + SELECT 1 FROM {_table_ref(self._memory_table)} + WHERE event_id = ? + ) + """ + inserted = 0 + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + try: + for entry in entries: + params: tuple[Any, ...] = ( + entry["id"], + entry["session_id"], + entry["app_name"], + entry["user_id"], + entry.get("scope", "user"), + entry["event_id"], + entry.get("author"), + entry["timestamp"], + to_json(entry["content_json"]), + entry["content_text"], + to_json(entry["metadata_json"]) if entry.get("metadata_json") is not None else None, + ) + if self._owner_id_column_name: + params = (*params, owner_id) + cursor.execute(sql, (*params, entry["event_id"])) + inserted += resolve_rowcount(cursor) + except Exception: + conn.rollback() + raise + conn.commit() + return inserted + + def search_entries( + self, + query: str, + app_name: str, + user_id: str, + limit: "int | None" = None, + scope_filter: Literal["all", "user", "app"] = "all", + embedding: "Sequence[float] | None" = None, + ) -> "list[StoredMemory]": + """Search memory entries by text query.""" + _ = embedding + if not self._enabled: + msg = "ADK memory store is disabled" + raise RuntimeError(msg) + limit_value = limit or self._max_results + where_scope, scope_params = _build_sqlanywhere_scope_where(app_name, user_id, scope_filter) + escaped_query = _escape_like_pattern(query) + sql = f""" + SELECT + id, session_id, app_name, user_id, scope, event_id, author, timestamp, + content_json, content_text, metadata_json, inserted_at + FROM {_table_ref(self._memory_table)} + WHERE {where_scope} AND content_text LIKE ? ESCAPE '\\' + ORDER BY timestamp DESC + LIMIT ? + """ + rows = self._execute_fetchall(sql, (*scope_params, f"%{escaped_query}%", limit_value)) + return [_memory_record_from_row(row) for row in rows] + + def delete_entries_by_session(self, session_id: str) -> int: + """Delete all memory entries for a specific session.""" + return self._execute( + f"DELETE FROM {_table_ref(self._memory_table)} WHERE session_id = ?", (session_id,), commit=True + ) + + def delete_entries_older_than(self, days: int, app_name: "str | None" = None, scope: "str | None" = None) -> int: + """Delete memory entries older than the retention window.""" + clauses = ["inserted_at < DATEADD(day, -?, CURRENT UTC TIMESTAMP)"] + params: list[Any] = [days] + if app_name is not None: + clauses.append("app_name = ?") + params.append(app_name) + if scope is not None: + clauses.append("scope = ?") + params.append(scope) + where_sql = " AND ".join(clauses) + return self._execute( + f"DELETE FROM {_table_ref(self._memory_table)} WHERE {where_sql}", tuple(params), commit=True + ) + + def _memory_table_ddl(self) -> str: + owner_line = f", {self._owner_id_column_ddl}" if self._owner_id_column_ddl else "" + table_ref = _table_ref(self._memory_table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "session_id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + "scope VARCHAR(16) DEFAULT 'user' NOT NULL, " + "event_id VARCHAR(128) NOT NULL, " + "author VARCHAR(256) NULL, " + "timestamp TIMESTAMP NOT NULL, " + f"content_json {JSON_COLUMN_TYPE} NOT NULL, " + "content_text LONG VARCHAR NOT NULL, " + f"metadata_json {JSON_COLUMN_TYPE} NULL, " + f"inserted_at TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL{owner_line}, " + f"CONSTRAINT {_constraint_ref('pk', self._memory_table, 'id')} PRIMARY KEY (id), " + f"CONSTRAINT {_constraint_ref('uq', self._memory_table, 'event_id')} UNIQUE (event_id)" + ");" + ) + + def _memory_index_specs(self) -> "list[tuple[str, str, str]]": + """Return ``(index_name, table, columns)`` specs for memory-table indexes.""" + _, table_name = split_schema_and_table(self._memory_table) + return [ + (f"idx_{table_name}_app_scope_user_time", self._memory_table, "app_name, scope, user_id, timestamp DESC"), + (f"idx_{table_name}_scope", self._memory_table, "app_name, scope"), + (f"idx_{table_name}_session", self._memory_table, "session_id"), + (f"idx_{table_name}_timestamp", self._memory_table, "timestamp DESC"), + ] + + def _drop_memory_table_sql(self) -> "list[str]": + return [f"DROP TABLE IF EXISTS {_table_ref(self._memory_table)}"] + + def _execute_fetchall(self, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, params) + return list(cursor.fetchall()) + + def _execute(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> int: + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, params) + rowcount = resolve_rowcount(cursor) + if commit: + conn.commit() + return rowcount + + +def _sessions_table_ddl(table: str, json_column_type: str, owner_id_column_ddl: "str | None") -> str: + owner_line = f", {owner_id_column_ddl}" if owner_id_column_ddl else "" + table_ref = _table_ref(table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + f"user_id VARCHAR(128) NOT NULL{owner_line}, " + f"state {json_column_type} NOT NULL, " + "create_time TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL, " + "update_time TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'id')} PRIMARY KEY (id)" + ");" + ) + + +def _sessions_index_specs(table: str) -> "list[tuple[str, str, str]]": + _, table_name = split_schema_and_table(table) + return [ + (f"idx_{table_name}_app_user", table, "app_name, user_id"), + (f"idx_{table_name}_update_time", table, "update_time DESC"), + ] + + +def _events_table_ddl(table: str, session_table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + session_ref = _table_ref(session_table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + "session_id VARCHAR(128) NOT NULL, " + "invocation_id VARCHAR(256) NOT NULL, " + "timestamp TIMESTAMP NOT NULL, " + f"event_data {json_column_type} NOT NULL, " + "actions LONG BINARY NULL, " + "partial BIT NULL, " + "turn_complete BIT NULL, " + "interrupted BIT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'id')} PRIMARY KEY (id), " + f"CONSTRAINT {_constraint_ref('fk', table, 'session')} FOREIGN KEY (session_id) REFERENCES {session_ref}(id)" + ");" + ) + + +def _events_index_specs(table: str) -> "list[tuple[str, str, str]]": + _, table_name = split_schema_and_table(table) + return [ + (f"idx_{table_name}_scope", table, "app_name, user_id, session_id, timestamp ASC"), + (f"idx_{table_name}_session", table, "session_id, timestamp ASC"), + (f"idx_{table_name}_invocation", table, "invocation_id"), + (f"idx_{table_name}_timestamp", table, "timestamp ASC"), + (f"idx_{table_name}_app_timestamp", table, "app_name, timestamp ASC"), + ] + + +def _app_states_table_ddl(table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "app_name VARCHAR(128) NOT NULL, " + f"state {json_column_type} NOT NULL, " + "update_time TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'app_name')} PRIMARY KEY (app_name)" + ");" + ) + + +def _user_states_table_ddl(table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + f"state {json_column_type} NOT NULL, " + "update_time TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'app_user')} PRIMARY KEY (app_name, user_id)" + ");" + ) + + +def _metadata_table_ddl(table: str) -> str: + table_ref = _table_ref(table) + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + '"key" VARCHAR(128) NOT NULL, ' + "value VARCHAR(512) NOT NULL, " + f'CONSTRAINT {_constraint_ref("pk", table, "key")} PRIMARY KEY ("key")' + ");" + ) + + +def _create_index_sql(table: str, index_name: str, columns: str) -> str: + return f"CREATE INDEX IF NOT EXISTS {quote_double_identifier(index_name)} ON {_table_ref(table)} ({columns})" + + +def _insert_event_sql(table: str) -> str: + return f""" + INSERT INTO {_table_ref(table)} ( + id, app_name, user_id, session_id, invocation_id, timestamp, event_data + ) + VALUES (?, ?, ?, ?, ?, ?, ?) + """ + + +def _upsert_state_sql(table: str, key_columns: "tuple[str, ...]") -> str: + table_ref = _table_ref(table) + insert_columns = ", ".join(quote_double_identifier(col) for col in (*key_columns, "state", "update_time")) + insert_placeholders = ", ".join(["?" for _ in key_columns] + ["?", "CURRENT UTC TIMESTAMP"]) + return f""" + INSERT INTO {table_ref} ({insert_columns}) + ON EXISTING UPDATE + VALUES ({insert_placeholders}) + """ + + +def _upsert_metadata_sql(table: str) -> str: + table_ref = _table_ref(table) + return f""" + INSERT INTO {table_ref} ("key", value) + ON EXISTING UPDATE + VALUES (?, ?) + """ + + +def _events_query( + table: str, app_name: str, user_id: str, session_id: str, after_timestamp: "datetime | None", limit: "int | None" +) -> "tuple[str, tuple[Any, ...]]": + params: list[Any] = [app_name, user_id, session_id] + after_clause = "" + if after_timestamp is not None: + after_clause = " AND timestamp > ?" + params.append(after_timestamp) + limit_clause = "" + if limit is not None: + limit_clause = "\n LIMIT ?" + params.append(limit) + sql = f""" + SELECT id, app_name, user_id, session_id, invocation_id, timestamp, event_data + FROM {_table_ref(table)} + WHERE app_name = ? AND user_id = ? AND session_id = ?{after_clause} + ORDER BY timestamp ASC{limit_clause} + """ + return sql, tuple(params) + + +def _event_insert_params(event_record: StoredEvent) -> "tuple[Any, ...]": + return ( + event_record["id"], + event_record["app_name"], + event_record["user_id"], + event_record["session_id"], + event_record["invocation_id"], + event_record["timestamp"], + to_json(event_record["event_data"]), + ) + + +def _row_value(row: object, key: str, index: int) -> Any: + if isinstance(row, dict): + if key in row: + return row[key] + upper_key = key.upper() + if upper_key in row: + return row[upper_key] + return None + if isinstance(row, (list, tuple)) and len(row) > index: + return row[index] + return getattr(row, key, None) + + +def _session_record_from_row(row: Any) -> StoredSession: + return StoredSession( + id=_row_value(row, "id", 0), + app_name=_row_value(row, "app_name", 1), + user_id=_row_value(row, "user_id", 2), + state=_json_dict(_row_value(row, "state", 3)), + create_time=_row_value(row, "create_time", 4), + update_time=_row_value(row, "update_time", 5), + ) + + +def _event_record_from_row(row: Any) -> StoredEvent: + return StoredEvent( + id=_row_value(row, "id", 0), + app_name=_row_value(row, "app_name", 1), + user_id=_row_value(row, "user_id", 2), + session_id=_row_value(row, "session_id", 3), + invocation_id=_row_value(row, "invocation_id", 4), + timestamp=_row_value(row, "timestamp", 5), + event_data=_json_dict(_row_value(row, "event_data", 6)), + ) + + +def _memory_record_from_row(row: Any) -> "StoredMemory": + metadata_raw = _row_value(row, "metadata_json", 10) + return cast( + "StoredMemory", + { + "id": _row_value(row, "id", 0), + "session_id": _row_value(row, "session_id", 1), + "app_name": _row_value(row, "app_name", 2), + "user_id": _row_value(row, "user_id", 3), + "scope": _row_value(row, "scope", 4), + "event_id": _row_value(row, "event_id", 5), + "author": _row_value(row, "author", 6), + "timestamp": _row_value(row, "timestamp", 7), + "content_json": _json_dict(_row_value(row, "content_json", 8)), + "content_text": _row_value(row, "content_text", 9), + "metadata_json": _json_dict(metadata_raw) if metadata_raw is not None else None, + "inserted_at": _row_value(row, "inserted_at", 11), + "embedding": None, + }, + ) + + +def _json_dict(value: Any) -> "dict[str, Any]": + if value is None: + return {} + if isinstance(value, dict): + return cast("dict[str, Any]", value) + if isinstance(value, bytearray): + value = bytes(value) + if isinstance(value, (bytes, str)): + return cast("dict[str, Any]", from_json(value)) + return cast("dict[str, Any]", from_json(str(value))) + + +def _is_sqlanywhere_table_missing(exc: BaseException) -> bool: + text = str(exc).lower() + _, sqlcode = extract_sqlstate_and_sqlcode(exc) + return "not found" in text or sqlcode == SQLANYWHERE_TABLE_NOT_FOUND_SQLCODE + + +def _table_ref(table: str) -> str: + schema_name, table_name = split_schema_and_table(table) + return f"{quote_double_identifier(schema_name)}.{quote_double_identifier(table_name)}" + + +def _constraint_ref(prefix: str, table: str, suffix: str) -> str: + _, table_name = split_schema_and_table(table) + return quote_double_identifier(f"{prefix}_{table_name}_{suffix}") + + +def _raise_session_not_found(session_id: str) -> None: + msg = f"Session {session_id} not found during append_event_and_update_state." + raise ValueError(msg) + + +def _build_sqlanywhere_scope_where( + app_name: str, user_id: str, scope_filter: Literal["all", "user", "app"] +) -> tuple[str, tuple[Any, ...]]: + if scope_filter == "all": + return "app_name = ? AND ((scope = 'user' AND user_id = ?) OR scope = 'app')", (app_name, user_id) + if scope_filter == "user": + return "app_name = ? AND scope = 'user' AND user_id = ?", (app_name, user_id) + return "app_name = ? AND scope = 'app'", (app_name,) + + +def _escape_like_pattern(value: str) -> str: + return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_") + + +def _session_list_query( + session_table: str, + app_name: str, + user_id: "str | None", + column: str, + direction: str, + limit: "int | None", + offset: int, +) -> "tuple[str, tuple[Any, ...]]": + """Return the bounded SQL Anywhere session-list query and its bound values.""" + params: list[Any] = [app_name] + where_clause = "app_name = ?" + if user_id is not None: + params.append(user_id) + where_clause = f"{where_clause} AND user_id = ?" + + page_clause = "" + if limit is not None: + params.extend((limit, offset)) + page_clause = "\n LIMIT ? OFFSET ?" + + sql = f""" + SELECT id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(session_table)} + WHERE {where_clause} + ORDER BY {column} {direction}, id {direction}{page_clause} + """ + return sql, tuple(params) diff --git a/sqlspec/adapters/sqlanywhere/config.py b/sqlspec/adapters/sqlanywhere/config.py new file mode 100644 index 000000000..fe635e405 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/config.py @@ -0,0 +1,235 @@ +"""SAP SQL Anywhere database configuration.""" + +from typing import TYPE_CHECKING, Any, ClassVar, Literal, NotRequired, TypedDict, cast + +from sqlspec.adapters.sqlanywhere._typing import ( + SqlanywhereConnection, + SqlanywhereCursor, + SqlanywhereRawCursor, + SqlanywhereSessionContext, +) +from sqlspec.adapters.sqlanywhere.core import apply_driver_features, build_connection_config, default_statement_config +from sqlspec.adapters.sqlanywhere.driver import SqlanywhereDriver, SqlanywhereExceptionHandler +from sqlspec.adapters.sqlanywhere.migrations import SqlanywhereSyncMigrationTracker +from sqlspec.adapters.sqlanywhere.pool import SqlanywhereConnectionPool +from sqlspec.config import ExtensionConfigs, SyncDatabaseConfig +from sqlspec.core import TypeCoercionCapabilities +from sqlspec.driver import SyncPoolConnectionContext, SyncPoolSessionFactory +from sqlspec.extensions.events import EventRuntimeHints +from sqlspec.utils.config_tools import normalize_connection_config + +if TYPE_CHECKING: + from collections.abc import Callable + + from sqlspec.core import StatementConfig + from sqlspec.observability import ObservabilityConfig + +__all__ = ( + "SqlanywhereConfig", + "SqlanywhereConnectionContext", + "SqlanywhereConnectionParams", + "SqlanywhereDriverFeatures", + "SqlanywherePoolParams", +) + + +class SqlanywhereConnectionParams(TypedDict): + """SAP SQL Anywhere connection parameters.""" + + uid: NotRequired[str] + """User ID / login name for authentication.""" + user: NotRequired[str] + """Alias for ``uid``.""" + username: NotRequired[str] + """Alias for ``uid``.""" + pwd: NotRequired[str] + """Password for authentication.""" + password: NotRequired[str] + """Alias for ``pwd``.""" + dbn: NotRequired[str] + """Database name on the SQL Anywhere server.""" + database: NotRequired[str] + """Alias for ``dbn``.""" + db: NotRequired[str] + """Alias for ``dbn``.""" + eng: NotRequired[str] + """SQL Anywhere database engine name.""" + server: NotRequired[str] + """Alias for ``eng``.""" + engine: NotRequired[str] + """Alias for ``eng``.""" + server_name: NotRequired[str] + """Alias for ``eng``.""" + dbf: NotRequired[str] + """Database file path for local or embedded engines.""" + database_file: NotRequired[str] + """Alias for ``dbf``.""" + host: NotRequired[str] + """TCP hostname or IP address of the server.""" + port: NotRequired[int | str] + """TCP port number of the server.""" + dsn: NotRequired[str] + """ODBC data source name.""" + links: NotRequired[str] + """Network communications links protocol configuration.""" + cset: NotRequired[str] + """Character set used for character data conversion.""" + extra: NotRequired["dict[str, Any]"] + """Additional connection parameters passed to the driver.""" + + +class SqlanywherePoolParams(SqlanywhereConnectionParams): + """SAP SQL Anywhere pool parameters.""" + + pool_min_size: NotRequired[int] + """Minimum number of connections maintained in the pool.""" + pool_max_size: NotRequired[int] + """Maximum number of connections allowed in the pool.""" + pool_timeout: NotRequired[float] + """Seconds to wait for a connection before timing out.""" + pool_recycle_seconds: NotRequired[int] + """Maximum age in seconds before a connection is recycled.""" + health_check_interval: NotRequired[float] + """Interval in seconds between background health checks.""" + + +class SqlanywhereDriverFeatures(TypedDict): + """SAP SQL Anywhere driver feature flags.""" + + json_serializer: NotRequired["Callable[[Any], str]"] + """Custom callable for serializing objects to JSON strings.""" + json_deserializer: NotRequired["Callable[[str], Any]"] + """Custom callable for deserializing JSON strings.""" + on_connection_create: "NotRequired[Callable[[SqlanywhereConnection], None]]" + """Callback invoked immediately when a new connection is established.""" + enable_events: NotRequired[bool] + """Enable database event listening.""" + events_backend: NotRequired[Literal["poll_queue"]] + """Backend mechanism used for dispatching events.""" + + +class SqlanywhereConnectionContext(SyncPoolConnectionContext): + """Context manager for SAP SQL Anywhere connections.""" + + __slots__ = () + + +class _SqlanywhereSessionConnectionHandler(SyncPoolSessionFactory): + __slots__ = () + + +class SqlanywhereConfig(SyncDatabaseConfig[SqlanywhereConnection, SqlanywhereConnectionPool, SqlanywhereDriver]): + """Configuration for SAP SQL Anywhere synchronous connections over sqlanydb.""" + + __slots__ = ( + "_health_check_interval", + "_pool_max_size", + "_pool_min_size", + "_pool_recycle_seconds", + "_pool_timeout", + "_user_connection_hook", + ) + + driver_type: "ClassVar[type[SqlanywhereDriver]]" = SqlanywhereDriver + connection_type: "ClassVar[type[SqlanywhereConnection]]" = cast( + "type[SqlanywhereConnection]", SqlanywhereConnection + ) + migration_tracker_type: "ClassVar[type[SqlanywhereSyncMigrationTracker]]" = SqlanywhereSyncMigrationTracker + supports_transactional_ddl: "ClassVar[bool]" = False + supports_migration_schemas: "ClassVar[bool]" = True + supports_native_arrow_export: "ClassVar[bool]" = True + supports_native_arrow_import: "ClassVar[bool]" = True + supports_native_parquet_export: "ClassVar[bool]" = True + supports_native_parquet_import: "ClassVar[bool]" = True + supports_native_row_streaming: "ClassVar[bool]" = True + type_coercion_capabilities: "ClassVar[TypeCoercionCapabilities]" = TypeCoercionCapabilities( + datetime_binding="native", timestamp_precision="microsecond", json_columns_decoded=False, uuid_binding="text" + ) + _connection_context_class: "ClassVar[type[SqlanywhereConnectionContext]]" = SqlanywhereConnectionContext + _session_factory_class: "ClassVar[type[_SqlanywhereSessionConnectionHandler]]" = ( + _SqlanywhereSessionConnectionHandler + ) + _session_context_class: "ClassVar[type[SqlanywhereSessionContext]]" = SqlanywhereSessionContext + _default_statement_config = default_statement_config + + def __init__( + self, + *, + connection_config: "SqlanywherePoolParams | dict[str, Any] | None" = None, + connection_instance: "SqlanywhereConnectionPool | None" = None, + migration_config: "dict[str, Any] | None" = None, + statement_config: "StatementConfig | None" = None, + driver_features: "SqlanywhereDriverFeatures | dict[str, Any] | None" = None, + bind_key: "str | None" = None, + extension_config: "ExtensionConfigs | None" = None, + observability_config: "ObservabilityConfig | None" = None, + **kwargs: Any, + ) -> None: + raw_config = dict(normalize_connection_config(connection_config)) + self._pool_min_size = int(raw_config.pop("pool_min_size", 1)) + self._pool_max_size = int(raw_config.pop("pool_max_size", 10)) + self._pool_timeout = float(raw_config.pop("pool_timeout", 30.0)) + self._pool_recycle_seconds = int(raw_config.pop("pool_recycle_seconds", 86400)) + self._health_check_interval = float(raw_config.pop("health_check_interval", 30.0)) + normalized_connection_config = build_connection_config(raw_config) + + statement_config = statement_config or default_statement_config + statement_config, driver_features = apply_driver_features(statement_config, driver_features) + + features_dict = dict(driver_features) if driver_features else {} + self._user_connection_hook: Callable[[SqlanywhereConnection], None] | None = features_dict.pop( + "on_connection_create", None + ) + + super().__init__( + connection_config=normalized_connection_config, + connection_instance=connection_instance, + migration_config=migration_config, + statement_config=statement_config, + driver_features=features_dict, + bind_key=bind_key, + extension_config=extension_config, + observability_config=observability_config, + **kwargs, + ) + + def create_connection(self) -> "SqlanywhereConnection": + """Open a standalone SAP SQL Anywhere connection owned by the caller.""" + return self.provide_pool().new_connection() + + def get_signature_namespace(self) -> "dict[str, Any]": + namespace = super().get_signature_namespace() + namespace.update({ + "SqlanywhereConfig": SqlanywhereConfig, + "SqlanywhereConnection": SqlanywhereConnection, + "SqlanywhereConnectionContext": SqlanywhereConnectionContext, + "SqlanywhereConnectionParams": SqlanywhereConnectionParams, + "SqlanywhereConnectionPool": SqlanywhereConnectionPool, + "SqlanywhereCursor": SqlanywhereCursor, + "SqlanywhereDriver": SqlanywhereDriver, + "SqlanywhereDriverFeatures": SqlanywhereDriverFeatures, + "SqlanywhereExceptionHandler": SqlanywhereExceptionHandler, + "SqlanywherePoolParams": SqlanywherePoolParams, + "SqlanywhereRawCursor": SqlanywhereRawCursor, + "SqlanywhereSessionContext": SqlanywhereSessionContext, + }) + return namespace + + def get_event_runtime_hints(self) -> "EventRuntimeHints": + return EventRuntimeHints(poll_interval=0.5, lease_seconds=30, select_for_update=False, skip_locked=False) + + def _create_pool(self) -> "SqlanywhereConnectionPool": + return SqlanywhereConnectionPool( + self.connection_config, + min_size=self._pool_min_size, + max_size=self._pool_max_size, + timeout=self._pool_timeout, + recycle_seconds=self._pool_recycle_seconds, + health_check_interval=self._health_check_interval, + on_connection_create=self._user_connection_hook, + ) + + def _close_pool(self) -> None: + if self.connection_instance: + self.connection_instance.close() + self.connection_instance = None diff --git a/sqlspec/adapters/sqlanywhere/core.py b/sqlspec/adapters/sqlanywhere/core.py new file mode 100644 index 000000000..f36e7240d --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/core.py @@ -0,0 +1,404 @@ +"""SAP SQL Anywhere adapter compiled helpers.""" + +from collections.abc import Callable, Mapping, Sized +from decimal import Decimal +from typing import TYPE_CHECKING, Any, Final, Literal + +from sqlspec.core import DriverParameterProfile, ParameterStyle, StatementConfig, build_statement_config_from_profile +from sqlspec.dialects.sybase import SQLAnywhere as _SQLAnywhereDialect +from sqlspec.exceptions import ( + CheckViolationError, + DatabaseConnectionError, + DataError, + DeadlockError, + ForeignKeyViolationError, + IntegrityError, + NotFoundError, + NotNullViolationError, + OperationalError, + PermissionDeniedError, + SerializationConflictError, + SQLParsingError, + SQLSpecError, + UniqueViolationError, +) +from sqlspec.utils.serializers import from_json, to_json +from sqlspec.utils.text import split_qualified_identifier +from sqlspec.utils.type_converters import build_uuid_coercions +from sqlspec.utils.type_guards import has_rowcount + +if TYPE_CHECKING: + from collections.abc import Sequence + from logging import Logger + +__all__ = ( + "DEFAULT_SQLANYWHERE_PORT", + "apply_driver_features", + "build_connection_config", + "build_insert_statement", + "build_profile", + "build_statement_config", + "collect_rows", + "create_mapped_exception", + "default_statement_config", + "driver_profile", + "escape_sql_literal", + "extract_sqlstate_and_sqlcode", + "format_identifier", + "normalize_execute_many_parameters", + "normalize_execute_parameters", + "quote_double_identifier", + "resolve_column_names", + "resolve_many_rowcount", + "resolve_rowcount", + "split_schema_and_table", +) + +DEFAULT_SQLANYWHERE_PORT: Final[int] = 2638 +_COLUMN_CACHE_MAX_SIZE: Final[int] = 256 +_SQLANYDB_FULL_ARGS_LEN: Final[int] = 3 +_SQLANYDB_PAIR_ARGS_LEN: Final[int] = 2 +_POOL_CONFIG_KEYS: Final[frozenset[str]] = frozenset({ + "pool_min_size", + "pool_max_size", + "pool_timeout", + "pool_recycle_seconds", + "health_check_interval", +}) +_SQLCODE_MAPPING: Final[dict[int, tuple[type[SQLSpecError], str]]] = { + -193: (UniqueViolationError, "primary key is not unique"), + -196: (UniqueViolationError, "unique index violation"), + -194: (ForeignKeyViolationError, "foreign key constraint violation"), + -198: (ForeignKeyViolationError, "primary key is referenced by foreign key"), + -195: (NotNullViolationError, "column cannot be NULL"), + -209: (CheckViolationError, "invalid value for column check constraint"), + -1091: (CheckViolationError, "check constraint violation"), + -306: (DeadlockError, "deadlock detected"), + -307: (DeadlockError, "all threads are blocked"), + -684: (DeadlockError, "rollback occurred due to deadlock"), + -210: (SerializationConflictError, "row is locked by another user"), + -1175: (SerializationConflictError, "snapshot isolation conflict"), + -131: (SQLParsingError, "SQL syntax error"), + -141: (NotFoundError, "table not found"), + -143: (NotFoundError, "column not found"), + -265: (NotFoundError, "procedure not found"), + -83: (DatabaseConnectionError, "specified database not found"), + -100: (DatabaseConnectionError, "database server not found"), + -103: (DatabaseConnectionError, "invalid user ID or password"), + -308: (DatabaseConnectionError, "connection was terminated"), + -832: (DatabaseConnectionError, "connection error"), + -101: (OperationalError, "not connected to a database"), + -157: (DataError, "data conversion error"), + -158: (DataError, "numeric value out of range"), + -638: (DataError, "right truncation of string data"), + -121: (PermissionDeniedError, "permission denied"), +} +_SQLSTATE_EXACT_MAPPING: Final[dict[str, tuple[type[SQLSpecError], str]]] = { + "23505": (UniqueViolationError, "unique constraint violation"), + "23503": (ForeignKeyViolationError, "foreign key constraint violation"), + "23502": (NotNullViolationError, "not-null constraint violation"), + "23506": (CheckViolationError, "check constraint violation"), + "23514": (CheckViolationError, "check constraint violation"), + "40001": (DeadlockError, "serialization/deadlock failure"), + "42000": (SQLParsingError, "SQL syntax error"), + "42S02": (NotFoundError, "table not found"), + "42S22": (NotFoundError, "column not found"), + "42501": (PermissionDeniedError, "permission denied"), + "28000": (DatabaseConnectionError, "invalid authorization specification"), +} + + +def quote_double_identifier(identifier: str) -> str: + """Quote a SQL Anywhere identifier with double quotes.""" + cleaned = identifier.strip() + if (cleaned.startswith('"') and cleaned.endswith('"')) or (cleaned.startswith("[") and cleaned.endswith("]")): + cleaned = cleaned[1:-1].replace('""', '"').replace("]]", "]") + return f'"{cleaned.replace(chr(34), chr(34) * 2)}"' + + +def escape_sql_literal(value: str) -> str: + """Escape single quotes inside a SQL string literal.""" + return value.replace("'", "''") + + +def split_schema_and_table(identifier: str, *, default_schema: str = "DBA") -> tuple[str, str]: + """Split a table reference into unquoted ``(schema, table)`` names.""" + cleaned = identifier.strip() + if not cleaned: + msg = "Table name must not be empty" + raise SQLSpecError(msg) + parts = split_qualified_identifier(cleaned, quote_chars='"', allow_bracket_quotes=True) + if not parts: + return default_schema, cleaned + if len(parts) == 1: + return default_schema, parts[0] + return parts[-2], parts[-1] + + +def format_identifier(identifier: str) -> str: + """Format a SQL Anywhere identifier with double-quote quoting.""" + cleaned = identifier.strip() + if not cleaned: + msg = "Table name must not be empty" + raise SQLSpecError(msg) + parts = split_qualified_identifier(cleaned, quote_chars='"', allow_bracket_quotes=True) + return ".".join(quote_double_identifier(part) for part in parts) + + +def build_insert_statement(table: str, columns: "list[str]") -> str: + """Build a QMARK-parameterized INSERT statement for SQL Anywhere.""" + column_clause = ", ".join(quote_double_identifier(column) for column in columns) + placeholders = ", ".join("?" for _ in columns) + return f"INSERT INTO {format_identifier(table)} ({column_clause}) VALUES ({placeholders})" + + +def normalize_execute_parameters(parameters: Any) -> Any: + """Normalize parameters for SQL Anywhere sqlanydb execute calls.""" + if parameters is None: + return () + if isinstance(parameters, list): + return tuple(parameters) + return parameters + + +def normalize_execute_many_parameters(parameters: Any) -> Any: + """Normalize parameters for SQL Anywhere sqlanydb executemany calls.""" + return parameters + + +def build_profile() -> "DriverParameterProfile": + """Create the SAP SQL Anywhere driver parameter profile.""" + return DriverParameterProfile( + name="sqlanywhere", + default_style=ParameterStyle.QMARK, + supported_styles={ + ParameterStyle.QMARK, + ParameterStyle.NUMERIC, + ParameterStyle.NAMED_COLON, + ParameterStyle.NAMED_PYFORMAT, + ParameterStyle.POSITIONAL_PYFORMAT, + }, + default_execution_style=ParameterStyle.QMARK, + supported_execution_styles={ParameterStyle.QMARK}, + has_native_list_expansion=False, + preserve_parameter_format=True, + needs_static_script_compilation=False, + allow_mixed_parameter_styles=False, + preserve_original_params_for_many=False, + json_serializer_strategy="helper", + custom_type_coercions=_custom_type_coercions(), + default_dialect="sqlanywhere", + ) + + +def build_statement_config( + *, json_serializer: "Callable[[Any], str] | None" = None, json_deserializer: "Callable[[str], Any] | None" = None +) -> "StatementConfig": + """Construct the SAP SQL Anywhere statement configuration.""" + return build_statement_config_from_profile( + driver_profile, + statement_overrides={"dialect": "sqlanywhere"}, + json_serializer=json_serializer or to_json, + json_deserializer=json_deserializer or from_json, + ) + + +def apply_driver_features( + statement_config: "StatementConfig", driver_features: "Mapping[str, Any] | None" +) -> "tuple[StatementConfig, dict[str, Any]]": + """Apply SQL Anywhere driver feature overrides to statement config.""" + features: dict[str, Any] = dict(driver_features) if driver_features else {} + json_serializer = features.setdefault("json_serializer", to_json) + json_deserializer = features.setdefault("json_deserializer", from_json) + + if json_serializer is not None: + parameter_config = statement_config.parameter_config.with_json_serializers( + json_serializer, deserializer=json_deserializer + ) + statement_config = statement_config.replace(parameter_config=parameter_config) + + return statement_config, features + + +def build_connection_config(connection_config: "Mapping[str, Any]") -> "dict[str, Any]": + """Normalize connection config into ``sqlanydb.connect(**kwargs)`` parameters. + + Maps friendly SQLSpec keys into SAP SQL Anywhere native connection string keys: + - ``user`` / ``username`` -> ``uid`` + - ``password`` -> ``pwd`` + - ``database`` / ``db`` -> ``dbn`` + - ``server`` / ``engine`` / ``server_name`` -> ``eng`` + - ``database_file`` -> ``dbf`` + - ``host`` + ``port`` -> ``host="host:port"`` (default port ``2638``) + """ + normalized: dict[str, Any] = dict(connection_config) + + extra = normalized.pop("extra", None) + if isinstance(extra, Mapping): + for key, value in extra.items(): + if value is not None: + normalized.setdefault(key, value) + + for pool_key in _POOL_CONFIG_KEYS: + normalized.pop(pool_key, None) + + for alias_key in ("user", "username"): + alias_val = normalized.pop(alias_key, None) + if alias_val is not None and "uid" not in normalized: + normalized["uid"] = alias_val + + password_val = normalized.pop("password", None) + if password_val is not None and "pwd" not in normalized: + normalized["pwd"] = password_val + + for db_key in ("database", "db"): + db_val = normalized.pop(db_key, None) + if db_val is not None and "dbn" not in normalized: + normalized["dbn"] = db_val + + for eng_key in ("server", "engine", "server_name"): + eng_val = normalized.pop(eng_key, None) + if eng_val is not None and "eng" not in normalized: + normalized["eng"] = eng_val + + dbf_val = normalized.pop("database_file", None) + if dbf_val is not None and "dbf" not in normalized: + normalized["dbf"] = dbf_val + + port_val = normalized.pop("port", None) + host_val = normalized.get("host") + if isinstance(host_val, str) and host_val and ":" not in host_val: + resolved_port = int(port_val) if port_val is not None else DEFAULT_SQLANYWHERE_PORT + normalized["host"] = f"{host_val}:{resolved_port}" + + return {key: value for key, value in normalized.items() if value is not None} + + +def resolve_column_names( + description: "Sequence[Any] | None", cache: "dict[int, tuple[Sequence[Any], list[str]]]" +) -> "list[str]": + """Resolve and cache column names from cursor description.""" + if not description: + return [] + desc_id = id(description) + cached = cache.get(desc_id) + if cached is not None and cached[0] is description: + return cached[1] + names = [col[0] for col in description] + if len(cache) >= _COLUMN_CACHE_MAX_SIZE: + cache.pop(next(iter(cache))) + cache[desc_id] = (description, names) + return names + + +def collect_rows( + fetched_data: "Sequence[Any] | None", + description: "Sequence[Any] | None", + cache: "dict[int, tuple[Sequence[Any], list[str]]]", +) -> "tuple[list[Any], list[str], Literal['dict', 'tuple']]": + """Collect SQL Anywhere fetched rows into a row list, column names, and row format.""" + if not description: + return [], [], "tuple" + column_names = resolve_column_names(description, cache) + if not fetched_data: + return [], column_names, "tuple" + data = fetched_data if isinstance(fetched_data, list) else list(fetched_data) + row_format: Literal["dict", "tuple"] = "dict" if isinstance(data[0], dict) else "tuple" + return data, column_names, row_format + + +def resolve_rowcount(cursor: Any) -> int: + """Resolve rowcount from a sqlanydb cursor.""" + if not has_rowcount(cursor): + return 0 + rowcount = cursor.rowcount + if isinstance(rowcount, int): + return max(rowcount, 0) + return 0 + + +def resolve_many_rowcount(cursor: Any, parameters: Any, *, fallback_count: int | None = None) -> int: + """Resolve affected row count for SQL Anywhere executemany.""" + rowcount = resolve_rowcount(cursor) + if rowcount > 0: + return rowcount + if fallback_count is not None: + return fallback_count + if isinstance(parameters, Sized): + return len(parameters) + return 0 + + +def extract_sqlstate_and_sqlcode(error: BaseException) -> tuple[str | None, int | None]: + """Extract ``(sqlstate, sqlcode)`` from a ``sqlanydb.Error`` instance.""" + sqlstate: str | None = getattr(error, "sqlstate", None) + sqlcode: int | None = getattr(error, "errorcode", getattr(error, "sqlcode", None)) + + raw_args = getattr(error, "args", None) + args: list[Any] = list(raw_args) if isinstance(raw_args, tuple) else [] + if len(args) >= _SQLANYDB_FULL_ARGS_LEN: + if sqlstate is None and isinstance(args[0], str): + sqlstate = args[0] + if sqlcode is None and isinstance(args[1], int): + sqlcode = args[1] + elif len(args) == _SQLANYDB_PAIR_ARGS_LEN: + if isinstance(args[0], int) and sqlcode is None: + sqlcode = args[0] + elif isinstance(args[0], str) and sqlstate is None: + sqlstate = args[0] + if isinstance(args[1], int) and sqlcode is None: + sqlcode = args[1] + elif len(args) == 1 and isinstance(args[0], int) and sqlcode is None: + sqlcode = args[0] + + return sqlstate, sqlcode + + +def create_mapped_exception(error: BaseException, *, logger: "Logger | None" = None) -> SQLSpecError: + """Map a sqlanydb exception into the SQLSpec exception hierarchy.""" + _ = logger + sqlstate, sqlcode = extract_sqlstate_and_sqlcode(error) + + if sqlcode is not None and sqlcode in _SQLCODE_MAPPING: + exc_cls, label = _SQLCODE_MAPPING[sqlcode] + return _create_error(error, sqlcode, sqlstate, exc_cls, label) + + if sqlstate is not None: + upper_state = sqlstate.upper() + if upper_state in _SQLSTATE_EXACT_MAPPING: + exc_cls, label = _SQLSTATE_EXACT_MAPPING[upper_state] + return _create_error(error, sqlcode, sqlstate, exc_cls, label) + if upper_state.startswith("08"): + return _create_error(error, sqlcode, sqlstate, DatabaseConnectionError, "connection error") + if upper_state.startswith("22"): + return _create_error(error, sqlcode, sqlstate, DataError, "data error") + if upper_state.startswith("23"): + return _create_error(error, sqlcode, sqlstate, IntegrityError, "integrity constraint violation") + + return _create_error(error, sqlcode, sqlstate, SQLSpecError, "database error") + + +def _bool_to_int(value: bool) -> int: + return int(value) + + +def _custom_type_coercions() -> "dict[type, Callable[[Any], Any]]": + return {bool: _bool_to_int, Decimal: str, **build_uuid_coercions()} + + +def _create_error( + error: BaseException, sqlcode: int | None, sqlstate: str | None, error_class: type[SQLSpecError], description: str +) -> SQLSpecError: + code_part = f"sqlcode={sqlcode}" if sqlcode is not None else "" + state_part = f"sqlstate={sqlstate}" if sqlstate is not None else "" + qualifier = ", ".join(part for part in (code_part, state_part) if part) + prefix = f"SQL Anywhere {description} [{qualifier}]" if qualifier else f"SQL Anywhere {description}" + msg = f"{prefix}: {error}" + exc = error_class(msg) + exc.__cause__ = error + return exc + + +_ = _SQLAnywhereDialect +driver_profile = build_profile() +default_statement_config = build_statement_config() diff --git a/sqlspec/adapters/sqlanywhere/data_dictionary.py b/sqlspec/adapters/sqlanywhere/data_dictionary.py new file mode 100644 index 000000000..4743347cc --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/data_dictionary.py @@ -0,0 +1,285 @@ +"""SAP SQL Anywhere database data dictionary implementation.""" + +import logging +from typing import TYPE_CHECKING, Any, ClassVar, cast + +from mypy_extensions import mypyc_attr + +from sqlspec.data_dictionary import ( + ColumnMetadata, + ForeignKeyMetadata, + IndexMetadata, + MetadataCapabilityProfile, + MetadataResult, + TableMetadata, +) +from sqlspec.data_dictionary.dialects.sqlanywhere import ( + SQLANYWHERE_CONFIG, + SQLANYWHERE_VERSION_PATTERN, + SqlanywhereVersionInfo, + build_sqlanywhere_metadata_capability_profile, + extract_sqlanywhere_version_value, + list_sqlanywhere_available_features, + merge_sqlanywhere_table_lists, + resolve_sqlanywhere_feature_flag, +) +from sqlspec.driver import SyncDataDictionaryBase +from sqlspec.utils.logging import get_logger, log_with_context + +if TYPE_CHECKING: + from collections.abc import Iterable, Sequence + + from sqlspec.adapters.sqlanywhere.driver import SqlanywhereDriver + +__all__ = ( + "SQLANYWHERE_CONFIG", + "SqlanywhereDataDictionary", + "SqlanywhereSyncDataDictionary", + "SqlanywhereVersionInfo", + "build_column_metadata", + "build_foreign_key_metadata", + "build_index_metadata", + "parse_sqlanywhere_version", +) + +logger = get_logger("sqlspec.adapters.sqlanywhere.data_dictionary") +_VERSION_PARTS_WITH_BUILD = 4 + + +def parse_sqlanywhere_version(row: "dict[str, Any] | None") -> "SqlanywhereVersionInfo | None": + """Parse the SQL Anywhere version string returned by the version query.""" + version_str = extract_sqlanywhere_version_value(row) + match = SQLANYWHERE_VERSION_PATTERN.search(version_str) if version_str else None + if match is None: + log_with_context( + logger, logging.DEBUG, "data_dictionary.version.unparsed", dialect="sqlanywhere", value=version_str + ) + return None + patch_val = int(match.group(3)) if match.group(3) else 0 + parts = version_str.split(".") if version_str else [] + build_number = parts[3].split()[0] if len(parts) >= _VERSION_PARTS_WITH_BUILD else None + return SqlanywhereVersionInfo(int(match.group(1)), int(match.group(2)), patch_val, build_number=build_number) + + +def build_column_metadata(rows: "Iterable[dict[str, Any]]") -> "list[ColumnMetadata]": + """Convert SQL Anywhere SYS.SYSTABCOL query rows to column metadata.""" + return [ + ColumnMetadata( + schema_name=str(r.get("schema_name", "")), + table_name=str(r.get("table_name", "")), + column_name=str(r.get("column_name", "")), + data_type=str(r.get("data_type", "")), + is_nullable=bool(r.get("is_nullable", True)), + column_default=str(r["column_default"]) if r.get("column_default") is not None else None, + ordinal_position=int(r.get("ordinal_position", 1)), + max_length=int(r.get("max_length", 0)) if r.get("max_length") is not None else 0, + numeric_precision=int(r.get("numeric_precision", 0)) if r.get("numeric_precision") is not None else 0, + numeric_scale=int(r.get("numeric_scale", 0)) if r.get("numeric_scale") is not None else 0, + is_primary=bool(r.get("is_primary")), + is_unique=bool(r.get("is_unique")), + is_identity=bool(r.get("is_identity")), + identity_generation=str(r["identity_generation"]) + if r.get("identity_generation") + else ("ALWAYS" if r.get("is_identity") else None), + is_generated=bool(r.get("is_generated")), + generation_expression=str(r["generation_expression"]) + if r.get("generation_expression") is not None + else None, + is_hidden=bool(r.get("is_hidden")), + ) + for r in rows + ] + + +def build_index_metadata(rows: "Iterable[dict[str, Any]]") -> "list[IndexMetadata]": + """Group per-column SQL Anywhere SYS.SYSIDX rows into index metadata.""" + indexes_map: dict[tuple[str, str, str], dict[str, Any]] = {} + for r in rows: + key = (str(r.get("schema_name", "")), str(r.get("table_name", "")), str(r.get("index_name", ""))) + if key not in indexes_map: + indexes_map[key] = { + "schema_name": key[0], + "table_name": key[1], + "index_name": key[2], + "columns": [], + "is_unique": bool(r.get("is_unique", 0)), + "is_primary": bool(r.get("is_primary", 0)), + } + col_name = str(r.get("column_name", "")) + if col_name: + indexes_map[key]["columns"].append(col_name) + + return [ + IndexMetadata( + schema_name=idx["schema_name"], + table_name=idx["table_name"], + index_name=idx["index_name"], + columns=idx["columns"], + is_unique=idx["is_unique"], + is_primary=idx["is_primary"], + ) + for idx in indexes_map.values() + ] + + +def build_foreign_key_metadata(rows: "Iterable[dict[str, Any]]") -> "list[ForeignKeyMetadata]": + """Convert SQL Anywhere SYS.SYSFOREIGNKEY query rows to foreign key metadata.""" + return [ + ForeignKeyMetadata( + table_name=str(r.get("table_name", "")), + column_name=str(r.get("column_name", "")), + referenced_table=str(r.get("referenced_table_name", "")), + referenced_column=str(r.get("referenced_column_name", "")), + constraint_name=str(r.get("constraint_name", "")), + schema=str(r["schema_name"]) if r.get("schema_name") else None, + referenced_schema=str(r["referenced_schema"]) if r.get("referenced_schema") else None, + ) + for r in rows + ] + + +@mypyc_attr(allow_interpreted_subclasses=True, native_class=False) +class SqlanywhereSyncDataDictionary(SyncDataDictionaryBase): + """Synchronous SAP SQL Anywhere data dictionary.""" + + dialect: ClassVar[str] = "sqlanywhere" + + def __init__(self) -> None: + super().__init__() + + def get_metadata_capabilities( + self, driver: "SqlanywhereDriver", domains: "Sequence[str] | None" = None + ) -> MetadataCapabilityProfile: + """Return the SQL Anywhere metadata capability profile.""" + _ = driver + return build_sqlanywhere_metadata_capability_profile(type(self).__name__, domains) + + def get_version(self, driver: "SqlanywhereDriver") -> "SqlanywhereVersionInfo | None": + """Detect and cache the SQL Anywhere server version.""" + driver_id = id(driver) + if driver_id in self._version_fetch_attempted: + return cast("SqlanywhereVersionInfo | None", self._version_cache.get(driver_id)) + + version_row = driver.select_one_or_none(self.get_query("version", "current")) + version_info = parse_sqlanywhere_version(version_row) + if version_info is None: + self._log_version_unavailable(type(self).dialect, "missing") + else: + self._log_version_detected(type(self).dialect, version_info) + self.cache_version(driver_id, version_info) + return version_info + + def get_feature_flag(self, driver: "SqlanywhereDriver", feature: str) -> bool: + """Check whether the connected SQL Anywhere server supports a feature.""" + version_info = self.get_version(driver) + return resolve_sqlanywhere_feature_flag(feature, version_info) + + def get_optimal_type(self, driver: "SqlanywhereDriver", type_category: str) -> str: + """Return the optimal SQL Anywhere SQL type for a logical category.""" + _ = driver + return SQLANYWHERE_CONFIG.get_optimal_type(type_category) + + def list_available_features(self) -> "list[str]": + """List feature flags defined for the SQL Anywhere dialect.""" + return list_sqlanywhere_available_features() + + def get_tables(self, driver: "SqlanywhereDriver", schema: "str | None" = None) -> "list[TableMetadata]": + """Return tables in topological dependency order for a SQL Anywhere schema.""" + schema_name = self.resolve_schema(schema) + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="tables") + ordered_rows = cast( + "list[TableMetadata]", + driver.select( + self.get_query("tables", "by_schema"), + schema_name=schema_name, + table_name=None, + schema_type=TableMetadata, + ), + ) + all_rows = cast( + "list[TableMetadata]", + driver.select( + self.get_query("tables", "all_by_schema"), + schema_name=schema_name, + table_name=None, + schema_type=TableMetadata, + ), + ) + return merge_sqlanywhere_table_lists(ordered_rows, all_rows) + + def get_columns( + self, driver: "SqlanywhereDriver", table: "str | None" = None, schema: "str | None" = None + ) -> "list[ColumnMetadata]": + """Return column metadata for a table or entire SQL Anywhere schema.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="columns") + rows = driver.select(self.get_query("columns", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="columns") + rows = driver.select(self.get_query("columns", "by_table"), schema_name=schema_name, table_name=table_name) + return build_column_metadata(rows) + + def get_indexes( + self, driver: "SqlanywhereDriver", table: "str | None" = None, schema: "str | None" = None + ) -> "list[IndexMetadata]": + """Return index metadata for a table or entire SQL Anywhere schema.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="indexes") + rows = driver.select(self.get_query("indexes", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="indexes") + rows = driver.select(self.get_query("indexes", "by_table"), schema_name=schema_name, table_name=table_name) + return build_index_metadata(rows) + + def get_foreign_keys( + self, driver: "SqlanywhereDriver", table: "str | None" = None, schema: "str | None" = None + ) -> "list[ForeignKeyMetadata]": + """Return foreign key metadata for a table or entire SQL Anywhere schema.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="foreign_keys") + rows = driver.select(self.get_query("foreign_keys", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="foreign_keys") + rows = driver.select( + self.get_query("foreign_keys", "by_table"), schema_name=schema_name, table_name=table_name + ) + return build_foreign_key_metadata(rows) + + def get_constraints( + self, driver: "SqlanywhereDriver", table: "str | None" = None, schema: "str | None" = None + ) -> MetadataResult: + """Get SQL Anywhere constraint metadata.""" + schema_name = self.resolve_schema(schema) + table_name = self.resolve_identifier(table) if table is not None else None + query = self.get_query("constraints", "by_table" if table_name else "by_schema") + rows = driver.select(query, schema_name=schema_name, table_name=table_name) + return MetadataResult("constraints", items=tuple(rows)) + + def get_views(self, driver: "SqlanywhereDriver", schema: "str | None" = None) -> MetadataResult: + """Get SQL Anywhere view metadata.""" + schema_name = self.resolve_schema(schema) + query = self.get_query("views", "by_schema") + rows = driver.select(query, schema_name=schema_name, view_name=None) + return MetadataResult("views", items=tuple(rows)) + + def get_objects(self, driver: "SqlanywhereDriver", schema: "str | None" = None) -> MetadataResult: + """Get tables, views, and procedures from the SQL Anywhere catalog.""" + rows = driver.select( + self.get_query("objects", "by_schema"), schema_name=self.resolve_schema(schema), object_name=None + ) + return MetadataResult("objects", items=tuple(rows)) + + def get_schemas(self, driver: "SqlanywhereDriver") -> MetadataResult: + """Get SQL Anywhere schema metadata.""" + query = self.get_query("schemas", "by_schema") + rows = driver.select(query, schema_name=None) + return MetadataResult("schemas", items=tuple(rows)) + + +SqlanywhereDataDictionary = SqlanywhereSyncDataDictionary diff --git a/sqlspec/adapters/sqlanywhere/driver.py b/sqlspec/adapters/sqlanywhere/driver.py new file mode 100644 index 000000000..908f800c3 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/driver.py @@ -0,0 +1,359 @@ +"""SAP SQL Anywhere database driver implementation over sqlanydb.""" + +import contextlib +from collections.abc import Sized +from typing import TYPE_CHECKING, Any, cast + +from sqlspec.adapters.sqlanywhere._typing import ( + SqlanywhereConnection, + SqlanywhereCursor, + SqlanywhereError, + SqlanywhereRawCursor, + SqlanywhereSessionContext, +) +from sqlspec.adapters.sqlanywhere.core import ( + build_insert_statement, + collect_rows, + create_mapped_exception, + default_statement_config, + driver_profile, + format_identifier, + normalize_execute_many_parameters, + normalize_execute_parameters, + resolve_column_names, + resolve_many_rowcount, + resolve_rowcount, +) +from sqlspec.adapters.sqlanywhere.data_dictionary import SqlanywhereSyncDataDictionary +from sqlspec.core import SQL, StatementConfig, get_cache_config, register_driver_profile +from sqlspec.driver import ( + BaseSyncExceptionHandler, + ExecutionResult, + SyncDriverAdapterBase, + SyncRowStream, + rows_to_dicts, + validate_savepoint_name, +) +from sqlspec.exceptions import SQLSpecError +from sqlspec.utils.logging import get_logger + +if TYPE_CHECKING: + from collections.abc import Sequence + + from sqlspec.adapters.sqlanywhere._typing import SqlanywhereQueryParams as QueryParams + from sqlspec.builder import QueryBuilder + from sqlspec.core import ArrowResult, SQLResult, Statement, StatementFilter + from sqlspec.driver import CachedQuery + from sqlspec.storage import StorageBridgeJob, StorageDestination, StorageFormat, StorageTelemetry + from sqlspec.typing import StatementParameters + +__all__ = ( + "SqlanywhereCursor", + "SqlanywhereDriver", + "SqlanywhereExceptionHandler", + "SqlanywhereSessionContext", + "SqlanywhereStreamSource", +) + +logger = get_logger("sqlspec.adapters.sqlanywhere") + + +class SqlanywhereExceptionHandler(BaseSyncExceptionHandler): + """Context manager for handling SAP SQL Anywhere sqlanydb exceptions.""" + + __slots__ = () + + def _handle_exception(self, exc_type: "type[BaseException] | None", exc_val: "BaseException") -> bool: + if exc_type is None: + return False + if isinstance(exc_val, (SqlanywhereError, Exception)) and not isinstance(exc_val, SQLSpecError): + self.pending_exception = create_mapped_exception(exc_val, logger=logger) + return True + return False + + +class SqlanywhereStreamSource: + """Native SQL Anywhere sqlanydb chunk source backed by ``cursor.fetchmany()``.""" + + __slots__ = ("_chunk_size", "_column_names", "_cursor_manager", "_driver", "_parameters", "_sql") + + def __init__(self, driver: "SqlanywhereDriver", sql: str, parameters: Any, chunk_size: int) -> None: + self._driver = driver + self._sql = sql + self._parameters = parameters + self._chunk_size = chunk_size + self._cursor_manager: SqlanywhereCursor | None = None + self._column_names: list[str] | None = None + + def start(self) -> None: + cursor_manager = self._driver.with_cursor(self._driver.connection) + try: + cursor = cursor_manager.__enter__() + handler = self._driver.handle_database_exceptions() + with handler: + normalized_params = normalize_execute_parameters(self._parameters) + if normalized_params: + cursor.execute(self._sql, normalized_params) + else: + cursor.execute(self._sql) + self._driver._check_pending_exception(handler) + except BaseException: + with contextlib.suppress(Exception): + cursor_manager.__exit__(None, None, None) + raise + self._cursor_manager = cursor_manager + + def fetch_chunk(self) -> "list[dict[str, Any]]": + cursor_manager = self._cursor_manager + if cursor_manager is None or cursor_manager.cursor is None: + return [] + cursor = cursor_manager.cursor + handler = self._driver.handle_database_exceptions() + rows: Any = [] + with handler: + rows = cursor.fetchmany(self._chunk_size) + self._driver._check_pending_exception(handler) + if not rows: + return [] + column_names = self._column_names + if column_names is None: + column_names = resolve_column_names(cursor.description or None, self._driver._column_name_cache) + self._column_names = column_names + return rows_to_dicts(rows, column_names) + + def close(self, error: bool = False) -> None: + _ = error + cursor_manager = self._cursor_manager + self._cursor_manager = None + if cursor_manager is not None: + with contextlib.suppress(Exception): + cursor_manager.__exit__(None, None, None) + + +class SqlanywhereDriver(SyncDriverAdapterBase): + """SAP SQL Anywhere database driver using sqlanydb.""" + + __slots__ = ("_column_name_cache", "_data_dictionary", "_transaction_active") + dialect = "sqlanywhere" + + def __init__( + self, + connection: "SqlanywhereConnection", + statement_config: "StatementConfig | None" = None, + driver_features: "dict[str, Any] | None" = None, + ) -> None: + if statement_config is None: + statement_config = default_statement_config.replace( + enable_caching=get_cache_config().compiled_cache_enabled + ) + + super().__init__(connection=connection, statement_config=statement_config, driver_features=driver_features) + self._data_dictionary: SqlanywhereSyncDataDictionary | None = None + self._column_name_cache: dict[int, tuple[Any, list[str]]] = {} + self._transaction_active = False + + @property + def data_dictionary(self) -> "SqlanywhereSyncDataDictionary": + if self._data_dictionary is None: + self._data_dictionary = SqlanywhereSyncDataDictionary() + return self._data_dictionary + + def dispatch_execute(self, cursor: "SqlanywhereRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + normalized_params = normalize_execute_parameters(prepared_parameters) + if normalized_params: + cursor.execute(sql, normalized_params) + else: + cursor.execute(sql) + + if statement.returns_rows(): + fetched_data = cursor.fetchall() + description = cursor.description or None + rows, column_names, row_format = collect_rows(fetched_data, description, self._column_name_cache) + return self.create_execution_result( + cursor, + selected_data=rows, + column_names=column_names, + data_row_count=len(rows), + is_select_result=True, + row_format=row_format, + ) + + return self.create_execution_result(cursor, rowcount_override=resolve_rowcount(cursor)) + + def dispatch_execute_many(self, cursor: "SqlanywhereRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + prepared_parameters = normalize_execute_many_parameters(prepared_parameters) + parameter_count = len(prepared_parameters) if isinstance(prepared_parameters, Sized) else None + cursor.executemany(sql, cast("Sequence[QueryParams]", prepared_parameters)) + + affected_rows = resolve_many_rowcount(cursor, prepared_parameters, fallback_count=parameter_count) + return self.create_execution_result(cursor, rowcount_override=affected_rows, is_many_result=True) + + def dispatch_execute_script(self, cursor: "SqlanywhereRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + statements = self.split_script_statements(sql, statement.statement_config, strip_trailing_semicolon=True) + normalized_params = normalize_execute_parameters(prepared_parameters) + + successful_count = 0 + for stmt in statements: + if normalized_params: + cursor.execute(stmt, normalized_params) + else: + cursor.execute(stmt) + successful_count += 1 + return self.create_execution_result( + cursor, statement_count=len(statements), successful_statements=successful_count, is_script_result=True + ) + + def begin(self) -> None: + """Mark a transaction as active on the SQL Anywhere DB-API 2.0 connection.""" + self._transaction_active = True + + def commit(self) -> None: + try: + self.connection.commit() + self._transaction_active = False + except Exception as exc: + msg = f"Failed to commit SQL Anywhere transaction: {exc}" + raise SQLSpecError(msg) from exc + + def rollback(self) -> None: + try: + self.connection.rollback() + except Exception as exc: + msg = f"Failed to rollback SQL Anywhere transaction: {exc}" + raise SQLSpecError(msg) from exc + finally: + self._transaction_active = False + + def _release_open_work(self) -> None: + """Roll back work left open before connection is returned to the pool.""" + with contextlib.suppress(SQLSpecError): + self.rollback() + + def with_cursor(self, connection: "SqlanywhereConnection") -> "SqlanywhereCursor": + return SqlanywhereCursor(connection) + + def handle_database_exceptions(self) -> "SqlanywhereExceptionHandler": + return SqlanywhereExceptionHandler() + + def dispatch_select_stream(self, statement: "SQL", chunk_size: int) -> "SyncRowStream[dict[str, Any]] | None": + """Return a native SQL Anywhere sqlanydb row stream backed by ``fetchmany()``.""" + if not statement.returns_rows(): + return None + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + return SyncRowStream(SqlanywhereStreamSource(self, sql, prepared_parameters, chunk_size)) + + def create_savepoint(self, name: str) -> None: + self.execute_script(f"SAVEPOINT {validate_savepoint_name(name)}") + + def release_savepoint(self, name: str) -> None: + self.execute_script(f"RELEASE SAVEPOINT {validate_savepoint_name(name)}") + + def rollback_to_savepoint(self, name: str) -> None: + self.execute_script(f"ROLLBACK TO SAVEPOINT {validate_savepoint_name(name)}") + + def has_schema(self, schema: str) -> bool: + """Return whether the specified user/schema exists in SYS.SYSUSER.""" + result = False + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.execute("SELECT 1 FROM SYS.SYSUSER WHERE user_name = ?", (schema,)) + result = cursor.fetchone() is not None + self._check_pending_exception(exc_handler) + return result + + def select_to_storage( + self, + statement: "Statement | QueryBuilder | SQL | str", + destination: "StorageDestination", + /, + *parameters: "StatementParameters | StatementFilter", + statement_config: "StatementConfig | None" = None, + partitioner: "dict[str, object] | None" = None, + format_hint: "StorageFormat | None" = None, + telemetry: "StorageTelemetry | None" = None, + **kwargs: Any, + ) -> "StorageBridgeJob": + """Execute a query and write the Arrow result to storage.""" + self._require_capability("arrow_export_enabled") + arrow_result = self.select_to_arrow(statement, *parameters, statement_config=statement_config, **kwargs) + pipeline = self._storage_pipeline() + telemetry_payload = self._write_storage_result( + arrow_result, destination, format_hint=format_hint, pipeline=pipeline + ) + self._attach_partition_telemetry(telemetry_payload, partitioner) + return self._storage_job(telemetry_payload, telemetry) + + def load_from_arrow( + self, + table: str, + source: "ArrowResult | Any", + *, + partitioner: "dict[str, object] | None" = None, + overwrite: bool = False, + telemetry: "StorageTelemetry | None" = None, + ) -> "StorageBridgeJob": + """Load Arrow data into the target SQL Anywhere table via executemany.""" + self._require_capability("arrow_import_enabled") + arrow_table = self._coerce_arrow_table(source) + if overwrite: + statement = f"DELETE FROM {format_identifier(table)}" + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.execute(statement) + self._check_pending_exception(exc_handler) + + columns, records = self._arrow_table_to_rows(arrow_table) + if records: + insert_sql = build_insert_statement(table, columns) + needs_preparation = self._arrow_rows_need_preparation(arrow_table) + prepared_records = ( + self.prepare_driver_parameters(records, self.statement_config, is_many=True) + if needs_preparation + else records + ) + normalized_records = normalize_execute_many_parameters(prepared_records) + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.executemany(insert_sql, cast("Sequence[QueryParams]", normalized_records)) + self._check_pending_exception(exc_handler) + + telemetry_payload = self._ingest_telemetry(arrow_table) + telemetry_payload["destination"] = table + self._attach_partition_telemetry(telemetry_payload, partitioner) + return self._storage_job(telemetry_payload, telemetry) + + def load_from_storage( + self, + table: str, + source: "StorageDestination", + *, + file_format: "StorageFormat", + partitioner: "dict[str, object] | None" = None, + overwrite: bool = False, + ) -> "StorageBridgeJob": + """Load staged artifacts from storage into the target SQL Anywhere table.""" + arrow_table, inbound = self._read_storage_arrow(source, file_format=file_format) + return self.load_from_arrow(table, arrow_table, partitioner=partitioner, overwrite=overwrite, telemetry=inbound) + + def collect_rows(self, cursor: "SqlanywhereRawCursor", fetched: "list[Any]") -> "tuple[list[Any], list[str], int]": + rows, column_names, _row_format = collect_rows(fetched, cursor.description or None, self._column_name_cache) + return rows, column_names, len(rows) + + def resolve_rowcount(self, cursor: "SqlanywhereRawCursor") -> int: + return resolve_rowcount(cursor) + + def _execute_cache_hit( + self, sql: str, params: "tuple[Any, ...] | list[Any] | dict[str, Any]", cached: "CachedQuery" + ) -> "SQLResult": + normalized_params = normalize_execute_parameters(params) if isinstance(params, list) else params + return super()._execute_cache_hit(sql, normalized_params, cached) + + def _connection_in_transaction(self) -> bool: + """Return whether a transaction opened by this driver remains active.""" + return self._transaction_active + + +register_driver_profile("sqlanywhere", driver_profile) diff --git a/sqlspec/adapters/sqlanywhere/events/__init__.py b/sqlspec/adapters/sqlanywhere/events/__init__.py new file mode 100644 index 000000000..70252c8ad --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/events/__init__.py @@ -0,0 +1,5 @@ +"""Events integration for SAP SQL Anywhere adapter.""" + +from sqlspec.adapters.sqlanywhere.events.store import SqlanywhereEventQueueStore, SqlanywhereSyncEventQueueStore + +__all__ = ("SqlanywhereEventQueueStore", "SqlanywhereSyncEventQueueStore") diff --git a/sqlspec/adapters/sqlanywhere/events/store.py b/sqlspec/adapters/sqlanywhere/events/store.py new file mode 100644 index 000000000..9a4e23b35 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/events/store.py @@ -0,0 +1,31 @@ +"""SAP SQL Anywhere event queue store with SQL Anywhere-compatible DDL.""" + +from sqlspec.adapters.sqlanywhere.config import SqlanywhereConfig +from sqlspec.extensions.events import BaseEventQueueStore + +__all__ = ("SqlanywhereEventQueueStore", "SqlanywhereSyncEventQueueStore") + +_VARCHAR_TEXT_THRESHOLD = 4000 + + +class SqlanywhereSyncEventQueueStore(BaseEventQueueStore[SqlanywhereConfig]): + """SAP SQL Anywhere DDL hooks for the event queue store.""" + + __slots__ = () + + def _column_types(self) -> tuple[str, str, str]: + return "LONG VARCHAR", "LONG VARCHAR", "TIMESTAMP" + + def _string_type(self, length: int) -> str: + if length >= _VARCHAR_TEXT_THRESHOLD: + return "LONG VARCHAR" + return f"VARCHAR({length})" + + def _integer_type(self) -> str: + return "INTEGER" + + def _timestamp_default(self) -> str: + return "CURRENT UTC TIMESTAMP" + + +SqlanywhereEventQueueStore = SqlanywhereSyncEventQueueStore diff --git a/sqlspec/adapters/sqlanywhere/litestar/__init__.py b/sqlspec/adapters/sqlanywhere/litestar/__init__.py new file mode 100644 index 000000000..d4b7031a6 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/litestar/__init__.py @@ -0,0 +1,5 @@ +"""Litestar integration for SAP SQL Anywhere adapter.""" + +from sqlspec.adapters.sqlanywhere.litestar.store import SqlanywhereStore + +__all__ = ("SqlanywhereStore",) diff --git a/sqlspec/adapters/sqlanywhere/litestar/store.py b/sqlspec/adapters/sqlanywhere/litestar/store.py new file mode 100644 index 000000000..e2b325534 --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/litestar/store.py @@ -0,0 +1,215 @@ +"""SAP SQL Anywhere Litestar Store implementation.""" + +from datetime import UTC, datetime, timedelta +from typing import TYPE_CHECKING, Any + +from sqlspec.adapters.sqlanywhere._typing import SqlanywhereCursor +from sqlspec.adapters.sqlanywhere.core import quote_double_identifier, split_schema_and_table +from sqlspec.extensions.litestar.store import BaseSQLSpecStore +from sqlspec.utils.sync_tools import async_ + +if TYPE_CHECKING: + from sqlspec.adapters.sqlanywhere.config import SqlanywhereConfig + +__all__ = ("SqlanywhereStore",) + + +class SqlanywhereStore(BaseSQLSpecStore["SqlanywhereConfig"]): + """SAP SQL Anywhere-backed session store using sqlanydb sync sessions.""" + + __slots__ = () + + def __init__(self, config: "SqlanywhereConfig") -> None: + super().__init__(config) + + async def create_table(self) -> None: + """Create the session table if it doesn't exist.""" + if not self.create_schema_enabled: + await self.reconcile_schema() + return + await async_(self._create_table)() + await self.reconcile_schema(assume_existing=True) + + async def get(self, key: str, renew_for: "int | timedelta | None" = None) -> "bytes | None": + """Get a session value by key.""" + return await async_(self._get)(key, renew_for) + + async def set(self, key: str, value: "str | bytes", expires_in: "int | timedelta | None" = None) -> None: + """Store a session value.""" + await async_(self._set)(key, value, expires_in) + + async def delete(self, key: str) -> None: + """Delete a session by key.""" + await async_(self._delete)(key) + + async def delete_all(self) -> None: + """Delete all sessions from the store.""" + await async_(self._delete_all)() + + async def exists(self, key: str) -> bool: + """Check if a session key exists and is not expired.""" + return await async_(self._exists)(key) + + async def expires_in(self, key: str) -> "int | None": + """Get the time in seconds until the session expires.""" + return await async_(self._expires_in)(key) + + async def delete_expired(self) -> int: + """Delete all expired sessions.""" + return await async_(self._delete_expired)() + + def _table_ddl(self) -> str: + """Get SQL Anywhere CREATE TABLE IF NOT EXISTS DDL.""" + _, table_name = split_schema_and_table(self._table_name) + table_ref = _table_ref(self._table_name) + index_name = quote_double_identifier(f"IX_{table_name}_expires_at") + return ( + f"CREATE TABLE IF NOT EXISTS {table_ref} (" + "session_id VARCHAR(255) NOT NULL PRIMARY KEY, " + "data LONG BINARY NOT NULL, " + "expires_at TIMESTAMP NULL, " + "created_at TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL, " + "updated_at TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP NOT NULL" + ");\n" + f"CREATE INDEX IF NOT EXISTS {index_name} ON {table_ref}(expires_at);" + ) + + def _drop_table_sql(self) -> "list[str]": + """Get SQL Anywhere DROP TABLE IF EXISTS statements.""" + table_ref = _table_ref(self._table_name) + return [f"DROP TABLE IF EXISTS {table_ref};"] + + def _create_table(self) -> None: + with self._config.provide_session() as driver: + driver.execute_script(self._table_ddl()) + driver.commit() + self._log_table_created() + + def _get(self, key: str, renew_for: "int | timedelta | None" = None) -> "bytes | None": + table_ref = _table_ref(self._table_name) + sql = f""" + SELECT data, expires_at FROM {table_ref} + WHERE session_id = ? + AND (expires_at IS NULL OR expires_at > CURRENT UTC TIMESTAMP) + """ + with self._config.provide_connection() as conn: + with SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, (key,)) + row = cursor.fetchone() + + if row is None: + return None + + expires_at = _normalize_utc(_row_value(row, "expires_at", 1)) + if renew_for is not None and expires_at is not None: + new_expires_at = self._calculate_expires_at(renew_for) + if new_expires_at is not None: + with SqlanywhereCursor(conn) as update_cursor: + update_cursor.execute( + f""" + UPDATE {table_ref} + SET expires_at = ?, updated_at = CURRENT UTC TIMESTAMP + WHERE session_id = ? + """, + (new_expires_at, key), + ) + conn.commit() + + return _coerce_bytes(_row_value(row, "data", 0)) + + def _set(self, key: str, value: "str | bytes", expires_in: "int | timedelta | None" = None) -> None: + data = self._value_to_bytes(value) + expires_at = self._calculate_expires_at(expires_in) + table_ref = _table_ref(self._table_name) + sql = f""" + INSERT INTO {table_ref} (session_id, data, expires_at, updated_at) + ON EXISTING UPDATE + VALUES (?, ?, ?, CURRENT UTC TIMESTAMP) + """ + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, (key, data, expires_at)) + conn.commit() + + def _delete(self, key: str) -> None: + table_ref = _table_ref(self._table_name) + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(f"DELETE FROM {table_ref} WHERE session_id = ?", (key,)) + conn.commit() + + def _delete_all(self) -> None: + table_ref = _table_ref(self._table_name) + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(f"TRUNCATE TABLE {table_ref}") + conn.commit() + self._log_delete_all() + + def _exists(self, key: str) -> bool: + table_ref = _table_ref(self._table_name) + sql = f""" + SELECT 1 FROM {table_ref} + WHERE session_id = ? + AND (expires_at IS NULL OR expires_at > CURRENT UTC TIMESTAMP) + """ + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, (key,)) + return cursor.fetchone() is not None + + def _expires_in(self, key: str) -> "int | None": + table_ref = _table_ref(self._table_name) + sql = f""" + SELECT expires_at FROM {table_ref} + WHERE session_id = ? + """ + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql, (key,)) + row = cursor.fetchone() + + if row is None: + return None + + expires_at = _normalize_utc(_row_value(row, "expires_at", 0)) + if expires_at is None: + return None + + now = datetime.now(UTC) + if expires_at <= now: + return 0 + return int((expires_at - now).total_seconds()) + + def _delete_expired(self) -> int: + table_ref = _table_ref(self._table_name) + sql = f"DELETE FROM {table_ref} WHERE expires_at <= CURRENT UTC TIMESTAMP" + with self._config.provide_connection() as conn, SqlanywhereCursor(conn) as cursor: + cursor.execute(sql) + conn.commit() + count = cursor.rowcount if cursor.rowcount and cursor.rowcount > 0 else 0 + if count > 0: + self._log_delete_expired(count) + return count + + +def _table_ref(table_name: str) -> str: + schema_name, bare_table_name = split_schema_and_table(table_name) + return f"{quote_double_identifier(schema_name)}.{quote_double_identifier(bare_table_name)}" + + +def _row_value(row: Any, key: str, index: int) -> Any: + if isinstance(row, dict): + return row.get(key) + return row[index] + + +def _coerce_bytes(value: Any) -> bytes: + if isinstance(value, bytes): + return value + if isinstance(value, (bytearray, memoryview)): + return bytes(value) + return str(value).encode("utf-8") + + +def _normalize_utc(value: "datetime | None") -> "datetime | None": + if value is None: + return None + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) diff --git a/sqlspec/adapters/sqlanywhere/migrations.py b/sqlspec/adapters/sqlanywhere/migrations.py new file mode 100644 index 000000000..767249bee --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/migrations.py @@ -0,0 +1,178 @@ +"""SAP SQL Anywhere-specific migration tracker.""" + +import logging +import os +from contextlib import suppress +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from collections.abc import Sequence + +from sqlspec.adapters.sqlanywhere.core import ( + escape_sql_literal, + format_identifier, + quote_double_identifier, + split_schema_and_table, +) +from sqlspec.builder import CreateTable, sql +from sqlspec.migrations.tracker import SyncMigrationTracker +from sqlspec.migrations.version import parse_version +from sqlspec.observability import resolve_db_system +from sqlspec.utils.logging import get_logger, log_with_context + +if TYPE_CHECKING: + from sqlspec.driver import SyncDriverAdapterBase + +__all__ = ("SqlanywhereMigrationTrackerMixin", "SqlanywhereSyncMigrationTracker") + +logger = get_logger("sqlspec.migrations.sqlanywhere") + + +class SqlanywhereMigrationTrackerMixin: + """SAP SQL Anywhere-specific migration table DDL and schema maintenance.""" + + __slots__ = () + + version_table: str + + def _tracking_table_ddl(self) -> CreateTable: + """Return SQL Anywhere-compatible migration tracking table DDL.""" + return ( + sql + .create_table(self.version_table) + .if_not_exists() + .column("version_num", "VARCHAR(32)", primary_key=True) + .column("version_type", "VARCHAR(16)") + .column("execution_sequence", "INTEGER") + .column("description", "LONG VARCHAR") + .column("applied_at", "TIMESTAMP", default="CURRENT UTC TIMESTAMP", not_null=True) + .column("execution_time_ms", "INTEGER") + .column("checksum", "VARCHAR(64)") + .column("applied_by", "VARCHAR(255)") + .column("replaces", "LONG VARCHAR") + ) + + def _idempotent_tracking_table_ddl_text(self) -> str: + """Render ``CREATE TABLE IF NOT EXISTS`` DDL with SQL Anywhere native column types.""" + column_lines: list[str] = [] + for column_def in self._tracking_table_ddl().columns: + default_clause = f" DEFAULT {column_def.default}" if column_def.default else "" + null_clause = " NOT NULL" if column_def.primary_key or column_def.not_null else " NULL" + primary_key_clause = " PRIMARY KEY" if column_def.primary_key else "" + column_lines.append( + f" {quote_double_identifier(column_def.name)} {column_def.dtype}{default_clause}{null_clause}{primary_key_clause}" + ) + return ( + f"CREATE TABLE IF NOT EXISTS {format_identifier(self.version_table)} (\n" + + ",\n".join(column_lines) + + "\n);" + ) + + def _existing_columns_query(self) -> str: + """Return SQL Anywhere catalog query text for migration tracking table columns.""" + schema_name, table_name = split_schema_and_table(self.version_table) + return f""" + SELECT c.column_name AS column_name + FROM SYS.SYSTABCOL c + INNER JOIN SYS.SYSTAB t ON t.table_id = c.table_id + INNER JOIN SYS.SYSUSER u ON u.user_id = t.creator + WHERE t.table_name = '{escape_sql_literal(table_name)}' + AND u.user_name = '{escape_sql_literal(schema_name)}' + """ + + def _add_column_statement_text(self, column_name: str) -> str | None: + """Return SQL Anywhere ALTER TABLE text for a missing migration column.""" + target_create = self._tracking_table_ddl() + column_def = next((col for col in target_create.columns if col.name.lower() == column_name), None) + if column_def is None: + return None + default_clause = f" DEFAULT {column_def.default}" if column_def.default else "" + null_clause = " NOT NULL" if column_def.not_null else " NULL" + return ( + f"ALTER TABLE {format_identifier(self.version_table)} " + f"ADD {quote_double_identifier(column_def.name)} {column_def.dtype}{default_clause}{null_clause}" + ) + + +class SqlanywhereSyncMigrationTracker(SqlanywhereMigrationTrackerMixin, SyncMigrationTracker): + """Sync migration tracker using SAP SQL Anywhere DDL.""" + + def ensure_tracking_table(self, driver: "SyncDriverAdapterBase") -> None: + """Create the migration tracking table if it does not exist.""" + driver.execute_script(self._idempotent_tracking_table_ddl_text()) + self._commit_if_needed(driver) + self._migrate_schema_if_needed(driver) + + def record_migration( + self, + driver: "SyncDriverAdapterBase", + version: str, + description: str, + execution_time_ms: int, + checksum: str, + *, + commit: bool = True, + replaces: "Sequence[str] | None" = None, + ) -> None: + """Record a successfully applied migration with SQL Anywhere-compatible metadata.""" + parsed_version = parse_version(version) + version_type = parsed_version.type.value + result = driver.execute(self._next_execution_sequence_query()) + next_sequence = result.get_data()[0]["next_seq"] if result.data else 1 + applied_by = os.environ.get("USER", "unknown") + if replaces: + statement = self._record_squashed_migration_statement( + version, + version_type, + next_sequence, + description, + execution_time_ms, + checksum, + applied_by, + ",".join(replaces), + ) + else: + statement = self._record_migration_statement( + version, version_type, next_sequence, description, execution_time_ms, checksum, applied_by + ) + driver.execute(statement) + if commit: + self._commit_if_needed(driver) + + def _commit_if_needed(self, driver: "SyncDriverAdapterBase") -> None: + if hasattr(driver, "driver_features"): + self._safe_commit(driver) + return + driver.commit() + + def _migrate_schema_if_needed(self, driver: "SyncDriverAdapterBase") -> None: + """Check and add missing tracking table columns through SQL Anywhere catalog tables.""" + try: + rows = driver.select(self._existing_columns_query()) + existing_columns = {str(row["column_name"]).lower() for row in rows if row.get("column_name") is not None} + missing_columns = self._detect_missing_columns(existing_columns) + if not missing_columns: + return + for column_name in sorted(missing_columns): + self._add_column(driver, column_name) + self._commit_if_needed(driver) + except Exception as exc: + with suppress(Exception): + driver.rollback() + log_with_context( + logger, + logging.ERROR, + "migration.track", + db_system=resolve_db_system(type(driver).__name__), + table=self.version_table, + operation="schema_check", + status="failed", + error_type=type(exc).__name__, + ) + + def _add_column(self, driver: "SyncDriverAdapterBase", column_name: str) -> None: + """Add a single missing migration tracking column.""" + add_column_sql = self._add_column_statement_text(column_name) + if add_column_sql is None: + return + driver.execute_script(add_column_sql) diff --git a/sqlspec/adapters/sqlanywhere/pool.py b/sqlspec/adapters/sqlanywhere/pool.py new file mode 100644 index 000000000..db1d4effb --- /dev/null +++ b/sqlspec/adapters/sqlanywhere/pool.py @@ -0,0 +1,237 @@ +"""SAP SQL Anywhere thread-safe connection pool.""" + +import contextlib +import logging +import threading +import time +from contextlib import contextmanager +from importlib import import_module +from typing import TYPE_CHECKING, Any, cast + +from sqlspec.adapters.sqlanywhere._typing import SqlanywhereConnection +from sqlspec.exceptions import MissingDependencyError +from sqlspec.utils.logging import POOL_LOGGER_NAME, get_logger, log_with_context +from sqlspec.utils.uuids import uuid4 + +if TYPE_CHECKING: + from collections.abc import Callable, Generator + +__all__ = ("SqlanywhereConnectionPool",) + +logger = get_logger(POOL_LOGGER_NAME) +_ADAPTER_NAME = "sqlanywhere" + + +def _load_sqlanydb_module() -> Any: + """Import ``sqlanydb`` lazily or raise ``MissingDependencyError``.""" + try: + return import_module("sqlanydb") + except ImportError as exc: + raise MissingDependencyError(package="sqlanydb", install_package="sqlanywhere") from exc + + +class SqlanywhereConnectionPool: + """Thread-local connection manager for SAP SQL Anywhere over sqlanydb.""" + + __slots__ = ( + "_checked_out", + "_connection_parameters", + "_connection_registry", + "_connection_threads", + "_generation", + "_health_check_interval", + "_max_size", + "_min_size", + "_on_connection_create", + "_pool_id", + "_recycle_seconds", + "_registry_lock", + "_thread_local", + "_timeout", + ) + + def __init__( + self, + connection_parameters: "dict[str, Any]", + min_size: int = 1, + max_size: int = 10, + timeout: float = 30.0, + recycle_seconds: int = 86400, + health_check_interval: float = 30.0, + on_connection_create: "Callable[[SqlanywhereConnection], None] | None" = None, + ) -> None: + self._connection_parameters = dict(connection_parameters) + self._min_size = min_size + self._max_size = max_size + self._timeout = timeout + self._thread_local = threading.local() + self._connection_registry: set[SqlanywhereConnection] = set() + self._connection_threads: dict[SqlanywhereConnection, threading.Thread] = {} + self._checked_out = 0 + self._generation = 0 + self._registry_lock = threading.Lock() + self._recycle_seconds = recycle_seconds + self._health_check_interval = health_check_interval + self._on_connection_create = on_connection_create + self._pool_id = str(uuid4())[:8] + + @property + def _database_name(self) -> str: + return str(self._connection_parameters.get("dbn", self._connection_parameters.get("eng", "unknown"))) + + def new_connection(self) -> SqlanywhereConnection: + """Open a standalone SAP SQL Anywhere connection via ``sqlanydb.connect``.""" + sqlanydb = _load_sqlanydb_module() + connection = sqlanydb.connect(**self._connection_parameters) + if self._on_connection_create is not None: + self._on_connection_create(connection) + return cast("SqlanywhereConnection", connection) + + @contextmanager + def get_connection(self) -> "Generator[SqlanywhereConnection, None, None]": + """Yield a thread-local SQL Anywhere connection.""" + connection = self.acquire() + try: + yield connection + except Exception: + with contextlib.suppress(Exception): + self._close_thread_connection() + raise + finally: + self.release(connection) + + def close(self) -> None: + """Close every connection opened by this pool.""" + self._close_thread_connection() + with self._registry_lock: + orphaned = list(self._connection_registry) + self._connection_registry.clear() + self._connection_threads.clear() + self._checked_out = 0 + self._generation += 1 + for connection in orphaned: + with contextlib.suppress(Exception): + connection.close() + + def acquire(self) -> SqlanywhereConnection: + """Acquire a healthy thread-local connection.""" + connection = self._get_thread_connection() + with self._registry_lock: + self._checked_out += 1 + return connection + + def release(self, connection: SqlanywhereConnection) -> None: + """Release a checked-out connection back to the thread-local slot.""" + _ = connection + with self._registry_lock: + if self._checked_out > 0: + self._checked_out -= 1 + + def size(self) -> int: + """Return the number of active connections registered with this pool.""" + self._prune_dead_thread_connections() + with self._registry_lock: + return len(self._connection_registry) + + def checked_out(self) -> int: + """Return the number of currently checked-out connections.""" + with self._registry_lock: + return self._checked_out + + def _prune_dead_thread_connections(self) -> None: + with self._registry_lock: + dead = [conn for conn, thread in self._connection_threads.items() if not thread.is_alive()] + for conn in dead: + self._connection_registry.discard(conn) + self._connection_threads.pop(conn, None) + for conn in dead: + with contextlib.suppress(Exception): + conn.close() + + def _create_connection(self) -> SqlanywhereConnection: + self._prune_dead_thread_connections() + connection = self.new_connection() + with self._registry_lock: + self._connection_registry.add(connection) + self._connection_threads[connection] = threading.current_thread() + return connection + + def _is_connection_alive(self, connection: SqlanywhereConnection) -> bool: + cursor = None + try: + cursor = connection.cursor() + cursor.execute("SELECT 1") + cursor.fetchone() + except Exception: + return False + else: + return True + finally: + if cursor is not None and hasattr(cursor, "close"): + with contextlib.suppress(Exception): + cursor.close() + + def _get_thread_connection(self) -> SqlanywhereConnection: + now = time.time() + current_gen = self._generation + conn = getattr(self._thread_local, "connection", None) + created_at = getattr(self._thread_local, "created_at", 0.0) + last_used = getattr(self._thread_local, "last_used", 0.0) + conn_gen = getattr(self._thread_local, "generation", -1) + + if conn is not None and conn_gen != current_gen: + self._close_thread_connection() + conn = None + + if conn is None: + conn = self._create_connection() + self._thread_local.connection = conn + self._thread_local.created_at = now + self._thread_local.last_used = now + self._thread_local.generation = current_gen + return conn + + if self._recycle_seconds > 0 and now - created_at >= self._recycle_seconds: + log_with_context( + logger, + logging.DEBUG, + "pool.connection.recycle", + adapter_name=_ADAPTER_NAME, + pool_id=self._pool_id, + database=self._database_name, + reason="exceeded_recycle_time", + ) + self._close_thread_connection() + conn = self._create_connection() + self._thread_local.connection = conn + self._thread_local.created_at = now + self._thread_local.last_used = now + self._thread_local.generation = current_gen + return conn + + if ( + self._health_check_interval > 0 + and now - last_used >= self._health_check_interval + and not self._is_connection_alive(conn) + ): + self._close_thread_connection() + conn = self._create_connection() + self._thread_local.connection = conn + self._thread_local.created_at = now + self._thread_local.generation = current_gen + + self._thread_local.last_used = now + return cast("SqlanywhereConnection", conn) + + def _close_thread_connection(self) -> None: + conn = getattr(self._thread_local, "connection", None) + if conn is not None: + with self._registry_lock: + self._connection_registry.discard(conn) + self._connection_threads.pop(conn, None) + with contextlib.suppress(Exception): + conn.close() + self._thread_local.connection = None + self._thread_local.created_at = 0.0 + self._thread_local.last_used = 0.0 + self._thread_local.generation = -1 diff --git a/sqlspec/adapters/sybase/__init__.py b/sqlspec/adapters/sybase/__init__.py new file mode 100644 index 000000000..f86b75c56 --- /dev/null +++ b/sqlspec/adapters/sybase/__init__.py @@ -0,0 +1,37 @@ +"""Sybase ASE adapter for SQLSpec.""" + +from sqlspec.adapters.sybase._typing import SybaseConnection, SybaseCursor +from sqlspec.adapters.sybase.config import SybaseConfig, SybaseConnectionParams, SybaseDriverFeatures, SybasePoolParams +from sqlspec.adapters.sybase.core import ( + DEFAULT_SYBASE_CONN_PROPERTIES, + DEFAULT_SYBASE_TDS_VERSION, + build_connection_config, + create_mapped_exception, + default_statement_config, + driver_profile, +) +from sqlspec.adapters.sybase.data_dictionary import SybaseDataDictionary, SybaseSyncDataDictionary +from sqlspec.adapters.sybase.driver import SybaseDriver, SybaseExceptionHandler +from sqlspec.adapters.sybase.migrations import SybaseSyncMigrationTracker +from sqlspec.adapters.sybase.pool import SybaseConnectionPool + +__all__ = ( + "DEFAULT_SYBASE_CONN_PROPERTIES", + "DEFAULT_SYBASE_TDS_VERSION", + "SybaseConfig", + "SybaseConnection", + "SybaseConnectionParams", + "SybaseConnectionPool", + "SybaseCursor", + "SybaseDataDictionary", + "SybaseDriver", + "SybaseDriverFeatures", + "SybaseExceptionHandler", + "SybasePoolParams", + "SybaseSyncDataDictionary", + "SybaseSyncMigrationTracker", + "build_connection_config", + "create_mapped_exception", + "default_statement_config", + "driver_profile", +) diff --git a/sqlspec/adapters/sybase/_typing.py b/sqlspec/adapters/sybase/_typing.py new file mode 100644 index 000000000..7b8826d9d --- /dev/null +++ b/sqlspec/adapters/sybase/_typing.py @@ -0,0 +1,134 @@ +"""Sybase ASE adapter type definitions. + +This module contains type aliases and classes that are excluded from mypyc +compilation to avoid ABI boundary issues. +""" + +import contextlib +from typing import TYPE_CHECKING, Any + +from mypy_extensions import mypyc_attr + +from sqlspec.utils.module_loader import import_optional_attr + +PYMSSQL_MODULE: Any = None +_DRIVER_CLS: Any = None + + +class SybaseError(Exception): + """Sybase ASE driver exception base.""" + + +if TYPE_CHECKING: + from collections.abc import Callable + from types import TracebackType + from typing import TypeAlias + + from pymssql import Connection as _SybaseConnection + from pymssql import Cursor as _SybaseRawCursor + from pymssql._pymssql import QueryParams as SybaseQueryParams + + from sqlspec.adapters.sybase.driver import SybaseDriver + from sqlspec.core import StatementConfig + + SybaseConnection: TypeAlias = _SybaseConnection + SybaseRawCursor: TypeAlias = _SybaseRawCursor + +if not TYPE_CHECKING: + SybaseQueryParams = Any + SybaseConnection = Any + SybaseRawCursor = Any + +__all__ = ( + "PYMSSQL_MODULE", + "SybaseConnection", + "SybaseCursor", + "SybaseError", + "SybaseQueryParams", + "SybaseRawCursor", + "SybaseSessionContext", +) + + +class SybaseCursor: + """Context manager for Sybase pymssql cursor operations.""" + + __slots__ = ("_as_dict", "connection", "cursor") + + def __init__(self, connection: "SybaseConnection", *, as_dict: bool | None = None) -> None: + self.connection = connection + self.cursor: SybaseRawCursor | None = None + self._as_dict = as_dict + + def __enter__(self) -> "SybaseRawCursor": + if self._as_dict is None: + self.cursor = self.connection.cursor() + else: + try: + raw_cursor: Any = ( + self.connection.cursor(as_dict=True) if self._as_dict else self.connection.cursor(as_dict=False) + ) + self.cursor = raw_cursor + except TypeError: + self.cursor = self.connection.cursor() + assert self.cursor is not None + return self.cursor + + def __exit__(self, *_: Any) -> None: + if self.cursor is not None: + with contextlib.suppress(Exception): + self.cursor.close() + + +@mypyc_attr(allow_interpreted_subclasses=True, native_class=False) +class SybaseSessionContext: + """Sync context manager for Sybase ASE sessions.""" + + __slots__ = ( + "_acquire_connection", + "_connection", + "_driver", + "_driver_features", + "_prepare_driver", + "_release_connection", + "_statement_config", + ) + + def __init__( + self, + acquire_connection: "Callable[[], Any]", + release_connection: "Callable[..., Any]", + statement_config: "StatementConfig", + driver_features: "dict[str, Any]", + prepare_driver: "Callable[[SybaseDriver], SybaseDriver]", + ) -> None: + self._acquire_connection = acquire_connection + self._release_connection = release_connection + self._statement_config = statement_config + self._driver_features = driver_features + self._prepare_driver = prepare_driver + self._connection: Any = None + self._driver: SybaseDriver | None = None + + def __enter__(self) -> "SybaseDriver": + global _DRIVER_CLS + if _DRIVER_CLS is None: + _DRIVER_CLS = import_optional_attr("sqlspec.adapters.sybase.driver", "SybaseDriver") + self._connection = self._acquire_connection() + self._driver = _DRIVER_CLS( + connection=self._connection, statement_config=self._statement_config, driver_features=self._driver_features + ) + assert self._driver is not None + return self._prepare_driver(self._driver) + + def __exit__( + self, exc_type: "type[BaseException] | None", exc_val: "BaseException | None", exc_tb: "TracebackType | None" + ) -> "bool | None": + if self._driver is not None and (exc_type is not None or self._driver._connection_in_transaction()): + with contextlib.suppress(Exception): + self._driver.rollback() + if self._connection is not None: + self._release_connection(self._connection, exc_type=exc_type, exc_val=exc_val, exc_tb=exc_tb) + self._connection = None + self._driver = None + return None diff --git a/sqlspec/adapters/sybase/adk/__init__.py b/sqlspec/adapters/sybase/adk/__init__.py new file mode 100644 index 000000000..c134a08d0 --- /dev/null +++ b/sqlspec/adapters/sybase/adk/__init__.py @@ -0,0 +1,5 @@ +"""Sybase ASE ADK extension.""" + +from sqlspec.adapters.sybase.adk.store import SybaseADKConfig, SybaseADKMemoryStore, SybaseADKStore + +__all__ = ("SybaseADKConfig", "SybaseADKMemoryStore", "SybaseADKStore") diff --git a/sqlspec/adapters/sybase/adk/store.py b/sqlspec/adapters/sybase/adk/store.py new file mode 100644 index 000000000..d009c1f78 --- /dev/null +++ b/sqlspec/adapters/sybase/adk/store.py @@ -0,0 +1,930 @@ +"""Sybase ASE ADK stores for Google Agent Development Kit session storage.""" + +from datetime import datetime +from typing import TYPE_CHECKING, Any, ClassVar, Final, Literal, cast + +from sqlspec.adapters.sybase._typing import SybaseCursor, SybaseError +from sqlspec.adapters.sybase.core import ( + escape_sql_literal, + extract_error_number, + quote_bracket_identifier, + resolve_rowcount, + split_schema_and_table, +) +from sqlspec.config import ADKConfig +from sqlspec.extensions.adk import BaseSyncADKStore, StoredEvent, StoredSession, normalize_session_list_options +from sqlspec.extensions.adk.memory.store import BaseSyncADKMemoryStore +from sqlspec.utils.serializers import from_json, to_json + +if TYPE_CHECKING: + from collections.abc import Sequence + from datetime import timedelta + + from sqlspec.adapters.sybase.config import SybaseConfig + from sqlspec.adapters.sybase.driver import SybaseDriver + from sqlspec.extensions.adk import SessionOrderBy + from sqlspec.extensions.adk.memory._types import StoredMemory + +__all__ = ("SybaseADKConfig", "SybaseADKMemoryStore", "SybaseADKStore") + +SYBASE_TABLE_NOT_FOUND_ERROR: Final[int] = 208 +SYBASE_SCHEMA: Final[str] = "dbo" +JSON_COLUMN_TYPE: Final[str] = "TEXT" + + +class SybaseADKConfig(ADKConfig): + """Sybase ASE ADK extension settings.""" + + +class SybaseADKStore(BaseSyncADKStore["SybaseConfig"]): + """Synchronous Sybase ASE ADK session/event store.""" + + connector_name: ClassVar[str] = "sybase" + __slots__ = () + + def __init__(self, config: "SybaseConfig") -> None: + super().__init__(config) + + def create_tables(self) -> None: + """Create ADK tables (idempotent Sybase ASE DDL) and DD-gated indexes.""" + if not self.create_schema_enabled: + self.reconcile_schema() + return + + with self._config.provide_session() as driver: + driver.execute_script(self._sessions_table_ddl()) + driver.execute_script(self._events_table_ddl()) + driver.execute_script(self._app_states_table_ddl()) + driver.execute_script(self._user_states_table_ddl()) + driver.execute_script(self._metadata_table_ddl()) + index_specs = self._index_specs() + existing_indexes = _existing_index_names(driver, index_specs) + for index_name, index_table, columns in index_specs: + if _bare_name(index_name) not in existing_indexes: + driver.execute(_create_index_sql(index_table, index_name, columns)) + driver.commit() + self.reconcile_schema(assume_existing=True) + + def create_session( + self, session_id: str, app_name: str, user_id: str, state: "dict[str, Any]", owner_id: "Any | None" = None + ) -> StoredSession: + """Create a new ADK session.""" + owner_column = f", {quote_bracket_identifier(self._owner_id_column_name)}" if self._owner_id_column_name else "" + owner_param = ", %s" if self._owner_id_column_name else "" + insert_sql = f""" + INSERT INTO {_table_ref(self._session_table)} ( + id, app_name, user_id{owner_column}, state, create_time, update_time + ) + VALUES (%s, %s, %s{owner_param}, %s, GETUTCDATE(), GETUTCDATE()) + """ + select_sql = f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = %s AND user_id = %s AND id = %s + """ + params: tuple[Any, ...] + if self._owner_id_column_name: + params = (session_id, app_name, user_id, owner_id, to_json(state)) + else: + params = (session_id, app_name, user_id, to_json(state)) + self._execute(insert_sql, params, commit=True) + row = self._execute_fetchone(select_sql, (app_name, user_id, session_id)) + if row is None: + msg = "Failed to fetch created session" + raise RuntimeError(msg) + return _session_record_from_row(row) + + def get_session( + self, app_name: str, user_id: str, session_id: str, *, renew_for: "int | timedelta | None" = None + ) -> "StoredSession | None": + """Return a scoped session or ``None`` if absent.""" + try: + if renew_for is not None and self._calculate_expires_at(renew_for) is not None: + self._execute( + f""" + UPDATE {_table_ref(self._session_table)} + SET update_time = GETUTCDATE() + WHERE app_name = %s AND user_id = %s AND id = %s + """, + (app_name, user_id, session_id), + commit=True, + ) + row = self._execute_fetchone( + f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = %s AND user_id = %s AND id = %s + """, + (app_name, user_id, session_id), + ) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return None + raise + return _session_record_from_row(row) if row is not None else None + + def update_session_state(self, app_name: str, user_id: str, session_id: str, state: "dict[str, Any]") -> None: + """Replace a session's durable state.""" + self._execute( + f""" + UPDATE {_table_ref(self._session_table)} + SET state = %s, update_time = GETUTCDATE() + WHERE app_name = %s AND user_id = %s AND id = %s + """, + (to_json(state), app_name, user_id, session_id), + commit=True, + ) + + def list_sessions( + self, + app_name: str, + user_id: "str | None" = None, + *, + order_by: "SessionOrderBy" = "update_time", + descending: bool = True, + limit: "int | None" = None, + offset: "int | None" = None, + ) -> "list[StoredSession]": + """List ADK sessions for an application, optionally scoped to a user.""" + column, direction, page_limit, page_offset = normalize_session_list_options(order_by, descending, limit, offset) + if page_limit == 0: + return [] + + sql, params = _session_list_query( + self._session_table, app_name, user_id, column, direction, page_limit, page_offset + ) + try: + rows = self._execute_fetchall(sql, params) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return [] + raise + return [_session_record_from_row(row) for row in rows] + + def delete_session(self, app_name: str, user_id: str, session_id: str) -> None: + """Delete a session and its associated events.""" + self._execute( + f"DELETE FROM {_table_ref(self._events_table)} WHERE app_name = %s AND user_id = %s AND session_id = %s", + (app_name, user_id, session_id), + commit=True, + ) + self._execute( + f"DELETE FROM {_table_ref(self._session_table)} WHERE app_name = %s AND user_id = %s AND id = %s", + (app_name, user_id, session_id), + commit=True, + ) + + def append_event(self, event_record: StoredEvent) -> None: + """Append an event to a session.""" + self._execute(_insert_event_sql(self._events_table), _event_insert_params(event_record), commit=True) + + def append_event_and_update_state( + self, + event_record: StoredEvent, + app_name: str, + user_id: str, + session_id: str, + state: "dict[str, Any]", + *, + app_state: "dict[str, Any] | None" = None, + user_state: "dict[str, Any] | None" = None, + ) -> StoredSession: + """Atomically append an event and update durable session/scoped state.""" + update_sql = f""" + UPDATE {_table_ref(self._session_table)} + SET state = %s, update_time = GETUTCDATE() + WHERE app_name = %s AND user_id = %s AND id = %s + """ + select_sql = f""" + SELECT TOP 1 id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(self._session_table)} + WHERE app_name = %s AND user_id = %s AND id = %s + """ + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + try: + cursor.execute(update_sql, (to_json(state), app_name, user_id, session_id)) + cursor.execute(select_sql, (app_name, user_id, session_id)) + row = cursor.fetchone() + if row is None: + _raise_session_not_found(session_id) + cursor.execute(_insert_event_sql(self._events_table), _event_insert_params(event_record)) + if app_state is not None: + app_json = to_json(app_state) + cursor.execute(self._upsert_app_state_sql(), (app_name, app_json, app_name, app_name, app_json)) + if user_state is not None: + user_json = to_json(user_state) + cursor.execute( + self._upsert_user_state_sql(), + (app_name, user_id, user_json, app_name, user_id, app_name, user_id, user_json), + ) + except Exception: + conn.rollback() + raise + conn.commit() + return _session_record_from_row(row) + + def get_events( + self, + app_name: str, + user_id: str, + session_id: str, + after_timestamp: "datetime | None" = None, + limit: "int | None" = None, + ) -> "list[StoredEvent]": + """Return events for a scoped session ordered by event timestamp.""" + if limit == 0: + return [] + sql, params = self._events_query(app_name, user_id, session_id, after_timestamp, limit) + try: + rows = self._execute_fetchall(sql, params) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return [] + raise + return [_event_record_from_row(row) for row in rows] + + def delete_expired_events(self, before: datetime, app_name: "str | None" = None) -> int: + """Delete events older than ``before``.""" + sql = f"DELETE FROM {_table_ref(self._events_table)} WHERE timestamp < %s" + params: list[Any] = [before] + if app_name is not None: + sql += " AND app_name = %s" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return 0 + raise + + def delete_idle_sessions(self, updated_before: datetime, app_name: "str | None" = None) -> int: + """Delete sessions whose update_time is older than ``updated_before``.""" + sql = f"DELETE FROM {_table_ref(self._session_table)} WHERE update_time < %s" + params: list[Any] = [updated_before] + if app_name is not None: + sql += " AND app_name = %s" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return 0 + raise + + def delete_idle_user_states(self, updated_before: datetime, app_name: "str | None" = None) -> int: + """Delete user state rows whose update_time is older than ``updated_before``.""" + sql = f"DELETE FROM {_table_ref(self._user_state_table)} WHERE update_time < %s" + params: list[Any] = [updated_before] + if app_name is not None: + sql += " AND app_name = %s" + params.append(app_name) + try: + return self._execute(sql, tuple(params), commit=True) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return 0 + raise + + def get_app_state(self, app_name: str) -> "dict[str, Any] | None": + """Return app-scoped state.""" + try: + row = self._execute_fetchone( + f"SELECT TOP 1 state FROM {_table_ref(self._app_state_table)} WHERE app_name = %s", (app_name,) + ) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return None + raise + return _json_dict(_row_value(row, "state", 0)) if row is not None else None + + def get_user_state(self, app_name: str, user_id: str) -> "dict[str, Any] | None": + """Return user-scoped state.""" + try: + row = self._execute_fetchone( + f""" + SELECT TOP 1 state + FROM {_table_ref(self._user_state_table)} + WHERE app_name = %s AND user_id = %s + """, + (app_name, user_id), + ) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return None + raise + return _json_dict(_row_value(row, "state", 0)) if row is not None else None + + def upsert_app_state(self, app_name: str, state: "dict[str, Any]") -> None: + """Insert or replace app-scoped state.""" + encoded = to_json(state) + self._execute(self._upsert_app_state_sql(), (app_name, encoded, app_name, app_name, encoded), commit=True) + + def upsert_user_state(self, app_name: str, user_id: str, state: "dict[str, Any]") -> None: + """Insert or replace user-scoped state.""" + encoded = to_json(state) + self._execute( + self._upsert_user_state_sql(), + (app_name, user_id, encoded, app_name, user_id, app_name, user_id, encoded), + commit=True, + ) + + def get_metadata(self, key: str) -> "str | None": + """Return an ADK metadata value.""" + try: + row = self._execute_fetchone( + f"SELECT TOP 1 value FROM {_table_ref(self._metadata_table)} WHERE [key] = %s", (key,) + ) + except SybaseError as exc: + if _is_sybase_table_missing(exc): + return None + raise + return str(_row_value(row, "value", 0)) if row is not None else None + + def set_metadata(self, key: str, value: str) -> None: + """Set an ADK metadata value.""" + self._execute(_upsert_metadata_sql(self._metadata_table), (key, value, key, key, value), commit=True) + + def _index_specs(self) -> "list[tuple[str, str, str]]": + """Return ``(index_name, table, columns)`` specs for session and event indexes.""" + return [*_sessions_index_specs(self._session_table), *_events_index_specs(self._events_table)] + + def _sessions_table_ddl(self) -> str: + """Return Sybase ASE DDL for the ADK session table.""" + return _sessions_table_ddl(self._session_table, JSON_COLUMN_TYPE, self._owner_id_column_ddl) + + def _events_table_ddl(self) -> str: + """Return Sybase ASE DDL for the ADK event table.""" + return _events_table_ddl(self._events_table, self._session_table, JSON_COLUMN_TYPE) + + def _app_states_table_ddl(self) -> str: + """Return Sybase ASE DDL for the app-scoped state table.""" + return _app_states_table_ddl(self._app_state_table, JSON_COLUMN_TYPE) + + def _user_states_table_ddl(self) -> str: + """Return Sybase ASE DDL for the user-scoped state table.""" + return _user_states_table_ddl(self._user_state_table, JSON_COLUMN_TYPE) + + def _metadata_table_ddl(self) -> str: + """Return Sybase ASE DDL for the ADK metadata table.""" + return _metadata_table_ddl(self._metadata_table) + + def _drop_app_states_table_sql(self) -> str: + ref = escape_sql_literal(_table_ref(self._app_state_table)) + return f"IF OBJECT_ID('{ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {ref}')" + + def _drop_user_states_table_sql(self) -> str: + ref = escape_sql_literal(_table_ref(self._user_state_table)) + return f"IF OBJECT_ID('{ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {ref}')" + + def _drop_metadata_table_sql(self) -> str: + ref = escape_sql_literal(_table_ref(self._metadata_table)) + return f"IF OBJECT_ID('{ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {ref}')" + + def _drop_tables_sql(self) -> "list[str]": + events_ref = escape_sql_literal(_table_ref(self._events_table)) + sessions_ref = escape_sql_literal(_table_ref(self._session_table)) + return [ + self._drop_metadata_table_sql(), + self._drop_user_states_table_sql(), + self._drop_app_states_table_sql(), + f"IF OBJECT_ID('{events_ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {events_ref}')", + f"IF OBJECT_ID('{sessions_ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {sessions_ref}')", + ] + + def _upsert_app_state_sql(self) -> str: + return _upsert_state_sql(self._app_state_table, ("app_name",)) + + def _upsert_user_state_sql(self) -> str: + return _upsert_state_sql(self._user_state_table, ("app_name", "user_id")) + + def _events_query( + self, + app_name: str, + user_id: str, + session_id: str, + after_timestamp: "datetime | None" = None, + limit: "int | None" = None, + ) -> "tuple[str, tuple[Any, ...]]": + return _events_query(self._events_table, app_name, user_id, session_id, after_timestamp, limit) + + def _execute_fetchone(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> "Any | None": + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + cursor.execute(sql, params) + row = cursor.fetchone() + if commit: + conn.commit() + return row + + def _execute_fetchall(self, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + cursor.execute(sql, params) + return list(cursor.fetchall()) + + def _execute(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> int: + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + cursor.execute(sql, params) + rowcount = resolve_rowcount(cursor) + if commit: + conn.commit() + return rowcount + + +class SybaseADKMemoryStore(BaseSyncADKMemoryStore["SybaseConfig"]): + """Sybase ASE ADK memory store using pymssql.""" + + __slots__ = () + + def __init__(self, config: "SybaseConfig") -> None: + super().__init__(config) + + def create_tables(self) -> None: + """Create the memory table (idempotent Sybase ASE DDL) and DD-gated indexes.""" + if not self.create_schema_enabled: + self.reconcile_schema() + return + + if not self._enabled: + return + with self._config.provide_session() as driver: + driver.execute_script(self._memory_table_ddl()) + index_specs = self._memory_index_specs() + existing_indexes = _existing_index_names(driver, index_specs) + for index_name, index_table, columns in index_specs: + if _bare_name(index_name) not in existing_indexes: + driver.execute(_create_index_sql(index_table, index_name, columns)) + driver.commit() + self.reconcile_schema(assume_existing=True) + + def insert_memory_entries(self, entries: "list[StoredMemory]", owner_id: "object | None" = None) -> int: + """Bulk insert memory entries with event-id deduplication.""" + if not self._enabled: + msg = "ADK memory store is disabled" + raise RuntimeError(msg) + if not entries: + return 0 + + owner_column = f", {quote_bracket_identifier(self._owner_id_column_name)}" if self._owner_id_column_name else "" + owner_value = ", %s" if self._owner_id_column_name else "" + sql = f""" + INSERT INTO {_table_ref(self._memory_table)} ( + id, session_id, app_name, user_id, scope, event_id, author, timestamp, + content_json, content_text, metadata_json{owner_column} + ) + SELECT %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s{owner_value} + WHERE NOT EXISTS ( + SELECT 1 FROM {_table_ref(self._memory_table)} HOLDLOCK + WHERE event_id = %s + ) + """ + inserted = 0 + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + try: + for entry in entries: + params: list[Any] = [ + entry["id"], + entry["session_id"], + entry["app_name"], + entry["user_id"], + entry.get("scope", "user"), + entry["event_id"], + entry.get("author"), + entry["timestamp"], + to_json(entry["content_json"]), + entry["content_text"], + to_json(entry["metadata_json"]) if entry.get("metadata_json") is not None else None, + ] + if self._owner_id_column_name: + params.append(owner_id) + params.append(entry["event_id"]) + cursor.execute(sql, tuple(params)) + inserted += resolve_rowcount(cursor) + except Exception: + conn.rollback() + raise + conn.commit() + return inserted + + def search_entries( + self, + query: str, + app_name: str, + user_id: str, + limit: "int | None" = None, + scope_filter: Literal["all", "user", "app"] = "all", + embedding: "Sequence[float] | None" = None, + ) -> "list[StoredMemory]": + """Search memory entries by text query.""" + _ = embedding + if not self._enabled: + msg = "ADK memory store is disabled" + raise RuntimeError(msg) + limit_value = limit or self._max_results + where_scope, scope_params = _build_sybase_scope_where(app_name, user_id, scope_filter) + escaped_query = _escape_like_pattern(query) + sql = f""" + SELECT + id, session_id, app_name, user_id, scope, event_id, author, timestamp, + content_json, content_text, metadata_json, inserted_at + FROM {_table_ref(self._memory_table)} + WHERE {where_scope} AND content_text LIKE %s ESCAPE '\\' + ORDER BY timestamp DESC + ROWS LIMIT %s + """ + rows = self._execute_fetchall(sql, (*scope_params, f"%{escaped_query}%", limit_value)) + return [_memory_record_from_row(row) for row in rows] + + def delete_entries_by_session(self, session_id: str) -> int: + """Delete all memory entries for a specific session.""" + return self._execute( + f"DELETE FROM {_table_ref(self._memory_table)} WHERE session_id = %s", (session_id,), commit=True + ) + + def delete_entries_older_than(self, days: int, app_name: "str | None" = None, scope: "str | None" = None) -> int: + """Delete memory entries older than the retention window.""" + clauses = ["inserted_at < DATEADD(day, -%s, GETUTCDATE())"] + params: list[Any] = [days] + if app_name is not None: + clauses.append("app_name = %s") + params.append(app_name) + if scope is not None: + clauses.append("scope = %s") + params.append(scope) + where_sql = " AND ".join(clauses) + return self._execute( + f"DELETE FROM {_table_ref(self._memory_table)} WHERE {where_sql}", tuple(params), commit=True + ) + + def _memory_table_ddl(self) -> str: + owner_line = f", {self._owner_id_column_ddl}" if self._owner_id_column_ddl else "" + table_ref = _table_ref(self._memory_table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "session_id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + f"scope VARCHAR(16) DEFAULT 'user' NOT NULL, " + "event_id VARCHAR(128) NOT NULL, " + "author VARCHAR(256) NULL, " + "timestamp BIGDATETIME NOT NULL, " + f"content_json {JSON_COLUMN_TYPE} NOT NULL, " + "content_text TEXT NOT NULL, " + f"metadata_json {JSON_COLUMN_TYPE} NULL, " + f"inserted_at BIGDATETIME DEFAULT GETUTCDATE() NOT NULL{owner_line}, " + f"CONSTRAINT {_constraint_ref('pk', self._memory_table, 'id')} PRIMARY KEY (id), " + f"CONSTRAINT {_constraint_ref('uq', self._memory_table, 'event_id')} UNIQUE (event_id)" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + def _memory_index_specs(self) -> "list[tuple[str, str, str]]": + """Return ``(index_name, table, columns)`` specs for memory-table indexes.""" + _, table_name = split_schema_and_table(self._memory_table) + return [ + (f"idx_{table_name}_app_scope_user_time", self._memory_table, "app_name, scope, user_id, timestamp DESC"), + (f"idx_{table_name}_scope", self._memory_table, "app_name, scope"), + (f"idx_{table_name}_session", self._memory_table, "session_id"), + (f"idx_{table_name}_timestamp", self._memory_table, "timestamp DESC"), + ] + + def _drop_memory_table_sql(self) -> "list[str]": + ref = escape_sql_literal(_table_ref(self._memory_table)) + return [f"IF OBJECT_ID('{ref}', 'U') IS NOT NULL EXECUTE('DROP TABLE {ref}')"] + + def _execute_fetchall(self, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + cursor.execute(sql, params) + return list(cursor.fetchall()) + + def _execute(self, sql: str, params: "tuple[Any, ...]" = (), *, commit: bool = False) -> int: + with self._config.provide_connection() as conn, SybaseCursor(conn, as_dict=False) as cursor: + cursor.execute(sql, params) + rowcount = resolve_rowcount(cursor) + if commit: + conn.commit() + return rowcount + + +def _existing_index_names(driver: "SybaseDriver", index_specs: "list[tuple[str, str, str]]") -> "set[str]": + schemas = {split_schema_and_table(index_table)[0] for _, index_table, _ in index_specs} or {SYBASE_SCHEMA} + existing: set[str] = set() + for schema_name in schemas: + existing.update(_casefold_names(driver.data_dictionary.get_indexes(driver, schema=schema_name), "index_name")) + return existing + + +def _sessions_table_ddl(table: str, json_column_type: str, owner_id_column_ddl: "str | None") -> str: + owner_line = f", {owner_id_column_ddl}" if owner_id_column_ddl else "" + table_ref = _table_ref(table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + f"user_id VARCHAR(128) NOT NULL{owner_line}, " + f"state {json_column_type} NOT NULL, " + "create_time BIGDATETIME DEFAULT GETUTCDATE() NOT NULL, " + "update_time BIGDATETIME DEFAULT GETUTCDATE() NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'id')} PRIMARY KEY (id)" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + +def _sessions_index_specs(table: str) -> "list[tuple[str, str, str]]": + _, table_name = split_schema_and_table(table) + return [ + (f"idx_{table_name}_app_user", table, "app_name, user_id"), + (f"idx_{table_name}_update_time", table, "update_time DESC"), + ] + + +def _events_table_ddl(table: str, session_table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + session_ref = _table_ref(session_table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "id VARCHAR(128) NOT NULL, " + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + "session_id VARCHAR(128) NOT NULL, " + "invocation_id VARCHAR(256) NOT NULL, " + "timestamp BIGDATETIME NOT NULL, " + f"event_data {json_column_type} NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'id')} PRIMARY KEY (id), " + f"CONSTRAINT {_constraint_ref('fk', table, 'session')} FOREIGN KEY (session_id) REFERENCES {session_ref}(id)" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + +def _events_index_specs(table: str) -> "list[tuple[str, str, str]]": + _, table_name = split_schema_and_table(table) + return [ + (f"idx_{table_name}_scope", table, "app_name, user_id, session_id, timestamp ASC"), + (f"idx_{table_name}_session", table, "session_id, timestamp ASC"), + (f"idx_{table_name}_invocation", table, "invocation_id"), + (f"idx_{table_name}_timestamp", table, "timestamp ASC"), + (f"idx_{table_name}_app_timestamp", table, "app_name, timestamp ASC"), + ] + + +def _app_states_table_ddl(table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "app_name VARCHAR(128) NOT NULL, " + f"state {json_column_type} NOT NULL, " + "update_time BIGDATETIME DEFAULT GETUTCDATE() NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'app_name')} PRIMARY KEY (app_name)" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + +def _user_states_table_ddl(table: str, json_column_type: str) -> str: + table_ref = _table_ref(table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "app_name VARCHAR(128) NOT NULL, " + "user_id VARCHAR(128) NOT NULL, " + f"state {json_column_type} NOT NULL, " + "update_time BIGDATETIME DEFAULT GETUTCDATE() NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'app_user')} PRIMARY KEY (app_name, user_id)" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + +def _metadata_table_ddl(table: str) -> str: + table_ref = _table_ref(table) + escaped_ref = escape_sql_literal(table_ref) + create_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "[key] VARCHAR(128) NOT NULL, " + "value VARCHAR(512) NOT NULL, " + f"CONSTRAINT {_constraint_ref('pk', table, 'key')} PRIMARY KEY ([key])" + ")" + ) + return f"IF OBJECT_ID('{escaped_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + +def _create_index_sql(table: str, index_name: str, columns: str) -> str: + return f"CREATE INDEX {quote_bracket_identifier(index_name)} ON {_table_ref(table)} ({columns})" + + +def _casefold_names(rows: "list[Any]", key: str) -> "set[str]": + """Collapse data-dictionary rows into a case-folded, schema-stripped name set.""" + return {str(row.get(key, "")).rsplit(".", 1)[-1].casefold() for row in rows} + + +def _bare_name(name: str) -> str: + """Return the case-folded, schema-stripped object name for membership checks.""" + return name.rsplit(".", 1)[-1].casefold() + + +def _insert_event_sql(table: str) -> str: + return f""" + INSERT INTO {_table_ref(table)} ( + id, app_name, user_id, session_id, invocation_id, timestamp, event_data + ) + VALUES (%s, %s, %s, %s, %s, %s, %s) + """ + + +def _upsert_state_sql(table: str, key_columns: "tuple[str, ...]") -> str: + table_ref = _table_ref(table) + where_clause = " AND ".join(f"{quote_bracket_identifier(col)} = %s" for col in key_columns) + insert_columns = ", ".join(quote_bracket_identifier(col) for col in (*key_columns, "state", "update_time")) + insert_placeholders = ", ".join(["%s" for _ in key_columns] + ["%s", "GETUTCDATE()"]) + return f""" + IF EXISTS (SELECT 1 FROM {table_ref} HOLDLOCK WHERE {where_clause}) + UPDATE {table_ref} SET state = %s, update_time = GETUTCDATE() WHERE {where_clause} + ELSE + INSERT INTO {table_ref} ({insert_columns}) VALUES ({insert_placeholders}) + """ + + +def _upsert_metadata_sql(table: str) -> str: + table_ref = _table_ref(table) + return f""" + IF EXISTS (SELECT 1 FROM {table_ref} HOLDLOCK WHERE [key] = %s) + UPDATE {table_ref} SET value = %s WHERE [key] = %s + ELSE + INSERT INTO {table_ref} ([key], value) VALUES (%s, %s) + """ + + +def _events_query( + table: str, app_name: str, user_id: str, session_id: str, after_timestamp: "datetime | None", limit: "int | None" +) -> "tuple[str, tuple[Any, ...]]": + params: list[Any] = [app_name, user_id, session_id] + after_clause = "" + if after_timestamp is not None: + after_clause = " AND timestamp > %s" + params.append(after_timestamp) + limit_clause = "" + if limit is not None: + limit_clause = "\n ROWS LIMIT %s" + params.append(limit) + sql = f""" + SELECT id, app_name, user_id, session_id, invocation_id, timestamp, event_data + FROM {_table_ref(table)} + WHERE app_name = %s AND user_id = %s AND session_id = %s{after_clause} + ORDER BY timestamp ASC{limit_clause} + """ + return sql, tuple(params) + + +def _event_insert_params(event_record: StoredEvent) -> "tuple[Any, ...]": + return ( + event_record["id"], + event_record["app_name"], + event_record["user_id"], + event_record["session_id"], + event_record["invocation_id"], + event_record["timestamp"], + to_json(event_record["event_data"]), + ) + + +def _row_value(row: object, key: str, index: int) -> Any: + if isinstance(row, dict): + if key in row: + return row[key] + upper_key = key.upper() + if upper_key in row: + return row[upper_key] + return None + if isinstance(row, (list, tuple)) and len(row) > index: + return row[index] + return getattr(row, key, None) + + +def _session_record_from_row(row: Any) -> StoredSession: + return StoredSession( + id=_row_value(row, "id", 0), + app_name=_row_value(row, "app_name", 1), + user_id=_row_value(row, "user_id", 2), + state=_json_dict(_row_value(row, "state", 3)), + create_time=_row_value(row, "create_time", 4), + update_time=_row_value(row, "update_time", 5), + ) + + +def _event_record_from_row(row: Any) -> StoredEvent: + return StoredEvent( + id=_row_value(row, "id", 0), + app_name=_row_value(row, "app_name", 1), + user_id=_row_value(row, "user_id", 2), + session_id=_row_value(row, "session_id", 3), + invocation_id=_row_value(row, "invocation_id", 4), + timestamp=_row_value(row, "timestamp", 5), + event_data=_json_dict(_row_value(row, "event_data", 6)), + ) + + +def _memory_record_from_row(row: Any) -> "StoredMemory": + metadata_raw = _row_value(row, "metadata_json", 10) + return cast( + "StoredMemory", + { + "id": _row_value(row, "id", 0), + "session_id": _row_value(row, "session_id", 1), + "app_name": _row_value(row, "app_name", 2), + "user_id": _row_value(row, "user_id", 3), + "scope": _row_value(row, "scope", 4), + "event_id": _row_value(row, "event_id", 5), + "author": _row_value(row, "author", 6), + "timestamp": _row_value(row, "timestamp", 7), + "content_json": _json_dict(_row_value(row, "content_json", 8)), + "content_text": _row_value(row, "content_text", 9), + "metadata_json": _json_dict(metadata_raw) if metadata_raw is not None else None, + "inserted_at": _row_value(row, "inserted_at", 11), + "embedding": None, + }, + ) + + +def _json_dict(value: Any) -> "dict[str, Any]": + if value is None: + return {} + if isinstance(value, dict): + return cast("dict[str, Any]", value) + if isinstance(value, bytearray): + value = bytes(value) + if isinstance(value, (bytes, str)): + return cast("dict[str, Any]", from_json(value)) + return cast("dict[str, Any]", from_json(str(value))) + + +def _is_sybase_table_missing(exc: BaseException) -> bool: + text = str(exc).lower() + return ( + "not found" in text + or "invalid object name" in text + or extract_error_number(exc) == SYBASE_TABLE_NOT_FOUND_ERROR + ) + + +def _table_ref(table: str) -> str: + schema_name, table_name = split_schema_and_table(table) + return f"{quote_bracket_identifier(schema_name)}.{quote_bracket_identifier(table_name)}" + + +def _constraint_ref(prefix: str, table: str, suffix: str) -> str: + _, table_name = split_schema_and_table(table) + return quote_bracket_identifier(f"{prefix}_{table_name}_{suffix}") + + +def _raise_session_not_found(session_id: str) -> None: + msg = f"Session {session_id} not found during append_event_and_update_state." + raise ValueError(msg) + + +def _build_sybase_scope_where( + app_name: str, user_id: str, scope_filter: Literal["all", "user", "app"] +) -> tuple[str, tuple[Any, ...]]: + if scope_filter == "all": + return "app_name = %s AND ((scope = 'user' AND user_id = %s) OR scope = 'app')", (app_name, user_id) + if scope_filter == "user": + return "app_name = %s AND scope = 'user' AND user_id = %s", (app_name, user_id) + return "app_name = %s AND scope = 'app'", (app_name,) + + +def _escape_like_pattern(value: str) -> str: + return value.replace("\\", "\\\\").replace("%", "\\%").replace("_", "\\_").replace("[", "\\[") + + +def _session_list_query( + session_table: str, + app_name: str, + user_id: "str | None", + column: str, + direction: str, + limit: "int | None", + offset: int, +) -> "tuple[str, tuple[Any, ...]]": + """Return the bounded Sybase ASE session-list query and its bound values.""" + params: list[Any] = [app_name] + where_clause = "app_name = %s" + if user_id is not None: + params.append(user_id) + where_clause = f"{where_clause} AND user_id = %s" + + page_clause = "" + if limit is not None: + params.extend((limit, offset)) + page_clause = "\n ROWS LIMIT %s OFFSET %s" + + sql = f""" + SELECT id, app_name, user_id, state, create_time, update_time + FROM {_table_ref(session_table)} + WHERE {where_clause} + ORDER BY {column} {direction}, id {direction}{page_clause} + """ + return sql, tuple(params) diff --git a/sqlspec/adapters/sybase/config.py b/sqlspec/adapters/sybase/config.py new file mode 100644 index 000000000..70bac45c2 --- /dev/null +++ b/sqlspec/adapters/sybase/config.py @@ -0,0 +1,215 @@ +"""Sybase ASE database configuration.""" + +from collections.abc import Callable, Mapping +from typing import TYPE_CHECKING, Any, ClassVar, Literal, NotRequired, TypedDict, cast + +from sqlspec.adapters.sybase._typing import SybaseConnection, SybaseCursor, SybaseRawCursor, SybaseSessionContext +from sqlspec.adapters.sybase.core import apply_driver_features, build_connection_config, default_statement_config +from sqlspec.adapters.sybase.driver import SybaseDriver, SybaseExceptionHandler +from sqlspec.adapters.sybase.migrations import SybaseSyncMigrationTracker +from sqlspec.adapters.sybase.pool import SybaseConnectionPool +from sqlspec.config import ExtensionConfigs, SyncDatabaseConfig +from sqlspec.core import TypeCoercionCapabilities +from sqlspec.driver import SyncPoolConnectionContext, SyncPoolSessionFactory +from sqlspec.extensions.events import EventRuntimeHints +from sqlspec.utils.config_tools import normalize_connection_config + +if TYPE_CHECKING: + from sqlspec.core import StatementConfig + from sqlspec.observability import ObservabilityConfig + +__all__ = ( + "SybaseConfig", + "SybaseConnectionContext", + "SybaseConnectionParams", + "SybaseDriverFeatures", + "SybasePoolParams", + "SybaseTimeout", +) + +SybaseTimeout = int | float + + +class SybaseConnectionParams(TypedDict): + """Sybase ASE connection parameters.""" + + server: NotRequired[str] + """Database server host, IP, or FreeTDS server definition name.""" + host: NotRequired[str] + """Alias for ``server``.""" + freetds_server: NotRequired[str] + """FreeTDS server configuration section name.""" + server_alias: NotRequired[str] + """Alias for ``server``.""" + user: NotRequired[str] + """Login user name.""" + username: NotRequired[str] + """Alias for ``user``.""" + password: NotRequired[str] + """Authentication password.""" + database: NotRequired[str] + """Initial database name to connect to.""" + db: NotRequired[str] + """Alias for ``database``.""" + port: NotRequired[int | str] + """TCP port number of the Sybase ASE server.""" + timeout: NotRequired[SybaseTimeout] + """Query execution timeout in seconds.""" + login_timeout: NotRequired[SybaseTimeout] + """Login and connection timeout in seconds.""" + charset: NotRequired[str] + """Client character set encoding.""" + as_dict: NotRequired[bool] + """Return query result rows as dictionaries.""" + appname: NotRequired[str] + """Application name sent to the server for connection identification.""" + conn_properties: NotRequired[str] + """Additional connection properties string.""" + autocommit: NotRequired[bool] + """Enable autocommit mode for transactions.""" + tds_version: NotRequired[str] + """TDS protocol version (such as ``'5.0'`` or ``'7.4'``).""" + encryption: NotRequired[Literal["off", "request", "require"]] + """TDS connection encryption level.""" + arraysize: NotRequired[int] + """Default cursor fetch array size.""" + conv: NotRequired[Mapping[int | type[Any], Callable[..., Any]]] + """Custom datatype conversion dictionary mapping TDS types to python types.""" + read_only: NotRequired[bool] + """Enforce read-only transactions on connections.""" + pool_recycle_seconds: NotRequired[int] + """Maximum age in seconds before a pooled connection is recycled.""" + health_check_interval: NotRequired[float] + """Interval in seconds between background health checks.""" + extra: NotRequired["dict[str, Any]"] + """Additional driver connection arguments.""" + + +class SybasePoolParams(SybaseConnectionParams): + """Sybase ASE pool parameters.""" + + +class SybaseDriverFeatures(TypedDict): + """Sybase ASE driver feature flags.""" + + json_serializer: NotRequired["Callable[[Any], str]"] + """Custom callable for serializing objects to JSON strings.""" + json_deserializer: NotRequired["Callable[[str], Any]"] + """Custom callable for deserializing JSON strings.""" + on_connection_create: "NotRequired[Callable[[SybaseConnection], None]]" + """Callback invoked immediately when a new connection is established.""" + enable_events: NotRequired[bool] + """Enable database event listening.""" + events_backend: NotRequired[Literal["poll_queue"]] + """Backend mechanism used for dispatching events.""" + + +class SybaseConnectionContext(SyncPoolConnectionContext): + """Context manager for Sybase ASE connections.""" + + __slots__ = () + + +class _SybaseSessionConnectionHandler(SyncPoolSessionFactory): + __slots__ = () + + +class SybaseConfig(SyncDatabaseConfig[SybaseConnection, SybaseConnectionPool, SybaseDriver]): + """Configuration for Sybase ASE synchronous connections over pymssql (FreeTDS TDS 5.0).""" + + __slots__ = ("_user_connection_hook",) + + driver_type: "ClassVar[type[SybaseDriver]]" = SybaseDriver + connection_type: "ClassVar[type[SybaseConnection]]" = cast("type[SybaseConnection]", SybaseConnection) + migration_tracker_type: "ClassVar[type[SybaseSyncMigrationTracker]]" = SybaseSyncMigrationTracker + supports_transactional_ddl: "ClassVar[bool]" = False + supports_migration_schemas: "ClassVar[bool]" = False + supports_native_arrow_export: "ClassVar[bool]" = True + supports_native_arrow_import: "ClassVar[bool]" = True + supports_native_parquet_export: "ClassVar[bool]" = True + supports_native_parquet_import: "ClassVar[bool]" = True + supports_native_row_streaming: "ClassVar[bool]" = True + type_coercion_capabilities: "ClassVar[TypeCoercionCapabilities]" = TypeCoercionCapabilities( + datetime_binding="native", timestamp_precision="microsecond", json_columns_decoded=False, uuid_binding="text" + ) + _connection_context_class: "ClassVar[type[SybaseConnectionContext]]" = SybaseConnectionContext + _session_factory_class: "ClassVar[type[_SybaseSessionConnectionHandler]]" = _SybaseSessionConnectionHandler + _session_context_class: "ClassVar[type[SybaseSessionContext]]" = SybaseSessionContext + _default_statement_config = default_statement_config + + def __init__( + self, + *, + connection_config: "SybasePoolParams | dict[str, Any] | None" = None, + connection_instance: "SybaseConnectionPool | None" = None, + migration_config: "dict[str, Any] | None" = None, + statement_config: "StatementConfig | None" = None, + driver_features: "SybaseDriverFeatures | dict[str, Any] | None" = None, + bind_key: "str | None" = None, + extension_config: "ExtensionConfigs | None" = None, + observability_config: "ObservabilityConfig | None" = None, + **kwargs: Any, + ) -> None: + connection_config = build_connection_config(normalize_connection_config(connection_config)) + + statement_config = statement_config or default_statement_config + statement_config, driver_features = apply_driver_features(statement_config, driver_features) + + features_dict = dict(driver_features) if driver_features else {} + self._user_connection_hook: Callable[[SybaseConnection], None] | None = features_dict.pop( + "on_connection_create", None + ) + + super().__init__( + connection_config=connection_config, + connection_instance=connection_instance, + migration_config=migration_config, + statement_config=statement_config, + driver_features=features_dict, + bind_key=bind_key, + extension_config=extension_config, + observability_config=observability_config, + **kwargs, + ) + + def create_connection(self) -> "SybaseConnection": + """Open a standalone Sybase ASE connection owned by the caller.""" + return self.provide_pool().new_connection() + + def get_signature_namespace(self) -> "dict[str, Any]": + namespace = super().get_signature_namespace() + namespace.update({ + "SybaseConfig": SybaseConfig, + "SybaseConnection": SybaseConnection, + "SybaseConnectionContext": SybaseConnectionContext, + "SybaseConnectionParams": SybaseConnectionParams, + "SybaseConnectionPool": SybaseConnectionPool, + "SybaseCursor": SybaseCursor, + "SybaseDriver": SybaseDriver, + "SybaseDriverFeatures": SybaseDriverFeatures, + "SybaseExceptionHandler": SybaseExceptionHandler, + "SybasePoolParams": SybasePoolParams, + "SybaseRawCursor": SybaseRawCursor, + "SybaseSessionContext": SybaseSessionContext, + }) + return namespace + + def get_event_runtime_hints(self) -> "EventRuntimeHints": + """Return runtime hints for Sybase ASE event channels.""" + return EventRuntimeHints(poll_interval=0.25, lease_seconds=5) + + def _create_pool(self) -> "SybaseConnectionPool": + config = dict(self.connection_config) + pool_recycle = config.pop("pool_recycle_seconds", 86400) + health_check = config.pop("health_check_interval", 30.0) + return SybaseConnectionPool( + config, + recycle_seconds=pool_recycle, + health_check_interval=health_check, + on_connection_create=self._user_connection_hook, + ) + + def _close_pool(self) -> None: + if self.connection_instance: + self.connection_instance.close() + self.connection_instance = None diff --git a/sqlspec/adapters/sybase/core.py b/sqlspec/adapters/sybase/core.py new file mode 100644 index 000000000..37d946fcf --- /dev/null +++ b/sqlspec/adapters/sybase/core.py @@ -0,0 +1,411 @@ +"""Sybase ASE adapter compiled helpers.""" + +import re +from collections.abc import Callable, Mapping, Sized +from typing import TYPE_CHECKING, Any, Final, Literal + +from sqlspec.core import DriverParameterProfile, ParameterStyle, StatementConfig, build_statement_config_from_profile +from sqlspec.dialects.sybase import Sybase as _SybaseDialect +from sqlspec.exceptions import ( + CheckViolationError, + DatabaseConnectionError, + DataError, + DeadlockError, + ForeignKeyViolationError, + IntegrityError, + NotFoundError, + NotNullViolationError, + OperationalError, + PermissionDeniedError, + QueryTimeoutError, + SQLParsingError, + SQLSpecError, + UniqueViolationError, +) +from sqlspec.utils.serializers import from_json, to_json +from sqlspec.utils.text import split_qualified_identifier +from sqlspec.utils.type_converters import build_uuid_coercions +from sqlspec.utils.type_guards import has_rowcount + +if TYPE_CHECKING: + from collections.abc import Sequence + from logging import Logger + + from sqlspec.core.parameters import ParameterValidator + +__all__ = ( + "DEFAULT_SYBASE_CONN_PROPERTIES", + "DEFAULT_SYBASE_TDS_VERSION", + "apply_driver_features", + "build_connection_config", + "build_insert_statement", + "build_profile", + "build_statement_config", + "collect_rows", + "create_mapped_exception", + "default_statement_config", + "driver_profile", + "escape_literal_percent", + "escape_sql_literal", + "extract_error_number", + "format_identifier", + "normalize_execute_many_parameters", + "normalize_execute_parameters", + "quote_bracket_identifier", + "resolve_column_names", + "resolve_many_rowcount", + "resolve_rowcount", + "split_schema_and_table", +) + +DEFAULT_SYBASE_CONN_PROPERTIES: Final[str] = "SET QUOTED_IDENTIFIER ON; SET ANSINULL ON; SET TEXTSIZE 2147483647;" +DEFAULT_SYBASE_TDS_VERSION: Final[str] = "5.0" + +_ERROR_NUMBER_PATTERN: Final[re.Pattern[str]] = re.compile(r"(?:\(([-]?\d+)(?:,|\))|\bMsg\s+([-]?\d+)\b)") +_SYBASE_CONSTRAINT_547: Final[int] = 547 +_COLUMN_CACHE_MAX_SIZE: Final[int] = 256 +_ERROR_CODE_MAPPING: Final[dict[int, tuple[type[SQLSpecError], str]]] = { + 2601: (UniqueViolationError, "unique index violation"), + 2627: (UniqueViolationError, "unique constraint violation"), + 546: (ForeignKeyViolationError, "foreign key constraint violation"), + 548: (CheckViolationError, "check constraint violation"), + 233: (NotNullViolationError, "not-null constraint violation"), + 515: (NotNullViolationError, "not-null constraint violation"), + 1205: (DeadlockError, "deadlock detected"), + 12205: (QueryTimeoutError, "lock wait period exceeded"), + -2: (QueryTimeoutError, "query timeout"), + 1222: (QueryTimeoutError, "lock request time out period exceeded"), + 102: (SQLParsingError, "SQL syntax error"), + 105: (SQLParsingError, "unclosed quotation mark"), + 156: (SQLParsingError, "incorrect syntax near keyword"), + 11021: (SQLParsingError, "invalid function invocation"), + 207: (NotFoundError, "invalid column name"), + 208: (NotFoundError, "object not found"), + 4002: (DatabaseConnectionError, "login failed"), + 4060: (DatabaseConnectionError, "cannot open database"), + 18456: (DatabaseConnectionError, "login failed"), + 20002: (DatabaseConnectionError, "Adaptive Server connection failed"), + 20009: (DatabaseConnectionError, "Adaptive Server is unavailable or does not exist"), + 1105: (OperationalError, "cannot allocate space in database"), + 2762: (OperationalError, "DDL command is not allowed within a multi-statement transaction"), + 20047: (OperationalError, "DBPROCESS is dead or not enabled"), + 168: (DataError, "numeric scale conversion error"), + 220: (DataError, "arithmetic overflow error"), + 241: (DataError, "datetime conversion error"), + 242: (DataError, "datetime conversion out of range"), + 245: (DataError, "data conversion error"), + 247: (DataError, "arithmetic overflow during conversion"), + 3606: (DataError, "arithmetic overflow occurred"), + 9502: (DataError, "character or binary data would be truncated"), + 229: (PermissionDeniedError, "permission denied"), + 230: (PermissionDeniedError, "permission denied"), + 262: (PermissionDeniedError, "permission denied"), + 10330: (PermissionDeniedError, "object permission denied"), + 10331: (PermissionDeniedError, "database permission denied"), + 10332: (PermissionDeniedError, "execute permission denied"), +} + + +def quote_bracket_identifier(identifier: str) -> str: + """Quote a Sybase T-SQL identifier with square brackets.""" + cleaned = identifier.strip() + if (cleaned.startswith("[") and cleaned.endswith("]")) or (cleaned.startswith('"') and cleaned.endswith('"')): + cleaned = cleaned[1:-1].replace("]]", "]").replace('""', '"') + return f"[{cleaned.replace(']', ']]')}]" + + +def escape_sql_literal(value: str) -> str: + """Escape single quotes inside a SQL string literal.""" + return value.replace("'", "''") + + +def split_schema_and_table(identifier: str, *, default_schema: str = "dbo") -> tuple[str, str]: + """Split a table reference into unquoted ``(schema, table)`` names.""" + cleaned = identifier.strip() + if not cleaned: + msg = "Table name must not be empty" + raise SQLSpecError(msg) + parts = split_qualified_identifier(cleaned, quote_chars='"', allow_bracket_quotes=True) + if not parts: + return default_schema, cleaned + if len(parts) == 1: + return default_schema, parts[0] + return parts[-2], parts[-1] + + +def format_identifier(identifier: str) -> str: + """Format a Sybase T-SQL identifier with bracket quoting.""" + cleaned = identifier.strip() + if not cleaned: + msg = "Table name must not be empty" + raise SQLSpecError(msg) + parts = split_qualified_identifier(cleaned, quote_chars='"', allow_bracket_quotes=True) + return ".".join(quote_bracket_identifier(part) for part in parts) + + +def build_insert_statement(table: str, columns: "list[str]") -> str: + """Build a pymssql-compatible INSERT statement for Sybase ASE.""" + column_clause = ", ".join(quote_bracket_identifier(column) for column in columns) + placeholders = ", ".join("%s" for _ in columns) + return f"INSERT INTO {format_identifier(table)} ({column_clause}) VALUES ({placeholders})" + + +def escape_literal_percent(sql: str, parameters: Any, validator: "ParameterValidator") -> str: + """Escape literal percent characters before driver-side interpolation.""" + if not parameters or "%" not in sql: + return sql + keep = {info.position for info in validator.extract_parameters(sql)} + return "".join("%%" if char == "%" and index not in keep else char for index, char in enumerate(sql)) + + +def normalize_execute_parameters(parameters: Any) -> Any: + """Normalize parameters for Sybase pymssql execute calls.""" + if parameters is None: + return None + if isinstance(parameters, list): + return tuple(parameters) + return parameters + + +def normalize_execute_many_parameters(parameters: Any) -> Any: + """Normalize parameters for Sybase pymssql executemany calls.""" + return parameters + + +def build_profile() -> "DriverParameterProfile": + """Create the Sybase ASE driver parameter profile.""" + return DriverParameterProfile( + name="sybase", + default_style=ParameterStyle.QMARK, + supported_styles={ParameterStyle.QMARK, ParameterStyle.NAMED_PYFORMAT}, + default_execution_style=ParameterStyle.POSITIONAL_PYFORMAT, + supported_execution_styles={ParameterStyle.POSITIONAL_PYFORMAT}, + has_native_list_expansion=False, + preserve_parameter_format=True, + needs_static_script_compilation=False, + allow_mixed_parameter_styles=False, + preserve_original_params_for_many=False, + json_serializer_strategy="helper", + custom_type_coercions=_custom_type_coercions(), + default_dialect="sybase", + ) + + +def build_statement_config( + *, json_serializer: "Callable[[Any], str] | None" = None, json_deserializer: "Callable[[str], Any] | None" = None +) -> "StatementConfig": + """Construct the Sybase ASE statement configuration.""" + return build_statement_config_from_profile( + driver_profile, + statement_overrides={"dialect": "sybase"}, + json_serializer=json_serializer or to_json, + json_deserializer=json_deserializer or from_json, + ) + + +def apply_driver_features( + statement_config: "StatementConfig", driver_features: "Mapping[str, Any] | None" +) -> "tuple[StatementConfig, dict[str, Any]]": + """Apply Sybase ASE driver feature defaults to statement config.""" + features: dict[str, Any] = dict(driver_features) if driver_features else {} + json_serializer = features.setdefault("json_serializer", to_json) + json_deserializer = features.setdefault("json_deserializer", from_json) + + if json_serializer is not None and ( + json_serializer is not to_json + or json_deserializer is not from_json + or statement_config.parameter_config.json_serializer is not to_json + ): + parameter_config = statement_config.parameter_config.with_json_serializers( + json_serializer, deserializer=json_deserializer + ) + statement_config = statement_config.replace(parameter_config=parameter_config) + + return statement_config, features + + +def create_mapped_exception(error: BaseException, *, logger: "Logger | None" = None) -> SQLSpecError: + """Map a Sybase ASE exception to SQLSpec's exception hierarchy.""" + error_number = extract_error_number(error) + + if error_number == _SYBASE_CONSTRAINT_547: + message = str(error) + if "check constraint" in message.lower(): + return CheckViolationError(f"Sybase ASE error 547: check constraint violation. Original error: {error}") + return ForeignKeyViolationError( + f"Sybase ASE error 547: foreign key constraint violation. Original error: {error}" + ) + if error_number is not None: + mapping = _ERROR_CODE_MAPPING.get(error_number) + if mapping is not None: + error_class, description = mapping + return error_class(f"Sybase ASE error {error_number}: {description}. Original error: {error}") + if logger is not None: + logger.debug("Unmapped Sybase ASE error number: %s", error_number) + + constraint_exception = _constraint_exception_from_message(error) + if constraint_exception is not None: + return constraint_exception + + exc_name = type(error).__name__ + if exc_name == "IntegrityError": + return IntegrityError(f"Sybase ASE integrity error. Original error: {error}") + if exc_name == "OperationalError": + return OperationalError(f"Sybase ASE operational error. Original error: {error}") + if exc_name == "DataError": + return DataError(f"Sybase ASE data error. Original error: {error}") + return SQLSpecError(f"Sybase ASE database error. Original error: {error}") + + +def resolve_column_names( + description: "Sequence[Any] | None", column_name_cache: "dict[int, tuple[Any, list[str]]] | None" = None +) -> "list[str]": + """Resolve ordered column names from cursor metadata.""" + if not description: + return [] + if column_name_cache is None: + return [desc[0] for desc in description] + cache_key = id(description) + cached = column_name_cache.get(cache_key) + if cached is not None and cached[0] is description: + return cached[1] + column_names = [desc[0] for desc in description] + if len(column_name_cache) >= _COLUMN_CACHE_MAX_SIZE: + column_name_cache.pop(next(iter(column_name_cache))) + column_name_cache[cache_key] = (description, column_names) + return column_names + + +def collect_rows( + fetched_data: "Sequence[Any] | None", + description: "Sequence[Any] | None", + column_name_cache: "dict[int, tuple[Any, list[str]]] | None" = None, +) -> "tuple[list[Any], list[str], Literal['dict', 'tuple', 'record']]": + """Collect Sybase pymssql rows, preserving dictionary or tuple row shape.""" + column_names = resolve_column_names(description, column_name_cache) + if not fetched_data: + return [], column_names, "tuple" + rows = fetched_data if isinstance(fetched_data, list) else list(fetched_data) + if isinstance(rows[0], dict): + return rows, column_names, "dict" + return rows, column_names, "tuple" + + +def resolve_rowcount(cursor: Any) -> int: + """Resolve rowcount from a Sybase pymssql cursor.""" + if not has_rowcount(cursor): + return 0 + rowcount = cursor.rowcount + if isinstance(rowcount, int) and rowcount >= 0: + return rowcount + return 0 + + +def resolve_many_rowcount(cursor: Any, parameters: Any, *, fallback_count: "int | None" = None) -> int: + """Resolve executemany rowcount using cursor metadata with payload fallback.""" + rowcount = resolve_rowcount(cursor) + if rowcount > 0: + return rowcount + if fallback_count is not None: + return fallback_count + if isinstance(parameters, Sized): + return len(parameters) + return 0 + + +def extract_error_number(exc: BaseException | None) -> int | None: + """Extract integer Sybase ASE error code from an exception if present. + + Checks native integer attributes (number, error_code, errno) or args[0] before regex. + """ + if exc is None: + return None + for attr in ("number", "error_code", "errno"): + val = getattr(exc, attr, None) + if isinstance(val, int) and not isinstance(val, bool) and val != 0: + return val + if exc.args: + first = exc.args[0] + if isinstance(first, int) and not isinstance(first, bool) and first != 0: + return first + if isinstance(first, tuple) and first: + nested = first[0] + if isinstance(nested, int) and not isinstance(nested, bool) and nested != 0: + return nested + matches = _ERROR_NUMBER_PATTERN.findall(str(exc)) + if not matches: + return None + last_match = matches[-1] + return int(last_match[0] or last_match[1]) + + +def build_connection_config(connection_config: "Mapping[str, Any]") -> "dict[str, Any]": + """Build a normalized Sybase ASE connection configuration dictionary. + + Injects ``DEFAULT_SYBASE_CONN_PROPERTIES``, ``autocommit=True``, ``port=5000``, + and ``charset='UTF-8'`` by default, and strips ``tds_version`` into + ``_sybase_tds_version`` so ``SybaseConnectionPool`` can negotiate TDS 5.0 + via ``os.environ['TDSVER']`` without triggering ``pymssql._mssql``'s Cython + ``tds_version`` validator. + + Args: + connection_config: Raw connection configuration mapping. + + Returns: + Normalized connection parameters. + """ + config = dict(connection_config) + extra = config.pop("extra", None) + if isinstance(extra, Mapping): + for extra_key, extra_value in extra.items(): + if extra_value is not None: + config.setdefault(str(extra_key), extra_value) + config.setdefault("server", config.pop("host", "localhost")) + config.setdefault("port", 5000) + config.setdefault("charset", "UTF-8") + config.setdefault("autocommit", True) + config.setdefault("conn_properties", DEFAULT_SYBASE_CONN_PROPERTIES) + db_alias = config.pop("db", None) + if "database" not in config and db_alias is not None: + config["database"] = db_alias + user_alias = config.pop("username", None) + if "user" not in config and user_alias is not None: + config["user"] = user_alias + tds_version = str(config.pop("tds_version", config.pop("_sybase_tds_version", DEFAULT_SYBASE_TDS_VERSION))) + config["_sybase_tds_version"] = tds_version + return config + + +def _bool_to_int(value: bool) -> int: + return int(value) + + +def _constraint_exception_from_message(error: BaseException) -> "SQLSpecError | None": + """Classify Sybase ASE constraint messages when a driver omits the native error number.""" + message = str(error) + normalized = message.lower() + if "unique key constraint" in normalized or "duplicate key" in normalized: + return UniqueViolationError(f"Sybase ASE unique constraint violation. Original error: {message}") + if ( + "cannot insert the value null" in normalized + or "does not allow nulls" in normalized + or "does not allow null values" in normalized + ): + return NotNullViolationError(f"Sybase ASE not-null constraint violation. Original error: {message}") + if "check constraint" in normalized: + return CheckViolationError(f"Sybase ASE check constraint violation. Original error: {message}") + if "foreign key constraint" in normalized: + return ForeignKeyViolationError(f"Sybase ASE foreign key constraint violation. Original error: {message}") + return None + + +def _custom_type_coercions() -> dict[type, Callable[[Any], Any]]: + """Return custom type coercions for Sybase ASE.""" + coercions: dict[type, Callable[[Any], Any]] = {bool: _bool_to_int} + coercions.update(build_uuid_coercions()) + return coercions + + +_ = _SybaseDialect +driver_profile = build_profile() +default_statement_config = build_statement_config() diff --git a/sqlspec/adapters/sybase/data_dictionary.py b/sqlspec/adapters/sybase/data_dictionary.py new file mode 100644 index 000000000..1c8200f2b --- /dev/null +++ b/sqlspec/adapters/sybase/data_dictionary.py @@ -0,0 +1,281 @@ +"""Sybase ASE database data dictionary implementation.""" + +import logging +from typing import TYPE_CHECKING, Any, ClassVar, cast + +from mypy_extensions import mypyc_attr + +from sqlspec.data_dictionary import ( + ColumnMetadata, + ForeignKeyMetadata, + IndexMetadata, + MetadataCapabilityProfile, + MetadataResult, + TableMetadata, +) +from sqlspec.data_dictionary.dialects.sybase import ( + SYBASE_CONFIG, + SYBASE_VERSION_PATTERN, + SybaseVersionInfo, + build_sybase_metadata_capability_profile, + extract_sybase_version_value, + list_sybase_available_features, + merge_sybase_table_lists, + resolve_sybase_feature_flag, +) +from sqlspec.driver import SyncDataDictionaryBase +from sqlspec.utils.logging import get_logger, log_with_context + +if TYPE_CHECKING: + from collections.abc import Iterable, Sequence + + from sqlspec.adapters.sybase.driver import SybaseDriver + +__all__ = ( + "SYBASE_CONFIG", + "SybaseDataDictionary", + "SybaseSyncDataDictionary", + "SybaseVersionInfo", + "build_column_metadata", + "build_foreign_key_metadata", + "build_index_metadata", + "parse_sybase_version", +) + +logger = get_logger("sqlspec.adapters.sybase.data_dictionary") + + +def parse_sybase_version(row: "dict[str, Any] | None") -> "SybaseVersionInfo | None": + """Parse the Sybase ASE version banner returned by the version query.""" + version_str = extract_sybase_version_value(row) + match = SYBASE_VERSION_PATTERN.search(version_str) if version_str else None + if match is None: + log_with_context(logger, logging.DEBUG, "data_dictionary.version.unparsed", dialect="sybase", value=version_str) + return None + sp_num = int(match.group(3)) if match.group(3) else 0 + sp_label = f"SP{sp_num:02d}" if match.group(3) else None + return SybaseVersionInfo(int(match.group(1)), int(match.group(2)), sp_num, service_pack=sp_label) + + +def build_column_metadata(rows: "Iterable[dict[str, Any]]") -> "list[ColumnMetadata]": + """Convert Sybase syscolumns query rows to column metadata.""" + return [ + ColumnMetadata( + schema_name=str(r.get("schema_name", "")), + table_name=str(r.get("table_name", "")), + column_name=str(r.get("column_name", "")), + data_type=str(r.get("data_type", "")), + is_nullable=bool(r.get("is_nullable", True)), + column_default=str(r["column_default"]) if r.get("column_default") is not None else None, + ordinal_position=int(r.get("ordinal_position", 1)), + max_length=int(r.get("max_length", 0)) if r.get("max_length") is not None else 0, + numeric_precision=int(r.get("numeric_precision", 0)) if r.get("numeric_precision") is not None else 0, + numeric_scale=int(r.get("numeric_scale", 0)) if r.get("numeric_scale") is not None else 0, + is_primary=bool(r.get("is_primary")), + is_unique=bool(r.get("is_unique")), + is_identity=bool(r.get("is_identity")), + identity_generation=str(r["identity_generation"]) + if r.get("identity_generation") + else ("ALWAYS" if r.get("is_identity") else None), + is_generated=bool(r.get("is_generated")), + generation_expression=str(r["generation_expression"]) + if r.get("generation_expression") is not None + else None, + is_hidden=bool(r.get("is_hidden")), + ) + for r in rows + ] + + +def build_index_metadata(rows: "Iterable[dict[str, Any]]") -> "list[IndexMetadata]": + """Group per-column Sybase sysindexes rows into index metadata.""" + indexes_map: dict[tuple[str, str, str], dict[str, Any]] = {} + for r in rows: + key = (str(r.get("schema_name", "")), str(r.get("table_name", "")), str(r.get("index_name", ""))) + if key not in indexes_map: + indexes_map[key] = { + "schema_name": key[0], + "table_name": key[1], + "index_name": key[2], + "columns": [], + "is_unique": bool(r.get("is_unique", 0)), + "is_primary": bool(r.get("is_primary", 0)), + } + col_name = str(r.get("column_name", "")) + if col_name: + indexes_map[key]["columns"].append(col_name) + + return [ + IndexMetadata( + schema_name=v["schema_name"], + table_name=v["table_name"], + index_name=v["index_name"], + columns=v["columns"], + is_unique=v["is_unique"], + is_primary=v["is_primary"], + ) + for v in indexes_map.values() + ] + + +def build_foreign_key_metadata(rows: "Iterable[dict[str, Any]]") -> "list[ForeignKeyMetadata]": + """Convert Sybase sysreferences query rows to foreign key metadata.""" + return [ + ForeignKeyMetadata( + schema=str(r.get("schema_name", "")), + table_name=str(r.get("table_name", "")), + constraint_name=str(r.get("constraint_name", "")), + column_name=str(r.get("column_name", "")), + referenced_schema=str(r.get("referenced_schema", "")), + referenced_table=str(r.get("referenced_table", "")), + referenced_column=str(r.get("referenced_column", "")), + ) + for r in rows + ] + + +@mypyc_attr(allow_interpreted_subclasses=True, native_class=False) +class SybaseSyncDataDictionary(SyncDataDictionaryBase): + """Sybase ASE sync data dictionary for metadata reflection.""" + + dialect: ClassVar[str] = "sybase" + + def __init__(self) -> None: + """Initialize Sybase ASE sync data dictionary.""" + super().__init__() + + def get_metadata_capabilities( + self, driver: "SybaseDriver", domains: "Sequence[str] | None" = None + ) -> MetadataCapabilityProfile: + """Get Sybase ASE data-dictionary capability profile.""" + _ = driver + return build_sybase_metadata_capability_profile(type(self).__name__, domains) + + def get_version(self, driver: "SybaseDriver") -> SybaseVersionInfo | None: + """Get Sybase ASE database version information.""" + driver_id = id(driver) + if driver_id in self._version_fetch_attempted: + return cast("SybaseVersionInfo | None", self._version_cache.get(driver_id)) + + version_info = parse_sybase_version(driver.select_one_or_none(self.get_query("version", "current"))) + self.cache_version(driver_id, version_info) + return version_info + + def get_feature_flag(self, driver: "SybaseDriver", feature: str) -> bool: + """Check whether Sybase ASE supports a feature.""" + version_info = self.get_version(driver) + return resolve_sybase_feature_flag(feature, version_info) + + def get_optimal_type(self, driver: "SybaseDriver", type_category: str) -> str: + """Get optimal Sybase ASE type for a category.""" + _ = driver + return SYBASE_CONFIG.get_optimal_type(type_category) + + def list_available_features(self) -> list[str]: + """List available feature flags for this dialect.""" + return list_sybase_available_features() + + def get_tables(self, driver: "SybaseDriver", schema: str | None = None) -> list[TableMetadata]: + """Get tables sorted by dependency order with catalog fallback.""" + schema_name = self.resolve_schema(schema) + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="tables") + + ordered_rows = cast( + "list[TableMetadata]", + driver.select( + self.get_query("tables", "by_schema"), + schema_name=schema_name, + table_name=None, + schema_type=TableMetadata, + ), + ) + all_rows = cast( + "list[TableMetadata]", + driver.select( + self.get_query("tables", "all_by_schema"), + schema_name=schema_name, + table_name=None, + schema_type=TableMetadata, + ), + ) + return merge_sybase_table_lists(ordered_rows, all_rows) + + def get_columns( + self, driver: "SybaseDriver", table: str | None = None, schema: str | None = None + ) -> list[ColumnMetadata]: + """Get columns for a table or schema from dbo.syscolumns.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="columns") + rows = driver.select(self.get_query("columns", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="columns") + rows = driver.select(self.get_query("columns", "by_table"), schema_name=schema_name, table_name=table_name) + + return build_column_metadata(rows) + + def get_indexes( + self, driver: "SybaseDriver", table: str | None = None, schema: str | None = None + ) -> list[IndexMetadata]: + """Get indexes for a table or schema from dbo.sysindexes.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="indexes") + rows = driver.select(self.get_query("indexes", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="indexes") + rows = driver.select(self.get_query("indexes", "by_table"), schema_name=schema_name, table_name=table_name) + + return build_index_metadata(rows) + + def get_foreign_keys( + self, driver: "SybaseDriver", table: str | None = None, schema: str | None = None + ) -> list[ForeignKeyMetadata]: + """Get foreign keys from dbo.sysreferences.""" + schema_name = self.resolve_schema(schema) + if table is None: + self._log_schema_introspect(driver, schema_name=schema_name, table_name=None, operation="foreign_keys") + rows = driver.select(self.get_query("foreign_keys", "by_schema"), schema_name=schema_name, table_name=None) + else: + table_name = self.resolve_identifier(table) + self._log_table_describe(driver, schema_name=schema_name, table_name=table_name, operation="foreign_keys") + rows = driver.select( + self.get_query("foreign_keys", "by_table"), schema_name=schema_name, table_name=table_name + ) + + return build_foreign_key_metadata(rows) + + def get_constraints( + self, driver: "SybaseDriver", table: str | None = None, schema: str | None = None + ) -> MetadataResult: + """Get Sybase ASE constraint metadata.""" + schema_name = self.resolve_schema(schema) + table_name = self.resolve_identifier(table) if table is not None else None + query = self.get_query("constraints", "by_table" if table_name else "by_schema") + rows = driver.select(query, schema_name=schema_name, table_name=table_name) + return MetadataResult("constraints", items=tuple(rows)) + + def get_views(self, driver: "SybaseDriver", schema: str | None = None) -> MetadataResult: + """Get Sybase ASE view metadata.""" + schema_name = self.resolve_schema(schema) + query = self.get_query("views", "by_schema") + rows = driver.select(query, schema_name=schema_name, view_name=None) + return MetadataResult("views", items=tuple(rows)) + + def get_objects(self, driver: "SybaseDriver", schema: "str | None" = None) -> MetadataResult: + """Get tables, views, and procedures from the Sybase ASE catalog.""" + rows = driver.select( + self.get_query("objects", "by_schema"), schema_name=self.resolve_schema(schema), object_name=None + ) + return MetadataResult("objects", items=tuple(rows)) + + def get_schemas(self, driver: "SybaseDriver") -> MetadataResult: + """Get Sybase ASE schema metadata.""" + query = self.get_query("schemas", "by_schema") + rows = driver.select(query, schema_name=None) + return MetadataResult("schemas", items=tuple(rows)) + + +SybaseDataDictionary = SybaseSyncDataDictionary diff --git a/sqlspec/adapters/sybase/driver.py b/sqlspec/adapters/sybase/driver.py new file mode 100644 index 000000000..1d6bf2885 --- /dev/null +++ b/sqlspec/adapters/sybase/driver.py @@ -0,0 +1,385 @@ +"""Sybase ASE database driver implementation over pymssql (FreeTDS TDS 5.0).""" + +import contextlib +from collections.abc import Sized +from typing import TYPE_CHECKING, Any, cast + +from sqlspec.adapters.sybase._typing import ( + SybaseConnection, + SybaseCursor, + SybaseError, + SybaseRawCursor, + SybaseSessionContext, +) +from sqlspec.adapters.sybase.core import ( + build_insert_statement, + collect_rows, + create_mapped_exception, + default_statement_config, + driver_profile, + escape_literal_percent, + format_identifier, + normalize_execute_many_parameters, + normalize_execute_parameters, + resolve_column_names, + resolve_many_rowcount, + resolve_rowcount, +) +from sqlspec.adapters.sybase.data_dictionary import SybaseSyncDataDictionary +from sqlspec.core import SQL, StatementConfig, get_cache_config, register_driver_profile +from sqlspec.driver import ( + BaseSyncExceptionHandler, + ExecutionResult, + SyncDriverAdapterBase, + SyncRowStream, + rows_to_dicts, + validate_savepoint_name, +) +from sqlspec.exceptions import SQLSpecError +from sqlspec.utils.logging import get_logger + +if TYPE_CHECKING: + from collections.abc import Sequence + + from sqlspec.adapters.sybase._typing import SybaseQueryParams as QueryParams + from sqlspec.builder import QueryBuilder + from sqlspec.core import ArrowResult, SQLResult, Statement, StatementFilter + from sqlspec.driver import CachedQuery + from sqlspec.storage import StorageBridgeJob, StorageDestination, StorageFormat, StorageTelemetry + from sqlspec.typing import StatementParameters + +__all__ = ("SybaseCursor", "SybaseDriver", "SybaseExceptionHandler", "SybaseSessionContext", "SybaseStreamSource") + +logger = get_logger("sqlspec.adapters.sybase") + + +class SybaseExceptionHandler(BaseSyncExceptionHandler): + """Context manager for handling Sybase pymssql exceptions.""" + + __slots__ = () + + def _handle_exception(self, exc_type: "type[BaseException] | None", exc_val: "BaseException") -> bool: + if exc_type is None: + return False + if isinstance(exc_val, (SybaseError, Exception)) and not isinstance(exc_val, SQLSpecError): + self.pending_exception = create_mapped_exception(exc_val, logger=logger) + return True + return False + + +class SybaseStreamSource: + """Native Sybase pymssql chunk source backed by ``cursor.fetchmany()``.""" + + __slots__ = ("_chunk_size", "_column_names", "_cursor_manager", "_driver", "_parameters", "_sql") + + def __init__(self, driver: "SybaseDriver", sql: str, parameters: Any, chunk_size: int) -> None: + self._driver = driver + self._sql = sql + self._parameters = parameters + self._chunk_size = chunk_size + self._cursor_manager: SybaseCursor | None = None + self._column_names: list[str] | None = None + + def start(self) -> None: + cursor_manager = self._driver.with_cursor(self._driver.connection) + try: + cursor = cursor_manager.__enter__() + handler = self._driver.handle_database_exceptions() + with handler: + normalized_params = normalize_execute_parameters(self._parameters) + sql = escape_literal_percent( + self._sql, normalized_params, self._driver.statement_config.parameter_validator + ) + cursor.execute(sql, normalized_params) + self._driver._check_pending_exception(handler) + except BaseException: + with contextlib.suppress(Exception): + cursor_manager.__exit__(None, None, None) + raise + self._cursor_manager = cursor_manager + + def fetch_chunk(self) -> "list[dict[str, Any]]": + cursor_manager = self._cursor_manager + if cursor_manager is None or cursor_manager.cursor is None: + return [] + cursor = cursor_manager.cursor + handler = self._driver.handle_database_exceptions() + rows: Any = [] + with handler: + rows = cursor.fetchmany(self._chunk_size) + self._driver._check_pending_exception(handler) + if not rows: + return [] + column_names = self._column_names + if column_names is None: + column_names = resolve_column_names(cursor.description or None, self._driver._column_name_cache) + self._column_names = column_names + return rows_to_dicts(rows, column_names) + + def close(self, error: bool = False) -> None: + _ = error + cursor_manager = self._cursor_manager + self._cursor_manager = None + if cursor_manager is not None: + with contextlib.suppress(Exception): + cursor_manager.__exit__(None, None, None) + + +class SybaseDriver(SyncDriverAdapterBase): + """Sybase Adaptive Server Enterprise database driver using pymssql over FreeTDS TDS 5.0.""" + + __slots__ = ("_column_name_cache", "_data_dictionary", "_explicit_transaction", "_transaction_active") + dialect = "sybase" + + def __init__( + self, + connection: "SybaseConnection", + statement_config: "StatementConfig | None" = None, + driver_features: "dict[str, Any] | None" = None, + ) -> None: + if statement_config is None: + statement_config = default_statement_config.replace( + enable_caching=get_cache_config().compiled_cache_enabled + ) + + super().__init__(connection=connection, statement_config=statement_config, driver_features=driver_features) + self._data_dictionary: SybaseSyncDataDictionary | None = None + self._column_name_cache: dict[int, tuple[Any, list[str]]] = {} + self._transaction_active = False + self._explicit_transaction = False + + @property + def data_dictionary(self) -> "SybaseSyncDataDictionary": + if self._data_dictionary is None: + self._data_dictionary = SybaseSyncDataDictionary() + return self._data_dictionary + + def dispatch_execute(self, cursor: "SybaseRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + normalized_params = normalize_execute_parameters(prepared_parameters) + sql = escape_literal_percent(sql, normalized_params, self.statement_config.parameter_validator) + cursor.execute(sql, normalized_params) + + if statement.returns_rows(): + fetched_data = cursor.fetchall() + description = cursor.description or None + rows, column_names, row_format = collect_rows(fetched_data, description, self._column_name_cache) + return self.create_execution_result( + cursor, + selected_data=rows, + column_names=column_names, + data_row_count=len(rows), + is_select_result=True, + row_format=row_format, + ) + + return self.create_execution_result(cursor, rowcount_override=resolve_rowcount(cursor)) + + def dispatch_execute_many(self, cursor: "SybaseRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + + prepared_parameters = normalize_execute_many_parameters(prepared_parameters) + sql = escape_literal_percent(sql, prepared_parameters, self.statement_config.parameter_validator) + parameter_count = len(prepared_parameters) if isinstance(prepared_parameters, Sized) else None + cursor.executemany(sql, cast("Sequence[QueryParams]", prepared_parameters)) + + affected_rows = resolve_many_rowcount(cursor, prepared_parameters, fallback_count=parameter_count) + return self.create_execution_result(cursor, rowcount_override=affected_rows, is_many_result=True) + + def dispatch_execute_script(self, cursor: "SybaseRawCursor", statement: "SQL") -> "ExecutionResult": + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + statements = self.split_script_statements(sql, statement.statement_config, strip_trailing_semicolon=True) + normalized_params = normalize_execute_parameters(prepared_parameters) + + successful_count = 0 + for stmt in statements: + stmt_sql = escape_literal_percent(stmt, normalized_params, self.statement_config.parameter_validator) + cursor.execute(stmt_sql, normalized_params) + successful_count += 1 + return self.create_execution_result( + cursor, statement_count=len(statements), successful_statements=successful_count, is_script_result=True + ) + + def begin(self) -> None: + """Begin a transaction on the Sybase ASE connection.""" + try: + explicit = bool(getattr(self.connection, "autocommit_state", True)) + if explicit: + with SybaseCursor(self.connection) as cursor: + cursor.execute("BEGIN TRANSACTION") + self._explicit_transaction = explicit + self._transaction_active = True + except SybaseError as exc: + msg = f"Failed to begin Sybase ASE transaction: {exc}" + raise SQLSpecError(msg) from exc + + def commit(self) -> None: + try: + if self._explicit_transaction: + with SybaseCursor(self.connection) as cursor: + cursor.execute("IF @@TRANCOUNT > 0 COMMIT TRANSACTION") + else: + self.connection.commit() + self._explicit_transaction = False + self._transaction_active = False + except SybaseError as exc: + msg = f"Failed to commit Sybase ASE transaction: {exc}" + raise SQLSpecError(msg) from exc + + def rollback(self) -> None: + try: + if self._explicit_transaction: + with SybaseCursor(self.connection) as cursor: + cursor.execute("IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION") + else: + self.connection.rollback() + except SybaseError as exc: + msg = f"Failed to rollback Sybase ASE transaction: {exc}" + raise SQLSpecError(msg) from exc + finally: + self._explicit_transaction = False + self._transaction_active = False + + def with_cursor(self, connection: "SybaseConnection") -> "SybaseCursor": + return SybaseCursor(connection) + + def handle_database_exceptions(self) -> "SybaseExceptionHandler": + return SybaseExceptionHandler() + + def dispatch_select_stream(self, statement: "SQL", chunk_size: int) -> "SyncRowStream[dict[str, Any]] | None": + """Return a native Sybase pymssql row stream backed by ``fetchmany()``.""" + if not statement.returns_rows(): + return None + sql, prepared_parameters = self._compiled_sql(statement, self.statement_config) + return SyncRowStream(SybaseStreamSource(self, sql, prepared_parameters, chunk_size)) + + def create_savepoint(self, name: str) -> None: + self.execute_script(f"SAVE TRANSACTION {validate_savepoint_name(name)}") + + def release_savepoint(self, name: str) -> None: + validate_savepoint_name(name) + + def rollback_to_savepoint(self, name: str) -> None: + self.execute_script(f"ROLLBACK TRANSACTION {validate_savepoint_name(name)}") + + def has_schema(self, schema: str) -> bool: + """Return whether the specified user/schema exists in dbo.sysusers.""" + result = False + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.execute("SELECT 1 FROM dbo.sysusers WHERE name = %s", (schema,)) + result = cursor.fetchone() is not None + self._check_pending_exception(exc_handler) + return result + + def select_to_storage( + self, + statement: "Statement | QueryBuilder | SQL | str", + destination: "StorageDestination", + /, + *parameters: "StatementParameters | StatementFilter", + statement_config: "StatementConfig | None" = None, + partitioner: "dict[str, object] | None" = None, + format_hint: "StorageFormat | None" = None, + telemetry: "StorageTelemetry | None" = None, + **kwargs: Any, + ) -> "StorageBridgeJob": + """Execute a query and write the Arrow result to storage.""" + self._require_capability("arrow_export_enabled") + arrow_result = self.select_to_arrow(statement, *parameters, statement_config=statement_config, **kwargs) + pipeline = self._storage_pipeline() + telemetry_payload = self._write_storage_result( + arrow_result, destination, format_hint=format_hint, pipeline=pipeline + ) + self._attach_partition_telemetry(telemetry_payload, partitioner) + return self._storage_job(telemetry_payload, telemetry) + + def load_from_arrow( + self, + table: str, + source: "ArrowResult | Any", + *, + partitioner: "dict[str, object] | None" = None, + overwrite: bool = False, + telemetry: "StorageTelemetry | None" = None, + ) -> "StorageBridgeJob": + """Load Arrow data into the target Sybase ASE table via executemany.""" + self._require_capability("arrow_import_enabled") + arrow_table = self._coerce_arrow_table(source) + if overwrite: + statement = f"DELETE FROM {format_identifier(table)}" + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.execute(statement) + self._check_pending_exception(exc_handler) + + columns, records = self._arrow_table_to_rows(arrow_table) + if records: + insert_sql = escape_literal_percent( + build_insert_statement(table, columns), records, self.statement_config.parameter_validator + ) + needs_preparation = self._arrow_rows_need_preparation(arrow_table) + prepared_records = ( + self.prepare_driver_parameters(records, self.statement_config, is_many=True) + if needs_preparation + else records + ) + normalized_records = normalize_execute_many_parameters(prepared_records) + exc_handler = self.handle_database_exceptions() + with exc_handler, self.with_cursor(self.connection) as cursor: + cursor.executemany(insert_sql, cast("Sequence[QueryParams]", normalized_records)) + self._check_pending_exception(exc_handler) + + telemetry_payload = self._ingest_telemetry(arrow_table) + telemetry_payload["destination"] = table + self._attach_partition_telemetry(telemetry_payload, partitioner) + return self._storage_job(telemetry_payload, telemetry) + + def load_from_storage( + self, + table: str, + source: "StorageDestination", + *, + file_format: "StorageFormat", + partitioner: "dict[str, object] | None" = None, + overwrite: bool = False, + ) -> "StorageBridgeJob": + """Load staged artifacts from storage into the target Sybase ASE table.""" + arrow_table, inbound = self._read_storage_arrow(source, file_format=file_format) + return self.load_from_arrow(table, arrow_table, partitioner=partitioner, overwrite=overwrite, telemetry=inbound) + + def collect_rows(self, cursor: "SybaseRawCursor", fetched: "list[Any]") -> "tuple[list[Any], list[str], int]": + rows, column_names, _row_format = collect_rows(fetched, cursor.description or None, self._column_name_cache) + return rows, column_names, len(rows) + + def resolve_rowcount(self, cursor: "SybaseRawCursor") -> int: + return resolve_rowcount(cursor) + + def _execute_cache_hit( + self, sql: str, params: "tuple[Any, ...] | list[Any] | dict[str, Any]", cached: "CachedQuery" + ) -> "SQLResult": + normalized_params = normalize_execute_parameters(params) if isinstance(params, list) else params + if not normalized_params or ( + "%" in cached.compiled_sql + and escape_literal_percent( + cached.compiled_sql, normalized_params, self.statement_config.parameter_validator + ) + != cached.compiled_sql + ): + statement = self._cached_statement( + sql, + normalized_params, + cached, + normalized_params or None, + params_are_simple=True, + compiled_sql=cached.compiled_sql, + ) + return self._execute_cached_statement(statement) + return super()._execute_cache_hit(sql, normalized_params, cached) + + def _connection_in_transaction(self) -> bool: + """Return whether a transaction opened by this driver remains active.""" + return self._transaction_active + + +register_driver_profile("sybase", driver_profile) diff --git a/sqlspec/adapters/sybase/events/__init__.py b/sqlspec/adapters/sybase/events/__init__.py new file mode 100644 index 000000000..7bf12a0d3 --- /dev/null +++ b/sqlspec/adapters/sybase/events/__init__.py @@ -0,0 +1,5 @@ +"""Sybase ASE event extension.""" + +from sqlspec.adapters.sybase.events.store import SybaseEventQueueStore, SybaseSyncEventQueueStore + +__all__ = ("SybaseEventQueueStore", "SybaseSyncEventQueueStore") diff --git a/sqlspec/adapters/sybase/events/store.py b/sqlspec/adapters/sybase/events/store.py new file mode 100644 index 000000000..d5871ac5d --- /dev/null +++ b/sqlspec/adapters/sybase/events/store.py @@ -0,0 +1,83 @@ +"""Sybase ASE event queue store with ASE-compatible DDL.""" + +from sqlspec.adapters.sybase.config import SybaseConfig +from sqlspec.adapters.sybase.core import escape_sql_literal, quote_bracket_identifier, split_schema_and_table +from sqlspec.extensions.events import BaseEventQueueStore + +__all__ = ("SybaseEventQueueStore", "SybaseSyncEventQueueStore") + +_VARCHAR_TEXT_THRESHOLD = 4000 + + +class SybaseSyncEventQueueStore(BaseEventQueueStore[SybaseConfig]): + """Sybase ASE DDL hooks for the event queue store.""" + + __slots__ = () + + def _column_types(self) -> tuple[str, str, str]: + return "TEXT", "TEXT", "BIGDATETIME" + + def _string_type(self, length: int) -> str: + if length >= _VARCHAR_TEXT_THRESHOLD: + return "TEXT" + return f"VARCHAR({length})" + + def _integer_type(self) -> str: + return "INT" + + def _timestamp_default(self) -> str: + return "GETUTCDATE()" + + def _table_ddl(self) -> str: + """Build Sybase ASE-compatible CREATE TABLE SQL with DEFAULT before NOT NULL and explicit NULL.""" + payload_type, metadata_type, timestamp_type = self._column_types() + string_64 = self._string_type(64) + string_128 = self._string_type(128) + string_32 = self._string_type(32) + integer_type = self._integer_type() + ts_default = self._timestamp_default() + pk_inline = self._primary_key_syntax() + table_clause = self._table_clause() + + pk_column = " PRIMARY KEY" if not pk_inline else "" + + return ( + f"CREATE TABLE {self.table_name} (" + f"event_id {string_64}{pk_column} NOT NULL, " + f"channel {string_128} NOT NULL, " + f"payload_json {payload_type} NOT NULL, " + f"metadata_json {metadata_type} NULL, " + f"status {string_32} DEFAULT 'pending' NOT NULL, " + f"available_at {timestamp_type} DEFAULT {ts_default} NOT NULL, " + f"lease_expires_at {timestamp_type} NULL, " + f"attempts {integer_type} DEFAULT 0 NOT NULL, " + f"created_at {timestamp_type} DEFAULT {ts_default} NOT NULL, " + f"acknowledged_at {timestamp_type} NULL" + f"){pk_inline}{table_clause}" + ) + + def _wrap_create_statement(self, statement: str, object_type: str) -> str: + object_name = escape_sql_literal(_object_name(self.table_name)) + escaped_statement = escape_sql_literal(statement.rstrip().rstrip(";")) + if object_type == "table": + return f"IF OBJECT_ID('{object_name}', 'U') IS NULL EXECUTE('{escaped_statement}')" + if object_type == "index": + index_name = escape_sql_literal(self._index_name()) + return ( + f"IF NOT EXISTS (SELECT 1 FROM dbo.sysindexes WHERE name = '{index_name}' " + f"AND id = OBJECT_ID('{object_name}')) EXECUTE('{escaped_statement}')" + ) + return statement + + def _wrap_drop_statement(self, statement: str) -> str: + object_name = escape_sql_literal(_object_name(self.table_name)) + escaped_statement = escape_sql_literal(statement.rstrip().rstrip(";")) + return f"IF OBJECT_ID('{object_name}', 'U') IS NOT NULL EXECUTE('{escaped_statement}');" + + +SybaseEventQueueStore = SybaseSyncEventQueueStore + + +def _object_name(table_name: str) -> str: + schema_name, bare_table_name = split_schema_and_table(table_name) + return f"{quote_bracket_identifier(schema_name)}.{quote_bracket_identifier(bare_table_name)}" diff --git a/sqlspec/adapters/sybase/litestar/__init__.py b/sqlspec/adapters/sybase/litestar/__init__.py new file mode 100644 index 000000000..800850dd5 --- /dev/null +++ b/sqlspec/adapters/sybase/litestar/__init__.py @@ -0,0 +1,5 @@ +"""Sybase ASE Litestar extension.""" + +from sqlspec.adapters.sybase.litestar.store import SybaseStore + +__all__ = ("SybaseStore",) diff --git a/sqlspec/adapters/sybase/litestar/store.py b/sqlspec/adapters/sybase/litestar/store.py new file mode 100644 index 000000000..e81a4fe0d --- /dev/null +++ b/sqlspec/adapters/sybase/litestar/store.py @@ -0,0 +1,241 @@ +"""Sybase ASE Litestar Store implementation.""" + +from datetime import UTC, datetime, timedelta +from typing import TYPE_CHECKING, Any + +from sqlspec.adapters.sybase._typing import SybaseCursor +from sqlspec.adapters.sybase.core import escape_sql_literal, quote_bracket_identifier, split_schema_and_table +from sqlspec.extensions.litestar.store import BaseSQLSpecStore +from sqlspec.utils.sync_tools import async_ + +if TYPE_CHECKING: + from sqlspec.adapters.sybase.config import SybaseConfig + +__all__ = ("SybaseStore",) + + +class SybaseStore(BaseSQLSpecStore["SybaseConfig"]): + """Sybase ASE-backed session store using pymssql sync sessions.""" + + __slots__ = () + + def __init__(self, config: "SybaseConfig") -> None: + super().__init__(config) + + async def create_table(self) -> None: + """Create the session table if it doesn't exist.""" + if not self.create_schema_enabled: + await self.reconcile_schema() + return + await async_(self._create_table)() + await self.reconcile_schema(assume_existing=True) + + async def get(self, key: str, renew_for: "int | timedelta | None" = None) -> "bytes | None": + """Get a session value by key.""" + return await async_(self._get)(key, renew_for) + + async def set(self, key: str, value: "str | bytes", expires_in: "int | timedelta | None" = None) -> None: + """Store a session value.""" + await async_(self._set)(key, value, expires_in) + + async def delete(self, key: str) -> None: + """Delete a session by key.""" + await async_(self._delete)(key) + + async def delete_all(self) -> None: + """Delete all sessions from the store.""" + await async_(self._delete_all)() + + async def exists(self, key: str) -> bool: + """Check if a session key exists and is not expired.""" + return await async_(self._exists)(key) + + async def expires_in(self, key: str) -> "int | None": + """Get the time in seconds until the session expires.""" + return await async_(self._expires_in)(key) + + async def delete_expired(self) -> int: + """Delete all expired sessions.""" + return await async_(self._delete_expired)() + + def _table_ddl(self) -> str: + """Get Sybase ASE CREATE TABLE SQL with OBJECT_ID existence guards.""" + _, table_name = split_schema_and_table(self._table_name) + table_ref = _table_ref(self._table_name) + escaped_table_ref = escape_sql_literal(table_ref) + index_bare = f"IX_{table_name}_expires_at" + escaped_index_bare = escape_sql_literal(index_bare) + index_name = quote_bracket_identifier(index_bare) + create_table_sql = escape_sql_literal( + f"CREATE TABLE {table_ref} (" + "session_id VARCHAR(255) NOT NULL PRIMARY KEY, " + "data IMAGE NOT NULL, " + "expires_at BIGDATETIME NULL, " + "created_at BIGDATETIME DEFAULT GETUTCDATE() NOT NULL, " + "updated_at BIGDATETIME DEFAULT GETUTCDATE() NOT NULL" + ")" + ) + create_index_sql = escape_sql_literal(f"CREATE INDEX {index_name} ON {table_ref}(expires_at)") + return ( + f"IF OBJECT_ID('{escaped_table_ref}', 'U') IS NULL EXECUTE('{create_table_sql}');\n" + f"IF NOT EXISTS (SELECT 1 FROM dbo.sysindexes WHERE name = '{escaped_index_bare}' " + f"AND id = OBJECT_ID('{escaped_table_ref}')) EXECUTE('{create_index_sql}');" + ) + + def _drop_table_sql(self) -> "list[str]": + """Get Sybase ASE DROP TABLE statements.""" + table_ref = _table_ref(self._table_name) + escaped = escape_sql_literal(table_ref) + return [f"IF OBJECT_ID('{escaped}', 'U') IS NOT NULL EXECUTE('DROP TABLE {escaped}');"] + + def _create_table(self) -> None: + with self._config.provide_session() as driver: + driver.execute_script(self._table_ddl()) + driver.commit() + self._log_table_created() + + def _get(self, key: str, renew_for: "int | timedelta | None" = None) -> "bytes | None": + table_ref = _table_ref(self._table_name) + sql = f""" + SELECT data, expires_at FROM {table_ref} + WHERE session_id = %s + AND (expires_at IS NULL OR expires_at > GETUTCDATE()) + """ + with self._config.provide_connection() as conn: + with SybaseCursor(conn) as cursor: + cursor.execute(sql, (key,)) + row = cursor.fetchone() + + if row is None: + return None + + expires_at = _normalize_utc(_row_value(row, "expires_at", 1)) + if renew_for is not None and expires_at is not None: + new_expires_at = self._calculate_expires_at(renew_for) + if new_expires_at is not None: + with SybaseCursor(conn) as update_cursor: + update_cursor.execute( + f""" + UPDATE {table_ref} + SET expires_at = %s, updated_at = GETUTCDATE() + WHERE session_id = %s + """, + (new_expires_at, key), + ) + conn.commit() + + return _coerce_bytes(_row_value(row, "data", 0)) + + def _set(self, key: str, value: "str | bytes", expires_in: "int | timedelta | None" = None) -> None: + data = self._value_to_bytes(value) + expires_at = self._calculate_expires_at(expires_in) + table_ref = _table_ref(self._table_name) + sql = f""" + IF EXISTS (SELECT 1 FROM {table_ref} HOLDLOCK WHERE session_id = %s) + UPDATE {table_ref} + SET data = %s, expires_at = %s, updated_at = GETUTCDATE() + WHERE session_id = %s + ELSE + INSERT INTO {table_ref} (session_id, data, expires_at) + VALUES (%s, %s, %s) + """ + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(sql, (key, data, expires_at, key, key, data, expires_at)) + conn.commit() + + def _delete(self, key: str) -> None: + table_ref = _table_ref(self._table_name) + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(f"DELETE FROM {table_ref} WHERE session_id = %s", (key,)) + conn.commit() + + def _delete_all(self) -> None: + table_ref = _table_ref(self._table_name) + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(f"TRUNCATE TABLE {table_ref}") + conn.commit() + self._log_delete_all() + + def _exists(self, key: str) -> bool: + table_ref = _table_ref(self._table_name) + sql = f""" + SELECT 1 + FROM {table_ref} + WHERE session_id = %s + AND (expires_at IS NULL OR expires_at > GETUTCDATE()) + """ + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(sql, (key,)) + return cursor.fetchone() is not None + + def _expires_in(self, key: str) -> "int | None": + table_ref = _table_ref(self._table_name) + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(f"SELECT expires_at FROM {table_ref} WHERE session_id = %s", (key,)) + row = cursor.fetchone() + + if row is None: + return None + expires_at = _normalize_utc(_row_value(row, "expires_at", 0)) + if expires_at is None: + return None + remaining = expires_at - datetime.now(UTC) + return max(0, int(remaining.total_seconds())) + + def _delete_expired(self) -> int: + table_ref = _table_ref(self._table_name) + sql = f""" + DELETE FROM {table_ref} + WHERE expires_at IS NOT NULL + AND expires_at < GETUTCDATE() + """ + with self._config.provide_connection() as conn, SybaseCursor(conn) as cursor: + cursor.execute(sql) + count = int(getattr(cursor, "rowcount", 0) or 0) + conn.commit() + if count > 0: + self._log_delete_expired(count) + return count + + +def _table_ref(table_name: str) -> str: + schema_name, bare_table_name = split_schema_and_table(table_name) + return f"{quote_bracket_identifier(schema_name)}.{quote_bracket_identifier(bare_table_name)}" + + +def _row_value(row: object, key: str, index: int) -> Any: + """Return a value from dict-like or sequence-like driver rows.""" + if isinstance(row, dict): + if key in row: + return row[key] + upper_key = key.upper() + if upper_key in row: + return row[upper_key] + return None + if isinstance(row, (list, tuple)) and len(row) > index: + return row[index] + return getattr(row, key, None) + + +def _normalize_utc(value: Any) -> "datetime | None": + if value is None: + return None + if not isinstance(value, datetime): + return None + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) + + +def _coerce_bytes(value: Any) -> bytes: + if value is None: + return b"" + if isinstance(value, bytes): + return value + if isinstance(value, bytearray): + return bytes(value) + if isinstance(value, memoryview): + return value.tobytes() + if isinstance(value, str): + return value.encode("utf-8") + return bytes(value) diff --git a/sqlspec/adapters/sybase/migrations.py b/sqlspec/adapters/sybase/migrations.py new file mode 100644 index 000000000..1e0a4e613 --- /dev/null +++ b/sqlspec/adapters/sybase/migrations.py @@ -0,0 +1,176 @@ +"""Sybase ASE-specific migration tracker.""" + +import logging +import os +from contextlib import suppress +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from collections.abc import Sequence + +from sqlspec.adapters.sybase.core import ( + escape_sql_literal, + format_identifier, + quote_bracket_identifier, + split_schema_and_table, +) +from sqlspec.builder import CreateTable, sql +from sqlspec.migrations.tracker import SyncMigrationTracker +from sqlspec.migrations.version import parse_version +from sqlspec.observability import resolve_db_system +from sqlspec.utils.logging import get_logger, log_with_context + +if TYPE_CHECKING: + from sqlspec.driver import SyncDriverAdapterBase + +__all__ = ("SybaseMigrationTrackerMixin", "SybaseSyncMigrationTracker") + +logger = get_logger("sqlspec.migrations.sybase") + + +class SybaseMigrationTrackerMixin: + """Sybase ASE-specific migration table DDL and schema maintenance.""" + + __slots__ = () + + version_table: str + + def _tracking_table_ddl(self) -> CreateTable: + """Return Sybase ASE-compatible migration tracking table DDL.""" + return ( + sql + .create_table(self.version_table) + .column("version_num", "VARCHAR(32)", primary_key=True) + .column("version_type", "VARCHAR(16)") + .column("execution_sequence", "INT") + .column("description", "TEXT") + .column("applied_at", "BIGDATETIME", default="GETUTCDATE()", not_null=True) + .column("execution_time_ms", "INT") + .column("checksum", "VARCHAR(64)") + .column("applied_by", "VARCHAR(255)") + .column("replaces", "TEXT") + ) + + def _idempotent_tracking_table_ddl_text(self) -> str: + """Wrap CREATE TABLE in a Sybase ASE OBJECT_ID existence probe with dynamic EXECUTE.""" + table_ref = escape_sql_literal(format_identifier(self.version_table)) + create_sql = escape_sql_literal(self._tracking_table_ddl_text().rstrip().rstrip(";")) + return f"IF OBJECT_ID('{table_ref}', 'U') IS NULL EXECUTE('{create_sql}');" + + def _tracking_table_ddl_text(self) -> str: + """Render CREATE TABLE text with Sybase ASE native column types.""" + column_lines: list[str] = [] + for column_def in self._tracking_table_ddl().columns: + default_clause = f" DEFAULT {column_def.default}" if column_def.default else "" + null_clause = " NOT NULL" if column_def.primary_key or column_def.not_null else " NULL" + primary_key_clause = " PRIMARY KEY" if column_def.primary_key else "" + column_lines.append( + f" {quote_bracket_identifier(column_def.name)} {column_def.dtype}{default_clause}{null_clause}{primary_key_clause}" + ) + return f"CREATE TABLE {format_identifier(self.version_table)} (\n" + ",\n".join(column_lines) + "\n)" + + def _existing_columns_query(self) -> str: + """Return Sybase ASE catalog query text for migration tracking table columns.""" + schema_name, table_name = split_schema_and_table(self.version_table) + return f""" + SELECT c.name AS column_name + FROM dbo.syscolumns c + INNER JOIN dbo.sysobjects o ON o.id = c.id AND o.type = 'U' + INNER JOIN dbo.sysusers u ON u.uid = o.uid + WHERE o.name = '{escape_sql_literal(table_name)}' + AND u.name = '{escape_sql_literal(schema_name)}' + """ + + def _add_column_statement_text(self, column_name: str) -> str | None: + """Return Sybase ASE ALTER TABLE text for a missing migration column.""" + target_create = self._tracking_table_ddl() + column_def = next((col for col in target_create.columns if col.name.lower() == column_name), None) + if column_def is None: + return None + default_clause = f" DEFAULT {column_def.default}" if column_def.default else "" + nullable_clause = " NOT NULL" if (column_def.not_null and column_def.default is not None) else " NULL" + return f"ALTER TABLE {format_identifier(self.version_table)} ADD {quote_bracket_identifier(column_def.name)} {column_def.dtype}{default_clause}{nullable_clause};" + + +class SybaseSyncMigrationTracker(SybaseMigrationTrackerMixin, SyncMigrationTracker): + """Sybase ASE sync migration tracker.""" + + def ensure_tracking_table(self, driver: "SyncDriverAdapterBase") -> None: + """Create the migration tracking table if it does not exist.""" + driver.execute_script(self._idempotent_tracking_table_ddl_text()) + self._commit_if_needed(driver) + self._migrate_schema_if_needed(driver) + + def record_migration( + self, + driver: "SyncDriverAdapterBase", + version: str, + description: str, + execution_time_ms: int, + checksum: str, + *, + commit: bool = True, + replaces: "Sequence[str] | None" = None, + ) -> None: + """Record a successfully applied migration with Sybase ASE-compatible metadata.""" + parsed_version = parse_version(version) + version_type = parsed_version.type.value + result = driver.execute(self._next_execution_sequence_query()) + next_sequence = result.get_data()[0]["next_seq"] if result.data else 1 + applied_by = os.environ.get("USER", "unknown") + if replaces: + statement = self._record_squashed_migration_statement( + version, + version_type, + next_sequence, + description, + execution_time_ms, + checksum, + applied_by, + ",".join(replaces), + ) + else: + statement = self._record_migration_statement( + version, version_type, next_sequence, description, execution_time_ms, checksum, applied_by + ) + driver.execute(statement) + if commit: + self._commit_if_needed(driver) + + def _commit_if_needed(self, driver: "SyncDriverAdapterBase") -> None: + if hasattr(driver, "driver_features"): + self._safe_commit(driver) + return + driver.commit() + + def _migrate_schema_if_needed(self, driver: "SyncDriverAdapterBase") -> None: + """Check and add missing tracking table columns through Sybase ASE catalog tables.""" + try: + rows = driver.select(self._existing_columns_query()) + existing_columns = {str(row["column_name"]).lower() for row in rows if row.get("column_name") is not None} + missing_columns = self._detect_missing_columns(existing_columns) + if not missing_columns: + return + for column_name in sorted(missing_columns): + self._add_column(driver, column_name) + self._commit_if_needed(driver) + except Exception as exc: + with suppress(Exception): + driver.rollback() + log_with_context( + logger, + logging.ERROR, + "migration.track", + db_system=resolve_db_system(type(driver).__name__), + table=self.version_table, + operation="schema_check", + status="failed", + error_type=type(exc).__name__, + ) + + def _add_column(self, driver: "SyncDriverAdapterBase", column_name: str) -> None: + """Add a single missing migration tracking column.""" + add_column_sql = self._add_column_statement_text(column_name) + if add_column_sql is None: + return + driver.execute_script(add_column_sql) diff --git a/sqlspec/adapters/sybase/pool.py b/sqlspec/adapters/sybase/pool.py new file mode 100644 index 000000000..fe16526f7 --- /dev/null +++ b/sqlspec/adapters/sybase/pool.py @@ -0,0 +1,275 @@ +"""Sybase ASE connection pool with thread-locked TDSVER='5.0' negotiation.""" + +import contextlib +import logging +import os +import threading +import time +from contextlib import contextmanager +from importlib import import_module +from typing import TYPE_CHECKING, Any, cast + +from sqlspec.adapters.sybase._typing import PYMSSQL_MODULE, SybaseConnection +from sqlspec.adapters.sybase.core import DEFAULT_SYBASE_CONN_PROPERTIES, DEFAULT_SYBASE_TDS_VERSION +from sqlspec.exceptions import ImproperConfigurationError, MissingDependencyError +from sqlspec.utils.logging import POOL_LOGGER_NAME, get_logger, log_with_context +from sqlspec.utils.uuids import uuid4 + +if TYPE_CHECKING: + from collections.abc import Callable, Generator + +__all__ = ("SybaseConnectionPool",) + +logger = get_logger(POOL_LOGGER_NAME) +_ADAPTER_NAME = "sybase" +pymssql = PYMSSQL_MODULE + + +def _load_pymssql_module() -> Any: + """Import ``pymssql`` lazily or raise ``MissingDependencyError``.""" + try: + return import_module("pymssql") + except ImportError as exc: + raise MissingDependencyError(package="pymssql", install_package="sybase") from exc + + +class SybaseConnectionPool: + """Thread-local connection manager for Sybase ASE over pymssql (FreeTDS TDS 5.0).""" + + __slots__ = ( + "_checked_out", + "_connection_parameters", + "_connection_registry", + "_connection_threads", + "_generation", + "_health_check_interval", + "_on_connection_create", + "_pool_id", + "_recycle_seconds", + "_registry_lock", + "_tds_version", + "_thread_local", + ) + + def __init__( + self, + connection_parameters: "dict[str, Any]", + recycle_seconds: int = 86400, + health_check_interval: float = 30.0, + on_connection_create: "Callable[[SybaseConnection], None] | None" = None, + ) -> None: + """Initialize the thread-local Sybase ASE connection manager. + + Args: + connection_parameters: Normalized Sybase connection parameters. + recycle_seconds: Connection recycle time in seconds (default 24h). + health_check_interval: Seconds of idle time before running health check. + on_connection_create: Callback executed when a connection is created. + """ + params = dict(connection_parameters) + freetds_alias = params.get("freetds_server") or params.get("server_alias") or params.get("is_server_alias") + if freetds_alias and isinstance(freetds_alias, str): + params["server"] = freetds_alias + tdsver = os.environ.get("TDSVER") + if not freetds_alias and tdsver != "5.0": + msg = ( + "Sybase ASE requires FreeTDS TDS 5.0 protocol negotiation. " + "Either configure a FreeTDS server section alias via 'server_alias' (or 'freetds_server'), " + "or set the environment variable TDSVER=5.0 before process startup." + ) + raise ImproperConfigurationError(msg) + self._tds_version = str( + params.pop("_sybase_tds_version", params.pop("tds_version", DEFAULT_SYBASE_TDS_VERSION)) + ) + params.setdefault("conn_properties", DEFAULT_SYBASE_CONN_PROPERTIES) + params.setdefault("autocommit", True) + self._connection_parameters = params + self._thread_local = threading.local() + self._connection_registry: set[SybaseConnection] = set() + self._connection_threads: dict[SybaseConnection, threading.Thread] = {} + self._checked_out = 0 + self._generation = 0 + self._registry_lock = threading.Lock() + self._recycle_seconds = recycle_seconds + self._health_check_interval = health_check_interval + self._on_connection_create = on_connection_create + self._pool_id = str(uuid4())[:8] + + @property + def _database_name(self) -> str: + """Get sanitized database name for logging.""" + return str(self._connection_parameters.get("database", "unknown")) + + def new_connection(self) -> SybaseConnection: + """Open a standalone Sybase ASE connection using FreeTDS TDS 5.0 negotiation. + + Returns: + SybaseConnection: A newly opened, fully configured connection. + """ + module = pymssql + if module is None: + module = _load_pymssql_module() + connection = module.connect(**self._connection_parameters) + + if self._on_connection_create is not None: + self._on_connection_create(connection) + + return cast("SybaseConnection", connection) + + @contextmanager + def get_connection(self) -> "Generator[SybaseConnection, None, None]": + """Get a thread-local connection. + + Yields: + A thread-local database connection. + """ + connection = self.acquire() + try: + yield connection + except Exception: + with contextlib.suppress(Exception): + self._close_thread_connection() + raise + finally: + self.release(connection) + + def close(self) -> None: + """Close every connection this pool opened, on any thread.""" + self._close_thread_connection() + with self._registry_lock: + orphaned = list(self._connection_registry) + self._connection_registry.clear() + self._connection_threads.clear() + self._checked_out = 0 + self._generation += 1 + for connection in orphaned: + with contextlib.suppress(Exception): + connection.close() + + def acquire(self) -> SybaseConnection: + connection = self._get_thread_connection() + with self._registry_lock: + self._checked_out += 1 + return connection + + def release(self, connection: SybaseConnection) -> None: + _ = connection + with self._registry_lock: + if self._checked_out > 0: + self._checked_out -= 1 + + def size(self) -> int: + self._prune_dead_thread_connections() + with self._registry_lock: + return len(self._connection_registry) + + def checked_out(self) -> int: + with self._registry_lock: + return self._checked_out + + def _prune_dead_thread_connections(self) -> None: + dead_connections: list[SybaseConnection] = [] + with self._registry_lock: + for conn, owner_thread in list(self._connection_threads.items()): + if not owner_thread.is_alive(): + self._connection_registry.discard(conn) + self._connection_threads.pop(conn, None) + dead_connections.append(conn) + for conn in dead_connections: + with contextlib.suppress(Exception): + conn.close() + + def _create_connection(self) -> SybaseConnection: + self._prune_dead_thread_connections() + connection = self.new_connection() + + with self._registry_lock: + self._connection_registry.add(connection) + self._connection_threads[connection] = threading.current_thread() + + return connection + + def _is_connection_alive(self, connection: SybaseConnection) -> bool: + try: + cursor = connection.cursor() + try: + cursor.execute("SELECT 1") + cursor.fetchone() + finally: + cursor.close() + except Exception: + return False + return True + + def _get_thread_connection(self) -> SybaseConnection: + thread_state = self._thread_local.__dict__ + if thread_state.get("generation") != self._generation: + stale = thread_state.pop("connection", None) + if stale is not None: + self._retire_connection(cast("SybaseConnection", stale)) + thread_state.pop("created_at", None) + thread_state.pop("last_used", None) + self._thread_local.generation = self._generation + if "connection" not in thread_state: + self._thread_local.connection = self._create_connection() + self._thread_local.created_at = time.time() + self._thread_local.last_used = time.time() + return cast("SybaseConnection", self._thread_local.connection) + + if self._recycle_seconds > 0 and time.time() - self._thread_local.created_at > self._recycle_seconds: + log_with_context( + logger, + logging.DEBUG, + "pool.connection.recycle", + adapter=_ADAPTER_NAME, + pool_id=self._pool_id, + database=self._database_name, + recycle_seconds=self._recycle_seconds, + reason="exceeded_recycle_time", + ) + self._retire_connection(cast("SybaseConnection", self._thread_local.connection)) + self._thread_local.connection = self._create_connection() + self._thread_local.created_at = time.time() + self._thread_local.last_used = time.time() + return cast("SybaseConnection", self._thread_local.connection) + + idle_time = time.time() - thread_state.get("last_used", 0) + if ( + self._health_check_interval > 0 + and idle_time > self._health_check_interval + and not self._is_connection_alive(self._thread_local.connection) + ): + log_with_context( + logger, + logging.DEBUG, + "pool.connection.recycle", + adapter=_ADAPTER_NAME, + pool_id=self._pool_id, + database=self._database_name, + idle_seconds=round(idle_time, 1), + reason="failed_health_check", + ) + self._retire_connection(cast("SybaseConnection", self._thread_local.connection)) + self._thread_local.connection = self._create_connection() + self._thread_local.created_at = time.time() + + self._thread_local.last_used = time.time() + return cast("SybaseConnection", self._thread_local.connection) + + def _retire_connection(self, connection: SybaseConnection) -> None: + """Close a pool-owned connection and drop it from the shutdown registry.""" + with self._registry_lock: + self._connection_registry.discard(connection) + self._connection_threads.pop(connection, None) + with contextlib.suppress(Exception): + connection.close() + + def _close_thread_connection(self) -> None: + thread_state = self._thread_local.__dict__ + if "connection" in thread_state: + self._retire_connection(cast("SybaseConnection", self._thread_local.connection)) + del self._thread_local.connection + if "created_at" in thread_state: + del self._thread_local.created_at + if "last_used" in thread_state: + del self._thread_local.last_used diff --git a/sqlspec/builder/_merge.py b/sqlspec/builder/_merge.py index 6775de65e..b34279d2c 100644 --- a/sqlspec/builder/_merge.py +++ b/sqlspec/builder/_merge.py @@ -33,7 +33,7 @@ __all__ = ("Merge",) -MERGE_UNSUPPORTED_DIALECTS = frozenset({"mysql", "sqlite", "duckdb"}) +MERGE_UNSUPPORTED_DIALECTS = frozenset({"mysql", "sqlite", "duckdb", "sybase"}) _MERGE_SQL_KEYWORD_LITERALS = frozenset({"CURRENT_TIMESTAMP", "CURRENT_DATE", "CURRENT_TIME"}) _MERGE_ARITHMETIC_NODES = (exp.Add, exp.Sub, exp.Mul, exp.Div, exp.Mod, exp.Paren) _POSTGRES_TYPE_DISPATCHER = TypeDispatcher[str]() diff --git a/sqlspec/builder/_parsing_utils.py b/sqlspec/builder/_parsing_utils.py index f4a64750f..c53c9af6a 100644 --- a/sqlspec/builder/_parsing_utils.py +++ b/sqlspec/builder/_parsing_utils.py @@ -5,11 +5,13 @@ """ import contextlib +import importlib import re from collections.abc import Callable from typing import TYPE_CHECKING, Any, Final from sqlglot import exp, maybe_parse +from sqlglot.dialects.dialect import Dialect from sqlspec.builder._column import Column from sqlspec.builder._expression_wrappers import ExpressionWrapper @@ -144,18 +146,19 @@ def parse_table_expression( BigQuery's ``\`project.dataset.table\``` are parsed into their qualified parts instead of a single literal name. """ + resolved_dialect: DialectType | None = _normalize_dialect(dialect) if isinstance(dialect, str) else dialect if explicit_alias is None and " " in table_input.strip(): parts = table_input.strip().split(None, 1) if len(parts) == ALIAS_PARTS_EXPECTED_COUNT: base_table, alias = parts if _is_simple_identifier(base_table): - return exp.to_table(base_table, alias=alias, dialect=dialect) + return exp.to_table(base_table, alias=alias, dialect=resolved_dialect) if _is_simple_identifier(table_input): - return exp.to_table(table_input, alias=explicit_alias, dialect=dialect) + return exp.to_table(table_input, alias=explicit_alias, dialect=resolved_dialect) with contextlib.suppress(Exception): - parsed: exp.Expr | None = exp.maybe_parse(f"SELECT * FROM {table_input}", dialect=dialect) + parsed: exp.Expr | None = exp.maybe_parse(f"SELECT * FROM {table_input}", dialect=resolved_dialect) if isinstance(parsed, exp.Select): from_clause = parsed.find(exp.From) if from_clause is not None: @@ -166,7 +169,9 @@ def parse_table_expression( return exp.alias_(table_expr, explicit_alias) return table_expr # type: ignore[no-any-return] - return _mark_explicit_table_quotes(exp.to_table(table_input, alias=explicit_alias, dialect=dialect), table_input) + return _mark_explicit_table_quotes( + exp.to_table(table_input, alias=explicit_alias, dialect=resolved_dialect), table_input + ) def parse_order_expression(order_input: str | exp.Expr) -> exp.Expr: @@ -437,7 +442,17 @@ def _normalize_dialect(dialect: "DialectType | None") -> str | None: name = dialect.__name__.lower() else: name = type(dialect).__name__.lower() - return {"mssql": "tsql", "mariadb": "mysql", "cockroachdb": "postgres", "postgresql": "postgres"}.get(name, name) + resolved = { + "mssql": "tsql", + "mariadb": "mysql", + "cockroachdb": "postgres", + "postgresql": "postgres", + "ase": "sybase", + "sa": "sqlanywhere", + }.get(name, name) + if resolved in {"sybase", "sqlanywhere"} and resolved not in Dialect.classes: + importlib.import_module("sqlspec.dialects.sybase") + return resolved def _merge_sql_parameters(sql_obj: Any, builder: Any) -> dict[str, str]: diff --git a/sqlspec/data_dictionary/_registry.py b/sqlspec/data_dictionary/_registry.py index 74ba43e60..f5ee61c04 100644 --- a/sqlspec/data_dictionary/_registry.py +++ b/sqlspec/data_dictionary/_registry.py @@ -24,6 +24,8 @@ "cockroach": "cockroachdb", "tsql": "mssql", "sqlserver": "mssql", + "ase": "sybase", + "sa": "sqlanywhere", } DIALECT_MODE_ALIASES: dict[tuple[str, str], str] = { diff --git a/sqlspec/data_dictionary/_types.py b/sqlspec/data_dictionary/_types.py index fa022e5b7..57eabc489 100644 --- a/sqlspec/data_dictionary/_types.py +++ b/sqlspec/data_dictionary/_types.py @@ -1470,6 +1470,8 @@ class FeatureFlags(TypedDict, total=False): """Whether native JSON or JSONB column types and operators are supported.""" supports_maps: bool """Whether native key-value ``MAP`` column types are supported.""" + supports_merge: bool + """Whether SQL MERGE statements are supported.""" supports_on_conflict: bool """Whether ``INSERT ... ON CONFLICT`` conflict resolution syntax is supported.""" supports_partitioning: bool @@ -1492,6 +1494,8 @@ class FeatureFlags(TypedDict, total=False): """Whether ``FOR UPDATE SKIP LOCKED`` non-blocking queue scans are supported.""" supports_structs: bool """Whether composite ``STRUCT`` / ``ROW`` column types are supported.""" + supports_transactional_ddl: bool + """Whether schema DDL statements can be executed within a transaction.""" supports_transactions: bool """Whether multi-statement ACID transactions are supported.""" supports_update_from: bool @@ -1525,6 +1529,8 @@ class FeatureVersions(TypedDict, total=False): """Minimum engine version supporting native JSON types and functions.""" supports_jsonb: "VersionInfo" """Minimum engine version supporting binary JSONB storage.""" + supports_merge: "VersionInfo" + """Minimum engine version supporting SQL MERGE statements.""" supports_partitioning: "VersionInfo" """Minimum engine version supporting declarative table partitioning.""" supports_resource_groups: "VersionInfo" diff --git a/sqlspec/data_dictionary/dialects/__init__.py b/sqlspec/data_dictionary/dialects/__init__.py index 2117887c8..86f3135e6 100644 --- a/sqlspec/data_dictionary/dialects/__init__.py +++ b/sqlspec/data_dictionary/dialects/__init__.py @@ -9,7 +9,9 @@ from sqlspec.data_dictionary.dialects.oracle import ORACLE_CONFIG from sqlspec.data_dictionary.dialects.postgres import POSTGRES_CONFIG from sqlspec.data_dictionary.dialects.spanner import SPANNER_CONFIG +from sqlspec.data_dictionary.dialects.sqlanywhere import SQLANYWHERE_CONFIG from sqlspec.data_dictionary.dialects.sqlite import SQLITE_CONFIG +from sqlspec.data_dictionary.dialects.sybase import SYBASE_CONFIG __all__ = ( "BIGQUERY_CONFIG", @@ -22,5 +24,7 @@ "ORACLE_CONFIG", "POSTGRES_CONFIG", "SPANNER_CONFIG", + "SQLANYWHERE_CONFIG", "SQLITE_CONFIG", + "SYBASE_CONFIG", ) diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/__init__.py b/sqlspec/data_dictionary/dialects/sqlanywhere/__init__.py new file mode 100644 index 000000000..3356a932c --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/__init__.py @@ -0,0 +1,31 @@ +"""SAP SQL Anywhere dialect static configuration and reflection helpers.""" + +from sqlspec.data_dictionary.dialects.sqlanywhere.config import ( + SQLANYWHERE_CONFIG, + SQLANYWHERE_FEATURE_FLAGS, + SQLANYWHERE_FEATURE_VERSIONS, + SQLANYWHERE_METADATA_DOMAINS, + SQLANYWHERE_TYPE_MAPPINGS, + SQLANYWHERE_VERSION_PATTERN, + SqlanywhereVersionInfo, + build_sqlanywhere_metadata_capability_profile, + extract_sqlanywhere_version_value, + list_sqlanywhere_available_features, + merge_sqlanywhere_table_lists, + resolve_sqlanywhere_feature_flag, +) + +__all__ = ( + "SQLANYWHERE_CONFIG", + "SQLANYWHERE_FEATURE_FLAGS", + "SQLANYWHERE_FEATURE_VERSIONS", + "SQLANYWHERE_METADATA_DOMAINS", + "SQLANYWHERE_TYPE_MAPPINGS", + "SQLANYWHERE_VERSION_PATTERN", + "SqlanywhereVersionInfo", + "build_sqlanywhere_metadata_capability_profile", + "extract_sqlanywhere_version_value", + "list_sqlanywhere_available_features", + "merge_sqlanywhere_table_lists", + "resolve_sqlanywhere_feature_flag", +) diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/config.py b/sqlspec/data_dictionary/dialects/sqlanywhere/config.py new file mode 100644 index 000000000..51246d50a --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/config.py @@ -0,0 +1,191 @@ +"""SAP SQL Anywhere dialect static configuration and reflection helpers.""" + +import re +from collections.abc import Mapping, Sequence +from typing import TYPE_CHECKING, Final + +from sqlspec.data_dictionary import register_dialect +from sqlspec.data_dictionary._types import ( + DialectConfig, + FeatureFlags, + FeatureVersions, + MetadataCapability, + MetadataCapabilityProfile, + MetadataFidelity, + MetadataSource, + MetadataSupport, + VersionInfo, +) + +if TYPE_CHECKING: + from sqlspec.data_dictionary import TableMetadata + +__all__ = ( + "SQLANYWHERE_CONFIG", + "SQLANYWHERE_FEATURE_FLAGS", + "SQLANYWHERE_FEATURE_VERSIONS", + "SQLANYWHERE_METADATA_DOMAINS", + "SQLANYWHERE_TYPE_MAPPINGS", + "SQLANYWHERE_VERSION_PATTERN", + "SqlanywhereVersionInfo", + "build_sqlanywhere_metadata_capability_profile", + "extract_sqlanywhere_version_value", + "list_sqlanywhere_available_features", + "merge_sqlanywhere_table_lists", + "resolve_sqlanywhere_feature_flag", +) + +SQLANYWHERE_VERSION_PATTERN = re.compile(r"(\d+)\.(\d+)\.(\d+)") + +SQLANYWHERE_FEATURE_VERSIONS: FeatureVersions = { + "supports_cte": VersionInfo(10, 0, 0), + "supports_window_functions": VersionInfo(10, 0, 0), + "supports_merge": VersionInfo(11, 0, 0), +} + +SQLANYWHERE_FEATURE_FLAGS: FeatureFlags = { + "supports_clustering": False, + "supports_returning": False, + "supports_upsert": True, + "supports_on_conflict": False, + "supports_window_functions": True, + "supports_cte": True, + "supports_json": False, + "supports_arrays": False, + "supports_uuid": True, + "supports_schemas": True, + "supports_merge": True, + "supports_for_update": True, + "supports_skip_locked": True, + "supports_transactional_ddl": True, + "supports_transactions": True, + "supports_prepared_statements": True, + "supports_in_memory": False, + "supports_update_from": True, + "supports_sequences": True, +} + +SQLANYWHERE_TYPE_MAPPINGS: dict[str, str] = { + "uuid": "UNIQUEIDENTIFIER", + "boolean": "BIT", + "timestamp": "TIMESTAMP", + "datetime": "TIMESTAMP", + "date": "DATE", + "time": "TIME", + "text": "LONG VARCHAR", + "blob": "LONG BINARY", + "bytes": "VARBINARY(32767)", + "json": "LONG VARCHAR", + "decimal": "DECIMAL", + "float": "DOUBLE", + "integer": "INTEGER", + "bigint": "BIGINT", + "smallint": "SMALLINT", + "string": "VARCHAR(255)", + "varchar": "VARCHAR(255)", + "AutoIncrement": "INTEGER DEFAULT AUTOINCREMENT", +} + +SQLANYWHERE_CONFIG = DialectConfig( + name="sqlanywhere", + feature_versions=SQLANYWHERE_FEATURE_VERSIONS, + feature_flags=SQLANYWHERE_FEATURE_FLAGS, + type_mappings=SQLANYWHERE_TYPE_MAPPINGS, + version_pattern=SQLANYWHERE_VERSION_PATTERN, + default_schema="DBA", + parameter_style="qmark", +) + +register_dialect(SQLANYWHERE_CONFIG) + +SQLANYWHERE_METADATA_DOMAINS: Final[tuple[str, ...]] = ( + "schemas", + "objects", + "tables", + "columns", + "constraints", + "indexes", + "foreign_keys", + "views", + "version", +) + + +class SqlanywhereVersionInfo(VersionInfo): + """SAP SQL Anywhere database version info with optional build number.""" + + __slots__ = ("build_number",) + + def __init__(self, major: int, minor: int = 0, patch: int = 0, build_number: str | None = None) -> None: + """Initialize SAP SQL Anywhere version info.""" + super().__init__(major, minor, patch) + self.build_number = build_number + + def __str__(self) -> str: + """Format version string including build number when present.""" + base = f"{self.major}.{self.minor}.{self.patch}" + return f"{base} ({self.build_number})" if self.build_number else base + + +def extract_sqlanywhere_version_value(row: object) -> str | None: + """Extract a SAP SQL Anywhere version string from a row-like object.""" + if isinstance(row, Mapping): + for key in ("version_string", "VERSION_STRING", "version", "VERSION"): + value = row.get(key) + if value: + return str(value) + if isinstance(row, (list, tuple)) and row: + return str(row[0]) + if row is not None: + return str(row) + return None + + +def resolve_sqlanywhere_feature_flag(feature: str, version_info: VersionInfo | None) -> bool: + """Resolve a SAP SQL Anywhere feature flag using config and version details.""" + flag = SQLANYWHERE_CONFIG.get_feature_flag(feature) + if flag is not None: + return bool(flag) + required_version = SQLANYWHERE_CONFIG.get_feature_version(feature) + if required_version is None or version_info is None: + return False + return bool(version_info >= required_version) + + +def list_sqlanywhere_available_features() -> list[str]: + """List static and versioned SAP SQL Anywhere data-dictionary feature flags.""" + features = set(SQLANYWHERE_CONFIG.feature_flags.keys()) + features.update(SQLANYWHERE_CONFIG.feature_versions.keys()) + return sorted(features) + + +def merge_sqlanywhere_table_lists( + ordered: list["TableMetadata"], all_tables: list["TableMetadata"] +) -> list["TableMetadata"]: + """Merge dependency-ordered SAP SQL Anywhere tables with catalog remainder rows.""" + if not ordered: + return sorted(all_tables, key=lambda item: item.get("table_name") or "") + seen = {(item.get("schema_name"), item.get("table_name")) for item in ordered if item.get("table_name")} + remainder = [item for item in all_tables if (item.get("schema_name"), item.get("table_name")) not in seen] + return ordered + remainder + + +def build_sqlanywhere_metadata_capability_profile( + adapter: str | None, domains: Sequence[str] | None = None +) -> MetadataCapabilityProfile: + """Build SAP SQL Anywhere data-dictionary capability metadata.""" + requested_domains = tuple(domains) if domains is not None else SQLANYWHERE_METADATA_DOMAINS + capabilities: list[MetadataCapability] = [] + for domain in requested_domains: + if domain in SQLANYWHERE_METADATA_DOMAINS: + capabilities.append( + MetadataCapability( + domain=domain, + support=MetadataSupport.SUPPORTED, + fidelity=MetadataFidelity.NATIVE, + source=MetadataSource.CATALOG, + ) + ) + else: + capabilities.append(MetadataCapability.unsupported(domain)) + return MetadataCapabilityProfile("sqlanywhere", adapter=adapter, capabilities=tuple(capabilities)) diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/columns.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/columns.sql new file mode 100644 index 000000000..6c8602cc1 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/columns.sql @@ -0,0 +1,97 @@ +-- name: by_schema +-- dialect: sqlanywhere +WITH pk_cols AS ( + SELECT DISTINCT + i.table_id, + ic.column_id + FROM SYS.SYSIDX i + JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id + WHERE i.index_category = 1 +), +unique_cols AS ( + SELECT DISTINCT + i.table_id, + ic.column_id + FROM SYS.SYSIDX i + JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id + WHERE i."unique" IN (1, 2) +) +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + c.column_name AS column_name, + d.domain_name AS data_type, + CASE c.nulls WHEN 'Y' THEN 'YES' ELSE 'NO' END AS is_nullable, + c."default" AS column_default, + c.column_id AS ordinal_position, + c.width AS char_length, + c.width AS max_length, + c.width AS numeric_precision, + c.scale AS numeric_scale, + CASE WHEN pk.column_id IS NOT NULL THEN 1 ELSE 0 END AS is_primary, + CASE WHEN uq.column_id IS NOT NULL THEN 1 ELSE 0 END AS is_unique, + CASE WHEN c."default" LIKE '%autoincrement%' THEN 1 ELSE 0 END AS is_identity, + CASE WHEN c."default" LIKE '%autoincrement%' THEN 'ALWAYS' ELSE NULL END AS identity_generation, + CASE WHEN c.column_type = 'C' THEN 1 ELSE 0 END AS is_generated, + CASE WHEN c.column_type = 'C' THEN c."default" ELSE NULL END AS generation_expression, + 0 AS is_hidden, + NULL AS column_comment +FROM SYS.SYSTABCOL c +JOIN SYS.SYSTAB t ON c.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +JOIN SYS.SYSDOMAIN d ON c.domain_id = d.domain_id +LEFT JOIN pk_cols pk ON pk.table_id = t.table_id AND pk.column_id = c.column_id +LEFT JOIN unique_cols uq ON uq.table_id = t.table_id AND uq.column_id = c.column_id +WHERE t.table_type IN (1, 21) + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) +ORDER BY u.user_name, t.table_name, c.column_id; + +-- name: by_table +-- dialect: sqlanywhere +WITH pk_cols AS ( + SELECT DISTINCT + i.table_id, + ic.column_id + FROM SYS.SYSIDX i + JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id + WHERE i.index_category = 1 +), +unique_cols AS ( + SELECT DISTINCT + i.table_id, + ic.column_id + FROM SYS.SYSIDX i + JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id + WHERE i."unique" IN (1, 2) +) +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + c.column_name AS column_name, + d.domain_name AS data_type, + CASE c.nulls WHEN 'Y' THEN 'YES' ELSE 'NO' END AS is_nullable, + c."default" AS column_default, + c.column_id AS ordinal_position, + c.width AS char_length, + c.width AS max_length, + c.width AS numeric_precision, + c.scale AS numeric_scale, + CASE WHEN pk.column_id IS NOT NULL THEN 1 ELSE 0 END AS is_primary, + CASE WHEN uq.column_id IS NOT NULL THEN 1 ELSE 0 END AS is_unique, + CASE WHEN c."default" LIKE '%autoincrement%' THEN 1 ELSE 0 END AS is_identity, + CASE WHEN c."default" LIKE '%autoincrement%' THEN 'ALWAYS' ELSE NULL END AS identity_generation, + CASE WHEN c.column_type = 'C' THEN 1 ELSE 0 END AS is_generated, + CASE WHEN c.column_type = 'C' THEN c."default" ELSE NULL END AS generation_expression, + 0 AS is_hidden, + NULL AS column_comment +FROM SYS.SYSTABCOL c +JOIN SYS.SYSTAB t ON c.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +JOIN SYS.SYSDOMAIN d ON c.domain_id = d.domain_id +LEFT JOIN pk_cols pk ON pk.table_id = t.table_id AND pk.column_id = c.column_id +LEFT JOIN unique_cols uq ON uq.table_id = t.table_id AND uq.column_id = c.column_id +WHERE t.table_type IN (1, 21) + AND t.table_name = :table_name + AND u.user_name = COALESCE(:schema_name, CURRENT USER) +ORDER BY c.column_id; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/constraints.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/constraints.sql new file mode 100644 index 000000000..f33deff1c --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/constraints.sql @@ -0,0 +1,43 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + i.index_name AS constraint_name, + CASE i.index_category + WHEN 1 THEN 'PRIMARY KEY' + WHEN 2 THEN 'FOREIGN KEY' + ELSE 'UNIQUE' + END AS constraint_type, + NULL AS check_clause, + 'Y' AS is_enforced +FROM SYS.SYSIDX i +JOIN SYS.SYSTAB t ON i.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +WHERE t.table_type = 1 + AND (i.index_category IN (1, 2) OR i."unique" IN (1, 2)) + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) +ORDER BY u.user_name, t.table_name, i.index_name; + +-- name: by_table +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + i.index_name AS constraint_name, + CASE i.index_category + WHEN 1 THEN 'PRIMARY KEY' + WHEN 2 THEN 'FOREIGN KEY' + ELSE 'UNIQUE' + END AS constraint_type, + NULL AS check_clause, + 'Y' AS is_enforced +FROM SYS.SYSIDX i +JOIN SYS.SYSTAB t ON i.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +WHERE t.table_type = 1 + AND (i.index_category IN (1, 2) OR i."unique" IN (1, 2)) + AND t.table_name = :table_name + AND u.user_name = COALESCE(:schema_name, CURRENT USER) +ORDER BY i.index_name; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/foreign_keys.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/foreign_keys.sql new file mode 100644 index 000000000..22d98b4a7 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/foreign_keys.sql @@ -0,0 +1,45 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + fu.user_name AS schema_name, + ft.table_name AS table_name, + fi.index_name AS constraint_name, + fc.column_name AS column_name, + pu.user_name AS referenced_schema, + pt.table_name AS referenced_table, + pc.column_name AS referenced_column +FROM SYS.SYSFKEY fk +JOIN SYS.SYSTAB ft ON fk.foreign_table_id = ft.table_id +JOIN SYS.SYSUSER fu ON ft.creator = fu.user_id +JOIN SYS.SYSIDX fi ON fk.foreign_table_id = fi.table_id AND fk.foreign_index_id = fi.index_id +JOIN SYS.SYSIDXCOL fic ON fi.table_id = fic.table_id AND fi.index_id = fic.index_id +JOIN SYS.SYSTABCOL fc ON fic.table_id = fc.table_id AND fic.column_id = fc.column_id +JOIN SYS.SYSTAB pt ON fk.primary_table_id = pt.table_id +JOIN SYS.SYSUSER pu ON pt.creator = pu.user_id +JOIN SYS.SYSTABCOL pc ON pt.table_id = pc.table_id AND fic.primary_column_id = pc.column_id +WHERE fu.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR ft.table_name = :table_name) +ORDER BY fu.user_name, ft.table_name, fi.index_name, fic.sequence; + +-- name: by_table +-- dialect: sqlanywhere +SELECT + fu.user_name AS schema_name, + ft.table_name AS table_name, + fi.index_name AS constraint_name, + fc.column_name AS column_name, + pu.user_name AS referenced_schema, + pt.table_name AS referenced_table, + pc.column_name AS referenced_column +FROM SYS.SYSFKEY fk +JOIN SYS.SYSTAB ft ON fk.foreign_table_id = ft.table_id +JOIN SYS.SYSUSER fu ON ft.creator = fu.user_id +JOIN SYS.SYSIDX fi ON fk.foreign_table_id = fi.table_id AND fk.foreign_index_id = fi.index_id +JOIN SYS.SYSIDXCOL fic ON fi.table_id = fic.table_id AND fi.index_id = fic.index_id +JOIN SYS.SYSTABCOL fc ON fic.table_id = fc.table_id AND fic.column_id = fc.column_id +JOIN SYS.SYSTAB pt ON fk.primary_table_id = pt.table_id +JOIN SYS.SYSUSER pu ON pt.creator = pu.user_id +JOIN SYS.SYSTABCOL pc ON pt.table_id = pc.table_id AND fic.primary_column_id = pc.column_id +WHERE ft.table_name = :table_name + AND fu.user_name = COALESCE(:schema_name, CURRENT USER) +ORDER BY fi.index_name, fic.sequence; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/indexes.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/indexes.sql new file mode 100644 index 000000000..bfa60c3cb --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/indexes.sql @@ -0,0 +1,39 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + i.index_name AS index_name, + c.column_name AS column_name, + ic.sequence + 1 AS column_position, + CASE WHEN i."unique" IN (1, 2) THEN 1 ELSE 0 END AS is_unique, + CASE WHEN i.index_category = 1 THEN 1 ELSE 0 END AS is_primary +FROM SYS.SYSIDX i +JOIN SYS.SYSTAB t ON i.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id +JOIN SYS.SYSTABCOL c ON ic.table_id = c.table_id AND ic.column_id = c.column_id +WHERE t.table_type = 1 + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) +ORDER BY u.user_name, t.table_name, i.index_name, ic.sequence; + +-- name: by_table +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + i.index_name AS index_name, + c.column_name AS column_name, + ic.sequence + 1 AS column_position, + CASE WHEN i."unique" IN (1, 2) THEN 1 ELSE 0 END AS is_unique, + CASE WHEN i.index_category = 1 THEN 1 ELSE 0 END AS is_primary +FROM SYS.SYSIDX i +JOIN SYS.SYSTAB t ON i.table_id = t.table_id +JOIN SYS.SYSUSER u ON t.creator = u.user_id +JOIN SYS.SYSIDXCOL ic ON i.table_id = ic.table_id AND i.index_id = ic.index_id +JOIN SYS.SYSTABCOL c ON ic.table_id = c.table_id AND ic.column_id = c.column_id +WHERE t.table_type = 1 + AND t.table_name = :table_name + AND u.user_name = COALESCE(:schema_name, CURRENT USER) +ORDER BY i.index_name, ic.sequence; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/objects.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/objects.sql new file mode 100644 index 000000000..eba975706 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/objects.sql @@ -0,0 +1,18 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS object_name, + CASE t.table_type + WHEN 1 THEN 'TABLE' + WHEN 21 THEN 'VIEW' + ELSE 'OBJECT' + END AS object_type, + NULL AS created, + NULL AS remarks +FROM SYS.SYSTAB t +JOIN SYS.SYSUSER u ON t.creator = u.user_id +WHERE t.table_type IN (1, 21) + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:object_name IS NULL OR t.table_name = :object_name) +ORDER BY u.user_name, object_type, t.table_name; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/schemas.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/schemas.sql new file mode 100644 index 000000000..8feff6bbc --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/schemas.sql @@ -0,0 +1,15 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + u.user_name AS owner_name, + NULL AS create_time, + NULL AS comment +FROM SYS.SYSUSER u +WHERE u.user_name NOT IN ('SYS', 'PUBLIC', 'dbo', 'diagnostics', 'rs_systabgroup', 'SA_DEBUG', 'SYS_SPATIAL_ADMIN_ROLE') + AND (:schema_name IS NULL OR u.user_name = :schema_name) +ORDER BY u.user_name; + +-- name: current +-- dialect: sqlanywhere +SELECT CURRENT USER AS schema_name; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/tables.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/tables.sql new file mode 100644 index 000000000..c683d6bd0 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/tables.sql @@ -0,0 +1,86 @@ +-- name: by_schema +-- dialect: sqlanywhere +WITH RECURSIVE dependency_tree (table_id, schema_name, table_name, dependency_level) AS ( + SELECT + t.table_id, + u.user_name AS schema_name, + t.table_name AS table_name, + 0 AS dependency_level + FROM SYS.SYSTAB t + JOIN SYS.SYSUSER u ON t.creator = u.user_id + WHERE t.table_type = 1 + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) + AND NOT EXISTS ( + SELECT 1 + FROM SYS.SYSFKEY fk + WHERE fk.foreign_table_id = t.table_id + AND fk.primary_table_id <> t.table_id + ) + UNION ALL + SELECT + child.table_id, + u.user_name AS schema_name, + child.table_name AS table_name, + parent.dependency_level + 1 AS dependency_level + FROM SYS.SYSFKEY fk + JOIN SYS.SYSTAB child ON child.table_id = fk.foreign_table_id + JOIN SYS.SYSUSER u ON child.creator = u.user_id + JOIN dependency_tree parent ON parent.table_id = fk.primary_table_id + WHERE child.table_type = 1 + AND fk.primary_table_id <> fk.foreign_table_id + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR child.table_name = :table_name) + AND parent.dependency_level < 20 +), +dependency_levels AS ( + SELECT + table_id, + MAX(dependency_level) AS dependency_level + FROM dependency_tree + GROUP BY table_id +) +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + 'BASE TABLE' AS table_type, + COALESCE(dl.dependency_level, 0) AS dependency_level, + COALESCE(dl.dependency_level, 0) AS level +FROM SYS.SYSTAB t +JOIN SYS.SYSUSER u ON t.creator = u.user_id +LEFT JOIN dependency_levels dl ON dl.table_id = t.table_id +WHERE t.table_type = 1 + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) +ORDER BY dependency_level, u.user_name, t.table_name; + +-- name: all_by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS table_name, + CASE t.table_type + WHEN 1 THEN 'BASE TABLE' + WHEN 21 THEN 'VIEW' + ELSE 'TABLE' + END AS table_type, + 0 AS dependency_level, + 0 AS level, + t.count AS estimated_row_count, + NULL AS table_comment +FROM SYS.SYSTAB t +JOIN SYS.SYSUSER u ON t.creator = u.user_id +WHERE t.table_type IN (1, 21) + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:table_name IS NULL OR t.table_name = :table_name) +ORDER BY u.user_name, t.table_name; + +-- name: names_by_schema +-- dialect: sqlanywhere +SELECT + t.table_name AS table_name +FROM SYS.SYSTAB t +JOIN SYS.SYSUSER u ON t.creator = u.user_id +WHERE t.table_type = 1 + AND u.user_name = COALESCE(:schema_name, CURRENT USER) +ORDER BY t.table_name; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/version.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/version.sql new file mode 100644 index 000000000..225d43d69 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/version.sql @@ -0,0 +1,3 @@ +-- name: current +-- dialect: sqlanywhere +SELECT PROPERTY('ProductVersion') AS version_string; diff --git a/sqlspec/data_dictionary/dialects/sqlanywhere/sql/views.sql b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/views.sql new file mode 100644 index 000000000..5ba790188 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sqlanywhere/sql/views.sql @@ -0,0 +1,15 @@ +-- name: by_schema +-- dialect: sqlanywhere +SELECT + u.user_name AS schema_name, + t.table_name AS view_name, + v.view_def AS definition, + 0 AS is_read_only, + 1 AS is_valid +FROM SYS.SYSTAB t +JOIN SYS.SYSUSER u ON t.creator = u.user_id +LEFT JOIN SYS.SYSVIEW v ON t.object_id = v.view_object_id +WHERE t.table_type = 21 + AND u.user_name = COALESCE(:schema_name, CURRENT USER) + AND (:view_name IS NULL OR t.table_name = :view_name) +ORDER BY u.user_name, t.table_name; diff --git a/sqlspec/data_dictionary/dialects/sybase/__init__.py b/sqlspec/data_dictionary/dialects/sybase/__init__.py new file mode 100644 index 000000000..57369bf58 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/__init__.py @@ -0,0 +1,31 @@ +"""Sybase ASE dialect static configuration and reflection helpers.""" + +from sqlspec.data_dictionary.dialects.sybase.config import ( + SYBASE_CONFIG, + SYBASE_FEATURE_FLAGS, + SYBASE_FEATURE_VERSIONS, + SYBASE_METADATA_DOMAINS, + SYBASE_TYPE_MAPPINGS, + SYBASE_VERSION_PATTERN, + SybaseVersionInfo, + build_sybase_metadata_capability_profile, + extract_sybase_version_value, + list_sybase_available_features, + merge_sybase_table_lists, + resolve_sybase_feature_flag, +) + +__all__ = ( + "SYBASE_CONFIG", + "SYBASE_FEATURE_FLAGS", + "SYBASE_FEATURE_VERSIONS", + "SYBASE_METADATA_DOMAINS", + "SYBASE_TYPE_MAPPINGS", + "SYBASE_VERSION_PATTERN", + "SybaseVersionInfo", + "build_sybase_metadata_capability_profile", + "extract_sybase_version_value", + "list_sybase_available_features", + "merge_sybase_table_lists", + "resolve_sybase_feature_flag", +) diff --git a/sqlspec/data_dictionary/dialects/sybase/config.py b/sqlspec/data_dictionary/dialects/sybase/config.py new file mode 100644 index 000000000..e4df1d757 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/config.py @@ -0,0 +1,187 @@ +"""Sybase ASE dialect static configuration and reflection helpers.""" + +import re +from collections.abc import Mapping, Sequence +from typing import TYPE_CHECKING, Final + +from sqlspec.data_dictionary import register_dialect +from sqlspec.data_dictionary._types import ( + DialectConfig, + FeatureFlags, + FeatureVersions, + MetadataCapability, + MetadataCapabilityProfile, + MetadataFidelity, + MetadataSource, + MetadataSupport, + VersionInfo, +) + +if TYPE_CHECKING: + from sqlspec.data_dictionary import TableMetadata + +__all__ = ( + "SYBASE_CONFIG", + "SYBASE_FEATURE_FLAGS", + "SYBASE_FEATURE_VERSIONS", + "SYBASE_METADATA_DOMAINS", + "SYBASE_TYPE_MAPPINGS", + "SYBASE_VERSION_PATTERN", + "SybaseVersionInfo", + "build_sybase_metadata_capability_profile", + "extract_sybase_version_value", + "list_sybase_available_features", + "merge_sybase_table_lists", + "resolve_sybase_feature_flag", +) + +SYBASE_VERSION_PATTERN = re.compile(r"Adaptive Server Enterprise/(\d+)\.(\d+)(?:\s+SP(\d+)|\.\d+)?", re.IGNORECASE) + +SYBASE_FEATURE_VERSIONS: FeatureVersions = {} + +SYBASE_FEATURE_FLAGS: FeatureFlags = { + "supports_clustering": False, + "supports_returning": False, + "supports_upsert": False, + "supports_on_conflict": False, + "supports_window_functions": False, + "supports_cte": False, + "supports_json": False, + "supports_arrays": False, + "supports_uuid": False, + "supports_schemas": True, + "supports_merge": False, + "supports_for_update": False, + "supports_skip_locked": False, + "supports_transactional_ddl": False, + "supports_transactions": True, + "supports_prepared_statements": True, + "supports_in_memory": False, + "supports_update_from": True, + "supports_sequences": False, +} + +SYBASE_TYPE_MAPPINGS: dict[str, str] = { + "uuid": "VARCHAR(36)", + "boolean": "BIT", + "timestamp": "BIGDATETIME", + "datetime": "BIGDATETIME", + "date": "DATE", + "time": "BIGTIME", + "text": "TEXT", + "blob": "IMAGE", + "bytes": "VARBINARY(16384)", + "json": "TEXT", + "decimal": "DECIMAL", + "float": "FLOAT", + "integer": "INT", + "bigint": "BIGINT", + "smallint": "SMALLINT", + "string": "VARCHAR(255)", + "varchar": "VARCHAR(255)", + "AutoIncrement": "INT IDENTITY", +} + +SYBASE_CONFIG = DialectConfig( + name="sybase", + feature_versions=SYBASE_FEATURE_VERSIONS, + feature_flags=SYBASE_FEATURE_FLAGS, + type_mappings=SYBASE_TYPE_MAPPINGS, + version_pattern=SYBASE_VERSION_PATTERN, + default_schema="dbo", + parameter_style="pyformat_named", +) + +register_dialect(SYBASE_CONFIG) + +SYBASE_METADATA_DOMAINS: Final[tuple[str, ...]] = ( + "schemas", + "objects", + "tables", + "columns", + "constraints", + "indexes", + "foreign_keys", + "views", + "version", +) + + +class SybaseVersionInfo(VersionInfo): + """Sybase ASE database version info with optional service pack string.""" + + __slots__ = ("service_pack",) + + def __init__(self, major: int, minor: int = 0, patch: int = 0, service_pack: str | None = None) -> None: + """Initialize Sybase ASE version info.""" + super().__init__(major, minor, patch) + self.service_pack = service_pack + + def __str__(self) -> str: + """Format version string including service pack when present.""" + base = f"{self.major}.{self.minor}.{self.patch}" + return f"{base} ({self.service_pack})" if self.service_pack else base + + +def extract_sybase_version_value(row: object) -> str | None: + """Extract a Sybase ASE version string from a row-like object.""" + if isinstance(row, Mapping): + for key in ("version_string", "VERSION_STRING", "version", "VERSION"): + value = row.get(key) + if value: + return str(value) + if isinstance(row, (list, tuple)) and row: + return str(row[0]) + if row is not None: + return str(row) + return None + + +def resolve_sybase_feature_flag(feature: str, version_info: VersionInfo | None) -> bool: + """Resolve a Sybase ASE feature flag using config and version details.""" + flag = SYBASE_CONFIG.get_feature_flag(feature) + if flag is not None: + return bool(flag) + required_version = SYBASE_CONFIG.get_feature_version(feature) + if required_version is None or version_info is None: + return False + return bool(version_info >= required_version) + + +def list_sybase_available_features() -> list[str]: + """List static and versioned Sybase ASE data-dictionary feature flags.""" + features = set(SYBASE_CONFIG.feature_flags.keys()) + features.update(SYBASE_CONFIG.feature_versions.keys()) + return sorted(features) + + +def merge_sybase_table_lists( + ordered: list["TableMetadata"], all_tables: list["TableMetadata"] +) -> list["TableMetadata"]: + """Merge dependency-ordered Sybase ASE tables with catalog remainder rows.""" + if not ordered: + return sorted(all_tables, key=lambda item: item.get("table_name") or "") + seen = {(item.get("schema_name"), item.get("table_name")) for item in ordered if item.get("table_name")} + remainder = [item for item in all_tables if (item.get("schema_name"), item.get("table_name")) not in seen] + return ordered + remainder + + +def build_sybase_metadata_capability_profile( + adapter: str | None, domains: Sequence[str] | None = None +) -> MetadataCapabilityProfile: + """Build Sybase ASE data-dictionary capability metadata.""" + requested_domains = tuple(domains) if domains is not None else SYBASE_METADATA_DOMAINS + capabilities: list[MetadataCapability] = [] + for domain in requested_domains: + if domain in SYBASE_METADATA_DOMAINS: + capabilities.append( + MetadataCapability( + domain=domain, + support=MetadataSupport.SUPPORTED, + fidelity=MetadataFidelity.NATIVE, + source=MetadataSource.CATALOG, + ) + ) + else: + capabilities.append(MetadataCapability.unsupported(domain)) + return MetadataCapabilityProfile("sybase", adapter=adapter, capabilities=tuple(capabilities)) diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/columns.sql b/sqlspec/data_dictionary/dialects/sybase/sql/columns.sql new file mode 100644 index 000000000..574116a4a --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/columns.sql @@ -0,0 +1,123 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + c.name AS column_name, + t.name AS data_type, + CASE WHEN (c.status & 8) = 8 THEN 'YES' ELSE 'NO' END AS is_nullable, + cm.text AS column_default, + c.colid AS ordinal_position, + c.length AS char_length, + c.length AS max_length, + c.prec AS numeric_precision, + c.scale AS numeric_scale, + CASE WHEN pk.col_name IS NOT NULL THEN 1 ELSE 0 END AS is_primary, + CASE WHEN uq.col_name IS NOT NULL THEN 1 ELSE 0 END AS is_unique, + CASE WHEN (c.status & 128) = 128 THEN 1 ELSE 0 END AS is_identity, + CASE WHEN (c.status & 128) = 128 THEN 'ALWAYS' ELSE NULL END AS identity_generation, + 0 AS is_generated, + NULL AS generation_expression, + 0 AS is_hidden, + NULL AS column_comment +FROM dbo.syscolumns c +JOIN dbo.sysobjects o ON c.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.systypes t ON c.usertype = t.usertype +LEFT JOIN dbo.syscomments cm ON c.cdefault = cm.id AND cm.colid = 1 +LEFT JOIN ( + SELECT DISTINCT + i.id, + index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) AS col_name + FROM dbo.sysindexes i + JOIN dbo.sysobjects o ON i.id = o.id + CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 + ) k + WHERE (i.status & 2048) = 2048 + AND index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) IS NOT NULL +) pk ON pk.id = o.id AND pk.col_name = c.name +LEFT JOIN ( + SELECT DISTINCT + i.id, + index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) AS col_name + FROM dbo.sysindexes i + JOIN dbo.sysobjects o ON i.id = o.id + CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 + ) k + WHERE (i.status & 2) = 2 + AND index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) IS NOT NULL +) uq ON uq.id = o.id AND uq.col_name = c.name +WHERE o.type IN ('U', 'V') + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +ORDER BY u.name, o.name, c.colid; + +-- name: by_table +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + c.name AS column_name, + t.name AS data_type, + CASE WHEN (c.status & 8) = 8 THEN 'YES' ELSE 'NO' END AS is_nullable, + cm.text AS column_default, + c.colid AS ordinal_position, + c.length AS char_length, + c.length AS max_length, + c.prec AS numeric_precision, + c.scale AS numeric_scale, + CASE WHEN pk.col_name IS NOT NULL THEN 1 ELSE 0 END AS is_primary, + CASE WHEN uq.col_name IS NOT NULL THEN 1 ELSE 0 END AS is_unique, + CASE WHEN (c.status & 128) = 128 THEN 1 ELSE 0 END AS is_identity, + CASE WHEN (c.status & 128) = 128 THEN 'ALWAYS' ELSE NULL END AS identity_generation, + 0 AS is_generated, + NULL AS generation_expression, + 0 AS is_hidden, + NULL AS column_comment +FROM dbo.syscolumns c +JOIN dbo.sysobjects o ON c.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.systypes t ON c.usertype = t.usertype +LEFT JOIN dbo.syscomments cm ON c.cdefault = cm.id AND cm.colid = 1 +LEFT JOIN ( + SELECT DISTINCT + i.id, + index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) AS col_name + FROM dbo.sysindexes i + JOIN dbo.sysobjects o ON i.id = o.id + CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 + ) k + WHERE (i.status & 2048) = 2048 + AND index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) IS NOT NULL +) pk ON pk.id = o.id AND pk.col_name = c.name +LEFT JOIN ( + SELECT DISTINCT + i.id, + index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) AS col_name + FROM dbo.sysindexes i + JOIN dbo.sysobjects o ON i.id = o.id + CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 + ) k + WHERE (i.status & 2) = 2 + AND index_col(USER_NAME(o.uid) + '.' + o.name, i.indid, k.pos) IS NOT NULL +) uq ON uq.id = o.id AND uq.col_name = c.name +WHERE o.type IN ('U', 'V') + AND o.name = :table_name + AND u.name = COALESCE(:schema_name, USER_NAME()) +ORDER BY c.colid; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/constraints.sql b/sqlspec/data_dictionary/dialects/sybase/sql/constraints.sql new file mode 100644 index 000000000..5354e5412 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/constraints.sql @@ -0,0 +1,95 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + co.name AS constraint_name, + CASE + WHEN (c.status & 1) = 1 THEN 'PRIMARY KEY' + WHEN (c.status & 2) = 2 THEN 'UNIQUE' + WHEN (c.status & 64) = 64 THEN 'FOREIGN KEY' + WHEN (c.status & 128) = 128 THEN 'CHECK' + ELSE 'CONSTRAINT' + END AS constraint_type, + cm.text AS check_clause, + 'Y' AS is_enforced +FROM dbo.sysconstraints c +JOIN dbo.sysobjects o ON c.tableid = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.sysobjects co ON c.constrid = co.id +LEFT JOIN dbo.syscomments cm ON co.id = cm.id AND cm.colid = 1 +WHERE u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +UNION +SELECT + u.name AS schema_name, + o.name AS table_name, + i.name AS constraint_name, + CASE + WHEN (i.status & 2048) = 2048 THEN 'PRIMARY KEY' + WHEN (i.status & 2) = 2 THEN 'UNIQUE' + END AS constraint_type, + NULL AS check_clause, + 'Y' AS is_enforced +FROM dbo.sysindexes i +JOIN dbo.sysobjects o ON i.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type = 'U' + AND ((i.status & 2048) = 2048 OR (i.status & 2) = 2) + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) + AND NOT EXISTS ( + SELECT 1 + FROM dbo.sysconstraints sc + JOIN dbo.sysobjects sco ON sc.constrid = sco.id + WHERE sc.tableid = o.id AND sco.name = i.name + ) +ORDER BY schema_name, table_name, constraint_name; + +-- name: by_table +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + co.name AS constraint_name, + CASE + WHEN (c.status & 1) = 1 THEN 'PRIMARY KEY' + WHEN (c.status & 2) = 2 THEN 'UNIQUE' + WHEN (c.status & 64) = 64 THEN 'FOREIGN KEY' + WHEN (c.status & 128) = 128 THEN 'CHECK' + ELSE 'CONSTRAINT' + END AS constraint_type, + cm.text AS check_clause, + 'Y' AS is_enforced +FROM dbo.sysconstraints c +JOIN dbo.sysobjects o ON c.tableid = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.sysobjects co ON c.constrid = co.id +LEFT JOIN dbo.syscomments cm ON co.id = cm.id AND cm.colid = 1 +WHERE o.name = :table_name + AND u.name = COALESCE(:schema_name, USER_NAME()) +UNION +SELECT + u.name AS schema_name, + o.name AS table_name, + i.name AS constraint_name, + CASE + WHEN (i.status & 2048) = 2048 THEN 'PRIMARY KEY' + WHEN (i.status & 2) = 2 THEN 'UNIQUE' + END AS constraint_type, + NULL AS check_clause, + 'Y' AS is_enforced +FROM dbo.sysindexes i +JOIN dbo.sysobjects o ON i.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type = 'U' + AND ((i.status & 2048) = 2048 OR (i.status & 2) = 2) + AND o.name = :table_name + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND NOT EXISTS ( + SELECT 1 + FROM dbo.sysconstraints sc + JOIN dbo.sysobjects sco ON sc.constrid = sco.id + WHERE sc.tableid = o.id AND sco.name = i.name + ) +ORDER BY constraint_name; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/foreign_keys.sql b/sqlspec/data_dictionary/dialects/sybase/sql/foreign_keys.sql new file mode 100644 index 000000000..82d483c08 --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/foreign_keys.sql @@ -0,0 +1,77 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + co.name AS constraint_name, + fc.name AS column_name, + ru.name AS referenced_schema, + ro.name AS referenced_table, + rc.name AS referenced_column +FROM dbo.sysreferences r +JOIN dbo.sysobjects o ON r.tableid = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.sysobjects co ON r.constrid = co.id +JOIN dbo.sysobjects ro ON r.reftabid = ro.id +JOIN dbo.sysusers ru ON ro.uid = ru.uid +CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 +) k +JOIN dbo.syscolumns fc ON fc.id = o.id AND fc.colid = CASE k.pos + WHEN 1 THEN r.fokey1 WHEN 2 THEN r.fokey2 WHEN 3 THEN r.fokey3 WHEN 4 THEN r.fokey4 + WHEN 5 THEN r.fokey5 WHEN 6 THEN r.fokey6 WHEN 7 THEN r.fokey7 WHEN 8 THEN r.fokey8 + WHEN 9 THEN r.fokey9 WHEN 10 THEN r.fokey10 WHEN 11 THEN r.fokey11 WHEN 12 THEN r.fokey12 + WHEN 13 THEN r.fokey13 WHEN 14 THEN r.fokey14 WHEN 15 THEN r.fokey15 WHEN 16 THEN r.fokey16 +END +JOIN dbo.syscolumns rc ON rc.id = ro.id AND rc.colid = CASE k.pos + WHEN 1 THEN r.refkey1 WHEN 2 THEN r.refkey2 WHEN 3 THEN r.refkey3 WHEN 4 THEN r.refkey4 + WHEN 5 THEN r.refkey5 WHEN 6 THEN r.refkey6 WHEN 7 THEN r.refkey7 WHEN 8 THEN r.refkey8 + WHEN 9 THEN r.refkey9 WHEN 10 THEN r.refkey10 WHEN 11 THEN r.refkey11 WHEN 12 THEN r.refkey12 + WHEN 13 THEN r.refkey13 WHEN 14 THEN r.refkey14 WHEN 15 THEN r.refkey15 WHEN 16 THEN r.refkey16 +END +WHERE k.pos <= r.frgcnt + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +ORDER BY u.name, o.name, co.name, k.pos; + +-- name: by_table +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + co.name AS constraint_name, + fc.name AS column_name, + ru.name AS referenced_schema, + ro.name AS referenced_table, + rc.name AS referenced_column +FROM dbo.sysreferences r +JOIN dbo.sysobjects o ON r.tableid = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +JOIN dbo.sysobjects co ON r.constrid = co.id +JOIN dbo.sysobjects ro ON r.reftabid = ro.id +JOIN dbo.sysusers ru ON ro.uid = ru.uid +CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 +) k +JOIN dbo.syscolumns fc ON fc.id = o.id AND fc.colid = CASE k.pos + WHEN 1 THEN r.fokey1 WHEN 2 THEN r.fokey2 WHEN 3 THEN r.fokey3 WHEN 4 THEN r.fokey4 + WHEN 5 THEN r.fokey5 WHEN 6 THEN r.fokey6 WHEN 7 THEN r.fokey7 WHEN 8 THEN r.fokey8 + WHEN 9 THEN r.fokey9 WHEN 10 THEN r.fokey10 WHEN 11 THEN r.fokey11 WHEN 12 THEN r.fokey12 + WHEN 13 THEN r.fokey13 WHEN 14 THEN r.fokey14 WHEN 15 THEN r.fokey15 WHEN 16 THEN r.fokey16 +END +JOIN dbo.syscolumns rc ON rc.id = ro.id AND rc.colid = CASE k.pos + WHEN 1 THEN r.refkey1 WHEN 2 THEN r.refkey2 WHEN 3 THEN r.refkey3 WHEN 4 THEN r.refkey4 + WHEN 5 THEN r.refkey5 WHEN 6 THEN r.refkey6 WHEN 7 THEN r.refkey7 WHEN 8 THEN r.refkey8 + WHEN 9 THEN r.refkey9 WHEN 10 THEN r.refkey10 WHEN 11 THEN r.refkey11 WHEN 12 THEN r.refkey12 + WHEN 13 THEN r.refkey13 WHEN 14 THEN r.refkey14 WHEN 15 THEN r.refkey15 WHEN 16 THEN r.refkey16 +END +WHERE k.pos <= r.frgcnt + AND o.name = :table_name + AND u.name = COALESCE(:schema_name, USER_NAME()) +ORDER BY co.name, k.pos; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/indexes.sql b/sqlspec/data_dictionary/dialects/sybase/sql/indexes.sql new file mode 100644 index 000000000..c78a3853c --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/indexes.sql @@ -0,0 +1,51 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + i.name AS index_name, + index_col(u.name + '.' + o.name, i.indid, k.pos) AS column_name, + k.pos AS column_position, + CASE WHEN (i.status & 2) = 2 THEN 1 ELSE 0 END AS is_unique, + CASE WHEN (i.status & 2048) = 2048 THEN 1 ELSE 0 END AS is_primary +FROM dbo.sysindexes i +JOIN dbo.sysobjects o ON i.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 +) k +WHERE o.type = 'U' + AND i.indid BETWEEN 1 AND 254 + AND index_col(u.name + '.' + o.name, i.indid, k.pos) IS NOT NULL + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +ORDER BY u.name, o.name, i.name, k.pos; + +-- name: by_table +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + i.name AS index_name, + index_col(u.name + '.' + o.name, i.indid, k.pos) AS column_name, + k.pos AS column_position, + CASE WHEN (i.status & 2) = 2 THEN 1 ELSE 0 END AS is_unique, + CASE WHEN (i.status & 2048) = 2048 THEN 1 ELSE 0 END AS is_primary +FROM dbo.sysindexes i +JOIN dbo.sysobjects o ON i.id = o.id +JOIN dbo.sysusers u ON o.uid = u.uid +CROSS JOIN ( + SELECT 1 AS pos UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 + UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 + UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12 + UNION ALL SELECT 13 UNION ALL SELECT 14 UNION ALL SELECT 15 UNION ALL SELECT 16 +) k +WHERE o.type = 'U' + AND i.indid BETWEEN 1 AND 254 + AND index_col(u.name + '.' + o.name, i.indid, k.pos) IS NOT NULL + AND o.name = :table_name + AND u.name = COALESCE(:schema_name, USER_NAME()) +ORDER BY i.name, k.pos; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/objects.sql b/sqlspec/data_dictionary/dialects/sybase/sql/objects.sql new file mode 100644 index 000000000..7dcdd41bb --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/objects.sql @@ -0,0 +1,21 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS object_name, + CASE o.type + WHEN 'U' THEN 'TABLE' + WHEN 'V' THEN 'VIEW' + WHEN 'P' THEN 'PROCEDURE' + WHEN 'SF' THEN 'FUNCTION' + WHEN 'TR' THEN 'TRIGGER' + ELSE o.type + END AS object_type, + o.crdate AS created, + NULL AS remarks +FROM dbo.sysobjects o +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type IN ('U', 'V', 'P', 'SF', 'TR') + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:object_name IS NULL OR o.name = :object_name) +ORDER BY u.name, object_type, o.name; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/schemas.sql b/sqlspec/data_dictionary/dialects/sybase/sql/schemas.sql new file mode 100644 index 000000000..83379acae --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/schemas.sql @@ -0,0 +1,16 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + u.name AS owner_name, + NULL AS create_time, + NULL AS comment +FROM dbo.sysusers u +WHERE u.uid < 16384 + AND u.name NOT IN ('public', 'probe', 'guest') + AND (:schema_name IS NULL OR u.name = :schema_name) +ORDER BY u.name; + +-- name: current +-- dialect: sybase +SELECT USER_NAME() AS schema_name; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/tables.sql b/sqlspec/data_dictionary/dialects/sybase/sql/tables.sql new file mode 100644 index 000000000..0df8acb6d --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/tables.sql @@ -0,0 +1,46 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + 'BASE TABLE' AS table_type, + 0 AS dependency_level, + 0 AS level +FROM dbo.sysobjects o +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type = 'U' + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +ORDER BY u.name, o.name; + +-- name: all_by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS table_name, + CASE o.type + WHEN 'U' THEN 'BASE TABLE' + WHEN 'V' THEN 'VIEW' + WHEN 'S' THEN 'SYSTEM TABLE' + ELSE 'TABLE' + END AS table_type, + 0 AS dependency_level, + 0 AS level, + NULL AS estimated_row_count, + NULL AS table_comment +FROM dbo.sysobjects o +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type IN ('U', 'V') + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:table_name IS NULL OR o.name = :table_name) +ORDER BY u.name, o.name; + +-- name: names_by_schema +-- dialect: sybase +SELECT + o.name AS table_name +FROM dbo.sysobjects o +JOIN dbo.sysusers u ON o.uid = u.uid +WHERE o.type = 'U' + AND u.name = COALESCE(:schema_name, USER_NAME()) +ORDER BY o.name; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/version.sql b/sqlspec/data_dictionary/dialects/sybase/sql/version.sql new file mode 100644 index 000000000..fd62b988a --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/version.sql @@ -0,0 +1,3 @@ +-- name: current +-- dialect: sybase +SELECT @@version AS version_string; diff --git a/sqlspec/data_dictionary/dialects/sybase/sql/views.sql b/sqlspec/data_dictionary/dialects/sybase/sql/views.sql new file mode 100644 index 000000000..03472e96f --- /dev/null +++ b/sqlspec/data_dictionary/dialects/sybase/sql/views.sql @@ -0,0 +1,15 @@ +-- name: by_schema +-- dialect: sybase +SELECT + u.name AS schema_name, + o.name AS view_name, + cm.text AS definition, + 0 AS is_read_only, + 1 AS is_valid +FROM dbo.sysobjects o +JOIN dbo.sysusers u ON o.uid = u.uid +LEFT JOIN dbo.syscomments cm ON o.id = cm.id AND cm.colid = 1 +WHERE o.type = 'V' + AND u.name = COALESCE(:schema_name, USER_NAME()) + AND (:view_name IS NULL OR o.name = :view_name) +ORDER BY u.name, o.name; diff --git a/sqlspec/dialects/__init__.py b/sqlspec/dialects/__init__.py index 05d2a2acb..8ce8a02f1 100644 --- a/sqlspec/dialects/__init__.py +++ b/sqlspec/dialects/__init__.py @@ -17,16 +17,19 @@ from sqlspec.dialects.db2 import DB2 from sqlspec.dialects.postgres import ParadeDB, PGTextSearch, PGVector from sqlspec.dialects.spanner import Spangres, Spanner + from sqlspec.dialects.sybase import SQLAnywhere, Sybase -__all__ = ("DB2", "PGTextSearch", "PGVector", "ParadeDB", "Spangres", "Spanner") +__all__ = ("DB2", "PGTextSearch", "PGVector", "ParadeDB", "SQLAnywhere", "Spangres", "Spanner", "Sybase") _DIALECT_MODULES = { "DB2": "sqlspec.dialects.db2", "PGTextSearch": "sqlspec.dialects.postgres", "PGVector": "sqlspec.dialects.postgres", "ParadeDB": "sqlspec.dialects.postgres", + "SQLAnywhere": "sqlspec.dialects.sybase", "Spangres": "sqlspec.dialects.spanner", "Spanner": "sqlspec.dialects.spanner", + "Sybase": "sqlspec.dialects.sybase", } diff --git a/sqlspec/dialects/sybase/__init__.py b/sqlspec/dialects/sybase/__init__.py new file mode 100644 index 000000000..f3c5c83a5 --- /dev/null +++ b/sqlspec/dialects/sybase/__init__.py @@ -0,0 +1,115 @@ +"""Sybase ASE and SAP SQL Anywhere sqlglot dialects. + +Both dialects inherit from sqlglot's ``TSQL`` dialect with +``NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE`` and install +per-instance generator dispatch overlays (``sybase_dispatch`` and +``sqlanywhere_dispatch``) so shared sqlglot generator classes are never +mutated or subclassed. +""" + +from typing import TYPE_CHECKING, Any + +from sqlglot import TokenType, exp, generator +from sqlglot.dialects.dialect import NormalizationStrategy +from sqlglot.dialects.tsql import TSQL + +from sqlspec.dialects.sybase._generators import sqlanywhere_dispatch, sybase_dispatch +from sqlspec.dialects.sybase._parsers import ( + normalize_sqlanywhere_expression, + normalize_sqlanywhere_tokens, + normalize_sybase_expression, + normalize_sybase_tokens, +) + +if TYPE_CHECKING: + from sqlglot.tokenizer_core import Token + +__all__ = ("SQLAnywhere", "SQLAnywhereTokenizer", "Sybase", "SybaseTokenizer") + +_DISPATCH_ATTR = "_dispatch" + + +class SybaseTokenizer(TSQL.Tokenizer): + """Tokenizer for SAP Adaptive Server Enterprise (Sybase ASE) SQL syntax.""" + + KEYWORDS = { + **TSQL.Tokenizer.KEYWORDS, + "BIGDATETIME": TokenType.DATETIME, + "BIGTIME": TokenType.TIME, + "BLOB": TokenType.BLOB, + } + + def tokenize(self, sql: str) -> "list[Token]": + """Tokenize Sybase ASE SQL and normalize ASE-only token sequences.""" + return normalize_sybase_tokens(super().tokenize(sql)) + + +class Sybase(TSQL): + """SAP Adaptive Server Enterprise (Sybase ASE) SQL dialect.""" + + NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE + + Tokenizer = SybaseTokenizer + Generator = TSQL.Generator + + def generator(self, **opts: Any) -> "generator.Generator": + """Return a generator whose dispatch table carries the Sybase ASE handlers.""" + instance = TSQL.Generator(dialect=self, **opts) + base_dispatch = getattr(instance, _DISPATCH_ATTR) + setattr(instance, _DISPATCH_ATTR, sybase_dispatch(base_dispatch)) + return instance + + def parse(self, sql: str, **opts: Any) -> "list[exp.Expr | None]": + """Parse Sybase ASE SQL and normalize ASE function nodes.""" + parsed = self.parser(**opts).parse(self.tokenize(sql), sql) + return [normalize_sybase_expression(expr) if expr is not None else None for expr in parsed] + + def parse_into(self, expression_type: Any, sql: str, **opts: Any) -> "list[exp.Expr | None]": + """Parse Sybase ASE SQL into a target expression type.""" + parsed = self.parser(**opts).parse_into(expression_type, self.tokenize(sql), sql) + return [normalize_sybase_expression(expr) if expr is not None else None for expr in parsed] + + +class SQLAnywhereTokenizer(TSQL.Tokenizer): + """Tokenizer for SAP SQL Anywhere SQL syntax.""" + + KEYWORDS = { + **TSQL.Tokenizer.KEYWORDS, + "BLOB": TokenType.BLOB, + "CURRENT DATE": TokenType.CURRENT_DATE, + "CURRENT TIME": TokenType.CURRENT_TIME, + "CURRENT TIMESTAMP": TokenType.CURRENT_TIMESTAMP, + "CURRENT USER": TokenType.CURRENT_USER, + "TIMESTAMP": TokenType.TIMESTAMP, + "UNIQUEIDENTIFIER": TokenType.UUID, + } + + def tokenize(self, sql: str) -> "list[Token]": + """Tokenize SAP SQL Anywhere SQL and normalize SQL Anywhere token sequences.""" + return normalize_sqlanywhere_tokens(super().tokenize(sql)) + + +class SQLAnywhere(TSQL): + """SAP SQL Anywhere SQL dialect.""" + + NORMALIZATION_STRATEGY = NormalizationStrategy.CASE_SENSITIVE + + Tokenizer = SQLAnywhereTokenizer + Generator = TSQL.Generator + + def generator(self, **opts: Any) -> "generator.Generator": + """Return a generator whose dispatch table carries the SAP SQL Anywhere handlers.""" + instance = TSQL.Generator(dialect=self, **opts) + base_dispatch = getattr(instance, _DISPATCH_ATTR) + setattr(instance, _DISPATCH_ATTR, sqlanywhere_dispatch(base_dispatch)) + return instance + + def parse(self, sql: str, **opts: Any) -> "list[exp.Expr | None]": + """Parse SAP SQL Anywhere SQL and normalize SQL Anywhere function nodes.""" + parsed = self.parser(**opts).parse(self.tokenize(sql), sql) + return [normalize_sqlanywhere_expression(expr) if expr is not None else None for expr in parsed] + + def parse_into(self, expression_type: Any, sql: str, **opts: Any) -> "list[exp.Expr | None]": + """Parse SAP SQL Anywhere SQL into a target expression type.""" + parsed = self.parser(**opts).parse_into(expression_type, self.tokenize(sql), sql) + return [normalize_sqlanywhere_expression(expr) if expr is not None else None for expr in parsed] diff --git a/sqlspec/dialects/sybase/_generators.py b/sqlspec/dialects/sybase/_generators.py new file mode 100644 index 000000000..4ea032b2a --- /dev/null +++ b/sqlspec/dialects/sybase/_generators.py @@ -0,0 +1,379 @@ +"""Sybase ASE and SAP SQL Anywhere render handlers layered onto per-instance sqlglot generator dispatch tables.""" + +from collections.abc import Callable +from typing import Any, Final, cast + +from sqlglot import exp, generator + +from sqlspec.dialects.sybase._transforms import ( + clone_with_detached_pagination, + is_literal_integer_limit, + render_char_length, + render_charindex, + render_ilike, +) + +__all__ = ( + "SQLANYWHERE_DEFAULT_TYPE_LENGTHS", + "SQLANYWHERE_TRANSFORMS", + "SQLANYWHERE_TYPE_MAPPING", + "SYBASE_DEFAULT_TYPE_LENGTHS", + "SYBASE_TRANSFORMS", + "SYBASE_TYPE_MAPPING", + "sqlanywhere_anonymous_sql", + "sqlanywhere_autoincrement_sql", + "sqlanywhere_create_sql", + "sqlanywhere_current_timestamp_sql", + "sqlanywhere_datatype_sql", + "sqlanywhere_dispatch", + "sqlanywhere_drop_sql", + "sqlanywhere_generated_identity_sql", + "sqlanywhere_select_sql", + "sqlanywhere_set_operation_sql", + "sybase_autoincrement_sql", + "sybase_columndef_sql", + "sybase_current_timestamp_sql", + "sybase_datatype_sql", + "sybase_dispatch", + "sybase_generated_identity_sql", + "sybase_returning_sql", + "sybase_select_sql", + "sybase_set_operation_sql", +) + +SYBASE_TYPE_MAPPING: Final[dict[exp.DType, str]] = { + **generator.Generator.TYPE_MAPPING, + exp.DType.BOOLEAN: "BIT", + exp.DType.BIT: "BIT", + exp.DType.BLOB: "IMAGE", + exp.DType.IMAGE: "IMAGE", + exp.DType.TEXT: "TEXT", + exp.DType.JSON: "TEXT", + exp.DType.JSONB: "TEXT", + exp.DType.UUID: "VARCHAR(36)", + exp.DType.VARCHAR: "VARCHAR", + exp.DType.NVARCHAR: "UNIVARCHAR", + exp.DType.CHAR: "CHAR", + exp.DType.NCHAR: "UNICHAR", + exp.DType.BINARY: "BINARY", + exp.DType.VARBINARY: "VARBINARY", + exp.DType.TIMESTAMP: "BIGDATETIME", + exp.DType.TIMESTAMPTZ: "BIGDATETIME", + exp.DType.TIMESTAMPLTZ: "BIGDATETIME", + exp.DType.TIMESTAMPNTZ: "BIGDATETIME", + exp.DType.DATETIME: "DATETIME", + exp.DType.DATETIME2: "BIGDATETIME", + exp.DType.ROWVERSION: "TIMESTAMP", + exp.DType.DATE: "DATE", + exp.DType.TIME: "BIGTIME", + exp.DType.TIMETZ: "BIGTIME", + exp.DType.TINYINT: "TINYINT", + exp.DType.SMALLINT: "SMALLINT", + exp.DType.INT: "INT", + exp.DType.BIGINT: "BIGINT", + exp.DType.FLOAT: "FLOAT", + exp.DType.DOUBLE: "DOUBLE PRECISION", + exp.DType.DECIMAL: "DECIMAL", +} + +SYBASE_DEFAULT_TYPE_LENGTHS: Final[dict[exp.DType, str]] = { + exp.DType.VARCHAR: "VARCHAR(16384)", + exp.DType.NVARCHAR: "UNIVARCHAR(8192)", + exp.DType.VARBINARY: "VARBINARY(16384)", +} + +SQLANYWHERE_TYPE_MAPPING: Final[dict[exp.DType, str]] = { + **generator.Generator.TYPE_MAPPING, + exp.DType.BOOLEAN: "BIT", + exp.DType.BIT: "BIT", + exp.DType.BLOB: "LONG BINARY", + exp.DType.IMAGE: "LONG BINARY", + exp.DType.TEXT: "LONG VARCHAR", + exp.DType.JSON: "LONG VARCHAR", + exp.DType.JSONB: "LONG VARCHAR", + exp.DType.UUID: "UNIQUEIDENTIFIER", + exp.DType.VARCHAR: "VARCHAR", + exp.DType.NVARCHAR: "NVARCHAR", + exp.DType.CHAR: "CHAR", + exp.DType.NCHAR: "NCHAR", + exp.DType.BINARY: "BINARY", + exp.DType.VARBINARY: "VARBINARY", + exp.DType.TIMESTAMP: "TIMESTAMP", + exp.DType.TIMESTAMPNTZ: "TIMESTAMP", + exp.DType.DATETIME: "TIMESTAMP", + exp.DType.DATETIME2: "TIMESTAMP", + exp.DType.TIMESTAMPTZ: "TIMESTAMP WITH TIME ZONE", + exp.DType.TIMESTAMPLTZ: "TIMESTAMP WITH TIME ZONE", + exp.DType.DATE: "DATE", + exp.DType.TIME: "TIME", + exp.DType.TIMETZ: "TIME", + exp.DType.TINYINT: "TINYINT", + exp.DType.SMALLINT: "SMALLINT", + exp.DType.INT: "INT", + exp.DType.BIGINT: "BIGINT", + exp.DType.FLOAT: "DOUBLE", + exp.DType.DOUBLE: "DOUBLE", + exp.DType.DECIMAL: "DECIMAL", +} + +SQLANYWHERE_DEFAULT_TYPE_LENGTHS: Final[dict[exp.DType, str]] = { + exp.DType.VARCHAR: "VARCHAR(32767)", + exp.DType.NVARCHAR: "NVARCHAR(32767)", + exp.DType.VARBINARY: "VARBINARY(32767)", +} + +_BOOLEAN_TYPES: Final[frozenset[exp.DType]] = frozenset({exp.DType.BOOLEAN, exp.DType.BIT}) + + +def sybase_select_sql(gen: "generator.Generator", expression: exp.Select) -> str: + """Render a Sybase ASE SELECT with copy-on-write TOP or ROWS LIMIT/OFFSET pagination.""" + limit_arg = expression.args.get("limit") + offset_arg = expression.args.get("offset") + if offset_arg is None and is_literal_integer_limit(limit_arg): + cloned = expression.copy() + return generator.Generator.select_sql(gen, cloned) + detached, limit_node, offset_node = clone_with_detached_pagination(expression) + base_sql = generator.Generator.select_sql(gen, cast("exp.Select", detached)) + tail = _render_sybase_pagination_tail(gen, limit_node, offset_node) + return f"{base_sql} {tail}" if tail else base_sql + + +def sybase_set_operation_sql(gen: "generator.Generator", expression: exp.SetOperation) -> str: + """Render Sybase ASE set operations with copy-on-write ROWS LIMIT/OFFSET pagination.""" + detached, limit_node, offset_node = clone_with_detached_pagination(expression) + base_sql = gen.set_operations(cast("exp.SetOperation", detached)) + tail = _render_sybase_pagination_tail(gen, limit_node, offset_node) + return f"{base_sql} {tail}" if tail else base_sql + + +def sybase_datatype_sql(gen: "generator.Generator", expression: exp.DataType) -> str: + """Render a data type using Sybase ASE type names and MAX / nullable BIT rules.""" + if _is_max_length_datatype(expression): + if expression.this in {exp.DType.NVARCHAR, exp.DType.NCHAR}: + return "UNITEXT" + if expression.this in {exp.DType.VARBINARY, exp.DType.BINARY, exp.DType.BLOB}: + return "IMAGE" + return "TEXT" + if expression.this in _BOOLEAN_TYPES and _is_explicitly_nullable_columndef(expression.parent): + return "TINYINT" + type_str = SYBASE_TYPE_MAPPING.get(expression.this) + if not type_str: + return generator.Generator.datatype_sql(gen, expression) + if expression.expressions: + params = ", ".join(gen.sql(e) for e in expression.expressions) + return f"{type_str}({params})" + return SYBASE_DEFAULT_TYPE_LENGTHS.get(expression.this, type_str) + + +def sybase_columndef_sql(gen: "generator.Generator", expression: exp.ColumnDef) -> str: + """Render a Sybase ASE column definition.""" + return generator.Generator.columndef_sql(gen, expression) + + +def sybase_generated_identity_sql( + gen: "generator.Generator", expression: exp.GeneratedAsIdentityColumnConstraint +) -> str: + """Render GeneratedAsIdentityColumnConstraint as bare Sybase ASE IDENTITY.""" + _ = (gen, expression) + return "IDENTITY" + + +def sybase_autoincrement_sql(gen: "generator.Generator", expression: exp.AutoIncrementColumnConstraint) -> str: + """Render AutoIncrementColumnConstraint as bare Sybase ASE IDENTITY.""" + _ = (gen, expression) + return "IDENTITY" + + +def sybase_current_timestamp_sql(gen: "generator.Generator", expression: exp.CurrentTimestamp) -> str: + """Render CurrentTimestamp as GETDATE() or GETUTCDATE() when UTC is specified.""" + _ = gen + zone = expression.args.get("zone") + if isinstance(expression, exp.UtcTimestamp) or (zone is not None and "utc" in str(zone).lower()): + return "GETUTCDATE()" + return "GETDATE()" + + +def sybase_returning_sql(gen: "generator.Generator", expression: exp.Returning) -> str: + """Omit RETURNING / OUTPUT clauses in Sybase ASE.""" + _ = expression + gen.unsupported("Sybase ASE does not support RETURNING or OUTPUT clauses") + return "" + + +def sqlanywhere_select_sql(gen: "generator.Generator", expression: exp.Select) -> str: + """Render a SAP SQL Anywhere SELECT with copy-on-write trailing LIMIT/OFFSET pagination.""" + detached, limit_node, offset_node = clone_with_detached_pagination(expression) + base_sql = generator.Generator.select_sql(gen, cast("exp.Select", detached)) + tail = _render_sqlanywhere_pagination_tail(gen, limit_node, offset_node) + return f"{base_sql} {tail}" if tail else base_sql + + +def sqlanywhere_set_operation_sql(gen: "generator.Generator", expression: exp.SetOperation) -> str: + """Render SAP SQL Anywhere set operations with copy-on-write trailing LIMIT/OFFSET pagination.""" + detached, limit_node, offset_node = clone_with_detached_pagination(expression) + base_sql = gen.set_operations(cast("exp.SetOperation", detached)) + tail = _render_sqlanywhere_pagination_tail(gen, limit_node, offset_node) + return f"{base_sql} {tail}" if tail else base_sql + + +def sqlanywhere_datatype_sql(gen: "generator.Generator", expression: exp.DataType) -> str: + """Render a data type using SAP SQL Anywhere type names.""" + if _is_max_length_datatype(expression): + if expression.this in {exp.DType.NVARCHAR, exp.DType.NCHAR}: + return "LONG NVARCHAR" + if expression.this in {exp.DType.VARBINARY, exp.DType.BINARY, exp.DType.BLOB}: + return "LONG BINARY" + return "LONG VARCHAR" + type_str = SQLANYWHERE_TYPE_MAPPING.get(expression.this) + if not type_str: + return generator.Generator.datatype_sql(gen, expression) + if expression.expressions: + params = ", ".join(gen.sql(e) for e in expression.expressions) + return f"{type_str}({params})" + return SQLANYWHERE_DEFAULT_TYPE_LENGTHS.get(expression.this, type_str) + + +def sqlanywhere_create_sql(gen: "generator.Generator", expression: exp.Create) -> str: + """Render CREATE statements using native SQL Anywhere IF NOT EXISTS syntax.""" + return generator.Generator.create_sql(gen, expression) + + +def sqlanywhere_drop_sql(gen: "generator.Generator", expression: exp.Drop) -> str: + """Render DROP statements using native SQL Anywhere IF EXISTS syntax.""" + return generator.Generator.drop_sql(gen, expression) + + +def sqlanywhere_current_timestamp_sql(gen: "generator.Generator", expression: exp.CurrentTimestamp) -> str: + """Render CurrentTimestamp as CURRENT TIMESTAMP or CURRENT UTC TIMESTAMP when UTC is specified.""" + _ = gen + zone = expression.args.get("zone") + if isinstance(expression, exp.UtcTimestamp) or (zone is not None and "utc" in str(zone).lower()): + return "CURRENT UTC TIMESTAMP" + return "CURRENT TIMESTAMP" + + +def sqlanywhere_anonymous_sql(gen: "generator.Generator", expression: exp.Anonymous) -> str: + """Render Anonymous functions, mapping GETUTCDATE to CURRENT UTC TIMESTAMP.""" + if str(expression.this).upper() == "GETUTCDATE": + return "CURRENT UTC TIMESTAMP" + return generator.Generator.anonymous_sql(gen, expression) + + +def sqlanywhere_generated_identity_sql( + gen: "generator.Generator", expression: exp.GeneratedAsIdentityColumnConstraint +) -> str: + """Render GeneratedAsIdentityColumnConstraint as DEFAULT AUTOINCREMENT.""" + _ = (gen, expression) + return "DEFAULT AUTOINCREMENT" + + +def sqlanywhere_autoincrement_sql(gen: "generator.Generator", expression: exp.AutoIncrementColumnConstraint) -> str: + """Render AutoIncrementColumnConstraint as DEFAULT AUTOINCREMENT.""" + _ = (gen, expression) + return "DEFAULT AUTOINCREMENT" + + +SYBASE_TRANSFORMS: Final[dict[type[exp.Expr], Callable[[Any, Any], str]]] = { + exp.Select: sybase_select_sql, + exp.Union: sybase_set_operation_sql, + exp.Intersect: sybase_set_operation_sql, + exp.Except: sybase_set_operation_sql, + exp.DataType: sybase_datatype_sql, + exp.ColumnDef: sybase_columndef_sql, + exp.GeneratedAsIdentityColumnConstraint: sybase_generated_identity_sql, + exp.AutoIncrementColumnConstraint: sybase_autoincrement_sql, + exp.CurrentTimestamp: sybase_current_timestamp_sql, + exp.Returning: sybase_returning_sql, + exp.Length: render_char_length, + exp.StrPosition: render_charindex, + exp.ILike: render_ilike, +} + +SQLANYWHERE_TRANSFORMS: Final[dict[type[exp.Expr], Callable[[Any, Any], str]]] = { + exp.Anonymous: sqlanywhere_anonymous_sql, + exp.Select: sqlanywhere_select_sql, + exp.Union: sqlanywhere_set_operation_sql, + exp.Intersect: sqlanywhere_set_operation_sql, + exp.Except: sqlanywhere_set_operation_sql, + exp.DataType: sqlanywhere_datatype_sql, + exp.Create: sqlanywhere_create_sql, + exp.Drop: sqlanywhere_drop_sql, + exp.GeneratedAsIdentityColumnConstraint: sqlanywhere_generated_identity_sql, + exp.AutoIncrementColumnConstraint: sqlanywhere_autoincrement_sql, + exp.CurrentTimestamp: sqlanywhere_current_timestamp_sql, + exp.Length: render_char_length, + exp.StrPosition: render_charindex, + exp.ILike: render_ilike, +} + +_sybase_overlay_cache: "tuple[dict[type[exp.Expr], Callable[..., str]], dict[type[exp.Expr], Callable[..., str]]] | None" = None +_sqlanywhere_overlay_cache: "tuple[dict[type[exp.Expr], Callable[..., str]], dict[type[exp.Expr], Callable[..., str]]] | None" = None + + +def sybase_dispatch(base: "dict[type[exp.Expr], Callable[..., str]]") -> "dict[type[exp.Expr], Callable[..., str]]": + """Return the Sybase ASE dispatch table layered over a root generator dispatch table.""" + global _sybase_overlay_cache + cache = _sybase_overlay_cache + if cache is not None and cache[0] is base: + return cache[1] + overlay = {**base, **SYBASE_TRANSFORMS} + _sybase_overlay_cache = (base, overlay) + return overlay + + +def sqlanywhere_dispatch( + base: "dict[type[exp.Expr], Callable[..., str]]", +) -> "dict[type[exp.Expr], Callable[..., str]]": + """Return the SAP SQL Anywhere dispatch table layered over a root generator dispatch table.""" + global _sqlanywhere_overlay_cache + cache = _sqlanywhere_overlay_cache + if cache is not None and cache[0] is base: + return cache[1] + overlay = {**base, **SQLANYWHERE_TRANSFORMS} + _sqlanywhere_overlay_cache = (base, overlay) + return overlay + + +def _is_max_length_datatype(expression: exp.DataType) -> bool: + if len(expression.expressions) != 1: + return False + first = expression.expressions[0] + return str(first).upper() == "MAX" or (isinstance(first, exp.DataTypeParam) and str(first.this).upper() == "MAX") + + +def _is_explicitly_nullable_columndef(parent: exp.Expr | None) -> bool: + if not isinstance(parent, exp.ColumnDef): + return False + for constraint in parent.constraints: + kind = constraint.kind + if isinstance(kind, exp.NotNullColumnConstraint) and bool(kind.args.get("allow_null")): + return True + return False + + +def _render_sybase_pagination_tail( + gen: "generator.Generator", limit_node: exp.Limit | None, offset_node: exp.Offset | None +) -> str: + limit_sql = gen.sql(limit_node, "expression") if limit_node is not None else "" + offset_sql = gen.sql(offset_node, "expression") if offset_node is not None else "" + if limit_sql and offset_sql: + return f"ROWS LIMIT {limit_sql} OFFSET {offset_sql}" + if limit_sql: + return f"ROWS LIMIT {limit_sql}" + if offset_sql: + return f"ROWS OFFSET {offset_sql}" + return "" + + +def _render_sqlanywhere_pagination_tail( + gen: "generator.Generator", limit_node: exp.Limit | None, offset_node: exp.Offset | None +) -> str: + limit_sql = gen.sql(limit_node, "expression") if limit_node is not None else "" + offset_sql = gen.sql(offset_node, "expression") if offset_node is not None else "" + if limit_sql and offset_sql: + return f"LIMIT {limit_sql} OFFSET {offset_sql}" + if limit_sql: + return f"LIMIT {limit_sql}" + if offset_sql: + return f"OFFSET {offset_sql}" + return "" diff --git a/sqlspec/dialects/sybase/_parsers.py b/sqlspec/dialects/sybase/_parsers.py new file mode 100644 index 000000000..fe6d51940 --- /dev/null +++ b/sqlspec/dialects/sybase/_parsers.py @@ -0,0 +1,225 @@ +"""Sybase ASE and SAP SQL Anywhere token and AST normalization helpers.""" + +from typing import Final + +from sqlglot import exp +from sqlglot.tokenizer_core import Token, TokenType + +__all__ = ( + "normalize_sqlanywhere_expression", + "normalize_sqlanywhere_tokens", + "normalize_sybase_expression", + "normalize_sybase_tokens", +) + +_QUOTED_TOKEN_TYPES: Final[frozenset[TokenType]] = frozenset({ + TokenType.STRING, + TokenType.IDENTIFIER, + TokenType.NATIONAL_STRING, + TokenType.RAW_STRING, + TokenType.BYTE_STRING, + TokenType.HEX_STRING, + TokenType.BIT_STRING, + TokenType.HEREDOC_STRING, + TokenType.UNICODE_STRING, +}) +_CHARINDEX_MIN_ARGS: Final[int] = 2 +_CHARINDEX_MAX_ARGS: Final[int] = 3 +_TOP_START_AT_SPAN: Final[int] = 5 + + +def normalize_sybase_tokens(tokens: "list[Token]") -> "list[Token]": + """Rewrite Sybase ASE token sequences into forms accepted by the T-SQL parser. + + Trailing ``ROWS LIMIT [OFFSET ]`` and ``ROWS OFFSET `` sequences + have their leading ``ROWS`` token removed so sqlglot parses them into + canonical ``exp.Limit`` and ``exp.Offset`` nodes, and ``UNITEXT`` tokens + are expanded to ``NVARCHAR(MAX)`` for round-trip preservation. + """ + result: list[Token] = [] + index = 0 + total = len(tokens) + while index < total: + token = tokens[index] + if ( + token.token_type == TokenType.ROWS + and index + 1 < total + and tokens[index + 1].token_type in {TokenType.LIMIT, TokenType.OFFSET} + ): + index += 1 + continue + if token.token_type not in _QUOTED_TOKEN_TYPES and token.text.upper() == "UNITEXT": + result.extend(_expand_nvarchar_max(token)) + index += 1 + continue + result.append(token) + index += 1 + return result + + +def normalize_sqlanywhere_tokens(tokens: "list[Token]") -> "list[Token]": + """Rewrite SAP SQL Anywhere token sequences into forms accepted by the T-SQL parser. + + Normalizes ``LONG VARCHAR``, ``LONG NVARCHAR``, ``LONG BINARY``, and + ``CURRENT UTC TIMESTAMP`` tokens, and rewrites Watcom-SQL + ``SELECT [DISTINCT|ALL] TOP START AT ...`` headers into trailing + 0-based ``LIMIT OFFSET `` clauses. + """ + rewritten = _normalize_sqlanywhere_multiword_tokens(tokens) + result: list[Token] = [] + statement: list[Token] = [] + for token in rewritten: + if token.token_type in {TokenType.SEMICOLON, TokenType.UNION, TokenType.INTERSECT, TokenType.EXCEPT}: + result.extend(_rewrite_top_start_at_statement(statement)) + result.append(token) + statement = [] + else: + statement.append(token) + result.extend(_rewrite_top_start_at_statement(statement)) + return result + + +def normalize_sybase_expression(expression: exp.Expr) -> exp.Expr: + """Normalize Sybase ASE function nodes into canonical sqlglot AST expressions.""" + return expression.transform(_canonical_sybase_node, copy=False) + + +def normalize_sqlanywhere_expression(expression: exp.Expr) -> exp.Expr: + """Normalize SAP SQL Anywhere function nodes into canonical sqlglot AST expressions.""" + return expression.transform(_canonical_sybase_node, copy=False) + + +def _expand_nvarchar_max(token: Token) -> "list[Token]": + return [ + Token(TokenType.NVARCHAR, "NVARCHAR", token.line, token.col, token.start, token.end, token.comments), + Token(TokenType.L_PAREN, "(", token.line, token.col, token.start, token.end), + Token(TokenType.VAR, "MAX", token.line, token.col, token.start, token.end), + Token(TokenType.R_PAREN, ")", token.line, token.col, token.start, token.end), + ] + + +def _normalize_sqlanywhere_multiword_tokens(tokens: "list[Token]") -> "list[Token]": + result: list[Token] = [] + index = 0 + total = len(tokens) + while index < total: + token = tokens[index] + if token.token_type not in _QUOTED_TOKEN_TYPES: + upper = token.text.upper() + if upper == "LONG" and index + 1 < total and tokens[index + 1].token_type not in _QUOTED_TOKEN_TYPES: + second_upper = tokens[index + 1].text.upper() + if second_upper == "VARCHAR": + result.append( + Token(TokenType.TEXT, "TEXT", token.line, token.col, token.start, tokens[index + 1].end) + ) + index += 2 + continue + if second_upper == "NVARCHAR": + result.extend(_expand_nvarchar_max(token)) + index += 2 + continue + if second_upper == "BINARY": + result.append( + Token(TokenType.BLOB, "BLOB", token.line, token.col, token.start, tokens[index + 1].end) + ) + index += 2 + continue + if ( + upper == "CURRENT" + and index + 2 < total + and tokens[index + 1].token_type not in _QUOTED_TOKEN_TYPES + and tokens[index + 2].token_type not in _QUOTED_TOKEN_TYPES + and tokens[index + 1].text.upper() == "UTC" + and tokens[index + 2].text.upper() in {"TIMESTAMP", "CURRENT_TIMESTAMP"} + ): + result.append( + Token( + TokenType.CURRENT_TIMESTAMP, + "CURRENT_TIMESTAMP", + token.line, + token.col, + token.start, + tokens[index + 2].end, + ) + ) + index += 3 + continue + result.append(token) + index += 1 + return result + + +def _rewrite_top_start_at_statement(statement: "list[Token]") -> "list[Token]": + if len(statement) < _TOP_START_AT_SPAN + 1 or statement[0].token_type != TokenType.SELECT: + return statement + header_index = 1 + if statement[header_index].token_type in {TokenType.DISTINCT, TokenType.ALL}: + header_index += 1 + if header_index + _TOP_START_AT_SPAN > len(statement): + return statement + top_tok = statement[header_index] + limit_tok = statement[header_index + 1] + start_word_tok = statement[header_index + 2] + at_word_tok = statement[header_index + 3] + start_val_tok = statement[header_index + 4] + if ( + top_tok.token_type != TokenType.TOP + or start_word_tok.token_type in _QUOTED_TOKEN_TYPES + or at_word_tok.token_type in _QUOTED_TOKEN_TYPES + or start_word_tok.text.upper() != "START" + or at_word_tok.text.upper() != "AT" + ): + return statement + if start_val_tok.token_type == TokenType.NUMBER and start_val_tok.text.isdigit(): + offset_str = str(max(int(start_val_tok.text) - 1, 0)) + offset_tokens = [ + Token( + TokenType.NUMBER, + offset_str, + start_val_tok.line, + start_val_tok.col, + start_val_tok.start, + start_val_tok.end, + ) + ] + else: + offset_tokens = [ + Token( + TokenType.L_PAREN, "(", start_val_tok.line, start_val_tok.col, start_val_tok.start, start_val_tok.start + ), + start_val_tok, + Token(TokenType.DASH, "-", start_val_tok.line, start_val_tok.col, start_val_tok.start, start_val_tok.end), + Token(TokenType.NUMBER, "1", start_val_tok.line, start_val_tok.col, start_val_tok.end, start_val_tok.end), + Token(TokenType.R_PAREN, ")", start_val_tok.line, start_val_tok.col, start_val_tok.end, start_val_tok.end), + ] + body = statement[header_index + _TOP_START_AT_SPAN :] + last = statement[-1] + tail = [ + Token(TokenType.LIMIT, "LIMIT", last.line, last.col, last.end, last.end), + limit_tok, + Token(TokenType.OFFSET, "OFFSET", last.line, last.col, last.end, last.end), + *offset_tokens, + ] + insert_idx = len(body) + for idx in range(len(body) - 1, -1, -1): + if body[idx].token_type == TokenType.FOR and idx + 1 < len(body): + next_text = body[idx + 1].text.upper() + if next_text in {"UPDATE", "READ"}: + insert_idx = idx + break + return [*statement[:header_index], *body[:insert_idx], *tail, *body[insert_idx:]] + + +def _canonical_sybase_node(node: exp.Expr) -> exp.Expr: + if isinstance(node, exp.Anonymous): + func_name = str(node.this).upper() + if func_name == "CHAR_LENGTH" and len(node.expressions) == 1: + return exp.Length(this=node.expressions[0]) + if func_name == "CHARINDEX" and _CHARINDEX_MIN_ARGS <= len(node.expressions) <= _CHARINDEX_MAX_ARGS: + substr = node.expressions[0] + this = node.expressions[1] + position = node.expressions[2] if len(node.expressions) == _CHARINDEX_MAX_ARGS else None + return exp.StrPosition(this=this, substr=substr, position=position) + if func_name in {"GETUTCDATE", "CURRENT UTC TIMESTAMP"}: + return exp.UtcTimestamp() + return node diff --git a/sqlspec/dialects/sybase/_transforms.py b/sqlspec/dialects/sybase/_transforms.py new file mode 100644 index 000000000..8efef9fe9 --- /dev/null +++ b/sqlspec/dialects/sybase/_transforms.py @@ -0,0 +1,68 @@ +"""Shared AST transformation and rendering helpers for Sybase ASE and SAP SQL Anywhere dialects.""" + +from typing import Any + +from sqlglot import exp + +__all__ = ( + "clone_with_detached_pagination", + "is_literal_integer_limit", + "render_char_length", + "render_charindex", + "render_ilike", +) + + +def is_literal_integer_limit(limit_expr: Any) -> bool: + """Return True when limit_expr is a literal integer exp.Limit or exp.Fetch without parameters.""" + if isinstance(limit_expr, exp.Fetch): + count_expr = limit_expr.args.get("count") + return isinstance(count_expr, exp.Literal) and not count_expr.is_string + if not isinstance(limit_expr, exp.Limit): + return False + count_expr = limit_expr.expression + return isinstance(count_expr, exp.Literal) and not count_expr.is_string + + +def clone_with_detached_pagination(expression: exp.Expr) -> "tuple[exp.Expr, exp.Limit | None, exp.Offset | None]": + """Return a shallow copy of expression with limit, offset, and fetch removed if present.""" + limit_arg = expression.args.get("limit") + offset_arg = expression.args.get("offset") + fetch_arg = expression.args.get("fetch") + if limit_arg is None and offset_arg is None and fetch_arg is None: + return expression, None, None + cloned = expression.copy() + cloned.set("limit", None) + cloned.set("offset", None) + cloned.set("fetch", None) + limit_node: exp.Limit | None = None + if isinstance(limit_arg, exp.Limit): + limit_node = limit_arg + elif isinstance(limit_arg, exp.Fetch): + count_expr = limit_arg.args.get("count") + limit_node = exp.Limit(expression=count_expr.copy() if count_expr is not None else None) + elif isinstance(fetch_arg, exp.Fetch): + count_expr = fetch_arg.args.get("count") + limit_node = exp.Limit(expression=count_expr.copy() if count_expr is not None else None) + offset_node = offset_arg if isinstance(offset_arg, exp.Offset) else None + return cloned, limit_node, offset_node + + +def render_char_length(generator: Any, expression: exp.Length) -> str: + """Render exp.Length as CHAR_LENGTH(this).""" + return f"CHAR_LENGTH({generator.sql(expression, 'this')})" + + +def render_charindex(generator: Any, expression: exp.StrPosition) -> str: + """Render exp.StrPosition as CHARINDEX(substr, this[, position]).""" + this_sql = generator.sql(expression, "this") + substr_sql = generator.sql(expression, "substr") + position = expression.args.get("position") + if position is not None: + return f"CHARINDEX({substr_sql}, {this_sql}, {generator.sql(position)})" + return f"CHARINDEX({substr_sql}, {this_sql})" + + +def render_ilike(generator: Any, expression: exp.ILike) -> str: + """Render case-insensitive ILIKE using LOWER(this) LIKE LOWER(expression).""" + return f"LOWER({generator.sql(expression, 'this')}) LIKE LOWER({generator.sql(expression, 'expression')})" diff --git a/sqlspec/utils/fixtures.py b/sqlspec/utils/fixtures.py index 17b834f42..a1e0b378c 100644 --- a/sqlspec/utils/fixtures.py +++ b/sqlspec/utils/fixtures.py @@ -8,6 +8,7 @@ import array import base64 import gzip +import importlib import os import re import secrets @@ -19,7 +20,7 @@ from pathlib import Path from typing import TYPE_CHECKING, Any, Final, NamedTuple -from sqlglot import exp +from sqlglot import Dialect, exp from sqlspec import dialects as _dialects from sqlspec.core.parameters import TypedParameter @@ -78,7 +79,9 @@ "oracle", "postgres", "postgresql", + "sqlanywhere", "sqlite", + "sybase", "tsql", }) _MYSQL_DIALECTS: Final["frozenset[str]"] = frozenset({"mariadb", "mysql"}) @@ -90,6 +93,7 @@ "blob", "bytea", "image", + "long binary", "long raw", "longblob", "mediumblob", @@ -117,6 +121,8 @@ "mysql": frozenset({"blob", "geometry", "json", "longblob", "mediumblob", "point", "tinyblob", "vector"}), "oracle": frozenset({"bfile", "blob", "clob", "long", "long raw", "nclob", "vector", "xmltype"}), "spanner": frozenset({"json", "tokenlist"}), + "sqlanywhere": frozenset({"long binary", "long nvarchar", "long varchar", "xml"}), + "sybase": frozenset({"image", "long binary", "long nvarchar", "long varchar", "text", "unitext"}), "tsql": frozenset({"geography", "geometry", "image", "ntext", "text", "xml"}), } _IDENTITY_GENERATION_VALUES: Final["frozenset[str]"] = frozenset({"always", "by default", "by default on null"}) @@ -676,6 +682,8 @@ def _sqlglot_dialect(dialect: "DialectType", family: str) -> "DialectType": """ if family in {"postgres", "mysql", "tsql"}: return family + if family in {"sybase", "sqlanywhere"} and family not in Dialect.classes: + importlib.import_module("sqlspec.dialects.sybase") return dialect @@ -883,11 +891,15 @@ def _dialect_name(dialect: "DialectType") -> str: def _dialect_family(dialect_name: str) -> str: - """Return ``postgres`` or ``mysql`` for those dialect families, otherwise the dialect name.""" + """Return canonical dialect family for a dialect name.""" if dialect_name in _POSTGRES_DIALECTS: return "postgres" if dialect_name in _MYSQL_DIALECTS: return "mysql" + if dialect_name == "ase": + return "sybase" + if dialect_name == "sa": + return "sqlanywhere" return dialect_name @@ -1598,17 +1610,17 @@ def _column_value_decoder(family: str, data_type: str) -> "Callable[[Any], Any] return encode_json if family in _JSON_VALUE_FAMILIES else None if family == "sqlite": return None - if data_type.startswith(("timestamp", "datetime")): + if data_type.startswith(("timestamp", "datetime", "bigdatetime", "smalldatetime", "datetimetz")): return _decode_datetime if data_type == "date": return _decode_datetime if family == "oracle" else _decode_date - if data_type.startswith("time") and family != "mysql": + if data_type.startswith(("time", "bigtime")) and family != "mysql": return _decode_time if data_type.startswith("interval"): return _decode_duckdb_interval if family == "duckdb" else _decode_interval - if data_type.startswith(("numeric", "decimal")): + if data_type.startswith(("numeric", "decimal", "money", "smallmoney")): return _decode_decimal - if data_type == "uuid": + if data_type in {"uuid", "uniqueidentifier"}: return _decode_uuid return None @@ -1716,14 +1728,16 @@ def _table_insert_statement( def _identity_insert_statements( dialect: "DialectType", family: str, table: str, columns: "list[str]", table_columns: "list[_TableColumn]" ) -> "tuple[str, str] | None": - """Return the SQL Server statements that allow and then disallow explicit identity values. + """Return the SQL Server and Sybase ASE statements that allow and then disallow explicit identity values. - SQL Server rejects values for identity columns unless ``IDENTITY_INSERT`` is on for + SQL Server and Sybase ASE reject values for identity columns unless ``IDENTITY_INSERT`` is on for the table, so inserts that include an identity column run between these statements. Other dialects return ``None``. """ inserted = set(columns) - if family != "tsql" or not any(column.is_identity and column.name in inserted for column in table_columns): + if family not in {"tsql", "sybase"} or not any( + column.is_identity and column.name in inserted for column in table_columns + ): return None table_sql = _quoted_table(table).sql(dialect=_sqlglot_dialect(dialect, family)) return f"SET IDENTITY_INSERT {table_sql} ON", f"SET IDENTITY_INSERT {table_sql} OFF" @@ -1992,10 +2006,48 @@ def _sequence_resync_statements( return statements +def _resync_sybase_sequences_sync(driver: "SyncDriverAdapterBase", table: str, columns: "list[_TableColumn]") -> None: + """Resynchronize Sybase ASE identity columns for a table synchronously.""" + for column in columns: + if column.is_identity: + quoted_col = _quoted_identifier_sql(column.name) + query = f"SELECT coalesce(max({quoted_col}), 0) FROM {_quoted_table_name(table)}" + max_val = driver.select_value(query) + next_val = int(max_val or 0) + if next_val > 0: + escaped_table = table.replace("'", "''") + driver.execute(f"EXEC sp_chgattribute '{escaped_table}', 'identity_burn_max', 0, '{next_val}'") + + +def _resync_sqlanywhere_sequences_sync( + driver: "SyncDriverAdapterBase", table: str, columns: "list[_TableColumn]" +) -> None: + """Resynchronize SAP SQL Anywhere identity columns for a table synchronously.""" + for column in columns: + if column.is_identity: + quoted_col = _quoted_identifier_sql(column.name) + query = f"SELECT coalesce(max({quoted_col}), 0) FROM {_quoted_table_name(table)}" + max_val = driver.select_value(query) + next_val = int(max_val or 0) + schema_part, _, table_part = table.rpartition(".") + escaped_table = table_part.replace("'", "''") + escaped_schema = schema_part.replace("'", "''") + schema_sql = f"'{escaped_schema}'" if schema_part else "NULL" + driver.execute(f"CALL sa_reset_identity('{escaped_table}', {schema_sql}, {next_val})") + + def _resync_table_sequences_sync( driver: "SyncDriverAdapterBase", family: str, dialect_name: str, table: str, columns: "list[_TableColumn]" ) -> None: """Resynchronize sequences, identity columns, or auto-increment counters for a table synchronously.""" + if family == "sybase": + _resync_sybase_sequences_sync(driver, table, columns) + return + + if family == "sqlanywhere": + _resync_sqlanywhere_sequences_sync(driver, table, columns) + return + if family == "db2": quoted_table = _quoted_table_name(table) for column in columns: @@ -2092,10 +2144,50 @@ def _resync_table_sequences_sync( driver.execute(statement, parameters) +async def _resync_sybase_sequences_async( + driver: "AsyncDriverAdapterBase", table: str, columns: "list[_TableColumn]" +) -> None: + """Resynchronize Sybase ASE identity columns for a table asynchronously.""" + for column in columns: + if column.is_identity: + quoted_col = _quoted_identifier_sql(column.name) + query = f"SELECT coalesce(max({quoted_col}), 0) FROM {_quoted_table_name(table)}" + max_val = await driver.select_value(query) + next_val = int(max_val or 0) + if next_val > 0: + escaped_table = table.replace("'", "''") + await driver.execute(f"EXEC sp_chgattribute '{escaped_table}', 'identity_burn_max', 0, '{next_val}'") + + +async def _resync_sqlanywhere_sequences_async( + driver: "AsyncDriverAdapterBase", table: str, columns: "list[_TableColumn]" +) -> None: + """Resynchronize SAP SQL Anywhere identity columns for a table asynchronously.""" + for column in columns: + if column.is_identity: + quoted_col = _quoted_identifier_sql(column.name) + query = f"SELECT coalesce(max({quoted_col}), 0) FROM {_quoted_table_name(table)}" + max_val = await driver.select_value(query) + next_val = int(max_val or 0) + schema_part, _, table_part = table.rpartition(".") + escaped_table = table_part.replace("'", "''") + escaped_schema = schema_part.replace("'", "''") + schema_sql = f"'{escaped_schema}'" if schema_part else "NULL" + await driver.execute(f"CALL sa_reset_identity('{escaped_table}', {schema_sql}, {next_val})") + + async def _resync_table_sequences_async( driver: "AsyncDriverAdapterBase", family: str, dialect_name: str, table: str, columns: "list[_TableColumn]" ) -> None: """Resynchronize sequences, identity columns, or auto-increment counters for a table asynchronously.""" + if family == "sybase": + await _resync_sybase_sequences_async(driver, table, columns) + return + + if family == "sqlanywhere": + await _resync_sqlanywhere_sequences_async(driver, table, columns) + return + if family == "db2": quoted_table = _quoted_table_name(table) for column in columns: diff --git a/tests/unit/adapters/test_sqlanywhere.py b/tests/unit/adapters/test_sqlanywhere.py new file mode 100644 index 000000000..06da062cb --- /dev/null +++ b/tests/unit/adapters/test_sqlanywhere.py @@ -0,0 +1,489 @@ +"""Unit tests for the SAP SQL Anywhere (sqlanydb) adapter.""" + +from typing import Any, cast +from unittest.mock import MagicMock + +import pytest + +from sqlspec.adapters.sqlanywhere import ( + DEFAULT_SQLANYWHERE_PORT, + SqlanywhereConfig, + SqlanywhereConnection, + SqlanywhereConnectionPool, + SqlanywhereCursor, + SqlanywhereDataDictionary, + SqlanywhereDriver, + SqlanywhereExceptionHandler, + SqlanywhereSyncDataDictionary, + build_connection_config, + create_mapped_exception, + default_statement_config, + driver_profile, +) +from sqlspec.adapters.sqlanywhere.adk import SqlanywhereADKConfig, SqlanywhereADKMemoryStore, SqlanywhereADKStore +from sqlspec.adapters.sqlanywhere.events import SqlanywhereEventQueueStore, SqlanywhereSyncEventQueueStore +from sqlspec.adapters.sqlanywhere.litestar import SqlanywhereStore +from sqlspec.adapters.sqlanywhere.migrations import SqlanywhereSyncMigrationTracker +from sqlspec.core import SQL, ParameterStyle +from sqlspec.data_dictionary.dialects.sqlanywhere import SqlanywhereVersionInfo +from sqlspec.exceptions import ( + CheckViolationError, + DatabaseConnectionError, + DataError, + DeadlockError, + ForeignKeyViolationError, + MissingDependencyError, + NotFoundError, + NotNullViolationError, + OperationalError, + PermissionDeniedError, + SerializationConflictError, + SQLParsingError, + UniqueViolationError, +) + + +class FakeSqlanydbError(Exception): + """Fake sqlanydb Error for SQL Anywhere unit tests.""" + + +class FakeCursor: + """Minimal DB-API 2.0 cursor for SQL Anywhere unit tests.""" + + def __init__( + self, rows: "list[Any] | None" = None, description: "list[tuple[str, ...]] | None" = None, rowcount: int = -1 + ) -> None: + self.rows = rows or [] + self.description = description + self.rowcount = rowcount + self.closed = False + self.calls: list[tuple[str, Any]] = [] + self.many_calls: list[tuple[str, Any]] = [] + self.fetchmany_sizes: list[int] = [] + + def execute(self, sql: str, parameters: Any = None) -> None: + self.calls.append((sql, parameters)) + + def executemany(self, sql: str, parameters: Any = None) -> None: + self.many_calls.append((sql, parameters)) + + def fetchall(self) -> "list[Any]": + return self.rows + + def fetchmany(self, size: int) -> "list[Any]": + self.fetchmany_sizes.append(size) + chunk = self.rows[:size] + self.rows = self.rows[size:] + return chunk + + def fetchone(self) -> Any: + return self.rows[0] if self.rows else None + + def close(self) -> None: + self.closed = True + + +class FakeConnection: + """Minimal sqlanydb connection for SQL Anywhere unit tests.""" + + def __init__(self, cursor: "FakeCursor | None" = None) -> None: + self.cursor_obj = cursor or FakeCursor() + self.closed = False + self.commits = 0 + self.rollbacks = 0 + + def cursor(self, *args: Any, **kwargs: Any) -> FakeCursor: + self.cursor_args = args + self.cursor_kwargs = kwargs + return self.cursor_obj + + def commit(self) -> None: + self.commits += 1 + + def rollback(self) -> None: + self.rollbacks += 1 + + def close(self) -> None: + self.closed = True + + +def test_build_connection_config_normalizes_friendly_keys_and_strips_pool_keys() -> None: + """build_connection_config maps friendly keys into sqlanydb connection keys and strips pool parameters.""" + cfg = build_connection_config({ + "username": "dba", + "password": "sql", + "database": "demo", + "server": "demo17", + "database_file": "/var/sa/demo.db", + "host": "sa.internal", + "pool_min_size": 2, + "pool_max_size": 8, + "pool_timeout": 15.0, + "pool_recycle_seconds": 600, + "health_check_interval": 30.0, + "extra": {"cset": "UTF-8", "ignored": None}, + }) + + assert cfg["uid"] == "dba" + assert "username" not in cfg + assert "user" not in cfg + assert cfg["pwd"] == "sql" + assert "password" not in cfg + assert cfg["dbn"] == "demo" + assert "database" not in cfg + assert cfg["eng"] == "demo17" + assert "server" not in cfg + assert cfg["dbf"] == "/var/sa/demo.db" + assert "database_file" not in cfg + assert cfg["host"] == f"sa.internal:{DEFAULT_SQLANYWHERE_PORT}" + assert "port" not in cfg + assert cfg["cset"] == "UTF-8" + assert "pool_min_size" not in cfg + assert "pool_max_size" not in cfg + assert "pool_timeout" not in cfg + assert "pool_recycle_seconds" not in cfg + assert "health_check_interval" not in cfg + assert "extra" not in cfg + + +def test_build_connection_config_combines_host_and_explicit_port() -> None: + """build_connection_config combines host and explicit port into host:port.""" + cfg = build_connection_config({"user": "dba", "pwd": "sql", "host": "10.0.0.5", "port": 2640, "dsn": "SADemo"}) + assert cfg["uid"] == "dba" + assert cfg["pwd"] == "sql" + assert cfg["host"] == "10.0.0.5:2640" + assert "port" not in cfg + assert cfg["dsn"] == "SADemo" + + +def test_sqlanywhere_pool_checkout_release_and_close(monkeypatch: pytest.MonkeyPatch) -> None: + """SqlanywhereConnectionPool manages connection acquisition, release, and pool closure.""" + import sqlspec.adapters.sqlanywhere.pool as pool_module + + captured_kwargs: list[dict[str, Any]] = [] + fake_conn = FakeConnection(FakeCursor(rows=[(1,)], description=[("1",)])) + + class FakeSqlanydbModule: + def connect(self, **kwargs: Any) -> FakeConnection: + captured_kwargs.append(kwargs) + return fake_conn + + monkeypatch.setattr(pool_module, "_load_sqlanydb_module", lambda: FakeSqlanydbModule()) + + cfg = build_connection_config({"user": "dba", "password": "sql", "database": "demo", "host": "localhost"}) + pool = SqlanywhereConnectionPool(cfg, min_size=1, max_size=2) + conn = pool.acquire() + assert cast("object", conn) is fake_conn + assert captured_kwargs[0]["uid"] == "dba" + assert captured_kwargs[0]["pwd"] == "sql" + assert captured_kwargs[0]["dbn"] == "demo" + assert captured_kwargs[0]["host"] == "localhost:2638" + + pool.release(conn) + assert pool.size() == 1 + pool.close() + assert fake_conn.closed is True + + +def test_sqlanywhere_pool_raises_missing_dependency_when_sqlanydb_absent(monkeypatch: pytest.MonkeyPatch) -> None: + """SqlanywhereConnectionPool raises MissingDependencyError when sqlanydb is not installed.""" + import sqlspec.adapters.sqlanywhere.pool as pool_module + + def _raise_import_error() -> Any: + msg = "No module named 'sqlanydb'" + raise ImportError(msg) + + monkeypatch.setattr(pool_module, "import_module", lambda name: _raise_import_error()) + + pool = SqlanywhereConnectionPool({"uid": "dba", "pwd": "sql"}) + with pytest.raises(MissingDependencyError, match="sqlanydb"): + pool.acquire() + + +@pytest.mark.parametrize( + ("args", "expected_type"), + [ + (("23505", -193, "Primary key for table 'users' is not unique"), UniqueViolationError), + (("23505", -196, "Index 'ix_email' for table 'users' would not be unique"), UniqueViolationError), + (("23503", -194, "No primary key value for foreign key 'fk_user'"), ForeignKeyViolationError), + (("23503", -198, "Primary key for row in table 'users' is referenced"), ForeignKeyViolationError), + (("23502", -195, "Column 'email' in table 'users' cannot be NULL"), NotNullViolationError), + (("23506", -209, "Invalid value for column 'age'"), CheckViolationError), + ((None, -1091, "Constraint 'chk_age' violated"), CheckViolationError), + (("40001", -306, "Deadlock detected"), DeadlockError), + ((None, -307, "All threads are blocked"), DeadlockError), + ((None, -684, "Rollback occurred due to deadlock"), DeadlockError), + ((None, -210, "User 'another' has the row in 'users' locked"), SerializationConflictError), + (("42000", -131, "Syntax error near 'SELEC'"), SQLParsingError), + (("42S02", -141, "Table 'missing_tbl' not found"), NotFoundError), + (("42S22", -143, "Column 'missing_col' not found"), NotFoundError), + (("08001", -100, "Database server not found"), DatabaseConnectionError), + (("28000", -103, "Invalid user ID or password"), DatabaseConnectionError), + ((None, -83, "Specified database not found"), DatabaseConnectionError), + ((None, -308, "Connection was terminated"), DatabaseConnectionError), + ((None, -832, "Connection error"), DatabaseConnectionError), + ((None, -101, "Not connected to a database"), OperationalError), + (("22018", -157, "Cannot convert 'abc' to a numeric"), DataError), + (("22003", -158, "Value 999999 out of range for destination"), DataError), + (("42501", -121, "Permission denied: you do not have permission"), PermissionDeniedError), + ], +) +def test_create_mapped_exception_maps_sqlanywhere_sqlcode_and_sqlstate( + args: tuple[Any, ...], expected_type: type[Exception] +) -> None: + """create_mapped_exception maps SQL Anywhere negative SQLCODEs and SQLSTATEs to SQLSpec exceptions.""" + err = FakeSqlanydbError(*args) + mapped = create_mapped_exception(err) + assert isinstance(mapped, expected_type) + + +def test_sqlanywhere_driver_profile_and_statement_config() -> None: + """Sqlanywhere driver profile uses QMARK execution and sqlanywhere dialect.""" + assert driver_profile.name == "sqlanywhere" + assert driver_profile.default_style == ParameterStyle.QMARK + assert driver_profile.default_execution_style == ParameterStyle.QMARK + assert driver_profile.default_dialect == "sqlanywhere" + assert default_statement_config.dialect == "sqlanywhere" + + +def test_sqlanywhere_cursor_context_manager_closes_cursor() -> None: + """SqlanywhereCursor creates and closes the underlying DB-API cursor.""" + fake_cursor = FakeCursor() + fake_conn = FakeConnection(fake_cursor) + with SqlanywhereCursor(cast("SqlanywhereConnection", fake_conn)) as cursor: + assert cursor is fake_cursor + assert fake_cursor.closed is True + + +def test_sqlanywhere_exception_handler_defers_mapped_exception() -> None: + """SqlanywhereExceptionHandler captures DB-API errors into pending_exception.""" + handler = SqlanywhereExceptionHandler() + with handler: + raise FakeSqlanydbError("23505", -193, "Primary key not unique") + assert isinstance(handler.pending_exception, UniqueViolationError) + + +def test_sqlanywhere_driver_execute_many_script_and_transactions() -> None: + """SqlanywhereDriver executes SELECT, DML, execute_many, execute_script, and transactions.""" + select_cursor = FakeCursor(rows=[(1, "Ada")], description=[("id",), ("name",)], rowcount=1) + conn = FakeConnection(select_cursor) + driver = SqlanywhereDriver(cast("SqlanywhereConnection", conn)) + + select_result = driver.execute(SQL("SELECT id, name FROM DBA.users WHERE id = ?", 1)) + assert select_result.get_data() == [{"id": 1, "name": "Ada"}] + assert select_cursor.calls[0][1] == (1,) + + dml_cursor = FakeCursor(rows=[], description=None, rowcount=2) + conn.cursor_obj = dml_cursor + many_result = driver.execute_many(SQL("INSERT INTO DBA.users (id, name) VALUES (?, ?)"), [(1, "Ada"), (2, "Grace")]) + assert many_result.rows_affected == 2 + assert len(dml_cursor.many_calls) == 1 + + script_cursor = FakeCursor(rows=[], description=None, rowcount=1) + conn.cursor_obj = script_cursor + script_result = driver.execute_script("CREATE TABLE IF NOT EXISTS DBA.t1 (id INT); INSERT INTO DBA.t1 VALUES (1);") + assert script_result.total_statements == 2 + + driver.begin() + driver.commit() + assert conn.commits == 1 + + driver.begin() + driver.rollback() + assert conn.rollbacks == 1 + + +def test_sqlanywhere_driver_select_to_arrow() -> None: + """SqlanywhereDriver.select_to_arrow streams rows into a PyArrow table.""" + cursor = FakeCursor(rows=[(1, "Ada"), (2, "Grace")], description=[("id",), ("name",)], rowcount=2) + conn = FakeConnection(cursor) + driver = SqlanywhereDriver(cast("SqlanywhereConnection", conn)) + + arrow_result = driver.select_to_arrow("SELECT id, name FROM DBA.users") + assert arrow_result.rows_affected == 2 + assert arrow_result.to_dict() == [{"id": 1, "name": "Ada"}, {"id": 2, "name": "Grace"}] + + +def test_sqlanywhere_data_dictionary_version_and_metadata() -> None: + """SqlanywhereSyncDataDictionary parses version banners and builds table/column/index/FK metadata.""" + dd = SqlanywhereSyncDataDictionary() + assert SqlanywhereDataDictionary is SqlanywhereSyncDataDictionary + + fake_driver = MagicMock(spec=SqlanywhereDriver) + fake_driver.select_one_or_none.return_value = {"version_string": "17.0.11.7458"} + fake_driver.select_value_or_none.return_value = "DBA" + + def _select_side_effect(query: Any, *args: Any, **kwargs: Any) -> list[dict[str, Any]]: + _ = (args, kwargs) + sql_text = str(query) + if "SYS.SYSTABCOL" in sql_text or "SYS.SYSCOLUMN" in sql_text: + return [ + { + "schema_name": "DBA", + "table_name": "users", + "column_name": "id", + "data_type": "integer", + "is_nullable": 0, + "column_default": "autoincrement", + "ordinal_position": 1, + "max_length": 4, + "numeric_scale": 0, + "is_primary": 1, + "is_unique": 1, + "is_identity": 1, + "identity_generation": "BY DEFAULT", + "is_generated": 0, + }, + { + "schema_name": "DBA", + "table_name": "users", + "column_name": "payload", + "data_type": "long nvarchar", + "is_nullable": 1, + "column_default": None, + "ordinal_position": 2, + "max_length": 2147483647, + "numeric_scale": 0, + "is_primary": 0, + "is_unique": 0, + "is_identity": 0, + "identity_generation": None, + "is_generated": 0, + }, + ] + if "SYS.SYSTAB" in sql_text and "SYS.SYSIDX" not in sql_text: + return [{"schema_name": "DBA", "table_name": "users", "table_type": "BASE TABLE"}] + return [] + + fake_driver.select.side_effect = _select_side_effect + + version = dd.get_version(fake_driver) + assert isinstance(version, SqlanywhereVersionInfo) + assert (version.major, version.minor, version.patch, version.build_number) == (17, 0, 11, "7458") + assert dd.get_feature_flag(fake_driver, "supports_cte") is True + assert dd.get_feature_flag(fake_driver, "supports_merge") is True + assert dd.get_optimal_type(fake_driver, "uuid") == "UNIQUEIDENTIFIER" + assert dd.get_optimal_type(fake_driver, "blob") == "LONG BINARY" + + tables = dd.get_tables(fake_driver, schema="DBA") + assert len(tables) == 1 + assert tables[0]["table_name"] == "users" + + columns = dd.get_columns(fake_driver, table="users", schema="DBA") + assert len(columns) == 2 + assert columns[0]["identity_generation"] == "BY DEFAULT" + assert columns[1]["data_type"] == "long nvarchar" + + +def test_sqlanywhere_migration_tracker_and_extension_stores() -> None: + """SqlanywhereSyncMigrationTracker, SqlanywhereStore, SqlanywhereSyncEventQueueStore, and SqlanywhereADKStore emit SQL Anywhere DDL.""" + tracker = SqlanywhereSyncMigrationTracker(version_table_name="DBA.ddl_migrations") + ddl_text = tracker._idempotent_tracking_table_ddl_text() + assert "CREATE TABLE IF NOT EXISTS" in ddl_text + assert "VARCHAR(32)" in ddl_text + assert "TIMESTAMP" in ddl_text + assert "CURRENT UTC TIMESTAMP" in ddl_text + assert "LONG VARCHAR" in ddl_text + assert "NVARCHAR(MAX)" not in ddl_text + + config = SqlanywhereConfig( + connection_config={"user": "dba", "password": "sql", "database": "demo"}, + extension_config={ + "litestar": {"session_table": "litestar_session", "manage_schema": False}, + "events": {"queue_table": "event_queue"}, + "adk": { + "session_table": "adk_sessions", + "events_table": "adk_events", + "memory_table": "adk_memory", + "owner_id_column": "tenant_id INTEGER NOT NULL REFERENCES DBA.tenants(id)", + }, + }, + ) + + litestar_store = SqlanywhereStore(config) + create_litestar_sql = litestar_store._table_ddl() + assert "CREATE TABLE IF NOT EXISTS" in create_litestar_sql + assert "LONG BINARY" in create_litestar_sql + assert "TIMESTAMP" in create_litestar_sql + drop_litestar_sql = litestar_store._drop_table_sql() + assert "DROP TABLE IF EXISTS" in drop_litestar_sql[0] + + event_store = SqlanywhereSyncEventQueueStore(config) + assert SqlanywhereEventQueueStore is SqlanywhereSyncEventQueueStore + event_statements = event_store.create_statements() + assert "LONG VARCHAR" in event_statements[0] + assert "TIMESTAMP" in event_statements[0] + assert "CURRENT UTC TIMESTAMP" in event_statements[0] + + adk_store = SqlanywhereADKStore(config) + assert isinstance(adk_store.config, SqlanywhereConfig) + session_ddl = adk_store._sessions_table_ddl() + events_ddl = adk_store._events_table_ddl() + assert "CREATE TABLE IF NOT EXISTS" in session_ddl + assert "LONG VARCHAR" in session_ddl + assert "tenant_id INTEGER NOT NULL REFERENCES DBA.tenants(id)" in session_ddl + assert "LONG VARCHAR" in events_ddl + assert "LONG BINARY" in events_ddl + assert "BIT NULL" in events_ddl + + memory_store = SqlanywhereADKMemoryStore(config) + memory_ddl = memory_store._memory_table_ddl() + assert "CREATE TABLE IF NOT EXISTS" in memory_ddl + assert "LONG VARCHAR" in memory_ddl + assert "TIMESTAMP" in memory_ddl + assert isinstance(SqlanywhereADKConfig(), dict) + + +@pytest.mark.anyio +async def test_sqlanywhere_litestar_and_adk_stores_roundtrip_with_mocked_connection( + monkeypatch: pytest.MonkeyPatch, +) -> None: + """SqlanywhereStore, SqlanywhereADKStore, and SqlanywhereADKMemoryStore execute operations cleanly.""" + config = SqlanywhereConfig(connection_config={"user": "dba", "password": "sql", "database": "demo"}) + litestar_store = SqlanywhereStore(config) + + monkeypatch.setattr(SqlanywhereStore, "_get", MagicMock(return_value=b"payload")) + monkeypatch.setattr(SqlanywhereStore, "_set", MagicMock()) + monkeypatch.setattr(SqlanywhereStore, "_delete", MagicMock()) + monkeypatch.setattr(SqlanywhereStore, "_exists", MagicMock(return_value=True)) + monkeypatch.setattr(SqlanywhereStore, "_expires_in", MagicMock(return_value=90)) + monkeypatch.setattr(SqlanywhereStore, "_delete_expired", MagicMock(return_value=3)) + + await litestar_store.set("sess-1", "payload", expires_in=90) + assert await litestar_store.get("sess-1") == b"payload" + assert await litestar_store.exists("sess-1") is True + assert await litestar_store.expires_in("sess-1") == 90 + assert await litestar_store.delete_expired() == 3 + await litestar_store.delete("sess-1") + + captured: list[tuple[str, tuple[Any, ...]]] = [] + + def _capture(_self: SqlanywhereADKStore, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + captured.append((sql, tuple(params))) + return [] + + monkeypatch.setattr(SqlanywhereADKStore, "_execute_fetchall", _capture) + adk_store = SqlanywhereADKStore(config) + adk_store.list_sessions("app", "u1", order_by="create_time", descending=False, limit=10, offset=20) + sql, params = captured[0] + normalized_sql = " ".join(sql.split()) + assert "ORDER BY create_time ASC, id ASC LIMIT ? OFFSET ?" in normalized_sql + assert params == ("app", "u1", 10, 20) + + monkeypatch.setattr(SqlanywhereADKMemoryStore, "insert_memory_entries", MagicMock(return_value=1)) + memory_store = SqlanywhereADKMemoryStore(config) + assert memory_store.insert_memory_entries([]) == 1 + + +def test_sqlanywhere_cursor_handles_cursor_without_close() -> None: + """SqlanywhereCursor safely handles raw cursors that lack a close() method.""" + + class CursorWithoutClose: + pass + + class ConnWithBareCursor: + def cursor(self) -> CursorWithoutClose: + return CursorWithoutClose() + + with SqlanywhereCursor(cast("SqlanywhereConnection", ConnWithBareCursor())) as cursor: + assert isinstance(cursor, CursorWithoutClose) diff --git a/tests/unit/adapters/test_sybase.py b/tests/unit/adapters/test_sybase.py new file mode 100644 index 000000000..ef23069b8 --- /dev/null +++ b/tests/unit/adapters/test_sybase.py @@ -0,0 +1,521 @@ +"""Unit tests for the Sybase ASE (pymssql over FreeTDS TDS 5.0) adapter.""" + +import os +import socket +import threading +from datetime import UTC, datetime +from typing import Any, cast +from unittest.mock import AsyncMock, MagicMock + +import pytest + +from sqlspec.adapters.sybase import ( + DEFAULT_SYBASE_CONN_PROPERTIES, + DEFAULT_SYBASE_TDS_VERSION, + SybaseConfig, + SybaseConnection, + SybaseConnectionPool, + SybaseCursor, + SybaseDataDictionary, + SybaseDriver, + SybaseExceptionHandler, + SybaseSyncDataDictionary, + build_connection_config, + create_mapped_exception, + default_statement_config, + driver_profile, +) +from sqlspec.adapters.sybase._typing import SybaseRawCursor +from sqlspec.adapters.sybase.adk import SybaseADKConfig, SybaseADKMemoryStore, SybaseADKStore +from sqlspec.adapters.sybase.events import SybaseEventQueueStore, SybaseSyncEventQueueStore +from sqlspec.adapters.sybase.litestar import SybaseStore +from sqlspec.adapters.sybase.migrations import SybaseSyncMigrationTracker +from sqlspec.core import SQL, ParameterStyle +from sqlspec.data_dictionary.dialects.sybase import SybaseVersionInfo +from sqlspec.exceptions import ( + CheckViolationError, + DatabaseConnectionError, + DataError, + DeadlockError, + ForeignKeyViolationError, + ImproperConfigurationError, + NotFoundError, + NotNullViolationError, + OperationalError, + PermissionDeniedError, + QueryTimeoutError, + SQLParsingError, + TransactionError, + UniqueViolationError, +) + + +class FakeSybaseError(Exception): + """Fake pymssql error for Sybase unit tests.""" + + +class FakeCursor: + """Minimal DB-API cursor for Sybase unit tests.""" + + def __init__( + self, rows: "list[Any] | None" = None, description: "list[tuple[str, ...]] | None" = None, rowcount: int = -1 + ) -> None: + self.rows = rows or [] + self.description = description + self.rowcount = rowcount + self.closed = False + self.calls: list[tuple[str, Any]] = [] + self.many_calls: list[tuple[str, Any]] = [] + self.fetchmany_sizes: list[int] = [] + + def execute(self, sql: str, parameters: Any = None) -> None: + self.calls.append((sql, parameters)) + + def executemany(self, sql: str, parameters: Any = None) -> None: + self.many_calls.append((sql, parameters)) + + def fetchall(self) -> "list[Any]": + return self.rows + + def fetchmany(self, size: int) -> "list[Any]": + self.fetchmany_sizes.append(size) + chunk = self.rows[:size] + self.rows = self.rows[size:] + return chunk + + def fetchone(self) -> Any: + return self.rows[0] if self.rows else None + + def close(self) -> None: + self.closed = True + + +class FakeConnection: + """Minimal pymssql connection for Sybase unit tests.""" + + def __init__(self, cursor: "FakeCursor | None" = None) -> None: + self.cursor_obj = cursor or FakeCursor() + self.closed = False + self.commits = 0 + self.rollbacks = 0 + self.autocommit_values: list[bool] = [] + self.autocommit_state = True + + def cursor(self, *args: Any, **kwargs: Any) -> FakeCursor: + self.cursor_args = args + self.cursor_kwargs = kwargs + return self.cursor_obj + + def commit(self) -> None: + if not self.autocommit_state: + self.commits += 1 + + def rollback(self) -> None: + if not self.autocommit_state: + self.rollbacks += 1 + + def close(self) -> None: + self.closed = True + + def autocommit(self, value: bool) -> None: + self.autocommit_values.append(value) + self.autocommit_state = value + + +def test_build_connection_config_defaults_and_normalization() -> None: + """build_connection_config sets Sybase ASE defaults and strips tds_version='5.0' for pool TDSVER negotiation.""" + cfg = build_connection_config({ + "host": "ase.internal", + "username": "sa", + "db": "pubs2", + "tds_version": "5.0", + "extra": {"appname": "sqlspec_sybase", "ignored": None}, + }) + + assert cfg["server"] == "ase.internal" + assert "host" not in cfg + assert cfg["port"] == 5000 + assert cfg["user"] == "sa" + assert "username" not in cfg + assert cfg["database"] == "pubs2" + assert "db" not in cfg + assert cfg["charset"] == "UTF-8" + assert cfg["autocommit"] is True + assert cfg["conn_properties"] == DEFAULT_SYBASE_CONN_PROPERTIES + assert cfg["_sybase_tds_version"] == DEFAULT_SYBASE_TDS_VERSION + assert "tds_version" not in cfg + assert cfg["appname"] == "sqlspec_sybase" + assert "extra" not in cfg + + +def test_sybase_pool_requires_tdsver_or_server_alias(monkeypatch: pytest.MonkeyPatch) -> None: + """SybaseConnectionPool requires either TDSVER='5.0' or a server_alias at process start.""" + import sqlspec.adapters.sybase.pool as pool_module + + fake_conn = FakeConnection() + + class FakeModule: + def connect(self, **kwargs: Any) -> FakeConnection: + return fake_conn + + monkeypatch.setattr(pool_module, "pymssql", FakeModule()) + + monkeypatch.delenv("TDSVER", raising=False) + cfg_invalid = build_connection_config({"server": "ase.example.test", "user": "sa", "database": "master"}) + with pytest.raises(ImproperConfigurationError): + SybaseConnectionPool(cfg_invalid) + + monkeypatch.setenv("TDSVER", "5.0") + cfg_env = build_connection_config({"server": "ase.example.test", "user": "sa", "database": "master"}) + pool_env = SybaseConnectionPool(cfg_env) + conn = pool_env.acquire() + pool_env.release(conn) + assert cast("object", conn) is fake_conn + assert os.environ.get("TDSVER") == "5.0" + + monkeypatch.delenv("TDSVER", raising=False) + cfg_alias = build_connection_config({"server_alias": "SYBASE_ASE", "user": "sa", "database": "master"}) + pool_alias = SybaseConnectionPool(cfg_alias) + conn_alias = pool_alias.acquire() + pool_alias.release(conn_alias) + assert cast("object", conn_alias) is fake_conn + + +def test_sybase_pool_transmits_native_tds_50_login_packet_over_loopback(monkeypatch: pytest.MonkeyPatch) -> None: + """SybaseConnectionPool.new_connection transmits a native 619-byte TDS 5.0 login packet via FreeTDS.""" + monkeypatch.setenv("TDSVER", "5.0") + received_packets: list[bytes] = [] + ready = threading.Event() + + def _listener(server_sock: socket.socket) -> None: + ready.set() + try: + conn, _ = server_sock.accept() + with conn: + conn.settimeout(2.0) + data = conn.recv(4096) + received_packets.append(data) + except OSError: + return + + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as server_sock: + server_sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + server_sock.bind(("127.0.0.1", 0)) + port = server_sock.getsockname()[1] + server_sock.listen(1) + + listener_thread = threading.Thread(target=_listener, args=(server_sock,), daemon=True) + listener_thread.start() + ready.wait(timeout=2.0) + + cfg = build_connection_config({ + "server": "127.0.0.1", + "port": port, + "user": "sa", + "password": "secret", + "database": "master", + "login_timeout": 1, + "timeout": 1, + }) + pool = SybaseConnectionPool(cfg) + with pytest.raises(Exception): + pool.new_connection() + + listener_thread.join(timeout=2.0) + + assert len(received_packets) == 1 + packet = received_packets[0] + assert len(packet) >= 500 + assert packet[0] == 0x02 + assert 0xE2 in packet + + +@pytest.mark.parametrize( + ("code", "message", "expected_type"), + [ + (2601, "Attempt to insert duplicate key row in object 'users' with unique index (2601)", UniqueViolationError), + (2627, "Violation of UNIQUE KEY constraint (2627)", UniqueViolationError), + (546, "Foreign key constraint violation occurred, dbname = 'pubs2' (546)", ForeignKeyViolationError), + (547, "Dependent foreign key constraint violation (547)", ForeignKeyViolationError), + (547, "The UPDATE statement conflicted with the CHECK constraint (547)", CheckViolationError), + (548, "Check constraint violation occurred, dbname = 'pubs2' (548)", CheckViolationError), + (233, "The column id in table users does not allow null values (233)", NotNullViolationError), + (515, "Attempt to insert NULL value into column 'name' (515)", NotNullViolationError), + (1205, "Your server command (family id #0, process id #14) encountered a deadlock (1205)", DeadlockError), + (12205, "Could not acquire a lock within the specified wait period (12205)", QueryTimeoutError), + (102, "Incorrect syntax near 'FROM' (102)", SQLParsingError), + (156, "Incorrect syntax near the keyword 'SELECT' (156)", SQLParsingError), + (11021, "Function FOO invoked with wrong number or type of argument(s) (11021)", SQLParsingError), + (207, "Invalid column name 'missing_col' (207)", NotFoundError), + (208, "users not found. Specify owner.objectname (208)", NotFoundError), + (4002, "Login failed (4002)", DatabaseConnectionError), + (18456, "Login failed for user 'sa' (18456)", DatabaseConnectionError), + (20002, "Adaptive Server connection failed (20002)", DatabaseConnectionError), + (20047, "DBPROCESS is dead or not enabled (20047)", OperationalError), + ( + 2762, + "The 'CREATE TABLE' command is not allowed within a multi-statement transaction (2762)", + OperationalError, + ), + (1105, "Can't allocate space for object 'syslogs' in database 'pubs2' (1105)", OperationalError), + (247, "Arithmetic overflow during explicit conversion of INT value (247)", DataError), + (220, "Arithmetic overflow error for type tinyint (220)", DataError), + (168, "Scale error during implicit conversion of NUMERIC value (168)", DataError), + (3606, "Arithmetic overflow occurred (3606)", DataError), + (10330, "SELECT permission denied on object users, database pubs2, owner dbo (10330)", PermissionDeniedError), + (10331, "Permission denied, database pubs2, owner dbo (10331)", PermissionDeniedError), + (10332, "EXECUTE permission denied on object sp_proc (10332)", PermissionDeniedError), + ], +) +def test_create_mapped_exception_maps_sybase_ase_error_codes( + code: int, message: str, expected_type: type[Exception] +) -> None: + """Sybase ASE error codes map to their canonical SQLSpec exception classes.""" + mapped = create_mapped_exception(Exception(code, message)) + assert isinstance(mapped, expected_type) + + +def test_sybase_driver_profile_and_config_flags() -> None: + """SybaseConfig sets supports_transactional_ddl=False and registers the 'sybase' dialect.""" + assert SybaseConfig.supports_transactional_ddl is False + assert SybaseDriver.dialect == "sybase" + assert default_statement_config.dialect == "sybase" + assert driver_profile.default_execution_style is ParameterStyle.POSITIONAL_PYFORMAT + + +def test_sybase_driver_execute_many_script_and_transactions() -> None: + """SybaseDriver executes SELECT, DML, execute_many, execute_script, and transactions.""" + cursor = FakeCursor(rows=[(1, "Ada")], description=[("id",), ("name",)], rowcount=1) + connection = FakeConnection(cursor) + driver = SybaseDriver(cast("SybaseConnection", connection)) + + select_result = driver.execute("SELECT id, name FROM dbo.users WHERE id = ?", 1) + assert select_result.get_data() == [{"id": 1, "name": "Ada"}] + assert cursor.calls[-1] == ("SELECT id, name FROM dbo.users WHERE id = %s", (1,)) + + many_stmt = SQL( + "INSERT INTO dbo.users (id, name) VALUES (?, ?)", + [(1, "Ada"), (2, "Grace")], + statement_config=default_statement_config, + is_many=True, + ) + many_result = driver.dispatch_execute_many(cast("SybaseRawCursor", cursor), many_stmt) + assert many_result.rowcount_override == 1 + assert cursor.many_calls[-1] == ("INSERT INTO dbo.users (id, name) VALUES (%s, %s)", [(1, "Ada"), (2, "Grace")]) + + script_stmt = SQL( + "INSERT INTO dbo.users (id) VALUES (1); INSERT INTO dbo.users (id) VALUES (2);", + statement_config=default_statement_config, + is_script=True, + ) + script_result = driver.dispatch_execute_script(cast("SybaseRawCursor", cursor), script_stmt) + assert script_result.statement_count == 2 + assert script_result.successful_statements == 2 + + driver.begin() + assert driver._connection_in_transaction() is True + driver.commit() + assert driver._connection_in_transaction() is False + assert ("BEGIN TRANSACTION", None) in cursor.calls + assert ("IF @@TRANCOUNT > 0 COMMIT TRANSACTION", None) in cursor.calls + + with pytest.raises(TransactionError): + driver.create_savepoint("bad;name") + + +def test_sybase_driver_select_to_arrow_and_load_from_arrow() -> None: + """SybaseDriver converts SELECT results to Arrow and loads Arrow tables via executemany.""" + pa = pytest.importorskip("pyarrow") + cursor = FakeCursor(rows=[(1, "Ada"), (2, "Grace")], description=[("id",), ("name",)], rowcount=2) + connection = FakeConnection(cursor) + driver = SybaseDriver(cast("SybaseConnection", connection), driver_features=SybaseConfig().driver_features) + + arrow_result = driver.select_to_arrow("SELECT id, name FROM dbo.users") + assert arrow_result.rows_affected == 2 + + table = pa.table({"id": [1, 2], "name": ["Ada", "Grace"]}) + job = driver.load_from_arrow("dbo.users", table, overwrite=True) + assert job.telemetry["rows_processed"] == 2 + assert cursor.calls[-1] == ("DELETE FROM [dbo].[users]", None) + assert cursor.many_calls[-1] == ( + "INSERT INTO [dbo].[users] ([id], [name]) VALUES (%s, %s)", + [(1, "Ada"), (2, "Grace")], + ) + + +def test_sybase_data_dictionary_reflection() -> None: + """SybaseSyncDataDictionary parses ASE version strings and builds table/column/index/FK metadata.""" + assert SybaseDataDictionary is SybaseSyncDataDictionary + dd = SybaseSyncDataDictionary() + + class FakeDDDriver: + def select_one_or_none(self, _sql: Any, **_kwargs: Any) -> dict[str, Any]: + return {"version_string": "Adaptive Server Enterprise/16.0 SP04 PL04/EBF 30829 SMP/P/x86_64/Linux"} + + def select(self, _sql: Any, **kwargs: Any) -> list[dict[str, Any]]: + if kwargs.get("table_name") == "users": + return [ + { + "schema_name": "dbo", + "table_name": "users", + "column_name": "id", + "data_type": "numeric(10, 0)", + "is_nullable": False, + "column_default": None, + "ordinal_position": 1, + "max_length": 10, + "numeric_scale": 0, + "is_primary": True, + "is_unique": False, + "is_identity": True, + "identity_generation": "ALWAYS", + "is_generated": False, + }, + { + "schema_name": "dbo", + "table_name": "users", + "column_name": "bio", + "data_type": "unitext", + "is_nullable": True, + "column_default": None, + "ordinal_position": 2, + "max_length": 1073741823, + "numeric_scale": 0, + "is_primary": False, + "is_unique": False, + "is_identity": False, + "identity_generation": None, + "is_generated": False, + }, + ] + return [{"schema_name": "dbo", "table_name": "users", "table_type": "TABLE"}] + + fake_driver = cast("Any", FakeDDDriver()) + version = dd.get_version(fake_driver) + assert isinstance(version, SybaseVersionInfo) + assert (version.major, version.minor, version.patch) == (16, 0, 4) + assert dd.get_feature_flag(fake_driver, "supports_transactions") is True + assert dd.get_feature_flag(fake_driver, "supports_transactional_ddl") is False + assert dd.get_optimal_type(fake_driver, "timestamp") == "BIGDATETIME" + assert dd.get_optimal_type(fake_driver, "blob") == "IMAGE" + + tables = dd.get_tables(fake_driver, schema="dbo") + assert len(tables) == 1 + assert tables[0]["table_name"] == "users" + + columns = dd.get_columns(fake_driver, table="users", schema="dbo") + assert len(columns) == 2 + assert columns[0]["data_type"] == "numeric(10, 0)" + assert columns[0]["identity_generation"] == "ALWAYS" + assert columns[1]["data_type"] == "unitext" + + +def test_sybase_migration_tracker_and_extension_stores() -> None: + """SybaseSyncMigrationTracker, SybaseStore, SybaseSyncEventQueueStore, and SybaseADKStore emit ASE-safe SQL.""" + tracker = SybaseSyncMigrationTracker(version_table_name="dbo.ddl_migrations") + ddl_text = tracker._idempotent_tracking_table_ddl_text() + assert "IF OBJECT_ID(" in ddl_text + assert "EXECUTE(" in ddl_text + assert "VARCHAR(32)" in ddl_text + assert "BIGDATETIME" in ddl_text + assert "GETUTCDATE()" in ddl_text + assert "TEXT" in ddl_text + assert "NVARCHAR(MAX)" not in ddl_text + + config = SybaseConfig( + extension_config={ + "litestar": {"session_table": "litestar_session", "manage_schema": False}, + "events": {"queue_table": "event_queue"}, + "adk": {}, + } + ) + + litestar_store = SybaseStore(config) + litestar_ddl = litestar_store._table_ddl() + assert "IF OBJECT_ID(" in litestar_ddl + assert "IMAGE NOT NULL" in litestar_ddl + assert "BIGDATETIME" in litestar_ddl + assert "GETUTCDATE()" in litestar_ddl + + assert SybaseEventQueueStore is SybaseSyncEventQueueStore + event_store = SybaseSyncEventQueueStore(config) + event_stmts = event_store.create_statements() + assert "TEXT" in event_stmts[0] + assert "BIGDATETIME" in event_stmts[0] + assert "GETUTCDATE()" in event_stmts[0] + assert "IF OBJECT_ID(" in event_stmts[0] + + adk_store = SybaseADKStore(config) + sessions_ddl = adk_store._sessions_table_ddl() + events_ddl = adk_store._events_table_ddl() + assert "IF OBJECT_ID(" in sessions_ddl + assert "TEXT NOT NULL" in sessions_ddl + assert "BIGDATETIME" in sessions_ddl + assert "GETUTCDATE()" in sessions_ddl + assert "IF OBJECT_ID(" in events_ddl + + mem_store = SybaseADKMemoryStore(config) + mem_ddl = mem_store._memory_table_ddl() + assert "IF OBJECT_ID(" in mem_ddl + assert "TEXT NOT NULL" in mem_ddl + assert "BIGDATETIME" in mem_ddl + assert "GETUTCDATE()" in mem_ddl + + assert SybaseCursor.__slots__ == ("_as_dict", "connection", "cursor") + assert SybaseExceptionHandler.__slots__ == () + assert isinstance(SybaseADKConfig(), dict) + + +@pytest.mark.anyio +async def test_sybase_litestar_store_async_bridge(monkeypatch: pytest.MonkeyPatch) -> None: + """SybaseStore async methods bridge to sync execution via async_.""" + calls: list[str] = [] + monkeypatch.setattr(SybaseStore, "_create_table", lambda _self: calls.append("created")) + monkeypatch.setattr(SybaseStore, "reconcile_schema", AsyncMock()) + store = SybaseStore(SybaseConfig(extension_config={"litestar": {"session_table": "litestar_session"}})) + + await store.create_table() + assert calls == ["created"] + + +def test_sybase_adk_store_session_list_and_memory_operations(monkeypatch: pytest.MonkeyPatch) -> None: + """SybaseADKStore uses ROWS LIMIT ... OFFSET ... for pagination and SybaseADKMemoryStore inserts entries.""" + captured: list[tuple[str, tuple[Any, ...]]] = [] + + def _capture(_self: SybaseADKStore, sql: str, params: "tuple[Any, ...]" = ()) -> "list[Any]": + captured.append((sql, tuple(params))) + return [] + + monkeypatch.setattr(SybaseADKStore, "_execute_fetchall", _capture) + config = MagicMock() + config.extension_config = {"adk": {}} + store = SybaseADKStore(cast("Any", config)) + + store.list_sessions("app", "u1", order_by="create_time", descending=False, limit=10, offset=20) + sql, params = captured[0] + normalized_sql = " ".join(sql.split()) + assert "ORDER BY create_time ASC, id ASC ROWS LIMIT %s OFFSET %s" in normalized_sql + assert params == ("app", "u1", 10, 20) + + conn = config.provide_connection.return_value.__enter__.return_value + cursor = conn.cursor.return_value + cursor.rowcount = 1 + mem_store = SybaseADKMemoryStore(cast("Any", config)) + now = datetime.now(tz=UTC) + entry = cast( + "Any", + { + "id": "mem-1", + "session_id": "sess-1", + "app_name": "app", + "user_id": "user-1", + "event_id": "evt-1", + "timestamp": now, + "content_json": {"text": "hello"}, + "content_text": "hello", + "metadata_json": None, + "inserted_at": now, + }, + ) + inserted = mem_store.insert_memory_entries([entry]) + assert inserted == 1 diff --git a/tests/unit/builder/test_sybase_builder.py b/tests/unit/builder/test_sybase_builder.py new file mode 100644 index 000000000..936161145 --- /dev/null +++ b/tests/unit/builder/test_sybase_builder.py @@ -0,0 +1,82 @@ +"""Unit tests for SQLSpec QueryBuilder with Sybase ASE and SAP SQL Anywhere dialects.""" + +import pytest + +from sqlspec import sql +from sqlspec.core import get_cache +from sqlspec.exceptions import DialectNotSupportedError + +__all__ = () + + +def test_sybase_builder_select_limit_offset_and_optimizer_cache() -> None: + """Verify QueryBuilder with dialect='sybase' renders ROWS LIMIT/OFFSET and populates QueryCache.""" + cache = get_cache() + cache.clear() + + builder = ( + sql + .select("id", "name", dialect="sybase") + .from_("users") + .where_eq("status", "active") + .order_by("id") + .limit(10) + .offset(20) + ) + first = builder.build() + second = builder.build() + + assert "ROWS LIMIT" in first.sql + assert "OFFSET" in first.sql + assert first.sql == second.sql + assert cache.get_stats().hits >= 1 + + +def test_sqlanywhere_builder_select_limit_offset_and_optimizer_cache() -> None: + """Verify QueryBuilder with dialect='sqlanywhere' renders LIMIT/OFFSET and populates QueryCache.""" + cache = get_cache() + cache.clear() + + builder = ( + sql + .select("id", "name", dialect="sqlanywhere") + .from_("users") + .where_eq("status", "active") + .order_by("id") + .limit(10) + .offset(20) + ) + first = builder.build() + second = builder.build() + + assert "LIMIT" in first.sql + assert "OFFSET" in first.sql + assert "FETCH" not in first.sql + assert first.sql == second.sql + assert cache.get_stats().hits >= 1 + + +def test_sybase_merge_raises_dialect_not_supported() -> None: + """Verify sql.merge(dialect='sybase').build() raises DialectNotSupportedError while sqlanywhere is supported.""" + with pytest.raises(DialectNotSupportedError): + ( + sql + .merge(dialect="sybase") + .into("users", alias="t") + .using("staging_users", alias="s") + .on("t.id = s.id") + .when_matched_then_update(name="s.name") + .build() + ) + + merge_builder = ( + sql + .merge(dialect="sqlanywhere") + .into("users", alias="t") + .using("staging_users", alias="s") + .on("t.id = s.id") + .when_matched_then_update(name="s.name") + .when_not_matched_then_insert(id="s.id", name="s.name") + ) + compiled = merge_builder.build() + assert "MERGE INTO" in compiled.sql diff --git a/tests/unit/data_dictionary/test_sybase_sqlanywhere_dialects.py b/tests/unit/data_dictionary/test_sybase_sqlanywhere_dialects.py new file mode 100644 index 000000000..a2a72c5dd --- /dev/null +++ b/tests/unit/data_dictionary/test_sybase_sqlanywhere_dialects.py @@ -0,0 +1,162 @@ +"""Unit tests for Sybase ASE and SAP SQL Anywhere data dictionary configurations and SQL packs.""" + +import pytest + +from sqlspec.data_dictionary import DataDictionaryLoader, TableMetadata, VersionInfo, get_dialect_config +from sqlspec.data_dictionary.dialects import SQLANYWHERE_CONFIG, SYBASE_CONFIG +from sqlspec.data_dictionary.dialects.sqlanywhere import ( + SQLANYWHERE_METADATA_DOMAINS, + SqlanywhereVersionInfo, + build_sqlanywhere_metadata_capability_profile, + extract_sqlanywhere_version_value, + list_sqlanywhere_available_features, + merge_sqlanywhere_table_lists, + resolve_sqlanywhere_feature_flag, +) +from sqlspec.data_dictionary.dialects.sybase import ( + SYBASE_METADATA_DOMAINS, + SybaseVersionInfo, + build_sybase_metadata_capability_profile, + extract_sybase_version_value, + list_sybase_available_features, + merge_sybase_table_lists, + resolve_sybase_feature_flag, +) + + +def test_sybase_and_sqlanywhere_dialect_configs_registered() -> None: + """Both sybase and sqlanywhere dialect configs should resolve by canonical name and alias.""" + assert get_dialect_config("sybase") is SYBASE_CONFIG + assert get_dialect_config("ase") is SYBASE_CONFIG + assert get_dialect_config("sqlanywhere") is SQLANYWHERE_CONFIG + assert get_dialect_config("sa") is SQLANYWHERE_CONFIG + + assert SYBASE_CONFIG.name == "sybase" + assert SYBASE_CONFIG.default_schema == "dbo" + assert SYBASE_CONFIG.get_feature_flag("supports_cte") is False + assert SYBASE_CONFIG.get_feature_flag("supports_window_functions") is False + assert SYBASE_CONFIG.get_feature_flag("supports_merge") is False + assert SYBASE_CONFIG.get_feature_flag("supports_returning") is False + assert SYBASE_CONFIG.get_feature_flag("supports_transactional_ddl") is False + assert SYBASE_CONFIG.get_feature_flag("supports_transactions") is True + assert SYBASE_CONFIG.get_optimal_type("boolean") == "BIT" + assert SYBASE_CONFIG.get_optimal_type("timestamp") == "BIGDATETIME" + assert SYBASE_CONFIG.get_optimal_type("blob") == "IMAGE" + + assert SQLANYWHERE_CONFIG.name == "sqlanywhere" + assert SQLANYWHERE_CONFIG.default_schema == "DBA" + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_cte") is True + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_window_functions") is True + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_merge") is True + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_for_update") is True + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_skip_locked") is True + assert SQLANYWHERE_CONFIG.get_feature_flag("supports_transactional_ddl") is True + assert SQLANYWHERE_CONFIG.get_optimal_type("uuid") == "UNIQUEIDENTIFIER" + assert SQLANYWHERE_CONFIG.get_optimal_type("text") == "LONG VARCHAR" + assert SQLANYWHERE_CONFIG.get_optimal_type("blob") == "LONG BINARY" + + +def test_sybase_and_sqlanywhere_version_patterns() -> None: + """Version regexes should extract major, minor, and patch components for ASE and SQL Anywhere.""" + ase_banner = ( + "Adaptive Server Enterprise/16.0 SP04 PL04/EBF 30626 SMP/P/x86_64/" + "Enterprise Linux/ase160sp04pl04x/0/64-bit/FBO/Thu" + ) + ase_match = SYBASE_CONFIG.version_pattern.search(ase_banner) + assert ase_match is not None + assert tuple(int(part or 0) for part in ase_match.groups()) == (16, 0, 4) + + sa_banner = "17.0.11.7458" + sa_match = SQLANYWHERE_CONFIG.version_pattern.search(sa_banner) + assert sa_match is not None + assert tuple(int(part or 0) for part in sa_match.groups()) == (17, 0, 11) + + +def test_sybase_and_sqlanywhere_reflection_helpers() -> None: + """Reflection helpers should extract version values, resolve features, and merge table lists.""" + ase_version = SybaseVersionInfo(16, 0, 4, "SP04") + assert str(ase_version) == "16.0.4 (SP04)" + assert extract_sybase_version_value({"version_string": "Adaptive Server Enterprise/16.0 SP04"}) == ( + "Adaptive Server Enterprise/16.0 SP04" + ) + assert resolve_sybase_feature_flag("supports_transactions", ase_version) is True + assert resolve_sybase_feature_flag("supports_cte", ase_version) is False + assert "supports_transactions" in list_sybase_available_features() + + sa_version = SqlanywhereVersionInfo(17, 0, 11, "7458") + assert str(sa_version) == "17.0.11 (7458)" + assert extract_sqlanywhere_version_value({"version_string": "17.0.11.7458"}) == "17.0.11.7458" + assert resolve_sqlanywhere_feature_flag("supports_cte", sa_version) is True + assert resolve_sqlanywhere_feature_flag("supports_window_functions", VersionInfo(12, 0, 0)) is True + assert "supports_cte" in list_sqlanywhere_available_features() + + ordered: list[TableMetadata] = [{"schema_name": "dbo", "table_name": "users"}] + all_rows: list[TableMetadata] = [ + {"schema_name": "dbo", "table_name": "posts"}, + {"schema_name": "dbo", "table_name": "users"}, + ] + assert merge_sybase_table_lists(ordered, all_rows) == [ + {"schema_name": "dbo", "table_name": "users"}, + {"schema_name": "dbo", "table_name": "posts"}, + ] + assert merge_sqlanywhere_table_lists(ordered, all_rows) == [ + {"schema_name": "dbo", "table_name": "users"}, + {"schema_name": "dbo", "table_name": "posts"}, + ] + + sybase_profile = build_sybase_metadata_capability_profile("sybase") + assert sybase_profile.dialect == "sybase" + assert len(sybase_profile.capabilities) == len(SYBASE_METADATA_DOMAINS) + + sa_profile = build_sqlanywhere_metadata_capability_profile("sqlanywhere") + assert sa_profile.dialect == "sqlanywhere" + assert len(sa_profile.capabilities) == len(SQLANYWHERE_METADATA_DOMAINS) + + +@pytest.mark.parametrize("dialect", ["sybase", "sqlanywhere"]) +@pytest.mark.parametrize( + ("domain", "query_names"), + [ + ("version", ("current",)), + ("schemas", ("by_schema", "current")), + ("tables", ("by_schema", "all_by_schema", "names_by_schema")), + ("columns", ("by_schema", "by_table")), + ("indexes", ("by_schema", "by_table")), + ("foreign_keys", ("by_schema", "by_table")), + ("constraints", ("by_schema", "by_table")), + ("objects", ("by_schema",)), + ("views", ("by_schema",)), + ], +) +def test_all_domain_queries_load_for_sybase_and_sqlanywhere( + dialect: str, domain: str, query_names: tuple[str, ...] +) -> None: + """DataDictionaryLoader should load all 9 domains and named queries for sybase and sqlanywhere.""" + loader = DataDictionaryLoader() + assert loader.has_domain_queries(dialect, domain) is True + for query_name in query_names: + query = loader.get_domain_query(dialect, domain, query_name) + assert query.is_supported is True + assert query.query_text is not None + assert query.query_text.strip() != "" + + +def test_sybase_and_sqlanywhere_catalog_tables_referenced() -> None: + """Sybase queries should use dbo.sys* catalogs and SQL Anywhere queries should use SYS.SYS* catalogs.""" + loader = DataDictionaryLoader() + + sybase_tables = loader.get_domain_query_text("sybase", "tables", "by_schema") or "" + sybase_columns = loader.get_domain_query_text("sybase", "columns", "by_table") or "" + sybase_fks = loader.get_domain_query_text("sybase", "foreign_keys", "by_table") or "" + assert "dbo.sysobjects" in sybase_tables + assert "dbo.syscolumns" in sybase_columns + assert "dbo.systypes" in sybase_columns + assert "dbo.sysreferences" in sybase_fks + + sa_tables = loader.get_domain_query_text("sqlanywhere", "tables", "by_schema") or "" + sa_columns = loader.get_domain_query_text("sqlanywhere", "columns", "by_table") or "" + sa_fks = loader.get_domain_query_text("sqlanywhere", "foreign_keys", "by_table") or "" + assert "SYS.SYSTAB" in sa_tables + assert "SYS.SYSTABCOL" in sa_columns + assert "SYS.SYSDOMAIN" in sa_columns + assert "SYS.SYSFKEY" in sa_fks diff --git a/tests/unit/dialects/test_sqlanywhere.py b/tests/unit/dialects/test_sqlanywhere.py new file mode 100644 index 000000000..c7221a7ac --- /dev/null +++ b/tests/unit/dialects/test_sqlanywhere.py @@ -0,0 +1,116 @@ +"""Unit tests for the custom SAP SQL Anywhere sqlglot dialect.""" + +import sqlglot +from sqlglot import exp, parse_one, transpile +from sqlglot.dialects.dialect import NormalizationStrategy + +from sqlspec.dialects.sybase import SQLAnywhere, SQLAnywhereTokenizer + +__all__ = () + + +def test_sqlanywhere_dialect_registration() -> None: + """Verify SQLAnywhere dialect is registered and resolvable by name.""" + dialect = sqlglot.Dialect.get_or_raise("sqlanywhere") + assert dialect.__class__ is SQLAnywhere + assert dialect.tokenizer_class is SQLAnywhereTokenizer + assert dialect.NORMALIZATION_STRATEGY == NormalizationStrategy.CASE_SENSITIVE + + +def test_sqlanywhere_limit_and_offset_rendering() -> None: + """Verify SQLAnywhere renders trailing LIMIT n and LIMIT n OFFSET m without TOP or FETCH.""" + limit_only = transpile("SELECT id, name FROM users LIMIT 10", write="sqlanywhere")[0] + assert limit_only == "SELECT id, name FROM users LIMIT 10" + + limit_offset = transpile("SELECT id, name FROM users ORDER BY id LIMIT 10 OFFSET 20", write="sqlanywhere")[0] + assert limit_offset == "SELECT id, name FROM users ORDER BY id LIMIT 10 OFFSET 20" + + offset_only = transpile("SELECT id FROM users ORDER BY id OFFSET 5", write="sqlanywhere")[0] + assert offset_only == "SELECT id FROM users ORDER BY id OFFSET 5" + + +def test_sqlanywhere_parse_top_start_at() -> None: + """Verify Watcom-SQL SELECT TOP n START AT m parses into 0-based Limit and Offset.""" + parsed = parse_one("SELECT TOP 10 START AT 21 id, name FROM users ORDER BY id", dialect="sqlanywhere") + assert isinstance(parsed, exp.Select) + assert isinstance(parsed.args.get("limit"), exp.Limit) + assert isinstance(parsed.args.get("offset"), exp.Offset) + assert parsed.sql(dialect="sqlanywhere") == "SELECT id, name FROM users ORDER BY id LIMIT 10 OFFSET 20" + + +def test_sqlanywhere_copy_false_preserves_ast_pagination() -> None: + """Verify expression.sql('sqlanywhere', copy=False) never mutates limit or offset in place.""" + select_expr = parse_one("SELECT id FROM users ORDER BY id LIMIT 10 OFFSET 5") + first_sql = select_expr.sql(dialect="sqlanywhere", copy=False) + second_sql = select_expr.sql(dialect="sqlanywhere", copy=False) + assert first_sql == "SELECT id FROM users ORDER BY id LIMIT 10 OFFSET 5" + assert second_sql == first_sql + assert isinstance(select_expr.args.get("limit"), exp.Limit) + assert isinstance(select_expr.args.get("offset"), exp.Offset) + + +def test_sqlanywhere_create_and_drop_if_exists() -> None: + """Verify SQLAnywhere emits native single-statement CREATE TABLE IF NOT EXISTS and DROP TABLE IF EXISTS.""" + create_sql = transpile( + "CREATE TABLE IF NOT EXISTS sessions (id VARCHAR(64) PRIMARY KEY, data TEXT)", + read="postgres", + write="sqlanywhere", + )[0] + assert create_sql == "CREATE TABLE IF NOT EXISTS sessions (id VARCHAR(64) PRIMARY KEY, data LONG VARCHAR)" + assert "OBJECT_ID" not in create_sql + + drop_sql = transpile("DROP TABLE IF EXISTS sessions", read="postgres", write="sqlanywhere")[0] + assert drop_sql == "DROP TABLE IF EXISTS sessions" + + +def test_sqlanywhere_type_mappings() -> None: + """Verify SQLAnywhere type mappings for LONG VARCHAR, LONG BINARY, TIMESTAMP, UNIQUEIDENTIFIER, and BIT.""" + assert parse_one("CAST(x AS TEXT)", read="postgres").sql("sqlanywhere") == "CAST(x AS LONG VARCHAR)" + assert parse_one("CAST(x AS JSONB)", read="postgres").sql("sqlanywhere") == "CAST(x AS LONG VARCHAR)" + assert parse_one("CAST(x AS BLOB)", dialect="sqlanywhere").sql("sqlanywhere") == "CAST(x AS LONG BINARY)" + assert parse_one("CAST(x AS BYTEA)", read="postgres").sql("sqlanywhere") == "CAST(x AS VARBINARY(32767))" + assert parse_one("CAST(x AS TIMESTAMP)", read="postgres").sql("sqlanywhere") == "CAST(x AS TIMESTAMP)" + assert parse_one("CAST(x AS TIMESTAMPTZ)", read="postgres").sql("sqlanywhere") == ( + "CAST(x AS TIMESTAMP WITH TIME ZONE)" + ) + assert parse_one("CAST(x AS UUID)", read="postgres").sql("sqlanywhere") == "CAST(x AS UNIQUEIDENTIFIER)" + assert parse_one("CAST(x AS BOOLEAN)", read="postgres").sql("sqlanywhere") == "CAST(x AS BIT)" + assert parse_one("CAST(x AS VARCHAR)", read="postgres").sql("sqlanywhere") == "CAST(x AS VARCHAR(32767))" + assert parse_one("CAST(x AS NVARCHAR(MAX))", read="tsql").sql("sqlanywhere") == "CAST(x AS LONG NVARCHAR)" + + +def test_sqlanywhere_long_types_and_utc_timestamp_parsing() -> None: + """Verify LONG VARCHAR, LONG BINARY, and CURRENT UTC TIMESTAMP parse cleanly in SQLAnywhere.""" + parsed = parse_one( + "CREATE TABLE IF NOT EXISTS docs (id UNIQUEIDENTIFIER, body LONG VARCHAR, raw_bytes LONG BINARY, created_at TIMESTAMP DEFAULT CURRENT UTC TIMESTAMP)", + dialect="sqlanywhere", + ) + rendered = parsed.sql(dialect="sqlanywhere") + assert "LONG VARCHAR" in rendered + assert "LONG BINARY" in rendered + assert "UNIQUEIDENTIFIER" in rendered + + +def test_sqlanywhere_parse_top_start_at_non_literal() -> None: + """Verify Watcom-SQL SELECT TOP n START AT ? parses with offset decrement.""" + parsed = parse_one("SELECT TOP 10 START AT ? id, name FROM users ORDER BY id", dialect="sqlanywhere") + assert parsed.sql(dialect="sqlanywhere") == "SELECT id, name FROM users ORDER BY id LIMIT 10 OFFSET (? - 1)" + + +def test_sqlanywhere_current_timestamp_local_and_utc() -> None: + """Verify CURRENT_TIMESTAMP renders local and GETUTCDATE renders UTC.""" + assert transpile("SELECT CURRENT_TIMESTAMP", read="postgres", write="sqlanywhere")[0] == "SELECT CURRENT TIMESTAMP" + assert transpile("SELECT GETUTCDATE()", read="tsql", write="sqlanywhere")[0] == "SELECT CURRENT UTC TIMESTAMP" + + +def test_sqlanywhere_identity_column_rendering() -> None: + """Verify generated identity and autoincrement constraints render DEFAULT AUTOINCREMENT.""" + create_sql = parse_one("CREATE TABLE t (id INT GENERATED ALWAYS AS IDENTITY, name VARCHAR(50))").sql("sqlanywhere") + assert "DEFAULT AUTOINCREMENT" in create_sql + + +def test_sqlanywhere_fetch_pagination_preservation() -> None: + """Verify FETCH FIRST n ROWS ONLY and OFFSET m ROWS FETCH NEXT n are preserved and not dropped.""" + fetch_expr = parse_one("SELECT id FROM users ORDER BY id OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY", read="tsql") + rendered = fetch_expr.sql("sqlanywhere") + assert rendered == "SELECT id FROM users ORDER BY id LIMIT 10 OFFSET 5" diff --git a/tests/unit/dialects/test_sybase.py b/tests/unit/dialects/test_sybase.py new file mode 100644 index 000000000..f185f0347 --- /dev/null +++ b/tests/unit/dialects/test_sybase.py @@ -0,0 +1,137 @@ +"""Unit tests for the custom SAP Adaptive Server Enterprise (Sybase ASE) sqlglot dialect.""" + +import sqlglot +from sqlglot import exp, parse_one, transpile +from sqlglot.dialects.dialect import NormalizationStrategy + +from sqlspec.dialects.sybase import Sybase, SybaseTokenizer + +__all__ = () + + +def test_sybase_dialect_registration() -> None: + """Verify Sybase dialect is registered and resolvable by name.""" + dialect = sqlglot.Dialect.get_or_raise("sybase") + assert dialect.__class__ is Sybase + assert dialect.tokenizer_class is SybaseTokenizer + assert dialect.NORMALIZATION_STRATEGY == NormalizationStrategy.CASE_SENSITIVE + + +def test_sybase_top_limit_without_offset() -> None: + """Verify literal integer LIMIT without OFFSET renders unparenthesized SELECT TOP n.""" + result = transpile("SELECT id, name FROM users LIMIT 10", read="postgres", write="sybase")[0] + assert result == "SELECT TOP 10 id, name FROM users" + + +def test_sybase_rows_limit_offset_rendering() -> None: + """Verify LIMIT with OFFSET renders SAP ASE 16.0 SP03+ ROWS LIMIT n OFFSET m.""" + result = transpile("SELECT id, name FROM users ORDER BY id LIMIT 10 OFFSET 20", write="sybase")[0] + assert result == "SELECT id, name FROM users ORDER BY id ROWS LIMIT 10 OFFSET 20" + + +def test_sybase_rows_offset_only_rendering() -> None: + """Verify OFFSET without LIMIT renders ROWS OFFSET m.""" + result = transpile("SELECT id FROM users ORDER BY id OFFSET 15", write="sybase")[0] + assert result == "SELECT id FROM users ORDER BY id ROWS OFFSET 15" + + +def test_sybase_parameterized_limit_renders_rows_limit() -> None: + """Verify non-literal LIMIT (placeholder/parameter) renders ROWS LIMIT instead of TOP.""" + expr = exp.select("id").from_("users").limit(exp.Placeholder(this="limit")) + rendered = expr.sql(dialect="sybase") + assert rendered == "SELECT id FROM users ROWS LIMIT :limit" + + +def test_sybase_parse_rows_limit_offset_roundtrip() -> None: + """Verify SAP ASE ROWS LIMIT n OFFSET m parses into canonical Limit/Offset AST nodes.""" + parsed = parse_one("SELECT id, name FROM users ORDER BY id ROWS LIMIT 10 OFFSET 20", dialect="sybase") + assert isinstance(parsed, exp.Select) + limit_node = parsed.args.get("limit") + offset_node = parsed.args.get("offset") + assert isinstance(limit_node, exp.Limit) + assert isinstance(offset_node, exp.Offset) + assert parsed.sql(dialect="sybase") == "SELECT id, name FROM users ORDER BY id ROWS LIMIT 10 OFFSET 20" + + +def test_sybase_copy_false_preserves_ast_pagination() -> None: + """Verify expression.sql('sybase', copy=False) never mutates limit or offset in place.""" + select_limit = parse_one("SELECT id FROM users LIMIT 5", read="postgres") + first_limit_sql = select_limit.sql(dialect="sybase", copy=False) + second_limit_sql = select_limit.sql(dialect="sybase", copy=False) + assert first_limit_sql == "SELECT TOP 5 id FROM users" + assert second_limit_sql == first_limit_sql + assert isinstance(select_limit.args.get("limit"), exp.Limit) + + select_page = parse_one("SELECT id FROM users ORDER BY id LIMIT 5 OFFSET 10") + first_page_sql = select_page.sql(dialect="sybase", copy=False) + second_page_sql = select_page.sql(dialect="sybase", copy=False) + assert first_page_sql == "SELECT id FROM users ORDER BY id ROWS LIMIT 5 OFFSET 10" + assert second_page_sql == first_page_sql + assert isinstance(select_page.args.get("limit"), exp.Limit) + assert isinstance(select_page.args.get("offset"), exp.Offset) + + +def test_sybase_set_operation_pagination() -> None: + """Verify UNION pagination renders ROWS LIMIT n OFFSET m and preserves AST under copy=False.""" + union_expr = parse_one("SELECT 1 AS a UNION ALL SELECT 2 AS a LIMIT 5 OFFSET 2") + first_sql = union_expr.sql(dialect="sybase", copy=False) + second_sql = union_expr.sql(dialect="sybase", copy=False) + assert first_sql == "SELECT 1 AS a UNION ALL SELECT 2 AS a ROWS LIMIT 5 OFFSET 2" + assert second_sql == first_sql + + +def test_sybase_type_mappings() -> None: + """Verify Sybase ASE type mappings for TEXT, IMAGE, BIGDATETIME, BIGTIME, UUID, and VARCHAR.""" + assert parse_one("CAST(x AS TEXT)", read="postgres").sql("sybase") == "CAST(x AS TEXT)" + assert parse_one("CAST(x AS JSONB)", read="postgres").sql("sybase") == "CAST(x AS TEXT)" + assert parse_one("CAST(x AS BLOB)", dialect="sybase").sql("sybase") == "CAST(x AS IMAGE)" + assert parse_one("CAST(x AS BYTEA)", read="postgres").sql("sybase") == "CAST(x AS VARBINARY(16384))" + assert parse_one("CAST(x AS TIMESTAMPTZ)", read="postgres").sql("sybase") == "CAST(x AS BIGDATETIME)" + assert parse_one("CAST(x AS TIMESTAMP)", read="postgres").sql("sybase") == "CAST(x AS BIGDATETIME)" + assert parse_one("CAST(x AS TIME)", read="postgres").sql("sybase") == "CAST(x AS BIGTIME)" + assert parse_one("CAST(x AS UUID)", read="postgres").sql("sybase") == "CAST(x AS VARCHAR(36))" + assert parse_one("CAST(x AS VARCHAR)", read="postgres").sql("sybase") == "CAST(x AS VARCHAR(16384))" + assert parse_one("CAST(x AS VARCHAR(MAX))", read="tsql").sql("sybase") == "CAST(x AS TEXT)" + assert parse_one("CAST(x AS NVARCHAR(MAX))", read="tsql").sql("sybase") == "CAST(x AS UNITEXT)" + assert parse_one("CAST(x AS VARBINARY(MAX))", read="tsql").sql("sybase") == "CAST(x AS IMAGE)" + + +def test_sybase_columndef_identity_and_nullable_boolean() -> None: + """Verify Sybase ASE renders bare IDENTITY and converts nullable BIT/BOOLEAN to TINYINT NULL.""" + ddl = parse_one( + "CREATE TABLE items (id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, active BOOLEAN NOT NULL, flag BOOLEAN NULL)", + read="postgres", + ).sql("sybase") + assert "id INT IDENTITY PRIMARY KEY" in ddl + assert "IDENTITY(1, 1)" not in ddl + assert "active BIT NOT NULL" in ddl + assert "flag TINYINT NULL" in ddl + + +def test_sybase_functions_and_operators() -> None: + """Verify CHAR_LENGTH, CHARINDEX, GETUTCDATE, and ILIKE rendering and parsing in Sybase.""" + assert transpile("SELECT LENGTH(name) FROM users", read="postgres", write="sybase")[0] == ( + "SELECT CHAR_LENGTH(name) FROM users" + ) + assert transpile("SELECT STRPOS(name, 'abc') FROM users", read="postgres", write="sybase")[0] == ( + "SELECT CHARINDEX('abc', name) FROM users" + ) + assert transpile("SELECT CURRENT_TIMESTAMP", read="postgres", write="sybase")[0] == "SELECT GETDATE()" + assert transpile("SELECT GETUTCDATE()", read="tsql", write="sybase")[0] == "SELECT GETUTCDATE()" + assert transpile("SELECT * FROM users WHERE name ILIKE '%alice%'", read="postgres", write="sybase")[0] == ( + "SELECT * FROM users WHERE LOWER(name) LIKE LOWER('%alice%')" + ) + + parsed_len = parse_one("SELECT CHAR_LENGTH(name) FROM users", dialect="sybase") + assert parsed_len.find(exp.Length) is not None + parsed_pos = parse_one("SELECT CHARINDEX('abc', name) FROM users", dialect="sybase") + pos_node = parsed_pos.find(exp.StrPosition) + assert pos_node is not None + assert pos_node.sql(dialect="sybase") == "CHARINDEX('abc', name)" + + +def test_sybase_fetch_pagination_preservation() -> None: + """Verify FETCH FIRST n ROWS ONLY and OFFSET m ROWS FETCH NEXT n are preserved and not dropped.""" + fetch_expr = parse_one("SELECT id FROM users ORDER BY id OFFSET 5 ROWS FETCH NEXT 10 ROWS ONLY", read="tsql") + rendered = fetch_expr.sql("sybase") + assert rendered == "SELECT id FROM users ORDER BY id ROWS LIMIT 10 OFFSET 5" diff --git a/tests/unit/test_mypyc_sybase_surface.py b/tests/unit/test_mypyc_sybase_surface.py new file mode 100644 index 000000000..2575c641f --- /dev/null +++ b/tests/unit/test_mypyc_sybase_surface.py @@ -0,0 +1,104 @@ +"""Unit tests for Sybase and SQL Anywhere packaging, MyPyC inventory, and import boundaries.""" + +import subprocess +import sys +import tomllib +from pathlib import Path + +from tools.scripts.mypyc_inventory import build_inventory, validate_inventory +from tools.scripts.mypyc_smoke import SMOKE_IMPORTS + +PROJECT_ROOT = Path(__file__).resolve().parents[2] + + +def test_pyproject_registers_sybase_and_sqlanywhere_extras_entrypoints_and_markers() -> None: + """pyproject.toml registers optional dependencies, sqlglot dialect entry points, and pytest markers.""" + config = tomllib.loads((PROJECT_ROOT / "pyproject.toml").read_text()) + + optional_deps = config["project"]["optional-dependencies"] + assert optional_deps.get("sybase") == ["pymssql"] + assert optional_deps.get("sqlanywhere") == ["sqlanydb>=1.0.14"] + + dialect_entrypoints = config["project"]["entry-points"]["sqlglot.dialects"] + assert dialect_entrypoints.get("sybase") == "sqlspec.dialects.sybase:Sybase" + assert dialect_entrypoints.get("sqlanywhere") == "sqlspec.dialects.sybase:SQLAnywhere" + + markers = config["tool"]["pytest"]["ini_options"]["markers"] + assert any(marker.startswith("sybase:") for marker in markers) + assert any(marker.startswith("sqlanywhere:") for marker in markers) + + mypy_overrides = config["tool"]["mypy"]["overrides"] + ignored_modules: set[str] = set() + for override in mypy_overrides: + if override.get("ignore_missing_imports"): + modules = override.get("module", []) + if isinstance(modules, list): + ignored_modules.update(modules) + assert "sqlanydb" in ignored_modules + assert "sqlanydb.*" in ignored_modules + + +def test_mypyc_inventory_includes_sybase_and_sqlanywhere_hot_surfaces() -> None: + """mypyc_inventory validates cleanly and classifies Sybase and SQL Anywhere modules accurately.""" + errors = validate_inventory(PROJECT_ROOT) + assert errors == [] + + inventory = build_inventory(PROJECT_ROOT) + compiled = set(inventory["compiled_modules"]) + interpreted = set(inventory["interpreted_modules"]) + hot_surfaces = inventory["hot_surfaces"] + + expected_compiled = { + "sqlspec/dialects/sybase/_generators.py", + "sqlspec/dialects/sybase/_parsers.py", + "sqlspec/dialects/sybase/_transforms.py", + "sqlspec/data_dictionary/dialects/sybase/config.py", + "sqlspec/data_dictionary/dialects/sqlanywhere/config.py", + "sqlspec/adapters/sybase/core.py", + "sqlspec/adapters/sybase/pool.py", + "sqlspec/adapters/sqlanywhere/core.py", + "sqlspec/adapters/sqlanywhere/pool.py", + } + for module_path in expected_compiled: + assert module_path in compiled, f"Expected compiled module: {module_path}" + assert module_path in hot_surfaces, f"Expected hot_surfaces entry: {module_path}" + assert hot_surfaces[module_path]["status"] == "compiled" + + expected_interpreted = { + "sqlspec/dialects/sybase/__init__.py", + "sqlspec/adapters/sybase/_typing.py", + "sqlspec/adapters/sybase/config.py", + "sqlspec/adapters/sybase/data_dictionary.py", + "sqlspec/adapters/sybase/driver.py", + "sqlspec/adapters/sqlanywhere/_typing.py", + "sqlspec/adapters/sqlanywhere/config.py", + "sqlspec/adapters/sqlanywhere/data_dictionary.py", + "sqlspec/adapters/sqlanywhere/driver.py", + } + for module_path in expected_interpreted: + assert module_path in interpreted, f"Expected interpreted module: {module_path}" + + +def test_mypyc_smoke_imports_include_sybase_and_sqlanywhere() -> None: + """SMOKE_IMPORTS includes dialect and compiled helper/pool entries for Sybase and SQL Anywhere.""" + smoke_by_name = {entry.name: entry for entry in SMOKE_IMPORTS} + assert "sybase_dialect" in smoke_by_name + assert "sqlanywhere_dialect" in smoke_by_name + assert "sybase_generators" in smoke_by_name + assert smoke_by_name["sybase_generators"].require_compiled is True + assert "sybase_pool" in smoke_by_name + assert smoke_by_name["sybase_pool"].require_compiled is True + assert "sqlanywhere_pool" in smoke_by_name + assert smoke_by_name["sqlanywhere_pool"].require_compiled is True + + +def test_importing_sybase_and_sqlanywhere_adapters_does_not_eagerly_import_driver_packages() -> None: + """Importing sqlspec.adapters.sybase and sqlspec.adapters.sqlanywhere defers pymssql and sqlanydb.""" + script = ( + "import sys\n" + "import sqlspec.adapters.sybase\n" + "import sqlspec.adapters.sqlanywhere\n" + "leaked = [name for name in ('pymssql', 'sqlanydb', 'rich') if name in sys.modules]\n" + "assert not leaked, f'Eagerly imported: {leaked}'\n" + ) + subprocess.run([sys.executable, "-c", script], capture_output=True, text=True, check=True) diff --git a/tests/unit/utils/test_fixtures.py b/tests/unit/utils/test_fixtures.py index e7b82f569..0890b8eb9 100644 --- a/tests/unit/utils/test_fixtures.py +++ b/tests/unit/utils/test_fixtures.py @@ -15,11 +15,11 @@ import struct import sys import zipfile -from datetime import datetime, time, timedelta, timezone +from datetime import UTC, datetime, time, timedelta, timezone from decimal import Decimal from pathlib import Path from typing import Any, cast -from unittest.mock import AsyncMock, MagicMock, Mock, patch +from unittest.mock import AsyncMock, MagicMock, Mock, call, patch from uuid import UUID import pytest @@ -2748,3 +2748,172 @@ def test_duckdb_sequence_resync_increment_and_schema(tmp_path: Path) -> None: assert empty_next_id == 1 finally: config.close_pool() + + +def test_sybase_identity_resync_executes_sp_chgattribute_sync(tmp_path: Path) -> None: + """Sybase fixture load with resync_sequences=True should execute sp_chgattribute when max > 0.""" + rows = [{"id": 42, "name": "Alice"}] + (tmp_path / "users.json").write_text(json.dumps(rows), encoding="utf-8") + driver = MagicMock() + driver.statement_config.dialect = "sybase" + driver.data_dictionary.get_columns.return_value = [ + { + "column_name": "id", + "data_type": "int", + "is_primary": True, + "identity_generation": "ALWAYS", + "is_generated": 0, + }, + { + "column_name": "name", + "data_type": "varchar", + "is_primary": False, + "identity_generation": None, + "is_generated": 0, + }, + ] + driver.select_value.return_value = 42 + + counts = load_table_fixtures_sync(driver, tmp_path, resync_sequences=True) + + assert counts == {"users": 1} + assert driver.execute_many.call_args.args[1] == [{"id": 42, "name": "Alice"}] + driver.select_value.assert_called_once_with('SELECT coalesce(max("id"), 0) FROM "users"') + assert call("EXEC sp_chgattribute 'users', 'identity_burn_max', 0, '42'") in driver.execute.call_args_list + + +def test_sybase_identity_resync_skips_when_max_is_zero(tmp_path: Path) -> None: + """Sybase fixture load should skip sp_chgattribute when table max is 0.""" + (tmp_path / "users.json").write_text("[]", encoding="utf-8") + driver = MagicMock() + driver.statement_config.dialect = "sybase" + driver.data_dictionary.get_columns.return_value = [ + { + "column_name": "id", + "data_type": "int", + "is_primary": True, + "identity_generation": "ALWAYS", + "is_generated": 0, + }, + { + "column_name": "name", + "data_type": "varchar", + "is_primary": False, + "identity_generation": None, + "is_generated": 0, + }, + ] + driver.select_value.return_value = 0 + + counts = load_table_fixtures_sync(driver, tmp_path, resync_sequences=True) + + assert counts == {"users": 0} + driver.select_value.assert_called_once_with('SELECT coalesce(max("id"), 0) FROM "users"') + driver.execute.assert_not_called() + + +async def test_sqlanywhere_identity_resync_executes_sa_reset_identity_async(tmp_path: Path) -> None: + """SQL Anywhere async fixture load should execute CALL sa_reset_identity with table, schema, and max.""" + rows = [{"id": 15, "name": "Bob"}] + (tmp_path / "DBA.users.json").write_text(json.dumps(rows), encoding="utf-8") + driver = MagicMock() + driver.statement_config.dialect = "sqlanywhere" + driver.data_dictionary.get_columns = AsyncMock( + return_value=[ + { + "column_name": "id", + "data_type": "integer", + "is_primary": True, + "identity_generation": "ALWAYS", + "is_generated": 0, + }, + { + "column_name": "name", + "data_type": "varchar", + "is_primary": False, + "identity_generation": None, + "is_generated": 0, + }, + ] + ) + driver.execute_many = AsyncMock() + driver.execute = AsyncMock() + driver.select_value = AsyncMock(return_value=15) + + counts = await load_table_fixtures_async(driver, tmp_path, resync_sequences=True) + + assert counts == {"DBA.users": 1} + assert driver.execute_many.call_args.args[1] == [{"id": 15, "name": "Bob"}] + driver.select_value.assert_awaited_once_with('SELECT coalesce(max("id"), 0) FROM "DBA"."users"') + driver.execute.assert_awaited_once_with("CALL sa_reset_identity('users', 'DBA', 15)") + + +def test_sybase_and_sqlanywhere_lob_types_excluded_from_export_order_by(tmp_path: Path) -> None: + """Unorderable LOB types in Sybase and SQL Anywhere should be excluded from export ORDER BY.""" + driver = MagicMock() + driver.statement_config.dialect = "sybase" + driver.data_dictionary.get_columns.return_value = [ + {"column_name": "code", "data_type": "varchar", "is_primary": False}, + {"column_name": "notes_text", "data_type": "text", "is_primary": False}, + {"column_name": "notes_unitext", "data_type": "unitext", "is_primary": False}, + {"column_name": "photo", "data_type": "image", "is_primary": False}, + {"column_name": "sa_clob", "data_type": "long varchar", "is_primary": False}, + {"column_name": "sa_nclob", "data_type": "long nvarchar", "is_primary": False}, + {"column_name": "sa_blob", "data_type": "long binary", "is_primary": False}, + ] + driver.select.return_value = [] + + export_table_fixtures_sync(driver, tmp_path, ["docs"], compress=False) + + select_query = driver.select.call_args.args[0] + order_by_clause = select_query.partition("ORDER BY")[2] + assert "[code]" in order_by_clause or '"code"' in order_by_clause + for lob_col in ("notes_text", "notes_unitext", "photo", "sa_clob", "sa_nclob", "sa_blob"): + assert lob_col not in order_by_clause + + +def test_sybase_and_sqlanywhere_value_decoders(tmp_path: Path) -> None: + """Sybase and SQL Anywhere temporal, monetary, UUID, and binary column types should decode on load.""" + rows = [ + { + "id": 1, + "big_dt": "2026-10-30T12:34:56.123456+00:00", + "small_dt": "2026-10-30T12:34:00+00:00", + "dt_tz": "2026-10-30T12:34:56+00:00", + "big_t": "12:34:56.123456", + "price": "19.95", + "small_price": "5.50", + "ext_id": "7f3c1d2e-9a4b-4c5d-8e6f-0a1b2c3d4e5f", + "raw_image": "AP8=", + "raw_long_bin": "AP8=", + } + ] + (tmp_path / "records.json").write_text(json.dumps(rows), encoding="utf-8") + columns = [ + {"column_name": "id", "data_type": "int", "is_primary": True}, + {"column_name": "big_dt", "data_type": "bigdatetime", "is_primary": False}, + {"column_name": "small_dt", "data_type": "smalldatetime", "is_primary": False}, + {"column_name": "dt_tz", "data_type": "datetimetz", "is_primary": False}, + {"column_name": "big_t", "data_type": "bigtime", "is_primary": False}, + {"column_name": "price", "data_type": "money", "is_primary": False}, + {"column_name": "small_price", "data_type": "smallmoney", "is_primary": False}, + {"column_name": "ext_id", "data_type": "uniqueidentifier", "is_primary": False}, + {"column_name": "raw_image", "data_type": "image", "is_primary": False}, + {"column_name": "raw_long_bin", "data_type": "long binary", "is_primary": False}, + ] + driver = MagicMock() + driver.statement_config.dialect = "sybase" + driver.data_dictionary.get_columns.return_value = columns + + load_table_fixtures_sync(driver, tmp_path) + + loaded_row = driver.execute_many.call_args.args[1][0] + assert loaded_row["big_dt"] == datetime(2026, 10, 30, 12, 34, 56, 123456, tzinfo=UTC) + assert loaded_row["small_dt"] == datetime(2026, 10, 30, 12, 34, 0, tzinfo=UTC) + assert loaded_row["dt_tz"] == datetime(2026, 10, 30, 12, 34, 56, tzinfo=UTC) + assert loaded_row["big_t"] == time(12, 34, 56, 123456) + assert loaded_row["price"] == Decimal("19.95") + assert loaded_row["small_price"] == Decimal("5.50") + assert loaded_row["ext_id"] == UUID("7f3c1d2e-9a4b-4c5d-8e6f-0a1b2c3d4e5f") + assert loaded_row["raw_image"] == b"\x00\xff" + assert loaded_row["raw_long_bin"] == b"\x00\xff" diff --git a/tools/scripts/mypyc_inventory.py b/tools/scripts/mypyc_inventory.py index b0c4aed56..0efd5cc60 100644 --- a/tools/scripts/mypyc_inventory.py +++ b/tools/scripts/mypyc_inventory.py @@ -111,6 +111,14 @@ "classification": "compile_now", "reason": "Spanner data-dictionary dialect configuration is in the compiled dialect surface.", }, + "sqlspec/data_dictionary/dialects/sqlanywhere/config.py": { + "classification": "compile_now", + "reason": "SQL Anywhere data-dictionary dialect configuration and feature helpers compile with the dialect surface.", + }, + "sqlspec/data_dictionary/dialects/sybase/config.py": { + "classification": "compile_now", + "reason": "Sybase ASE data-dictionary dialect configuration and feature helpers compile with the dialect surface.", + }, "sqlspec/data_dictionary/dialects/bigquery/config.py": { "classification": "compile_now", "reason": "Shared BigQuery INFORMATION_SCHEMA formatting helpers for native BigQuery and ADBC-as-BigQuery dictionaries.", @@ -161,6 +169,18 @@ }, "sqlspec/dialects/db2/_parsers.py": {"classification": "compile_now", "reason": "Db2 AST normalisation helpers."}, "sqlspec/dialects/db2/_transforms.py": {"classification": "compile_now", "reason": "Db2 render helper functions."}, + "sqlspec/dialects/sybase/_generators.py": { + "classification": "compile_now", + "reason": "Sybase and SQL Anywhere render handlers compile as explicit-argument callables with per-instance generator dispatch.", + }, + "sqlspec/dialects/sybase/_parsers.py": { + "classification": "compile_now", + "reason": "Sybase and SQL Anywhere AST normalisation helpers.", + }, + "sqlspec/dialects/sybase/_transforms.py": { + "classification": "compile_now", + "reason": "Sybase and SQL Anywhere render helper functions.", + }, "sqlspec/adapters/mysql_common.py": { "classification": "compile_now", "reason": "Shared MySQL-family adapter helpers compile with the adapter core surface.", @@ -251,6 +271,22 @@ "classification": "compile_now", "reason": "Pool runtime compiles without adapter driver exception-handler subclasses.", }, + "sqlspec/adapters/sqlanywhere/core.py": { + "classification": "compile_now", + "reason": "SQL Anywhere parameter normalization, row collection, and exception mapping helpers compile with the adapter core surface.", + }, + "sqlspec/adapters/sqlanywhere/pool.py": { + "classification": "compile_now", + "reason": "SQL Anywhere thread-local pool runtime compiles without adapter driver subclasses.", + }, + "sqlspec/adapters/sybase/core.py": { + "classification": "compile_now", + "reason": "Sybase ASE parameter normalization, percent escaping, row collection, and exception mapping helpers compile with the adapter core surface.", + }, + "sqlspec/adapters/sybase/pool.py": { + "classification": "compile_now", + "reason": "Sybase ASE thread-local pool runtime with TDSVER='5.0' negotiation compiles without adapter driver subclasses.", + }, "sqlspec/adapters/aiosqlite/pool.py": { "classification": "compile_now", "reason": "Async pool runtime compiles without adapter driver exception-handler subclasses.", diff --git a/tools/scripts/mypyc_smoke.py b/tools/scripts/mypyc_smoke.py index c53fe936a..81f15174f 100644 --- a/tools/scripts/mypyc_smoke.py +++ b/tools/scripts/mypyc_smoke.py @@ -44,11 +44,16 @@ class SmokeImport(NamedTuple): SmokeImport("storage_backend_fsspec", "sqlspec.storage.backends.fsspec", "FSSpecBackend", True), SmokeImport("storage_backend_obstore", "sqlspec.storage.backends.obstore", "ObStoreBackend", True), SmokeImport("sqlite_pool", "sqlspec.adapters.sqlite.pool", "SqliteConnectionPool", True), + SmokeImport("sybase_pool", "sqlspec.adapters.sybase.pool", "SybaseConnectionPool", True), + SmokeImport("sqlanywhere_pool", "sqlspec.adapters.sqlanywhere.pool", "SqlanywhereConnectionPool", True), SmokeImport("data_dictionary_registry", "sqlspec.data_dictionary._registry", "get_dialect_config", True), SmokeImport("data_dictionary_loader", "sqlspec.data_dictionary._loader", "DataDictionaryLoader", True), SmokeImport("pgvector_dialect", "sqlspec.dialects.postgres._pgvector", "PGVector"), SmokeImport("spanner_dialect", "sqlspec.dialects.spanner._spanner", "Spanner"), SmokeImport("db2_dialect", "sqlspec.dialects.db2", "DB2"), + SmokeImport("sybase_dialect", "sqlspec.dialects.sybase", "Sybase"), + SmokeImport("sqlanywhere_dialect", "sqlspec.dialects.sybase", "SQLAnywhere"), + SmokeImport("sybase_generators", "sqlspec.dialects.sybase._generators", "sybase_select_sql", True), SmokeImport("fastapi_providers", "sqlspec.extensions.fastapi.providers", "provide_filters", True, "fastapi"), SmokeImport( "litestar_providers", "sqlspec.extensions.litestar.providers", "create_filter_dependencies", True, "litestar" @@ -587,6 +592,7 @@ def _discover_adapter_config_classes(*, skipped: "list[str] | None" = None) -> " "psycopg_pool", "pymssql", "pymysql", + "sqlanydb", }: raise skipped.append(f"{module_name}: optional dependency missing: {exc.name}")