Skip to content

Commit 5012e2f

Browse files
committed
feat: PGQ enhancement - fulltext search, cursor pollution fix, and PG 19 compatibility (#49)
* feat(pgq): gate quantified path and comma-separated patterns as unsupported on PG 19 PostgreSQL 19 does not support: - Variable-length (quantified) path patterns (+/*/{n,m}) - Comma-separated multiple patterns in a single MATCH clause Add capability overrides returning False, document known limitations, and add tests for the new gating methods. * fix: use correct fixture names 'postgres_backend' / 'async_postgres_backend' in cursor pollution tests The cursor pollution tests referenced 'pg_backend' and 'async_pg_backend' which do not exist as pytest fixtures. The correct names are 'postgres_backend' and 'async_postgres_backend', matching the actual fixture definitions used by the rest of the test suite. * feat: sync bridge files with testsuite rename and add 4 async relation bridges - Update bridge files to match testsuite rename: test_sync_queries -> test_queries - Add 4 new async relation bridge files (test_base_async, test_cache_async, test_interfaces_async, test_nested_relationship_access_async) - Include pgq/ example directory * feat: add bridge files for derived_field, interface feature tests and relation benchmark - derived_field/conftest.py + test_derived_field_testsuite.py - interface/test_threadsafe_dict.py (sync + async combined) - benchmark/relation/conftest.py + test_cache_comparison.py * perf(backend): remove redundant fetchall() in get_server_version() execute() is atomic per DB-API 2.0 — the extra fetchall() after fetchone() is unnecessary and adds a round-trip for no benefit. Follow-up to commit ab3be40 which introduced these defensive calls. * refactor(postgres): merge composite_pk/derived_field, switch to expression-based DDL - Delete composite_pk/derived_field bridge files, schema SQL, and providers - Create tests/providers/fixtures/ with TABLE_EXPRESSIONS (basic, query, mixins, events) - Replace _load_postgres_schema with CreateTableExpression/DropTableExpression DDL - Update all query/relation bridge imports to new subdirectory paths - Fix _setup_model_async -> _setup_async_model typo in query provider * fix(postgres): move bridge files to subdirectories, add missing conftest, fix composite_pk setup - Move 70 bridge files from flat paths to testsuite-matching subdirectories - Create 6 missing bridge files (composite_pk crud/query/cte/set_operations, derived_field_basic, example_basic_fixtures) - Add missing basic/worker, query/worker, interface conftest bridge files - Rename test_worker_transaction_isolation -> test_transaction_isolation - Fix setup_order_item_model to use basic TABLE_EXPRESSIONS for composite PK order_items table * fix(postgres): fix PROVIDER_KEY_ASYNC and method names in relation conftest - Use PROVIDER_KEY_ASYNC instead of undefined PROVIDER_KEY - Fix method call names: setup_async_user_model -> setup_user_model (await) - Fix cleanup_after_test_async -> cleanup_after_test (await) - Align method calls with testsuite IRelationAsyncProvider interface * fix(postgres): remove non-existent _ensure_user_post_comment_async_schema call from relation conftest RelationAsyncProvider._setup_user_post_comment_async() already handles all three tables in a single call. * fix(postgres): extend cleanup_after_test table lists to ensure no cross-test contamination Add missing tables to PostgreSQL provider cleanup lists so all tables created during a test are reliably removed afterwards: - basic provider (sync + async): bulk_users - query provider (sync + async): profiles These were created by setup methods (setup_bulk_user_model, setup_profile_fixtures) but absent from the cleanup drop list, leading to orphaned tables across sequential test runs and cross-scenario contamination. * fix(postgres): add DROP TABLE to relation/connection/benchmark cleanup, fix bare excepts - RelationSyncProvider/RelationAsyncProvider cleanup_after_test now drops all 12 created tables (employees, departments, authors, books, chapters, profiles, users, posts, comments, boundary_*) with CASCADE before disconnect. - BasicConnectionProvider/QueryConnectionProvider cleanup now drops test_users table. - CrudBenchmarkProvider cleanup now drops benchmark_users table. - All 8 bare 'except:' -> 'except Exception:' across basic.py, query.py, mixins.py, events.py providers. * refactor(test): add missing _async bridge files (61), fix naming - Generate 61 _async.py bridge files for sync/async pairing compliance - Rename test_eager_loading_scenarios_sync.py -> test_eager_loading_scenarios.py - Rename test_relations_with_sync.py -> test_relations_with.py - Split test_derived_field_basic.py into sync-only + test_derived_field_basic_async.py - Add missing bridge files for basic/fields/ subcategories - Fix testsuite module path in relation/derived/test_derived_field.py * fix(test): remove conflicting test_derived_field bridges from basic/fields/ Same fix as python-activerecord-mysql: identical leaf filename test_derived_field.py from two different testsuite modules collide under pytest's import model. Drop the basic/fields duplicates to keep only the relation/derived/ bridges, matching SQLite. * fix(test): restore __init__.py under tests/benchmark/packages Same fix as python-activerecord-mysql: the four __init__.py markers under tests/benchmark/{,backend,activerecord_bulk,relation} were removed in commit f146d54, breaking pytest collection on tests that use relative imports (e.g. 'from .workloads import ...' in tests/benchmark/backend/test_execute_crud_*.py). Restore them so pytest treats those directories as packages and the relative imports resolve. * fix(test): resolve cursor pollution, DataType errors, and import-file-mismatch - Move cursor.description assertions before cursor.close() in all 8 cursor pollution tests (both sync and async) to fix assert None - Replace string type names ('INTEGER', 'VARCHAR', 'NUMERIC') with proper DataType instances (IntegerType, VarCharType, DecimalType) in test_recursive_cte_graph_traversal.py to fix TypeError - Rename test_transaction_isolation.py → test_worker_transaction_isolation.py to avoid import-file-mismatch collision with backend/transaction/ * fix(ci): update benchmark test file names to match testsuite (remove _sync suffix) * fix(test): replace FunctionCall('+') with infix BinaryArithmeticExpression for depth+1 in CTE tests PostgreSQL does not support prefix function-call syntax for arithmetic operators (+(a, b)). Use the overloaded + operator (creates BinaryArithmeticExpression) which generates correct infix a + b SQL. * fix(test): add GROUP BY to fund_trace_aggregation, fix pre-existing assertion bugs in CTE tests - Add GroupByHavingClause to fix column 'fund_trace.depth' must appear in GROUP BY - Fix depth 3 expected sum to 800000 (500000 Diana + 300000 Eve) - Fix social network assertions: Charlie at depth 2 (not 1), no Diana in results - These bugs were pre-existing: tests were never runnable before DataType fix * fix(test): track created tables per provider to eliminate xdist race condition in cleanup_after_test Instead of hardcoding tables_to_drop list in cleanup_after_test (which drops ALL tables including those used by concurrent xdist workers), track only tables that were actually created by _setup_{model,multiple} calls in each provider instance. Since each test fixture creates a fresh provider, this naturally isolates table cleanup to just the tables the current test created, eliminating the race where one test's teardown drops users/type_adapter_tests/pydantic_validated_models tables that another concurrent worker is still using. * feat(postgres): set supports_json_arrow_operators=True * feat(postgres): add ON CONFLICT capability switches PostgreSQL supports only a single ON CONFLICT clause per INSERT. Explicitly declare supports_on_conflict_clause=True and supports_multiple_on_conflict_clauses=False to enable the generic gate in DMLMixin.format_on_conflict_clauses. Add targeted tests for the capability switches and single-clause rendering (incl. EXCLUDED). * test(postgres): add DROP TABLE CASCADE/RESTRICT rendering test Confirm PostgreSQL inherits the correct optimistic True/True defaults from the core TableMixin and emits standard CASCADE/RESTRICT tokens. No code change required on the PostgreSQL backend. * fix(postgres): add missing supports_on_conflict_clause and supports_multiple_on_conflict_clauses to PostgresUpsertSupport protocol * fix(postgres): remove sync savepoint test import, only async tests apply * fix(postgres): pass scenario name to type_adapter_fixtures provider to avoid cross-scenario table conflicts * fix(postgres): import all testsuite basic fixtures in conftest while overriding type_adapter_fixtures * fix(postgres): revert conftest override, restore import * to avoid breaking other fixtures * fix(postgres): pass scenario name to sync type_adapter_fixtures to avoid cross-scenario table collisions * fix(postgres): remove test bridge files accidentally added in protocol fix * fix(postgres): remove shadowing fixture import from type_adapter bridge file The bridge file imported type_adapter_fixtures directly from the testsuite conftest into the test module namespace. pytest registered that fixture with the test module as its most specific baseid, shadowing the conftest override (feature/basic/conftest.py) that passes the scenario name to the provider. As a result, the sync [postgres_exts] type_adapter tests connected to the default scenario database (5432) instead of postgres_exts (5433), colliding with async [postgres_default] tests on the same type_adapter_tests table and causing 'relation does not exist' failures under xdist concurrency. Remove the import so pytest resolves the conftest override, which correctly passes the scenario name. * test(postgres): add unit tests for data type formatting and parsing - Fix relative imports in parse_type (..expression.types -> ...expression.types) that crashed serial/binary/jsonpath/uuid/net/geom/range/oid/ts/vector parsing with ModuleNotFoundError. - Fix DATETIME mis-parsed as DATE (dispatch order), and CHARACTER/ CHARACTER VARYING/TIMESTAMPTZ/TIMETZ falling back to CustomType due to \b boundary regex. - Add test_data_type_formatting.py covering all PostgreSQL type format and parse branches (data_type_formatting.py coverage 55.67% -> 98%). - Add test_expression_types.py covering synonyms/equality/hash/array equivalence (expression/types.py coverage 76.19% -> 100%). - Cover index fulltext PHRASE mode + multi-column, and upsert capability switches (index.py and upsert.py -> 100%). * chore: add changelog fragment for PR #49
1 parent 2003ec6 commit 5012e2f

238 files changed

Lines changed: 4451 additions & 1117 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/test.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -123,13 +123,13 @@ jobs:
123123
--rootdir=. \
124124
--confcutdir=. \
125125
"$TESTSUITE_DIR/benchmark/crud/test_delete_async.py" \
126-
"$TESTSUITE_DIR/benchmark/crud/test_delete_sync.py" \
126+
"$TESTSUITE_DIR/benchmark/crud/test_delete.py" \
127127
"$TESTSUITE_DIR/benchmark/crud/test_insert_async.py" \
128-
"$TESTSUITE_DIR/benchmark/crud/test_insert_sync.py" \
128+
"$TESTSUITE_DIR/benchmark/crud/test_insert.py" \
129129
"$TESTSUITE_DIR/benchmark/crud/test_query_async.py" \
130-
"$TESTSUITE_DIR/benchmark/crud/test_query_sync.py" \
130+
"$TESTSUITE_DIR/benchmark/crud/test_query.py" \
131131
"$TESTSUITE_DIR/benchmark/crud/test_update_async.py" \
132-
"$TESTSUITE_DIR/benchmark/crud/test_update_sync.py" \
132+
"$TESTSUITE_DIR/benchmark/crud/test_update.py" \
133133
"$TESTSUITE_DIR/benchmark/query" \
134134
"$TESTSUITE_DIR/benchmark/transaction" \
135135
"$TESTSUITE_DIR/benchmark/mixin" \

‎changelog.d/49.added.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Add PGQ enhancements: fulltext search capability, ON CONFLICT switches, JSON arrow operators, cursor result-set pollution fix, and PostgreSQL 19 compatibility gate for quantified path and comma-separated patterns

‎src/rhosocial/activerecord/backend/impl/postgres/backend/async_backend.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -346,8 +346,6 @@ async def get_server_version(self) -> Optional[Tuple[int, int, int]]:
346346
cursor = await self._get_cursor()
347347
await cursor.execute("SELECT version()")
348348
version_row = await cursor.fetchone()
349-
# Defensively consume remaining results to prevent cursor.description pollution
350-
remaining = await cursor.fetchall()
351349
if not version_row or not version_row[0]:
352350
self.log(logging.WARNING, "PostgreSQL version query returned no result")
353351
return None

‎src/rhosocial/activerecord/backend/impl/postgres/backend/sync.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -347,8 +347,6 @@ def get_server_version(self) -> Optional[Tuple[int, int, int]]:
347347
cursor = self._get_cursor()
348348
cursor.execute("SELECT version()")
349349
result = cursor.fetchone()
350-
# Defensively consume remaining results to prevent cursor.description pollution
351-
cursor.fetchall()
352350
if not result or not result[0]:
353351
self.log(logging.WARNING, "PostgreSQL version query returned no result")
354352
return None

‎src/rhosocial/activerecord/backend/impl/postgres/dialect.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -694,6 +694,10 @@ def supports_for_update(self) -> bool:
694694
"""
695695
return True
696696

697+
def supports_json_arrow_operators(self) -> bool:
698+
"""PostgreSQL supports -> and ->> operators for JSON/JSONB access."""
699+
return True
700+
697701
def supports_lock_strength(self, strength) -> bool:
698702
"""
699703
Check if a specific lock strength is supported.

‎src/rhosocial/activerecord/backend/impl/postgres/examples/conftest.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,14 @@
262262
'min_version': '9',
263263
'max_version': '18',
264264
},
265+
'pgq/recursive_cte_graph_traversal.py': {
266+
'title': 'Recursive CTE Graph Traversal (PGQ Quantified Path Alternative)',
267+
'dialect_protocols': [],
268+
'priority': 5,
269+
'min_version': '9',
270+
'max_version': '18',
271+
'note': 'Uses recursive CTE + UNION ALL to simulate PGQ quantified path patterns; works on all PG versions',
272+
},
265273
'transaction/basic.py': {
266274
'title': 'Transaction Control',
267275
'dialect_protocols': [],

‎src/rhosocial/activerecord/backend/impl/postgres/examples/pgq/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)