Skip to content

Commit bdaa142

Browse files
committed
feat(schema): implement schema-diff support and DataType refactoring for PostgreSQL backend (#46)
* feat(types): implement PostgreSQL DataType subclasses and type support mixin - Add ~45 PostgreSQL-specific DataType subclasses in expression/types.py - Add PostgresTypeFormatSupportMixin with TypeFormattingSupport + TypeParsingSupport - Integrate type support into PostgresDialect - Update exports in __init__.py, expression/__init__.py, mixins/__init__.py, mixins/types/__init__.py * refactor(types): add __eq__/__hash__, remove synonyms, add CharacterVaryingType - Replace _type_params() with __eq__/__hash__ in PostgresBitType, PostgresVarBitType, PostgresVectorType - Remove synonyms() from PostgresSmallSerialType, PostgresSerialType, PostgresBigSerialType - Add PostgresCharacterVaryingType(VarCharType) for CHARACTER VARYING(n) - Register new type in data_type_formatting mixin - Clean up unused imports (ClassVar, Set, Tuple) * feat(schema): add parsed_data_type to PostgreSQL _parse_columns Call DataType.parse_data_type_str(dialect, data_type) when constructing ColumnInfo so that introspection results carry structured DataType instances for schema comparison. * feat(types): add synonyms to PostgreSQL types - PostgresSerialType ↔ IntegerType/SerialType - PostgresSmallSerialType ↔ SmallIntType/SmallSerialType - PostgresBigSerialType ↔ BigIntType/BigSerialType - PostgresCharacterVaryingType ↔ VarCharType * feat(types): add synonyms to PostgresByteaType PostgresByteaType ↔ BlobType for cross-backend schema comparison. * refactor(types): inherit DDLTypeMixin in Postgres type formatting * refactor(types): update to DDLTypeSupport protocol * refactor(types): use @DDLTypeMixin.handles, remove format_data_type override - Add @DDLTypeMixin.handles decorators to all 56 formatter methods - Remove custom format_data_type() dispatch and supports_data_types() list - Move Postgres type imports from TYPE_CHECKING to runtime * fix(types): rewrite Postgres core type formatters, fix missing PostgresTypeFormatSupportMixin base - Replace _default_sql() calls with explicit PostgreSQL SQL strings for all 18 core type handlers (SmallIntType→SMALLINT, etc.) - Add PostgresTypeFormatSupportMixin as base of PostgresDialect (was imported but never used as base, causing format_data_type to resolve to protocol stub and return None) - Create test_core_type_mappings.py with 15 tests for core type→SQL rendering via PostgreSQL dialect * feat(postgres): implement array type support - Add PostgresArrayType parameterised container type (element_type, dimensions) - Add format_data_type_array handler that recursively formats element types - Update parse_type to detect ARRAY keyword and [] suffix before standard dispatch - Add 25 tests covering core type mappings and array rendering/parsing * refactor(postgres): use core ArrayType, keep PostgresArrayType as alias - PostgresArrayType is now just 'PostgresArrayType = ArrayType' - Update handler registration to use core ArrayType - Update parse_type to return ArrayType directly - Add test for is_element_type_equivalent * feat(postgres): add element type validation and PostgresArrayType subclass - format_data_type_array validates element_type is in _type_formatters - PostgresArrayType is now a proper subclass with is_equivalent ignoring dimensions (PG normalises multi-dim arrays to 1-D internally) - parse_type always creates PostgresArrayType with dimensions=1 - Update tests for new PostgresArrayType subclass and PG array semantics * feat(schema): add PostgresSchemaDiffer Column order has no semantic meaning in PostgreSQL; array dimension equivalence is handled by PostgresArrayType.is_equivalent(). * feat(schema): add schema-diff examples and refactor for PostgreSQL backend - Add 6 schema-diff example scripts: table_changes, column_order_irrelevant, postgres10_identity, array_type_diff, index_type_diff, serialization_roundtrip - All DDL uses expression objects, call introspect_and_adapt() after connect - Refactor format_column_definition to use col_def.data_type.to_sql() and add identity dialect_options support - Update existing DDL examples to use expression API (table param, QueryExpression for verification) * refactor(types): remove _default_sql(), make format_data_type return Tuple[str, tuple] - Delete all _default_sql() methods from Postgres expression types - Change all PostgresTypeFormatSupportMixin formatters to return Tuple[str, tuple] - Inline _default_sql() logic directly into each format_data_type_* method - Unpack format_data_type() call in array formatter * style(types): add complete type annotations to all format_data_type_* methods * feat(examples): strengthen serialization roundtrip with table data assertions * fix(tests): replace string data_type with DataType instances across 22 test files ColumnDefinition now validates that data_type must be a DataType instance. Replace all string data_type values (SERIAL, TEXT, INTEGER, etc.) with proper instance references. Also add PostgresCitextType, PostgresCubeType, PostgresLtreeType, PostgresRasterType missing extension DataType subclasses. Update security injection tests to verify TypeError at construction time instead of ValueError at format time. * docs: add changelog fragment for PR #46 * fix(tests): replace positional string data_type with DataType instances in 6 more files * fix(types): register PostgresCitextType, PostgresCubeType, PostgresLtreeType, PostgresRasterType formatters * fix: restore PostgresCidrType import accidentally removed in type formatter registration
1 parent 5d6850a commit bdaa142

52 files changed

Lines changed: 3137 additions & 386 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎changelog.d/46.added.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Added PostgreSQL-specific DataType subclasses and schema-diff support with `PostgresSchemaDiffer` for accurate schema introspection and column comparison.

‎src/rhosocial/activerecord/backend/impl/postgres/__init__.py‎

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,55 @@
2323
from .dialect import PostgresDialect
2424
from .transaction import PostgresTransactionManager, AsyncPostgresTransactionManager
2525
from .types import PostgresEnumType
26+
from .expression.types import (
27+
PostgresBigSerialType,
28+
PostgresBitType,
29+
PostgresBoxType,
30+
PostgresByteaType,
31+
PostgresCIDType,
32+
PostgresCircleType,
33+
PostgresDateMultirangeType,
34+
PostgresDateRangeType,
35+
PostgresGeographyType,
36+
PostgresGeometryType,
37+
PostgresHstoreType,
38+
PostgresInetType,
39+
PostgresInt4MultirangeType,
40+
PostgresInt4RangeType,
41+
PostgresInt8MultirangeType,
42+
PostgresInt8RangeType,
43+
PostgresJsonPathType,
44+
PostgresLineSegmentType,
45+
PostgresLineType,
46+
PostgresMacAddr8Type,
47+
PostgresMacAddrType,
48+
PostgresMoneyType,
49+
PostgresNumMultirangeType,
50+
PostgresNumRangeType,
51+
PostgresOIDType,
52+
PostgresPathType,
53+
PostgresPgLSNType,
54+
PostgresPointType,
55+
PostgresPolygonType,
56+
PostgresRegClassType,
57+
PostgresRegTypeType,
58+
PostgresSerialType,
59+
PostgresSmallSerialType,
60+
PostgresTIDType,
61+
PostgresTSQueryType,
62+
PostgresTSVectorType,
63+
PostgresTsMultirangeType,
64+
PostgresTsRangeType,
65+
PostgresTsTzMultirangeType,
66+
PostgresTsTzRangeType,
67+
PostgresUUIDType,
68+
PostgresVarBitType,
69+
PostgresVectorType,
70+
PostgresXID8Type,
71+
PostgresXIDType,
72+
PostgresXMLType,
73+
PostgresCidrType,
74+
)
2675
from .explain import PostgresExplainResult, PostgresExplainPlanLine
2776
from .expression.ddl import (
2877
PostgresCreateEnumTypeExpression,
@@ -505,4 +554,52 @@
505554
# hstore functions - Subscript
506555
"hstore_subscript_get",
507556
"hstore_subscript_set",
557+
# PostgreSQL DataType subclasses for DDL
558+
"PostgresBigSerialType",
559+
"PostgresBitType",
560+
"PostgresBoxType",
561+
"PostgresByteaType",
562+
"PostgresCIDType",
563+
"PostgresCircleType",
564+
"PostgresDateMultirangeType",
565+
"PostgresDateRangeType",
566+
"PostgresGeographyType",
567+
"PostgresGeometryType",
568+
"PostgresHstoreType",
569+
"PostgresInetType",
570+
"PostgresInt4MultirangeType",
571+
"PostgresInt4RangeType",
572+
"PostgresInt8MultirangeType",
573+
"PostgresInt8RangeType",
574+
"PostgresJsonPathType",
575+
"PostgresLineSegmentType",
576+
"PostgresLineType",
577+
"PostgresMacAddr8Type",
578+
"PostgresMacAddrType",
579+
"PostgresMoneyType",
580+
"PostgresNumMultirangeType",
581+
"PostgresNumRangeType",
582+
"PostgresOIDType",
583+
"PostgresPathType",
584+
"PostgresPgLSNType",
585+
"PostgresPointType",
586+
"PostgresPolygonType",
587+
"PostgresRegClassType",
588+
"PostgresRegTypeType",
589+
"PostgresSerialType",
590+
"PostgresSmallSerialType",
591+
"PostgresTIDType",
592+
"PostgresTSQueryType",
593+
"PostgresTSVectorType",
594+
"PostgresTsMultirangeType",
595+
"PostgresTsRangeType",
596+
"PostgresTsTzMultirangeType",
597+
"PostgresTsTzRangeType",
598+
"PostgresUUIDType",
599+
"PostgresVarBitType",
600+
"PostgresVectorType",
601+
"PostgresXID8Type",
602+
"PostgresXIDType",
603+
"PostgresXMLType",
604+
"PostgresCidrType",
508605
]

‎src/rhosocial/activerecord/backend/impl/postgres/dialect.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@
9090
IntrospectionSupport,
9191
TransactionControlSupport,
9292
SQLFunctionSupport,
93+
DDLTypeSupport,
9394
)
9495
from .mixins import (
9596
PostgresExtensionMixin,
@@ -169,6 +170,7 @@
169170
PostgresRangeTypeMixin,
170171
PostgresJSONBEnhancedMixin,
171172
PostgresArrayEnhancedMixin,
173+
PostgresTypeFormatSupportMixin,
172174
# DDL/DML operation mixins (new)
173175
PostgresExtendedStatisticsMixin,
174176
PostgresStoredProcedureMixin,
@@ -400,6 +402,7 @@ class PostgresDialect(
400402
PostgresFullTextSearchMixin,
401403
PostgresRangeTypeMixin,
402404
PostgresArrayEnhancedMixin,
405+
PostgresTypeFormatSupportMixin,
403406
# DDL/DML operation mixins (new)
404407
PostgresExtendedStatisticsMixin,
405408
PostgresStoredProcedureMixin,
@@ -524,6 +527,8 @@ class PostgresDialect(
524527
PostgresRangeTypeSupport,
525528
PostgresJSONBEnhancedSupport,
526529
PostgresArrayEnhancedSupport,
530+
# DataType Support Protocol
531+
DDLTypeSupport,
527532
# New feature protocols
528533
PostgresParallelQuerySupport,
529534
PostgresStoredProcedureSupport,

‎src/rhosocial/activerecord/backend/impl/postgres/examples/conftest.py‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,4 +416,46 @@
416416
'max_version': '18',
417417
'note': 'Requires superuser or pg_read_server_files role; only available in PG15+ docker images',
418418
},
419+
'schema_diff/table_changes.py': {
420+
'title': 'Schema Diff — Table Add/Remove',
421+
'dialect_protocols': [],
422+
'priority': 10,
423+
'min_version': '9',
424+
'max_version': '18',
425+
},
426+
'schema_diff/column_order_irrelevant.py': {
427+
'title': 'Schema Diff — Column Order Is Ignored',
428+
'dialect_protocols': [],
429+
'priority': 10,
430+
'min_version': '9',
431+
'max_version': '18',
432+
},
433+
'schema_diff/postgres10_identity.py': {
434+
'title': 'Schema Diff — PG9 SERIAL vs PG10+ IDENTITY',
435+
'dialect_protocols': [],
436+
'priority': 10,
437+
'min_version': '9',
438+
'max_version': '18',
439+
},
440+
'schema_diff/array_type_diff.py': {
441+
'title': 'Schema Diff — Array Type Dimension Changes',
442+
'dialect_protocols': [],
443+
'priority': 10,
444+
'min_version': '9',
445+
'max_version': '18',
446+
},
447+
'schema_diff/index_type_diff.py': {
448+
'title': 'Schema Diff — PostgreSQL Index Type Switches',
449+
'dialect_protocols': [],
450+
'priority': 10,
451+
'min_version': '9',
452+
'max_version': '18',
453+
},
454+
'schema_diff/serialization_roundtrip.py': {
455+
'title': 'Schema Diff — Snapshot Serialization Roundtrip',
456+
'dialect_protocols': [],
457+
'priority': 10,
458+
'min_version': '9',
459+
'max_version': '18',
460+
},
419461
}

‎src/rhosocial/activerecord/backend/impl/postgres/examples/ddl/alter_table.py‎

Lines changed: 18 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838

3939
drop_table = DropTableExpression(
4040
dialect=dialect,
41-
table_name='users',
41+
table='users',
4242
if_exists=True,
4343
cascade=True,
4444
)
@@ -47,7 +47,7 @@
4747

4848
create_table = CreateTableExpression(
4949
dialect=dialect,
50-
table_name='users',
50+
table='users',
5151
columns=[
5252
ColumnDefinition(
5353
'id',
@@ -148,13 +148,23 @@
148148
print("Column renamed successfully")
149149

150150
# Verify table structure
151-
options = ExecutionOptions(stmt_type=StatementType.DQL)
152-
result = backend.execute(
153-
"SELECT column_name, data_type, is_nullable "
154-
"FROM information_schema.columns "
155-
"WHERE table_name = 'users' ORDER BY ordinal_position",
156-
options=options
151+
from rhosocial.activerecord.backend.expression import QueryExpression, TableExpression
152+
from rhosocial.activerecord.backend.expression.core import WildcardExpression, Column
153+
from rhosocial.activerecord.backend.expression.query_parts import OrderByClause
154+
155+
verify_expr = QueryExpression(
156+
dialect=dialect,
157+
select=[
158+
Column(dialect, 'column_name'),
159+
Column(dialect, 'data_type'),
160+
Column(dialect, 'is_nullable'),
161+
],
162+
from_=TableExpression(dialect, 'columns', schema_name='information_schema'),
163+
where=Column(dialect, 'table_name') == Literal(dialect, 'users'),
164+
order_by=OrderByClause(dialect, [Column(dialect, 'ordinal_position')]),
157165
)
166+
options = ExecutionOptions(stmt_type=StatementType.DQL)
167+
result = backend.execute(*verify_expr.to_sql(), options=options)
158168
print("Table structure after alterations:")
159169
for row in result.data or []:
160170
print(f" {row}")

‎src/rhosocial/activerecord/backend/impl/postgres/examples/ddl/create_index.py‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333

3434
drop_table = DropTableExpression(
3535
dialect=dialect,
36-
table_name='products',
36+
table='products',
3737
if_exists=True,
3838
cascade=True,
3939
)
@@ -42,7 +42,7 @@
4242

4343
create_table = CreateTableExpression(
4444
dialect=dialect,
45-
table_name='products',
45+
table='products',
4646
columns=[
4747
ColumnDefinition(
4848
'id',
@@ -83,11 +83,17 @@
8383
print("Index created: idx_category_price")
8484

8585
# Verify index creation
86-
options = ExecutionOptions(stmt_type=StatementType.DQL)
87-
verify_result = backend.execute(
88-
"SELECT indexname FROM pg_indexes WHERE indexname = 'idx_category_price'",
89-
options=options
86+
from rhosocial.activerecord.backend.expression import QueryExpression, TableExpression
87+
from rhosocial.activerecord.backend.expression.core import Column, Literal
88+
89+
verify_expr = QueryExpression(
90+
dialect=dialect,
91+
select=[Column(dialect, 'indexname')],
92+
from_=TableExpression(dialect, 'pg_indexes'),
93+
where=Column(dialect, 'indexname') == Literal(dialect, 'idx_category_price'),
9094
)
95+
options = ExecutionOptions(stmt_type=StatementType.DQL)
96+
verify_result = backend.execute(*verify_expr.to_sql(), options=options)
9197
print(f"Index info: {verify_result.data}")
9298

9399
# ============================================================

0 commit comments

Comments
 (0)