Repository navigation
feat(postgres)!: own status families, storage split, one introspect question per family - #64
Merged
Merged
Conversation
…d answers per-table storage The four status families that were only ever PostgreSQL's -- WAL, replication, archive, security -- leave core and come back as typed fields on PostgreSQLServerOverview(ServerOverview); get_overview() keeps its core signature because the subclass is a legal covariant return, and core's to_dict() iterates fields so the subclass's facts serialize without being re-declared. The status introspector, its CLI and the module exports follow the types home. Per-table storage, which the feedback named as missing, answers through the existing table listing: pg_relation_size, pg_indexes_size and pg_total_relation_size become expression nodes measured on the row's own oid, and the row count travels as two signals -- reltuples, the planner's estimate, preferred, and pg_stat_user_tables.n_live_tup, taken only when the planner has none (PostgreSQL 14+ reports -1 for a never-analyzed relation, which is the absence of an estimate rather than an empty table). The parse_row_count reducer normalizes -1 to None, keeps 0 as a real estimate, and invents nothing when neither source answers. supports_table_storage_introspection answers True: PostgreSQL's storage is row-plus-index shaped, so all four narrow slots of IntrospectionTableInfo are answerable, and that is what the CLI checks before offering the family. Version numbers in pyproject.toml are untouched on this branch; the release line takes the break at merge. CI: the core pin in test.yml points at feature/introspection-enhancement for this cross-repository round, because the changes here consume core APIs that exist only on that branch; move the pin back to the release line once core merges. The CLI capability contract caught the new probe with no type behind it: supports_table_storage_introspection answered True while the CLI offered no question that consulted it -- answerable but unaskable, which the rules forbid. PostgreSQL gets the type: table-storage asks for one named table and answers with the storage figures the listing already projects (total, data, index bytes and the row estimate), reading the listing once rather than the four questions a per-table info answer would spend. The test's fake rows gain the attribute access the real IntrospectionTableInfo already has.
… on psycopg waits on the libpq socket with loop.add_reader/add_writer, which on Windows the platform default event loop (ProactorEventLoop, since 3.8) does not implement -- so every "await psycopg async under the default loop" path failed there. Measured on a Windows box: psycopg 3.3.6 refuses with "Psycopg cannot use the 'ProactorEventLoop' to run in async mode. Please use a compatible event loop, for instance by running 'asyncio.run(..., loop_factory=asyncio.SelectorEventLoop(selectors.SelectSelector()))'", and that suggested loop answers SELECT 42 against a live server. The product path gains one honest helper -- async_support.selector_event_loop()/run_coroutine(): the loop psycopg's async mode requires, constructed explicitly (asyncio.run's loop_factory argument is 3.13+, this backend still supports 3.8), with asyncio.run-grade lifecycle: cancel pending tasks on KeyboardInterrupt, shut async generators down, close and detach. CLI's three --async modes (introspect, query, status) run through it and now work on Windows out of the box; verified against a live server. The test path stops switching the global policy. Three conftests each set the deprecated WindowsSelectorEventLoopPolicy (announced for removal in Python 3.16, and a selector loop on Windows cannot serve asyncio subprocess or pipe I/O, so the switch could break what else runs in that process). They are replaced by pytest-asyncio 1.x's supported mechanism -- a module-level pytest_asyncio_loop_factories hook returning a selector loop, which scopes the choice to the tests' loops instead of the process. The async pool tests against a live server pass under it on Windows, and the deprecation warnings are gone. The unit tests assert the capability psycopg actually depends on (add_reader/add_writer on the constructed loop) rather than only a class name, plus the lifecycle: value propagation, exception propagation with the loop still closed, refusal inside a running loop, and interruption cancelling what is pending. AsyncPostgresBackend.connect() gains no guard of its own: psycopg 3.3+ already refuses with the fix named, and duplicating it would put the driver's evolution on this side of the seam. Version numbers in pyproject.toml are untouched on this branch. The first CI run caught the hook answering None on Linux: pytest-asyncio 1.4.1 requires a non-empty mapping from pytest_asyncio_loop_factories, so "nothing to do here" raised a UsageError in every async test's setup (2736 errors). The hook now exists only on Windows -- not defining it is how the hook says "no opinion", and everywhere else the default loop is already selector-based with the async tests' node ids unchanged.
… family, not after the sync Follows the core rename (53842f99) that took the `Sync` prefix off the introspection family's sync side, so the pair's shape now matches the backend family's: `PostgreSQLIntrospector` and `AsyncPostgreSQLIntrospector`, one plain name and one marked variant, the way `PostgresBackend` and `AsyncPostgresBackend` already read. Both concrete classes drop the prefix -- `SyncPostgreSQLIntrospector` and `SyncPostgreSQLStatusIntrospector` -- along with the three core base names they and their imports refer to (`SyncAbstractIntrospector`, `SyncAbstractStatusIntrospector`, `SyncIntrospectorExecutor`), which the core change renamed underneath us. Constructor call sites in the backend and sync backend, the package's exports, the introspection and dialect tests and the status tests follow. Alias-free by the same reasoning as the core change: a `SyncXxx = Xxx` abbreviation would put two names on one fact, and no other family here spells a pair that way. Method names and the sync/async parity they carry are untouched -- this commit only changes what the classes are called. Verified: imports of all four classes through the package's own export surface, black showing no new debt in any touched file (the dirt in these files exists at HEAD and renaming only ever shortens a line), ruff clean, and a pytest collection pass over the four touched test modules (11/74/29/15 cases) with no NameError or ImportError. The suite needs a live PostgreSQL server to run further, which CI provides.
…over-promising it `extension` is not SQL vocabulary, and the object it names differs per engine: PostgreSQL's are `CREATE EXTENSION` packages carrying a version and a schema, SQLite's are registered virtual table modules. The switch was proposed for withdrawal from core on exactly those grounds, and that argument loses. Withdrawing it would not make the semantics agree -- it would only make the question unaskable through one spelling, so a caller would have to already know the backend's private name (`supports_extension_introspection`) before it could ask anything. That is strictly worse than an answer that has to be read together with its backend's docs. The standing precedent is `supports_fulltext_search`: `MATCH ... AGAINST` and `CONTAINS` share nothing but the question, and the name stays in core because a caller wants one way to ask it. So the rule is now written down -- a capability switch buys *discoverability*, never *semantics* -- as section 2a of the introspection rules, with the test for deciding which of the two is uniform, and the consequences for anyone editing such a probe. Two things the rule caught while being written: * SQLite answered `True` on the strength of `PRAGMA module_list` alone, with no query formatting, no result type and no CLI type behind it. A caller was promised an answer it could not get, which is worse than `False` -- `False` is "I do not answer", whereas that is "I will" followed by nothing. It now answers `False`, with the honest name for the thing it would need to build. * PostgreSQL answered `True` through its own spelling while the core spelling still defaulted to `False`: one fact, two answers, so a caller could be told "unsupported" by whichever name it happened to use. It now answers through the core name too, and the CLI keeps working because it never used it. Verified: SQLite `supports_extensions` is False, PostgreSQL answers True under both spellings, ruff clean, and no new black debt in the touched files (the dirt in them is present at HEAD).
First backend answer for `supports_table_stats`, from the source the family was shaped on: `pg_stat_user_tables`. The projection is the collector's own columns under their own names -- `seq_scan`, `idx_scan`, `n_tup_ins`, `n_tup_upd`, `n_tup_del`, `n_live_tup` -- which is why the common shape reads them by those spellings first. What the listing deliberately does not do is flatten the view's other facts into the common shape: `n_dead_tup` and the `last_*` access times are real and useful, but they have no name every engine shares, so projecting them into the common slots would be the `extra`-as-parking-spot move the ownership rule forbids in the other direction. They stay on `pg_stat_user_tables`, for a backend reader that wants them. A table with no statistics row yet is absent from the view rather than zero-filled, and the docstring says so: the listing answers for the tables the collector has seen, which is the honest scope. The reset caveat -- crash or `pg_stat_reset()`, so every counter is "since the last reset" -- is stated on the capability probe, where a caller deciding what a zero means will look.
…tead of falling through The roundtrip walk in the PostgreSQL suite enumerates every core expression class and asks this dialect to render it, classifying each answer as a render, a declared `UnsupportedFeatureError`, or an entry in `LEGITIMATE_NON_RENDERS`. The new `UnusedIndexesQueryStatement` had none of those: it reached a `runtime_checkable` protocol stub sitting in the dialect's MRO, which answers `getattr` and then returns `None` -- reported as `ProtocolNotImplementedError`, a defect every job flagged. So PostgreSQL answers it properly rather than being excused from it: no vendor view exists, so the family is computed from `pg_stat_user_indexes.idx_scan` being zero over valid indexes, joined to `pg_namespace` for the schema. Only `idx_scan` is read -- an index is what makes a scan indexed -- and `indisvalid` filters the indexes the planner would never choose. The capability docstring says which tense PostgreSQL speaks, because it is not MySQL's: a zero means "no index scan since the last crash or `pg_stat_reset()`", an engine-computed claim rather than a vendor-maintained one, and the answer stays "worth reviewing" rather than "safe to drop". Rendered locally for the record: `SELECT n.nspname AS "schema_name", i.relname AS "table_name", i.indexrelname AS "index_name" FROM pg_stat_user_indexes AS i JOIN pg_namespace AS n ON i.schemaname = n.nspname WHERE n.nspname = %s AND i.idx_scan = %s AND i.indisvalid = TRUE ORDER BY i.relname, i.indexrelname`.
…capability in two The previous commit put the core spelling on the extension mixin, and CI refused it on two counts at once, which together say where the method belongs: * `PostgresExtensionMixin` must declare every public method it defines in the protocol it is paired with (`PostgresExtensionSupport`), so adding `supports_extensions` there obliges that protocol to name it; * two protocols may not declare the same method unless one derives from the other, and `IntrospectionSupport` names `supports_extensions` already -- so the pairing refuses it, because two protocols declaring one capability reads as two answers unless one derives from the other. So the method moves to `PostgresIntrospectionCapabilityMixin`, where the cross-engine probes already live, and delegates to `supports_extension_introspection()` rather than re-declaring `True`: one fact, and `pg_extension` is still the only place it is answered from. Calling through either spelling now gives the same reply, which is what closes the split -- a caller can no longer be told "unsupported" by the name it happened to use. `PostgresExtensionSupport` gains no member; its `supports_extension_introspection` docstring records why the other spelling is not declared there. Verified: both probes answer True on the dialect, the protocol conformance suite passes, ruff clean.
The common contract landed first; this is the first engine to answer it, and the whole family was written against PostgreSQL's catalogues, so each docstring can state the specifics rather than the possibilities. * runtime -- one row per backend from ``pg_stat_activity``, projected to what describes the server (application, state, wait, start). ``pg_stat_statements`` is deliberately not read: it is an extension, and answering from a catalogue that is only sometimes installed would make the family depend on a module rather than the engine. That decision is recorded on the probe. * index usage -- ``idx_scan`` and the tuple counters beside ``pg_relation_size(indexrelid)``, so "how much use" and "how much space" are one row instead of two queries a caller has to join. * dependencies -- ``pg_depend`` joined through to names, with ``deptype`` as the relation and ``action`` left unmapped: PostgreSQL has no per-edge action column (a dependant's drop behaviour is a property of its class), and inventing one from ``deptype`` would be a fact the catalog does not state. * DDL -- assembled, and it says so. ``supports_ddl_extraction`` is True while ``supports_ddl_extraction_native`` is False, and the query projects no ``ddl`` column at all: the parser drops textless rows, so ``get_ddl_statement`` returns None. An honest "cannot reproduce this" beats a plausible-looking fragment that renders as if it were complete. Two things CI-style checks caught while writing, both worth naming because they were my own and neither a syntax error: * ``Literal`` has no ``.as_``, so the two constant projections were invented API; the ``object_type`` is now read from the catalog's own ``relkind``. * the query passed ``table_name`` where the expression had been given ``object_name``, so the filter would have silently matched nothing. Both are now pinned by local renders for all four classes, and by the roundtrip walk (1146 passed) and protocol conformance (560 passed).
A batch rename meant for the DDL formatter's parameter reached four formatters
that never used it: TableInfo, ColumnInfo, IndexInfo and ForeignKey all key on
\ able_name\. Each then read \params.get("object_name", "")\ -- which is always
empty for those expressions, so all four rendered a query filtered by an
empty table name and answered nothing.
Damage was not theoretical: a pure-SQL test asserts
\params == ("public", "measurements")\ and got \("public", "")\, and the
list-columns read path returned an empty roster. Both are caught by tests that
do not need a server, which is why the local run found it while the first
green-looking one did not.
The object-dependency and DDL formatters are the two that legitimately key on
\object_name\, and both keep it. MySQL was checked for the same pattern; its
two \object_name\ reads are exactly those two families.
Core grew the three data-profile statements this round. None of these dialects implements them yet, so each takes the fourth answer the walk demands -- a named row with a reason -- rather than reading as an accident. The reasons are the audit's findings, written where the next reader will look: MySQL and BigQuery can only answer by aggregating and simply have not been written; MariaDB's mysql.column_stats is empty unless an operator asked for statistics, so answering from it would report an empty profile as a measurement; PostgreSQL's pg_stats carries a negative _distinct when it means a fraction and hides min/max inside histogram_bounds, so a naive projection would put a negative count and a string bound in slots that promise otherwise; Oracle's low_value/high_value are RAW and need per-type conversion; SQL Server keys its statistics by stats object rather than column; ClickHouse and Firebird keep no per-column statistics at all. Snowflake also needed the previous round's two rows: its collection error had been masking them, which is exactly what the matrix-exactness assertion is there to catch. MySQL's file uses a different table shape -- a string exception name rather than the class -- so its rows are recorded there in that form.
…imate The reference implementation of the data layer: pg_stats keeps one row per analysed column, so profiling a table is one catalog query however large the table is. That cheapness is exactly why source has to travel with the numbers -- every figure here is an estimate of the last ANALYZE, not a count. Three encodings had to be undone, and each needs the whole row: * _distinct is negative when it means a *fraction* of the row count, so the count is -n_distinct * reltuples; a positive value is already a count. Converting it in SQL would mean dividing two columns, which reads worse than doing it beside the row estimate it uses. * ull_frac is -1 on a column measured before the statistics were reset, and eltuples is -1 on a relation the planner has never scanned. Both are "unknown", and neither is minus one of anything. * min/max exist only inside histogram_bounds, an �nyarray. Its endpoints are read by subscript -- (arr)[1] and (arr)[ARRAY_LENGTH(arr, 1)] -- which is NULL-safe without a CASE and, unlike MIN(anyelement), does not require the column's type to have a btree opclass: a json column has statistics and no histogram, and a MIN() formulation would refuse to plan for it. histogram_bounds excludes the values ANALYZE retained as most-common, so its endpoints are the extremes of what is *left*. The parser merges the retained list back in, comparing only like kinds (two numbers, or two strings -- which covers text, and dates and uuids in their ISO spellings), so a status column whose minimum is also its commonest value reports that minimum rather than the nearest uncommon neighbour. The value listing is a projection rather than an aggregate: ANALYZE already keeps most_common_vals and most_common_freqs in lockstep. They travel as o_jsonb -- the one shape that survives every element type -- and are paired by position here. The requested limit is applied after they arrive, since what comes back is a fixed-size list there is nothing to slice in SQL. source is ESTIMATE when the analysis time is known and UNKNOWN when a pg_stat_reset() erased it: the numbers are estimates either way, but the rule for this family is that an estimate carries its vintage, and claiming a vintage the server did not report would be worse than admitting there isn't one. Join coverage stays unclaimed, with the probe answering False and saying why: it is a double scan rather than a catalogue read, and the statement has not been written. The roundtrip table drops the two rows that now render and keeps that one, which is what the matrix-exactness assertion exists to enforce.
The new statements rendered, which proves they are well-formed and nothing about whether PostgreSQL accepts them -- nor whether the parser's conversions land where a hand count says they should. Only a server answers that, and `pg_stats` is the part of this family that cannot be faked. The table is a thousand rows so every figure is arithmetic: a unique key, a three-value text column, a unique numeric column, and a text column that is NULL in 60% of rows. The size is deliberate -- with a handful of rows PostgreSQL may keep every value as a most-common value and no histogram, which would test one branch of the min/max merge and leave the other unexercised. A thousand rows guarantees a histogram for the unique columns and a most-common list for the low-cardinality one. Three things worth pinning beyond the arithmetic: * a unique column's `n_distinct` arrives as `-1`, so the count of 1000 is the negative-value conversion working rather than a raw number that happened to read right; * a never-analysed table answers with an empty list, not a zero-filled profile; * PostgreSQL's most-common list never contains NULL -- `null_frac` is where the nulls are -- while SQLite's grouped scan does return a NULL row. Same question, two shapes, which is why the caller reads `null_fraction` instead of looking for a NULL entry.
The new live test failed on the first real server, and the failure was the informative kind: `cannot subscript type anyarray`. `pg_stats` declares `histogram_bounds`, `most_common_vals` and `most_common_freqs` as `anyarray` -- a pseudo-type standing for "an array of whatever this column is" -- and PostgreSQL refuses both subscripting and `MIN(anyelement)` over a pseudo-type, since there is no element type to resolve an operator against. `array_to_json` is the one function declared over `anyarray` itself, so the arrays travel through it and the parser picks the endpoints out of the JSON. The cost, now stated where a reader will look: a bound arrives as a JSON scalar rather than the driver's own object. Numbers stay numbers; a date, a timestamp or a UUID reads as its ISO text. That is a limit of `pg_stats` rather than a choice -- nothing in the catalog can hand back a typed endpoint without knowing the column's type at plan time. Worth recording how this was found: the statement rendered perfectly, the roundtrip walk accepted it, and only executing it against a server exposed the pseudo-type. Rendering proves a statement is well-formed, not that it runs.
The live test's second finding. `ANALYZE` ran, the figures were right, and `source` came back `unknown` -- because `pg_stat_all_tables.last_analyze` is written by the statistics collector *asynchronously*, so a profile read immediately after the analysis usually finds no timestamp yet. That is not a bug to paper over: it is the family's own rule working. The rule says an `ESTIMATE` carries the time it was collected, precisely so a stale number cannot pass for a fresh one. When the server has not recorded that time, the honest answer is that the age is unknown -- calling it `ESTIMATE` with an empty timestamp would claim an age the server never reported. So the pairing stays as it is and the reasoning is written down: two things leave the time unrecorded, a `pg_stat_reset()` and the collector's own lag, and neither is an error. The test now pins the invariant rather than one value -- `(source == "estimate") == (collected_at is not None)` -- which is the property that matters and holds whatever the collector happens to have flushed.
With this statement the data-profile family is complete on PostgreSQL. What it answers is the overlap of two columns' value domains, and it answers in one row carrying both shares: how much of the left column the right one accounts for, and the same the other way round. Neither figure means much alone -- a left share of 1.0 beside a right share of 0.2 says every tested value exists over there while most of that domain is unaccounted for -- so a single similarity score would average away exactly the asymmetry a caller needs to decide whether a join is worth writing. The semantics are the ones the SQLite implementation established, spelled for this engine. Both fractions count distinct values rather than rows, because a key repeated a thousand times that matches once would otherwise read as well covered. NULLs are excluded from the numerators and from the divisors: a NULL is not a value that could ever join, and in the numerator the exclusion is load-bearing, since SELECT DISTINCT would otherwise treat it as one value and credit the tested side with a match it does not have. Each divisor is guarded by NULLIF, so a column with nothing to match answers NULL rather than a zero that would read as a real measurement; the parser already turns that into "no measurement". The `* 1.0` is not decoration either: PostgreSQL truncates integer division, so without it every share below one would come back as zero. This is the one formatter here whose identifiers are not the catalog's. Both sides are user tables, so the source and its columns are built as Table and Column nodes and the dialect quotes them, schema and aliases included; _catalog_column writes names the way pg_catalog spells them, which is how a catalog relation is spelled and not how a user's table is. The two sides get distinct aliases so the measurement can name which direction it is reporting. The price is stated rather than hidden. supports_join_coverage now answers True, and its docstring and the formatter's say the same thing: this is a double scan plus an intersection, it belongs behind an explicit call and never in an automatic pass, and its fractions are a lower bound when the two columns' collations differ, because values the server would compare as equal may be counted as distinct here. The roundtrip table drops its last introspection row. That entry existed only because the statement did not, and it is the table's exactness assertion that made removing it part of writing the formatter. The data-profile test gains a coverage class measured against two hand-sized tables -- 1..4 against 2..6 must answer 3/4 and 3/5 -- together with the three cases that separate a measurement from a plausible guess: repeated values counted once, disjoint domains answering 0.0 rather than None, and a side with no non-NULL value answering no fraction at all. No PostgreSQL server is reachable from this machine, so CI is where the statement meets the engine for the first time. Its arithmetic was checked locally by executing the rendered text against SQLite under an attached `public` schema, which answered 0.75/0.6, 0.5/0.5, 0.0/0.0, NULL/0.0 and NULL/NULL for the cases the tests assert.
The introspect mapping named eleven types while the dialect answered six operational and three data families that no type reached. A capability the CLI cannot reach is the same defect as a statement declared and never rendered: the dialect says the server in front of us can answer, and no question gets the answer out. Core's introspection rule states it in one line -- the type lands in the same change as the capability -- so the nine now ride _TYPE_CAPABILITIES beside the existing entries, through the same _require_capability gate, so a server whose probe says False is still refused rather than answered with an empty listing. unused-indexes reads list_unused_indexes, the indexes the statistics collector reports as unchosen since the statistics were last reset. table-stats reads the per-table usage counters and runtime reads the live measurements, both as the statistics collector keeps them, and index-stats answers "how much" per index where unused-indexes answers "ever". All four are listings over the connection's scope, so none takes a name. dependencies asks list_object_dependencies: the whole graph without a name, or one object's dependants with it, which is the "what breaks if I drop this" question. ddl asks get_ddl_statement for one named table and prints the statement itself in table output, because a CREATE is text and folding it into a one-column table buries it; the structured formats keep the whole row with object_type and native beside it. The three data families ask what the rows hold rather than what the schema declares. column-profiles asks list_column_data_profiles for every column of one table, value-samples asks sample_column_values for one column of it, and join-coverage asks estimate_join_coverage for both sides of a two-column measurement -- table and column on each side, four names in all. The column is named on --column and the right-hand side on --right-table and --right-column; a flag the chosen type has no question for is refused rather than ignored, the same way a name already was, and a measurement that comes back empty is reported as no measurement instead of a row of nulls that would read as zero overlap. Both dispatch chains carry the nine branches. The sync and async handlers are written out separately because the executor is the only thing that differs, and the tests drive both: one case per type asserts the method and every argument, and the refusal path is checked across the whole type list. The offline CLI tests' fake introspector gains one method per family, so the parametrized gates now cover every advertised type instead of the first eleven. Only the paths that need a live catalogue are left to CI; on this machine there is no PostgreSQL, so the answers are pinned at the dispatch boundary rather than read from a running server.
…ture branch ix/column-constraint-gates has been merged: its tree and elease/v1.0.0.dev17 are identical (the branch carries 21 commits the release commit squashes into one, and the content matches). Pinning CI to the feature branch was always a temporary arrangement -- it meant every job installed a ref that only existed to get one fix in, and the day that branch was deleted the suite would have started failing to install rather than failing a test. The release line is now the thing to test against, which is also what the core repository already pins.
Core now ships SetOperationStatement in backend.expression.statements, re-exported from backend.expression, with format_set_operation_statement on SetOperationMixin and a SetOperationSupport protocol. The roundtrip matrix enumerates every class in the package, so this class had to answer for itself in four parts: it renders and the rendered SQL plus its bind parameters are byte-identical after all three round-trips, or it refuses with UnsupportedFeatureError because the dialect lacks the feature, or it refuses with NotImplementedError because it is an abstract base, or it refuses for some other reason and is named in the not-rendered table with that reason. Every one of those is asserted exactly in both directions, so a class nobody listed is a failure and so is a row left behind after the class started working. Each backend here now gives the answer its own dialect earns, and pins the SQL where it renders, because a render verdict without the bytes says nothing about the dialect.
Provenance notes in src docstrings, test modules and docs examples now refer to the originating servers through scenario names instead of development-endpoint details; named-connection fixtures use neutral values. The scenario-to-endpoint mapping is defined only in the gitignored tests/config. No behaviour change; keeps dev-environment endpoints out of tracked source.
The profile parser already said, in its own docstring, that source is ESTIMATE whenever the vintage is known and UNKNOWN when it is not -- and then wrote the words "estimate" and "unknown" into the field. It now assigns ProfileSource.ESTIMATE and ProfileSource.UNKNOWN, which is the same two values a consumer reads and a closed set a checker can name; the JSON artifact is unchanged because the enum is a str subclass, and the parser test asserts the members by identity as well as by value. This repository is one of the engines that override core's sample_column_values so it can read pg_stats instead of scanning the rows, and both overrides wrote limit: int = 100 -- a third and fourth spelling of the value core already owns as VALUE_SAMPLE_LIMIT. Both now reference that one name, so the twins still offer a default and offer the same object; a new test reads both signatures by inspect and asserts identity against core's constant, because a second 100 would compare equal and still be a second place to move. No CLI here ever defined DEFAULT_VALUE_SAMPLE_LIMIT and no CLI passed a limit, so there was nothing to delete on that half, and this repository prints no aggregate-profile note, so there was no sentence to inline. The output choices were spelled four times: introspect, query and status each carried the four-format list and info carried a two-format list under the same name. The four-format list is now OUTPUT_CHOICES in cli/output.py beside the provider factory that must be able to build every member, the narrower one is INFO_OUTPUT_CHOICES in the same file so the name says which surface it belongs to, and each subcommand imports the list it offers. Ruff, the CLI suites and the roundtrip matrix pass; the data-profile tests and anything that needs a server are for CI.
This backend's introspection modules import AbstractIntrospector, ProfileSource, ColumnDataProfile, introspection.defaults.VALUE_SAMPLE_LIMIT and the nine new introspection query statement classes, none of which exist in core 1.0.0.dev29. The >=1.0.0.dev0 floor admitted dev29, which is the version PyPI publishes and what pip install resolved, so the install succeeded and the first import failed. The pyproject floor is now >=1.0.0.dev30,<2.0.0, the line that actually carries these symbols, so a fresh resolve either gets a core that imports or stops instead of quietly installing a broken one. requirements-3.8.txt, the Python 3.8 install path, also names core with no floor at all, which resolved dev29 by the same accident; it now carries the same honest floor. The extras deliberately carry no core floor of their own: pip installs extras on top of the main dependencies, so they inherit this floor rather than resolving core independently.
The NAME rule lived in twenty hand-written ifs, two per type per chain, each recomputed where it was needed and none of them asserted anywhere. Four of the listings over the statistics collector's scope -- unused-indexes, table-stats, runtime, index-stats -- had no check at all, so a name given beside one was parsed and dropped and the run printed the family listing under a one-object question. The module docstring already promised NAME was "never silently dropped"; the branches did not keep it. _NAME_RULE declares the answer once per type, and _validate_name reads it at one point in each chain, so a type added without a rule is a type nobody has decided for rather than one that happens to forget. The rule is now pinned three ways: as data (the whole table), as a set (every advertised type has an entry), and as behaviour (each category through a run). The four counter listings move from silently dropping a name to refusing it with exit 2 and the sentence the other family listings already used. Every type that was refused still is, with the same code and the same "does not take an object name" wording; every type that required a name still does, with the same code and the same "Table name is required" sentence. The refusal order is unchanged -- it stays after the capability gate and the stray-flag check, where the branches used to make it, so a run against an unreachable server still reports the connection rather than the argument. No changelog fragment: no user-visible behaviour changes except the four families that used to swallow a name.
…e it The floor is right; the order that enforced it was not. Each matrix job installed the requirements and extras first, and the Python 3.8 jobs install requirements-3.8.txt, which names the core floor directly. pip therefore resolved rhosocial-activerecord from PyPI, whose newest release is 1.0.0.dev29, and stopped with an unsatisfiable >=1.0.0.dev30,<2.0.0 before the step that installs the matching core from git ever ran. The other jobs had the same order without anything in their files naming the core, so the guarantee was an accident of which file each interpreter happened to read. The order was already wrong before the floor moved: the old >=1.0.0.dev0 floor let the wrong core install silently and moved the failure to import time. Every job in this workflow now installs the core branch and the testsuite pin from git before the requirements and extras. Those files are unchanged, constraints included; pip now finds an installed 1.0.0.dev30 that satisfies >=1.0.0.dev30,<2.0.0 and installs only the pieces still missing from the index, instead of looking for a core version PyPI does not have. The floor, the version, the git refs and the testsuite pin are untouched; the install order is the only change.
Core defines the probes and the layers; this backend's page now states what PostgreSQL itself answers. It adds the twenty-one-probe matrix, the six operational families read from the statistics collector, and the three data families read from pg_stats, including the four encodings that surface as traps: anyarray bounds through array_to_json, signed n_distinct, -1 as unknown, and a histogram that omits the kept most-common values. The operational section names each source and its tense, since the counters are only since the last statistics reset, and states that DDL is an assembly with supports_ddl_extraction_native false. The data section states when the profile reports ESTIMATE or UNKNOWN and why an asynchronous last_analyze usually leaves the timestamp unset, and why a never-analysed table answers with no rows. Join coverage is described as a two-sided measurement over distinct non-null values, with NULLIF so an empty column answers None and zero overlap answers 0.0, and as a double scan that is never implicit. The CLI documentation gains the nine family types plus table-storage, the NAME rule for each, and the note that ddl legitimately reports no available text and exits non-zero. The Chinese mirror follows the same headings and carries the same code blocks, now in English.
This corrects the page the earlier commit added. The operational section now says five of the six families read the statistics collector and that DDL is the exception reading pg_catalog, rather than generalising across all six. The data section now separates the two pg_stats-backed families from join coverage, which reads the rows of both tables because pg_stats keeps only a bounded sample. The CLI note now says the advertised type list is static and the capability switch is asked after the dialect has adapted, rather than implying each type is offered because its switch answers True. The stale version table, which claimed full partition information support against a probe that answers False, is replaced by the one version gate in this code: pg_sequences is read only on PostgreSQL 10 and later, and an unknown version takes the narrower form.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/v1.0.0.dev17 #64 +/- ##
========================================================
+ Coverage 77.58% 77.91% +0.33%
========================================================
Files 353 355 +2
Lines 17043 17504 +461
Branches 2752 2805 +53
========================================================
+ Hits 13222 13638 +416
- Misses 2834 2868 +34
- Partials 987 998 +11
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Completes core PR #128's backend side (merged into
release/v1.0.0.dev30at67a76716), which settled.claude/rules/introspection.md§5–§6: ownership is decided by divergence, never byextra.WALInfo,ReplicationInfo,ArchiveInfoandSecurityInfomove here besidePostgreSQLServerOverview;get_overview()keeps its signature.pg_relation_sizeandpg_indexes_sizebecome nodes bound to the row's oid, the total rides the table listing;reltuplesis preferred for row count withn_live_tupas fallback and-1→None; CLI gainstable-storage.pg_stat_user_indexes,pg_stat_user_tables,pg_stat_activity,pg_depend;pg_stat_statementsis not read. DDL is an assembly with no text:get_ddl_statementreturnsNone(CLI exit 1);supports_ddl_extraction_nativeisFalse.pg_stats. Profiles and samples cost one query, labelledESTIMATE/UNKNOWNby vintage; arrays useto_jsonb; coverage is exact._TYPE_CAPABILITIES, with per-type NAME;supports_extensionsdelegates to one answer.async_support.pysuppliesrun_coroutine()for the CLI's three--asyncmodes; tests use pytest-asyncio's loop factory rather than the deprecated policy.Type of change
feat: A new featuredocs: Documentation only changes (if related to the new feature)perf: A code change that improves performance (if part of the feature)test: Adding missing tests or correcting existing tests (for the new feature)chore: Changes to the build process or auxiliary tools and libraries suchs as documentation generation (if related to the new feature)ci: Changes to our CI configuration files and scripts (if related to the new feature)build: Changes that affect the build system or external dependencies (if related to the new feature)revert: Reverts a previous commitBreaking Change
Applicable
rhosocial-activerecordVersion Range:>=1.0.0.dev30,<2.0.0Impact on End-Users:
None: signatures unchanged; the CLI gains types and
--asyncon Windows.Impact on Backend Developers (Custom Implementations):
Status types import here;
get_overview()returnsPostgreSQLServerOverview;SyncPostgreSQL…becomesPostgreSQL…, and core'sIntrospectorExecutorreplaces the sync executor. Storage fills the slots or leaves themNone; async callers need a selector loop on Windows.Related Repositories/Packages
python-activerecord— #128 (rule and core minimum).python-activerecord-testsuite— CI here pinsrelease/v1.0.0.dev17.Testing
Test Plan:
16 test files touched, 4 new (storage, async, profile, profile source), plus conftests. No local full suites; CI covers PostgreSQL 9–18 on Python 3.8–3.15.
Checklist
README.md, etc.).changelog.d/<issue_number>.<type>.md).Additional Notes
requirements*.txt, pinned torelease/v1.0.0.dev30(which now carries #128) with testsuiterelease/v1.0.0.dev17, so this PR is verified against the merged core.>=1.0.0.dev30,<2.0.0inpyproject.tomlandrequirements-3.8.txt.