Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions demo/tour.sh
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ dry_run() {
else
assert_eq "dry-run refusal exit of [$sql]" 2 "$status"
fi
assert_eq "plan format_version of [$sql]" 10 "$(jq -r '.format_version' <<<"$out")"
assert_eq "plan format_version of [$sql]" 11 "$(jq -r '.format_version' <<<"$out")"
Comment thread
aparajon marked this conversation as resolved.
assert_eq "disposition of [$sql]" "$disposition" "$(jq -r '.disposition' <<<"$out")"
assert_eq "route of [$sql]" "$route" "$(jq -r '.statements[0].route' <<<"$out")"
assert_eq "reason of [$sql]" "$reason" "$(jq -r '.statements[0].decisions[0].reason' <<<"$out")"
Expand Down Expand Up @@ -180,7 +180,7 @@ diff_plan() {
if [ "$CHECK" = 1 ]; then
out=$("$PGS" diff --url "$PG_DSN" --desired "$desired" --schema public --json) || status=$?
assert_eq "diff exit of [$desired]" 0 "$status"
assert_eq "diff format_version of [$desired]" 10 "$(jq -r '.format_version' <<<"$out")"
assert_eq "diff format_version of [$desired]" 11 "$(jq -r '.format_version' <<<"$out")"
assert_eq "statement count of [$desired]" "$count" "$(jq -r '.statements | length' <<<"$out")"
case "$(jq -r '.statements[0].sql' <<<"$out")" in
*"$fragment"*) ;;
Expand Down Expand Up @@ -208,7 +208,7 @@ diff_refused() {
if [ "$CHECK" = 1 ]; then
out=$("$PGS" diff --url "$PG_DSN" --desired "$desired" --schema public --json) || status=$?
assert_eq "diff refusal exit of [$desired]" 2 "$status"
assert_eq "diff format_version of [$desired]" 10 "$(jq -r '.format_version' <<<"$out")"
assert_eq "diff format_version of [$desired]" 11 "$(jq -r '.format_version' <<<"$out")"
assert_eq "diff disposition of [$desired]" refuse "$(jq -r '.disposition' <<<"$out")"
assert_eq "diff reason of [$desired]" unsupported-statement "$(jq -r '.statements[0].reason' <<<"$out")"
assert_eq "diff class of [$desired]" "$class" "$(jq -r '.statements[0].class' <<<"$out")"
Expand Down Expand Up @@ -243,7 +243,7 @@ run_pull() {
out=$("$PGS" diff --url "$PG_DSN" --schema public --desired "$desired" --json) || status=$?
assert_eq "diff exit of [$desired]" 0 "$status"
# Plan report contract version (plan.FormatVersion), same pin as diff_plan.
assert_eq "diff format_version of [$desired]" 10 "$(jq -r '.format_version' <<<"$out")"
assert_eq "diff format_version of [$desired]" 11 "$(jq -r '.format_version' <<<"$out")"
assert_eq "diff disposition of [$desired]" execute "$(jq -r '.disposition' <<<"$out")"
assert_eq "zero diff of [$desired]" 0 "$(jq -r '.statements | length' <<<"$out")"
else
Expand Down Expand Up @@ -326,7 +326,7 @@ run_offline() {
if [ "$CHECK" = 1 ]; then
out=$("$PGS" lint --json risky.sql) || status=$?
assert_eq "lint exit" 1 "$status"
assert_eq "lint format_version" 4 "$(jq -r '.format_version' <<<"$out")"
assert_eq "lint format_version" 5 "$(jq -r '.format_version' <<<"$out")"
assert_eq "lint errors" 1 "$(jq -r '.errors' <<<"$out")"
else
"$PGS" lint risky.sql || echo "(exit $? — error findings gate, by design)"
Expand Down
6 changes: 3 additions & 3 deletions docs/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ or the only online form is a direct statement the operator runs).
| `ADD COLUMN ... GENERATED ... STORED` | 🟡 | copy-and-swap | Yes | Table rewrite; copy-and-swap route. The copy engine must **recompute, never copy,** generated columns on the shadow table |
| `ALTER COLUMN ... SET EXPRESSION` (changed stored generated expression) | 🟡 | copy-and-swap | Yes | Planned but not executed. `diff` shows a changed stored generated expression as one `SET EXPRESSION` statement of kind `set-generated-expression`, refused with `unsupported-statement`, rather than a drop and re-add that would also drop the column's indexes. It recomputes every row under `ACCESS EXCLUSIVE`, and the statement needs PostgreSQL 17 or later |
| `ADD COLUMN` with inline `UNIQUE`/`PRIMARY KEY`/`REFERENCES`/`CHECK` | 🟡 | native, planned flow | Yes | The inline constraint does its index build or validation scan under the `ADD COLUMN`'s `ACCESS EXCLUSIVE` lock; refused with guidance to add the column first, then build the constraint online |
| `DROP COLUMN` | ✅ | native, as-is | Yes | Metadata-only; flagged **destructive** in the plan report. Refused with `replica-identity` when a replica identity index the drop reaches keys on the column: the table's own, or one on a partition or inheritance child below it unless the statement says `ONLY`. Refused with `dependent-view`, naming the views, when a view or materialized view reads the column, on a table the drop reaches, unless the statement says `CASCADE` |
| `DROP COLUMN` | ✅ | native, as-is | Yes | Metadata-only; flagged **destructive** in the plan report. Refused with `replica-identity` when a replica identity index the drop reaches keys on the column: the table's own, or one on a partition or inheritance child below it unless the statement says `ONLY`. Refused with `dependent-view`, naming the views, when a view or materialized view reads the column, on a table the drop reaches. Refused with `cascading-drop` when the statement says `CASCADE`, which would also drop the views that read the column |
| `ALTER COLUMN TYPE`, binary-coercible (proven against live column facts) | ✅ | native, as-is | Yes | Catalog relabel, e.g. `varchar(50)` → `varchar(100)`, `varchar` → `text`; refused with `dependent-view`, naming the views, when a view or materialized view reads the column; PostgreSQL itself refuses the change when a rule or `STORED` generated column depends on it — see [binary-coercible-type-changes.md](binary-coercible-type-changes.md#no-rewrite-is-not-no-cost) |
| `ALTER COLUMN TYPE`, general (or with `USING`) | 🟡 | copy-and-swap | Yes | Table rewrite; copy-and-swap route, refused today. A `USING` expression will be admitted only when the copy can reproduce it: immutable, naming only the retyped column, not the primary key (see limitations) |
| `SET DEFAULT` / `DROP DEFAULT` / `DROP NOT NULL` | ✅ | native, as-is | Yes | Metadata-only |
Expand All @@ -228,7 +228,7 @@ or the only online form is a direct statement the operator runs).
| `ADD CONSTRAINT ... NOT VALID` / `... USING INDEX` / `VALIDATE CONSTRAINT` | ✅ | native, as-is | Yes | Already the online idiom; executed as-is |
| `ADD FOREIGN KEY ... NOT VALID` on a **partitioned parent** | ✅ | native, safer sequence | Yes — server version 18 or later | Supported on version 18 and later; refused on 14–17 with an environmental class |
| `EXCLUDE` constraints (and unrecognized constraint forms) | ❌ | — | Yes — unsolvable today | No online pattern exists in PostgreSQL — the build scans under `ACCESS EXCLUSIVE` with no `NOT VALID`/`USING INDEX` equivalent. Refused; revisit only if PostgreSQL grows one |
| `DROP CONSTRAINT` | ✅ | native, as-is | Yes | Metadata-only; flagged **destructive**. Refused with `replica-identity` when the constraint owns the table's replica identity index, or the index a partition's is attached under |
| `DROP CONSTRAINT` | ✅ | native, as-is | Yes | Metadata-only; flagged **destructive**. Refused with `replica-identity` when the constraint owns the table's replica identity index, or the index a partition's is attached under. Refused with `cascading-drop` when the statement says `CASCADE`, which would also drop other tables' foreign keys on the constraint |
<!-- capabilities:end constraints -->

### Indexes
Expand All @@ -237,7 +237,7 @@ or the only online form is a direct statement the operator runs).
| Operation | Status | Engine path | Online-safety problem? | Behavior and why |
| --- | --- | --- | --- | --- |
| `CREATE [UNIQUE] INDEX` on a plain table — including partial, expression, covering (`INCLUDE`), GIN/GiST/BRIN | ✅ | native, safer sequence | Yes | Executed as (or rewritten to) `CREATE INDEX CONCURRENTLY`, with validity verification, typed invalid-index outcomes, and a proven recovery for abandoned leftovers (`RebuildAbandonedIndex`, or `DropAbandonedIndex` to remove the leftover without rebuilding; both library-only; [runbook](invalid-index-recovery.md)) |
| `DROP INDEX` | ❌ | — | Yes — the online form is a direct `CONCURRENTLY` statement that `migrate` does not drive | `migrate` never runs this statement online-safely: the plain form is refused and names `DROP INDEX CONCURRENTLY`, and the concurrent form is refused as already safe — run it directly against the database. A single-index plain `DROP INDEX` without `CASCADE` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0); `--accept-blocking` checks the replica identity under the table's lock and refuses the drop as `unsupported-statement`, naming the identity index. `diff` reports an index to remove as a **destructive** `DROP INDEX` and hints to run it through `migrate --alter`, which ends in this `index-statement` refusal — run `DROP INDEX CONCURRENTLY` directly instead. `diff` refuses dropping the table's replica identity index, or one a partition's is attached under, with `replica-identity` |
| `DROP INDEX` | ❌ | — | Yes — the online form is a direct `CONCURRENTLY` statement that `migrate` does not drive | `migrate` never runs this statement online-safely: the plain form is refused and names `DROP INDEX CONCURRENTLY`, and the concurrent form is refused as already safe — run it directly against the database. A single-index plain `DROP INDEX` without `CASCADE` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0); `--accept-blocking` checks the replica identity under the table's lock and refuses the drop as `unsupported-statement`, naming the identity index. `diff` reports an index to remove as a **destructive** `DROP INDEX` and hints to run it through `migrate --alter`, which ends in this `index-statement` refusal — run `DROP INDEX CONCURRENTLY` directly instead. `diff` refuses dropping the table's replica identity index, or one a partition's is attached under, with `replica-identity`. `plan` and `lint` refuse `DROP INDEX … CASCADE` with `cascading-drop`, since `DROP INDEX CONCURRENTLY` does not accept `CASCADE` |
| `REINDEX` | ❌ | — | Yes — the online form is a direct `CONCURRENTLY` statement that `migrate` does not drive | `migrate` never runs this statement online-safely: the plain form is refused and names `REINDEX ... CONCURRENTLY`, and the concurrent form is refused as already safe — run it directly against the database. A plain `REINDEX INDEX` or `REINDEX TABLE` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0). `REINDEX SCHEMA`, `DATABASE`, and `SYSTEM` stay refused. A desired schema cannot express a reindex, so `diff` never produces one |
| Index build on a **partitioned parent** | 🟡 | native, planned flow | Yes | PostgreSQL has no parent-level `CONCURRENTLY`; the blocking form is refused by policy (`--force` does not bypass it, and `--accept-blocking` does not reach it yet). The partition-aware flow — `CREATE INDEX ON ONLY` → per-partition CIC → `ATTACH PARTITION`, with crash-resume per leaf — is planned |
| `ADD CONSTRAINT ... USING INDEX` on a partitioned parent | ❌ | — | Yes — unsolvable today | PostgreSQL does not support adopting an index on a partitioned parent in any supported version; refused before execution |
Expand Down
16 changes: 8 additions & 8 deletions docs/cli-output-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ itself.
```console
$ pg-sprite migrate --alter 'ALTER TABLE users ADD COLUMN note text' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "users",
Expand Down Expand Up @@ -188,7 +188,7 @@ plans the safer online sequence instead: the decision carries it in
```console
$ pg-sprite migrate --alter 'ALTER TABLE users ADD CONSTRAINT users_email_key UNIQUE (email)' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "users",
Expand Down Expand Up @@ -290,7 +290,7 @@ column first, then build the constraint as a separate, named
```console
$ pg-sprite migrate --alter 'ALTER TABLE users ADD COLUMN nickname text UNIQUE' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "users",
Expand Down Expand Up @@ -327,7 +327,7 @@ implemented yet.
```console
$ pg-sprite migrate --alter 'ALTER TABLE users ALTER COLUMN id TYPE text' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "users",
Expand Down Expand Up @@ -364,7 +364,7 @@ The refusal cause is the report-level `reason`.
```console
$ pg-sprite migrate --alter 'CREATE INDEX events_created_idx ON events (created)' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "events",
Expand Down Expand Up @@ -404,7 +404,7 @@ the reviewer or orchestrator to gate on; `migrate` itself does not block it.
```console
$ pg-sprite migrate --alter 'ALTER TABLE users DROP COLUMN email' --dry-run --json
{
"format_version": 10,
"format_version": 11,
"source": "alter",
"schema": "public",
"table": "users",
Expand Down Expand Up @@ -453,7 +453,7 @@ ALTER TABLE users ADD CONSTRAINT users_email_key UNIQUE (email);
```console
$ pg-sprite lint /tmp/changes.sql --json
{
"format_version": 4,
"format_version": 5,
"postgres_versions": "14-18",
"findings": [
{
Expand Down Expand Up @@ -531,7 +531,7 @@ CREATE TABLE users (
```console
$ pg-sprite diff --desired /tmp/users.sql --json
{
"format_version": 10,
"format_version": 11,
"source": "diff",
"schema": "public",
"table": "users",
Expand Down
3 changes: 2 additions & 1 deletion docs/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ not escape hatches:
| `ATTACH PARTITION` | Without `--force`, refused with `not-native-safe-rewrite-required`, nothing executed: the engine does not yet construct the validated `CHECK` that lets the attach skip its scan. Pre-prove the bound yourself, then run the attach directly. `--force SCHEMA.TABLE` runs the blocking `ATTACH PARTITION` as submitted under bounded budgets, taking the blocking lock. |
| `DETACH PARTITION` and `DETACH PARTITION ... CONCURRENTLY` | Without `--force`, refused with `unsupported-statement` before anything executes: the plain form is rewritten to `CONCURRENTLY`, and the executor does not drive that step because a cancelled concurrent detach leaves a detach-pending state it does not yet own recovering. Run `DETACH PARTITION ... CONCURRENTLY` directly. `--force SCHEMA.TABLE` runs a plain `DETACH PARTITION` as submitted under bounded budgets, taking the blocking lock; `DETACH ... CONCURRENTLY` stays refused under `--force`. `migrate --dry-run` reports the rewrite as a step to run and exits 0 while the real run refuses (exit 2), so the dry-run exit code is not a submit gate for a detach. |
| Dropping what a replica identity index is built on | `REPLICA IDENTITY USING INDEX` outlives its index, so dropping the index, the primary key or unique constraint that owns it, or a column it covers leaves the table with no usable identity, and every `UPDATE` and `DELETE` on it fails while a publication publishes them. The planner refuses such a drop with `replica-identity`, on the table and on what the drop reaches below it: a `DROP COLUMN` reaches partitions and inheritance children unless the statement says `ONLY`, and a `DROP CONSTRAINT` reaches partitions, whose indexes are attached under the parent's. The executor reads the identity again under the lock the drop takes, so an identity that moved onto the dropped object after planning still refuses the run before anything commits. `migrate --accept-blocking` makes the same check before a `DROP INDEX`. Point `REPLICA IDENTITY` at another unique index first, or `DEFAULT` if a usable primary key remains after the drop, then re-plan. |
| Retyping or dropping a column a view reads | PostgreSQL refuses `ALTER COLUMN TYPE` and `DROP COLUMN` on a column a view or materialized view reads, even for a type change it would relabel in place. Without `ONLY`, a type change reaches the column on every partition and inheritance child too, and a drop reaches every partition and each inheritance child that inherits the column from this one parent without declaring it; a child that declares the column or inherits it from a second parent keeps its copy. The planner reads the views on every copy the change reaches from the catalog and refuses the change with `dependent-view`, naming each view in the plan and in the verdict's `dependent_views`, instead of letting the statement fail at the server. `DROP COLUMN … CASCADE` is not refused: the server drops the views with the column, and the drop is flagged destructive. pg-sprite does not drop and recreate the views for you. Drop or redefine them first, then re-plan. |
| Retyping or dropping a column a view reads | PostgreSQL refuses `ALTER COLUMN TYPE` and `DROP COLUMN` on a column a view or materialized view reads, even for a type change it would relabel in place. Without `ONLY`, a type change reaches the column on every partition and inheritance child too, and a drop reaches every partition and each inheritance child that inherits the column from this one parent without declaring it; a child that declares the column or inherits it from a second parent keeps its copy. The planner reads the views on every copy the change reaches from the catalog and refuses the change with `dependent-view`, naming each view in the plan and in the verdict's `dependent_views`, instead of letting the statement fail at the server. `DROP COLUMN … CASCADE`, which would drop the views with the column, is refused with `cascading-drop`. pg-sprite does not drop and recreate the views for you. Drop or redefine them first, then re-plan. |
| Dropping with `CASCADE` | `CASCADE` on `DROP COLUMN`, `DROP CONSTRAINT`, or `DROP INDEX` also drops whatever depends on the target: a view that reads the column, or another table's foreign key on the constraint. Those objects are not in the statement, so the plan cannot show them and the table's lock does not cover them. The planner refuses every such drop with `cascading-drop`. Drop each dependent object as its own change, then resubmit without `CASCADE`; PostgreSQL then refuses the drop while anything still depends on the target. |
| `ADD COLUMN` on a hot standby, or by a role without `TEMPORARY` | Whether adding a column rewrites the table is the server's answer, read by adding the column to an empty temporary table. A server that accepts no writes, or a role whose `TEMPORARY` on the database was revoked, cannot create that table, so the plan fails with `ErrAddColumnProbeUnavailable` before anything runs. Plan against the primary, or grant `TEMPORARY` on the database to the engine's role. |
| Copy-and-swap | The copy-and-swap backend is not yet available. Statements that require it route to `refuse`; pg-sprite never falls through to a blocking rewrite. |
| `ALTER COLUMN TYPE ... USING <expr>` the copy cannot reproduce | The copy-and-swap route is still closed (above); when it opens, the shadow builder admits a `USING` expression only when it can evaluate it again, identically, in the chunk copy, the verifier's recopy, and every change the catch-up flushes. Three rules, each refused as `shadow-statement-conversion` before any shadow is committed: the expression must be **immutable** by the server's own classification (no `now()`, `random()`, `current_setting()`, session-dependent casts such as `text` to `timestamptz`, or functions declared `STABLE` or `VOLATILE`; checked by trying it as a stored generated column's expression on the empty shadow inside the build's transaction, then rolling that attempt back), it must **name only the column it retypes**, bare and unqualified (no neighbouring column, no `t.col`, no whole-row `t.*`), and it must not retype the **primary key**, whose values every chunk range, watermark, and resume clear are expressed in. The expression is also refused when the deparser cannot prove it rendered the text faithfully. Rewrite the expression in terms of the retyped column alone and of `IMMUTABLE` functions, or retype without `USING` so the server's assignment cast converts the value; the refusal names the column, the expression, and that rewrite. The spellings most often refused, and what to write instead, are in [`USING` expressions the copy reproduces](#using-expressions-the-copy-reproduces) below. |
Expand Down
Loading
Loading