Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,10 @@ refusal — never a silently wrong or incomplete result:
tooling, then state it in the file. A serial cannot state one, so export
refuses a serial whose sequence was declared otherwise.
- **Partitioned tables** support in-place statement changes, but cannot be
expressed in or exported to desired files.
expressed in or exported to desired files, and `migrate` refuses
`ATTACH PARTITION` and `DETACH PARTITION`.
- **Index maintenance** is not driven: `migrate` refuses `DROP INDEX` and
`REINDEX` (plain or `CONCURRENTLY`); see the `--accept-blocking` note below.
- **Unlogged tables and explicit column collations** are outside the
declarative model: converging either is a table (or column) rewrite, so
export and diff refuse rather than plan one.
Expand Down
11 changes: 8 additions & 3 deletions docs/capabilities-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ become `area`; the first column is named `operation`, `table_shape`, or
- `status_mark`: exactly `✅`, `🟡`, `⚪`, `🔵`, or `❌`. They mean, respectively,
supported today, planned with a typed refusal today, out of scope because there is
no online-safety problem, out of scope because another tool class owns the work, and
out of scope because PostgreSQL has no online mechanism.
out of scope because there is no online mechanism pg-sprite will drive: PostgreSQL
has none, or the only online form is a direct statement the operator runs.
- `tier`: `t1`, `t2`, or `t3`, derived consistently from the mark but stored and
validated so a consumer need not understand emoji. `✅` is T1, `🟡` is T2, and all
three remaining marks are T3.
Expand Down Expand Up @@ -214,9 +215,13 @@ row and a consumer reading a verdict reach the same route:
| T2 / `🟡` | `capability-boundary` |
| T3 / `⚪` | `no-online-safety-problem`, with `owner: direct-operator` |
| T3 / `🔵` | `no-online-safety-problem`, with `owner` naming the tool class |
| T3 / `❌` | `by-design` |
| T3 / `❌` | `by-design` (see below for index maintenance) |

T1 / `✅` has no capability refusal class. Two classes have no matrix row: `environmental`
The `❌` rows for index maintenance (`drop-index`, `reindex`) span two classes: the
plain form refuses as `by-design`, while the concurrent form, which is the direct
statement the operator runs, refuses as `no-online-safety-problem` with
`owner: direct-operator`. Both carry the row's `index-statement` reason. T1 / `✅` has
no capability refusal class. Two classes have no matrix row: `environmental`
describes the run site — privileges, contention, budgets, or other conditions — not whether
pg-sprite supports an operation, and `invariant-violation` reports a defect in pg-sprite,
not a property of the operation. The matrix and that contract must change together.
Expand Down
42 changes: 24 additions & 18 deletions docs/capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,9 @@ pg-sprite capabilities --json | jq '.capabilities[] | select(.engine_path == "co
# in diff, but can execute through migrate --desired.
pg-sprite capabilities --json | jq '.capabilities[] | select(.front_doors.diff == "refused")'

# Rows another tool class owns: the ⚪ and 🔵 rows. The ❌ rows name no owner, because
# PostgreSQL offers no online mechanism for them, so this is not the full out-of-scope set.
# Rows another tool class owns: the ⚪ and 🔵 rows. The ❌ rows name no owner: they have
# no online mechanism pg-sprite will drive (PostgreSQL has none, or the only online form
# is a direct statement the operator runs), so this is not the full out-of-scope set.
pg-sprite capabilities --json | jq '.capabilities[] | select(.owning_tool_class != null)'
```

Expand Down Expand Up @@ -96,9 +97,9 @@ Two consequences follow, and they explain most of this page:

| Tier | Meaning | What you see today |
| --- | --- | --- |
| **T1 — supported today** | The engine executes the change through an online-safe pattern | Execution (or the safer rewritten sequence), exit 0; where `migrate` refuses a plain index-maintenance form and names its `CONCURRENTLY` idiom, `--accept-blocking` runs the plain form under budgets and exits 3 |
| **T1 — supported today** | The engine executes the change through an online-safe pattern | Execution (or the safer rewritten sequence), exit 0 |
| **T2 — planned** | A known online pattern exists (or requires the copy-and-swap engine); building it is on the roadmap | A **typed refusal** naming the reason, exit 2 — never a silent fallback to a blocking form |
| **T3 — out of scope by design** | No online-safety problem to solve, solving it belongs to a different tool class, or PostgreSQL offers no online mechanism to build on | A typed refusal or a parse-level rejection, with the reason stating *why it is not planned* |
| **T3 — out of scope by design** | No online-safety problem to solve, solving it belongs to a different tool class, or there is no online mechanism pg-sprite will drive (PostgreSQL has none, or the only online form is a direct statement the operator runs) | A typed refusal or a parse-level rejection, with the reason stating *why it is not planned* |

T3 rows carry one of three marks, because they mean different things — and only one of
them is a limitation:
Expand All @@ -112,9 +113,12 @@ them is a limitation:
- **🔵 a different tool class owns it** — the job is real but belongs to another kind of
tool (data-change runners, provisioning/IaC, convergence planners, expand/contract
frameworks). The row's "Online-safety problem?" column names the class to look for.
- **❌ no online mechanism exists** — PostgreSQL itself provides no online pattern to
build on, so pg-sprite refuses rather than silently run the blocking form. These are
the only rows where "unsupported" is the honest reading.
- **❌ no online mechanism pg-sprite will drive** — PostgreSQL has none, or the only
online form is a direct statement the operator runs, so pg-sprite refuses rather than
silently run the blocking form. Index maintenance (`DROP INDEX`, `REINDEX`) is the
second case: the online form is a plain `CONCURRENTLY` statement, which `migrate`
refuses to wrap and names so you run it directly; for a single-relation plain form,
`--accept-blocking` runs it under budgets and exits 3.

Every matrix table carries an **"Online-safety problem?"** column: "Yes" means there is a
readers-and-writers problem for an online engine to solve (pg-sprite solves it, plans to,
Expand Down Expand Up @@ -151,7 +155,8 @@ engine:
shadow-copy engine; a typed refusal until that engine lands (see
[optimistic-attempt.md](optimistic-attempt.md) for how the router proves the rewrite).
- **—** — no engine path: either the job belongs to another tool class (⚪ / 🔵) or
PostgreSQL offers no online mechanism to build on (❌).
there is no online mechanism pg-sprite will drive (❌): PostgreSQL has none, or the
only online form is a direct statement the operator runs.

Path and tier are orthogonal on purpose: a 🟡 row's path says what kind of work lifts
the refusal — building a native flow or landing the copy engine — and a ✅ row's path
Expand Down Expand Up @@ -182,15 +187,16 @@ are the canonical example.
> `make gen-capabilities`; do not edit the generated regions below by hand.

<!-- capabilities:begin summary -->
**60 operations: 20 supported today, 24 planned behind a typed refusal, 14 out of scope
by design, and 2 with no online mechanism in PostgreSQL to build on.**
**60 operations: 16 supported today, 26 planned behind a typed refusal, 14 out of scope
by design, and 4 with no online mechanism for pg-sprite to build on.**
<!-- capabilities:end summary -->

Status legend: ✅ T1 (supported today) · 🟡 T2 (planned; typed refusal today) ·
⚪ T3 (out of scope; **no online-safety problem** — run directly, or through whatever
review the object warrants) ·
🔵 T3 (out of scope; **a different tool class owns it**) ·
❌ T3 (out of scope; **no online mechanism exists** in PostgreSQL).
❌ T3 (out of scope; **no online mechanism pg-sprite will drive**: PostgreSQL has none,
or the only online form is a direct statement the operator runs).

### Column changes

Expand Down Expand Up @@ -231,8 +237,8 @@ review the object warrants) ·
| Operation | Status | Engine path | Online-safety problem? | Behavior and why |
| --- | --- | --- | --- | --- |
| `CREATE [UNIQUE] INDEX` on a plain table — including partial, expression, covering (`INCLUDE`), GIN/GiST/BRIN | ✅ | native, safer sequence | Yes | Executed as (or rewritten to) `CREATE INDEX CONCURRENTLY`, with validity verification, typed invalid-index outcomes, and a proven recovery for abandoned leftovers (`RebuildAbandonedIndex`, or `DropAbandonedIndex` to remove the leftover without rebuilding; both library-only; [runbook](invalid-index-recovery.md)) |
| `DROP INDEX` | ✅ | native, safer sequence | Yes | Rewritten to `DROP INDEX CONCURRENTLY`; flagged **destructive**. `migrate` refuses the plain form and names the idiom; a single-index `DROP INDEX` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0). `diff` refuses dropping the table's replica identity index, or one a partition's is attached under, with `replica-identity`. `migrate` refuses every plain `DROP INDEX` before it reads the table, so its dry-run names the concurrent idiom; `--accept-blocking` checks the replica identity under the table's lock and refuses the drop as `unsupported-statement`, naming the identity index. pg-sprite never runs the concurrent drop itself |
| `REINDEX` | ✅ | native, safer sequence | Yes | Rewritten to `REINDEX ... CONCURRENTLY`. `migrate` refuses the plain form and names the idiom; a plain `REINDEX INDEX` or `REINDEX TABLE` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0). `REINDEX SCHEMA`, `DATABASE`, and `SYSTEM` stay refused |
| `DROP INDEX` | ❌ | — | Yes — the online form is a direct `CONCURRENTLY` statement that `migrate` does not drive | `migrate` never runs this statement online-safely: the plain form is refused and names `DROP INDEX CONCURRENTLY`, and the concurrent form is refused as already safe — run it directly against the database. A single-index plain `DROP INDEX` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0); `--accept-blocking` checks the replica identity under the table's lock and refuses the drop as `unsupported-statement`, naming the identity index. `diff` reports an index to remove as a **destructive** `DROP INDEX` and hints to run it through `migrate --alter`, which ends in this `index-statement` refusal — run `DROP INDEX CONCURRENTLY` directly instead. `diff` refuses dropping the table's replica identity index, or one a partition's is attached under, with `replica-identity` |
| `REINDEX` | ❌ | — | Yes — the online form is a direct `CONCURRENTLY` statement that `migrate` does not drive | `migrate` never runs this statement online-safely: the plain form is refused and names `REINDEX ... CONCURRENTLY`, and the concurrent form is refused as already safe — run it directly against the database. A plain `REINDEX INDEX` or `REINDEX TABLE` can be run as-is under bounded budgets with `--accept-blocking SCHEMA.TABLE` (exit 3, never 0). `REINDEX SCHEMA`, `DATABASE`, and `SYSTEM` stay refused. A desired schema cannot express a reindex, so `diff` never produces one |
| Index build on a **partitioned parent** | 🟡 | native, planned flow | Yes | PostgreSQL has no parent-level `CONCURRENTLY`; the blocking form is refused by policy (`--force` does not bypass it, and `--accept-blocking` does not reach it yet). The partition-aware flow — `CREATE INDEX ON ONLY` → per-partition CIC → `ATTACH PARTITION`, with crash-resume per leaf — is planned |
| `ADD CONSTRAINT ... USING INDEX` on a partitioned parent | ❌ | — | Yes — unsolvable today | PostgreSQL does not support adopting an index on a partitioned parent in any supported version; refused before execution |
<!-- capabilities:end indexes -->
Expand All @@ -243,8 +249,8 @@ review the object warrants) ·
| Operation | Status | Engine path | Online-safety problem? | Behavior and why |
| --- | --- | --- | --- | --- |
| `CREATE TABLE ... PARTITION OF` | 🟡 | native, planned flow | Yes | Typed refusal at both doors: the imperative door does not take `CREATE TABLE`, and the declarative create path refuses the form at plan time and re-checks it at apply — attaching a partition takes a brief `ACCESS EXCLUSIVE` on the **parent**, which the greenfield absence proof does not cover. A desired file for a partition that already exists is refused before planning builds anything, since the scratch build would attach to the live parent. The partition-aware flow is planned |
| `ATTACH PARTITION` | ✅ | native, as-is | Yes | Executed; the safer idiom (pre-prove the bound with a validated `CHECK` so the attach skips its scan) is surfaced as guidance. A classify-first flow that constructs the proof itself is planned |
| `DETACH PARTITION [CONCURRENTLY]` | ✅ | native, safer sequence | Yes | `CONCURRENTLY` is the idiom; the blocking form is rewritten to it |
| `ATTACH PARTITION` | 🟡 | native, planned flow | Yes | Without `--force`, a typed refusal with nothing executed: the attach scans the partition to validate its bound unless a validated `CHECK` already proves it, and the engine does not yet construct that proof. The safer idiom (pre-prove the bound with a validated `CHECK` so the attach skips its scan) is surfaced as guidance. `--force SCHEMA.TABLE` runs the blocking `ATTACH PARTITION` as submitted under bounded budgets, taking the blocking lock, and records the override. The classify-first flow that constructs the proof itself is planned. A desired schema cannot express an attach, so `diff` never produces one |
| `DETACH PARTITION [CONCURRENTLY]` | 🟡 | native, planned flow | Yes | `CONCURRENTLY` is the idiom and the blocking form is rewritten to it (`--dry-run` and `suggest` show the rewrite), but without `--force` `migrate` refuses both forms before anything executes: a cancelled concurrent detach leaves a detach-pending partition state the executor does not yet own recovering. `--force SCHEMA.TABLE` runs a plain `DETACH PARTITION` as submitted under bounded budgets, taking the blocking lock; `DETACH ... CONCURRENTLY` stays refused under `--force`. `migrate --dry-run` of either form reports the rewrite as a safer sequence to run and exits 0 while the real run refuses and exits 2, so the dry-run exit code is not a submit gate for a detach. A desired schema cannot express a detach, so `diff` never produces one |
| Partitioned parents in the **declarative model** | 🟡 | native, planned flow | Yes | Typed refusal: the model does not yet carry partition keys, and rendering a partitioned parent as a plain `CREATE TABLE` would be silently wrong |
| Partitioned tables in **copy-and-swap** | 🟡 | copy-and-swap | Yes | Root-vs-leaf publication semantics and per-partition swap; sequenced after the copy engine core |
<!-- capabilities:end partitioned_tables -->
Expand Down Expand Up @@ -274,7 +280,7 @@ review the object warrants) ·
| --- | --- | --- | --- | --- |
| Enum-typed columns on plain tables | 🟡 | native, planned flow | Yes | Tolerance end to end (introspection already canonicalizes via `format_type`; desired-file admission and transaction-scoped scratch-schema mechanics are being verified) |
| `ALTER TYPE ... ADD VALUE` | 🟡 | native, planned flow | Yes | Metadata-only and online-safe (PG 14+ allows it in a transaction; the value is usable after commit) — planned as an owned operation. No peer online executor owns it |
| Enum value rename / removal | 🟡 | copy-and-swap | Yes | PostgreSQL has no `DROP VALUE`; this is a type swap + table rewrite — routes to a typed refusal toward copy-and-swap |
| Enum value rename / removal | 🟡 | copy-and-swap | Yes | PostgreSQL has no `DROP VALUE`; this is a type swap + table rewrite, planned on copy-and-swap. Today the imperative door refuses `ALTER TYPE ... RENAME VALUE` as `unsupported-statement`, since it takes only `ALTER TABLE` and `CREATE INDEX`, and `DROP VALUE` does not parse |
| Enum/domain type creation and drop | ⚪ | — | No — owner tooling (psql, shipped with the code change) | Bootstrap/catalog work with no concurrent-access problem; owner tooling applies it in the same change that ships the code |
| Views, materialized views (create and replace) | ⚪ | — | No — owner tooling | Transactional catalog work, but `CREATE OR REPLACE VIEW` takes a brief `ACCESS EXCLUSIVE` on the view and queues behind in-flight readers — run it under a `lock_timeout` |
| `REFRESH MATERIALIZED VIEW` | 🔵 | — | No — data jobs / owner tooling | A data operation, not catalog work: the plain form holds `ACCESS EXCLUSIVE` on the matview for the whole rebuild (`CONCURRENTLY` needs a unique index and trades the lock for churn). Scheduling refreshes belongs to data jobs |
Expand Down Expand Up @@ -430,8 +436,8 @@ Three related jobs stay with humans on purpose:
the recovery handles, and what the rest license a human to do is
[invalid-index-recovery.md](invalid-index-recovery.md).
- **Index maintenance (`REINDEX` automation, bloat-driven rebuild scheduling).** The
engine executes `REINDEX CONCURRENTLY` when asked (see matrix); *deciding* when an
index needs rebuilding is monitoring-and-operations territory. Automating it is
engine does not drive `REINDEX` at all (see matrix) — run `REINDEX ... CONCURRENTLY`
directly; *deciding* when an index needs rebuilding is monitoring-and-operations territory. Automating it is
considered but on hold for the same ownership reasons as above.

---
Expand Down
Loading
Loading