fix(frontend): persist data branch database identity

[gouhongshen]

What type of PR is this?

• API-change
• BUG
• Improvement
• Documentation
• Feature
• Test and CI
• Code Refactoring

Which issue(s) this PR fixes:

Related to #26068

What this PR does / why we need it:

Root cause

Database-branch identity was inferred only from active ordinary-table rows in mo_branch_metadata. A successfully created branch database therefore had no durable database-level identity when its source had no ordinary tables, or after all cloned ordinary tables were dropped. DATA BRANCH DELETE DATABASE could not distinguish it from an ordinary database and rejected the delete.

Design-review status

This PR is Draft and its persistent-contract implementation remains blocked pending independent approval of design revision data-branch-database-identity-2026-09-07-r8 in docs/design/data_branch_privilege.md section 12.

The proposed contract uses mo_database.dat_type = 'data-branch' behind cluster-wide MORPCVersion56. Below v56, new CNs reject database-branch creation before DDL and all consumers retain legacy behavior. Revision r8 uses v56 because current main assigns v48 to the generation-aware CDC watermark catalog contract, v49 to vector-level grouping-set projection expansion, v50/v51 to the ordered ODKU protocol contracts, v52 to MySQL binary JSON subtype tags, v53 to ordered-stream distributed Top-N merge, v54 to catalog-authenticated proxy prepared-cache reuse, and v55 to session-owned temporary DDL with transactional data. The proposal also defines upgrade, mixed-version, downgrade/rollback, physical backup, logical snapshot/PITR restore, alternatives, and deterministic compatibility tests.

The existing production implementation at this head predates that capability and restore work and must not be approved or merged until the design is approved and the code/tests are brought into exact alignment.

Current implementation

• Add the categorical data-branch database type and attach it to the internal CREATE DATABASE, so mo_database.dat_type is written atomically with the database definition. This avoids a second catalog update in the clone transaction.
• Validate database-level branch identity during database delete. Marked branch databases may have zero ordinary tables; every ordinary table that is present must still have an active branch receipt, so locally added tables continue to prevent deletion.
• Preserve deletion compatibility for pre-change branch databases that still have active ordinary-table receipts. Unmarked empty databases remain rejected.
• Treat data-branch databases as user databases in create/alter publication validation.
• Add focused UT and BVT coverage, including cross-account creation.
• Add the proposed v56 persisted-state and rollout design amendment. Production gating, restore preservation, and mixed-version tests intentionally await design approval.

Issue-to-test proof already present

• Empty source: asserts dat_type = 'data-branch', deletes the branch, and asserts the destination is absent.
• View-only source: verifies the cloned view is queryable, deletes the branch, and asserts the destination is absent.
• Sequence-only source: verifies the sequence is cloned, deletes the branch, and asserts the destination is absent.
• All cloned tables dropped: verifies both tables were cloned, drops both normally, deletes the branch, and asserts the destination is absent.
• Safety boundaries: verifies a locally added ordinary table blocks database deletion; after dropping it, deletion succeeds. An ordinary empty database remains rejected and present.
• Cross-account creation: asserts the target account's database row carries data-branch.
• Unit tests cover marked and unmarked empty databases, marked/legacy active tables, locally added tables, missing databases, subscription/unknown types, internal create-type propagation, and publication create/alter compatibility.

After design approval, deterministic v55/v56 creation, deletion, publication, snapshot, and PITR cases listed in section 12.7 are mandatory before this PR returns to Ready for review.

Tests run

• make build
• .agents/skills/mo-dev/scripts/mo-cgo-test -count=1 -timeout=20m ./pkg/catalog
• .agents/skills/mo-dev/scripts/mo-cgo-test -count=1 -timeout=20m ./pkg/sql/compile
• .agents/skills/mo-dev/scripts/mo-cgo-test -count=1 -timeout=20m ./pkg/frontend
• branch_database_identity.sql: 62/62 passed twice on the same local service instance
• Existing branch_metadata.sql control selection: 59/59 passed
• branch/privilege/branch_privilege.sql: 258/258 passed
• Post-test cleanup query: 0 related databases and 0 related accounts remained
• GitHub CI at pre-rebase head eb637e0ec10d8e3a6f2dad128161ab32d14c661f: all 28 checks reached successful, skipped, or neutral terminal states
• Rebased onto origin/main 39225f5fbc63b44a190cdd43f954ee07d78e6cdc; range-diff shows both PR commits remain patch-identical
• The current base preserves terminal dispatch results for late remote receivers, orders snapshot/view lifecycle locks across DDL transactions, rejects non-finite linear-reference parameters, adds a functional-index RFC, makes temporary DDL transaction-independent, and adds MongoDB sort/unwind pipeline stages. The temporary-DDL change owns MORPCVersion55 and overlaps pkg/sql/compile/ddl.go; semantic inspection confirms its CREATE TABLE routing is independent of this PR's typed CREATE DATABASE path. The lifecycle change still does not preserve mo_database.dat_type during logical restore. The latest MongoDB delta is isolated to its query translator, tests, RFC, and E2E harness and does not overlap this PR or consume MORPCVersion56
• Post-rebase TestCreateDatabaseChecksExistingBeforeSerializingAbsence, all four focused session-temporary-DDL compile tests, TestRequireViewMetadataRevalidationRejectsMissingRefreshGate, and TestViewMetadataRevalidationActivationPropagatesCatalogErrors passed
• Post-rebase focused frontend tests passed for database identity validation, missing database handling, user database type classification, create/alter publication behavior, explicit-table deduplication, lifecycle-lock system-context handling, and temporary-DDL lifetime/reset behavior
• Post-rebase TestHandleGetProtocolVersionIncludesPendingCNs passed, proving pending CNs remain in common-protocol discovery
• Post-rebase TestCNStartsQueryServiceBeforeViewMetadataAdmission and TestCNStartPropagatesQueryServiceStartFailure passed
• Post-rebase TestLockViewMetadataLifecycleOrderAndErrors passed for the shared catalog lock helper
• Current head 9fd92cd87b675eb539afec0a7b3a12720f625cc3: git diff --check passed

Known blockers and residual risks

• The current code is not rollout-safe: it emits and consumes data-branch without a common-protocol gate, and logical snapshot/PITR restore does not preserve the marker. These are the active P1 blockers addressed by the proposed design, not accepted residual risk.
• A pre-change database branch that already has no active ordinary-table receipts has no trustworthy distinguishing metadata and cannot be migrated automatically. It remains rejected rather than risking deletion of an ordinary database.
• No catalog schema migration is proposed; the marker uses the existing mo_database.dat_type column.

[qodo-code-review]

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

[XuPeng-SH]

Deep-reviewed exact head 25fe6784bcfe31929ec646a9ca860a12d46d54fc across catalog ownership, delete safety, legacy fallback, cross-account creation, publication consumers, rollback, BVT isolation, and mixed-version operation. The local implementation is internally coherent: the marker is written atomically with CREATE DATABASE; ordinary empty/subscription/unknown targets remain rejected; every present ordinary table still needs an active receipt; and focused UT/BVT cover the intended success and safety cases. One mandatory persistent-contract/rollout blocker remains.

[P1] Design and gate the new persistent database type for mixed-version clusters

This PR introduces a new durable mo_database.dat_type value and changes DATA BRANCH DELETE semantics, but the repository’s approved docs/design/data_branch_privilege.md still fixes the opposite contract: empty/view-only/sequence-only databases must fail closed, and database-level branch identity requires a separate design/implementation step. The PR neither updates that versioned design nor records an approved replacement decision, migration/rollback contract, or capability gate.

This is already a concrete mixed-version failure, not documentation drift. A new CN writes dat_type = data-branch; an old CN treats every non-empty dat_type as a non-user database in create/alter publication validation, so the same branch database accepts or rejects publication operations depending on which CN receives the request. Conversely, an old CN can create an empty/view-only branch without the marker, and a new CN must permanently reject its DATA BRANCH DELETE because there is no trustworthy identity to recover. During rollout, persistent state and user-visible behavior are therefore CN-version-dependent.

Please update the design document and obtain the required design decision for the marker/identity model, including upgrade, downgrade/rollback, mixed-version creation/deletion/publication behavior, and backup/restore. Implement a rollout/capability rule that prevents new persistent semantics from being emitted until all eligible CNs can consume them (or provide another reviewed compatibility mechanism). Add deterministic mixed-version/capability tests for old-created and new-created empty and table-backed branches. Merely documenting the current inconsistency as a residual risk is not sufficient for a new catalog contract.

No separate hot-path, unbounded-state, wait, cleanup, privilege, or SQL-injection blocker was found in the reviewed implementation.

[gouhongshen]

Unrelated CI blocker: workflow run 34020511404 attempt 1 failed only in the existing pessimistic_transaction/vector/vector_hnsw.sql case. ALTER ... REINDEX returned transfer rowids failed, length mismatch, expect 2, got 1; the Coverage job then failed solely because that BVT coverage producer was unavailable. PR #28272 changes no vector, HNSW, disttae, or RowID-transfer code, while its data-branch paths and the parallel pessimistic BVT job passed. This recurrence is tracked by open issue #27878. The failed jobs are being rerun once per CI policy.

[gouhongshen]

[P1] Design and gate the new persistent database type for mixed-version clusters

The blocker remains valid and in scope. The latest main (a6ee717132) assigns MORPCVersion50 and MORPCVersion51 to the ordered ODKU protocol contracts, so I rebased and updated the still-proposed design to revision data-branch-database-identity-2026-09-06-r4 at fa77898595, using MORPCVersion52 and a deterministic v51/v52 compatibility matrix.

The implementation commit remains patch-identical. The rebase was conflict-free, and the focused compile/frontend tests passed against the new planner/compiler/protobuf baseline. Production gating and restore changes remain blocked pending approval of section 12; the PR remains Draft.

[gouhongshen]

[P1] Design and gate the new persistent database type for mixed-version clusters

The blocker remains valid and in scope. Latest main 6e5f82f568 assigns MORPCVersion52 to MySQL binary JSON subtype tags, so I rebased and updated the still-proposed design to revision data-branch-database-identity-2026-09-07-r5 at 2ce530beed, using MORPCVersion53 and a deterministic v52/v53 compatibility matrix.

The implementation commit remains patch-identical. The rebase was conflict-free, and the focused compile/frontend tests passed against the bytejson/protocol baseline. Production gating and restore changes remain blocked pending independent approval of section 12; the PR remains Draft.

[gouhongshen]

Ready-for-review gate evaluated at exact head 2406d9dbdc.

The final diff and PR description are internally consistent, the branch is conflict-free against main f14eb824df, and all 28 checks are in successful, skipped, or neutral terminal states. The existing review requests cover the changed CODEOWNERS paths.

I am keeping the PR Draft because the mandatory design gate is still open: revision data-branch-database-identity-2026-09-07-r5 has no independent approval, while production still lacks the proposed MORPCVersion53 creation/deletion/publication gate and logical snapshot/PITR marker preservation. Marking Ready now would present known mixed-version and restore failures as implementation-complete. Please review section 12 as the design decision; after approval I will align code and deterministic v52/v53 tests before requesting implementation review.

[gouhongshen]

Ready-for-review gate evaluated at exact head 7faf22d3c6.

The final diff and PR description are consistent, the branch is conflict-free against main 74668fc075, and all 28 checks are in successful, skipped, or neutral terminal states. Existing requests cover the changed CODEOWNERS paths: frontend/docs (XuPeng-SH), catalog/compile (XuPeng-SH, aptend, aunjgr), and BVT (XuPeng-SH, LeftHandCold, aptend).

I am keeping the PR Draft because the mandatory persistent-contract design gate remains open. Revision data-branch-database-identity-2026-09-07-r5 has no independent approval, while the current production code still emits and consumes data-branch without the proposed MORPCVersion53 gate and logical snapshot/PITR restore still does not preserve the marker. Marking Ready would present known mixed-version and restore failures as implementation-complete. Please review section 12 as the design decision; after approval I will align implementation and deterministic v52/v53 tests before requesting implementation review.

[gouhongshen]

Rebased onto latest main bcd5ed833b at head eb637e0ec1. The new base delta only adds stored-view sequence resolution in planner/function code and sequence BVTs; it does not consume MORPCVersion53 or touch catalog, CREATE DATABASE, restore, publication, or data-branch identity paths. The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical.

Post-rebase compile and frontend focused tests passed. The PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r5 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v52/v53 tests remain intentionally blocked on that decision.

[gouhongshen]

Rebased onto latest main 0e2393417f at head ffc5d7cdaa. The new base delta only fixes composite JSON_VALUE results in planner/function code and function BVTs; it does not consume MORPCVersion53 or touch catalog, CREATE DATABASE, restore, publication, or data-branch identity paths. The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical.

Post-rebase compile and frontend focused tests passed. The complete PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r5 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v52/v53 tests remain blocked on that decision.

[gouhongshen]

The latest main 4fdb9e9161 assigns MORPCVersion53 to the capability-gated ordered-stream distributed Top-N merge. I rebased at head 7d9ed373ed and advanced the still-proposed data-branch design to revision data-branch-database-identity-2026-09-07-r6, using MORPCVersion54 with a deterministic v53/v54 matrix.

The rebase was conflict-free. Range-diff shows the implementation commit remains patch-identical; the design commit changes only the necessary rollout version and records the v53 owner. Post-rebase compile and frontend focused tests passed, and the complete PR description is updated. Production gating and restore work remain blocked pending independent design approval; the PR remains Draft.

[gouhongshen]

Ready-for-review gate evaluated at exact head 7d9ed373ed.

The final diff and complete PR description are consistent, the branch is conflict-free and current on main 4fdb9e9161, and all 28 checks are in successful, skipped, or neutral terminal states. Existing requests cover every changed CODEOWNERS path: docs/frontend (XuPeng-SH), catalog/compile (XuPeng-SH, aptend, aunjgr), and BVT (XuPeng-SH, LeftHandCold, aptend).

I am keeping the PR Draft because the mandatory persistent-contract design gate remains open. Revision data-branch-database-identity-2026-09-07-r6 has no independent approval, while current production code still emits and consumes data-branch without the proposed MORPCVersion54 gate and logical snapshot/PITR restore does not preserve the marker. After design approval, implementation alignment and the deterministic v53/v54 compatibility matrix are required before implementation review.

[gouhongshen]

Rebased onto latest main 6eee64625e at head 4d7f1df530. The new base adds equal-value ODKU fulltext-maintenance elision in planner/index-plugin code; it does not consume a protocol version or modify this PR's catalog, CREATE DATABASE, restore, publication, or data-branch identity files.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Post-rebase compile DDL and frontend data-branch/publication focused tests passed. The complete PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r6 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v53/v54 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main 3d67664696 at head 0a27ef7c88. The new base contains two changes: raw-CN protocol discovery/QueryService startup ordering for failed-upgrade retry, and a shared unrestricted-DELETE predicate that preserves JOIN/USING scope for indexed deletes. It does not consume MORPCVersion54 or modify this PR's catalog, CREATE DATABASE, restore, publication, or data-branch identity files. The protocol change strengthens the control-plane inventory assumed by the proposed cluster-wide gate.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Post-rebase focused compile DDL, frontend data-branch/publication, protocol discovery, and CN startup-order tests passed. The complete PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r6 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v53/v54 tests remain blocked on that decision.

[gouhongshen]

The latest main e7cadcf031 assigns MORPCVersion54 to catalog-authenticated proxy prepared-cache reuse. I rebased at head f22bd0c09a and advanced the still-proposed data-branch design to revision data-branch-database-identity-2026-09-07-r7, using MORPCVersion55 with a deterministic v54/v55 matrix.

The new base also adds subscription index metadata routing and dependency-ordered standalone shutdown. The subscription change overlaps publication_subscription.go and its tests, but its new helper paths compose with this PR's user-database classification. The rebase was conflict-free; range-diff shows the implementation commit remains patch-identical, while the design commit changes only the required rollout version and v54 ownership record.

Post-rebase compile DDL and focused frontend data-branch/publication tests passed, including the upstream explicit-table deduplication control. The complete PR description now records the exact base/head and validation. The PR remains Draft because r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main 7e6cb6aaac at head 73210a8600. The new base only fixes exact integer midpoint precision in median aggregation and adds its focused BVT; it does not modify this PR's catalog, CREATE DATABASE, frontend, publication, restore, data-branch identity, or protocol-version paths. MORPCVersion55 remains unassigned on main.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Post-rebase focused compile DDL and frontend data-branch/publication tests passed. The complete PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main 8edac64737 at head 30ba00fcab. The new base only rejects non-sequence relations in SETVAL and adds focused sequence tests; it does not modify this PR's catalog, CREATE DATABASE, frontend, publication, restore, data-branch identity, or protocol-version paths. MORPCVersion55 remains unassigned on main.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Because the planner function change is in the frontend dependency closure, post-rebase focused compile DDL and frontend data-branch/publication tests were rerun and passed. The complete PR description now records the exact base/head and validation. The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main ad225722a0 at head 07cde932a9. The new base validates stale cross-CN fulltext plans before execution and supports altering temporary-table columns and names. It changes planner/compile code including pkg/sql/compile/ddl.go, but does not consume MORPCVersion55 or modify this PR's catalog, frontend, publication, restore, or data-branch identity contract.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Semantic inspection confirms the upstream DDL changes compose with the typed internal CREATE DATABASE path. Post-rebase focused compile DDL and frontend data-branch/publication tests passed, and git diff --check is clean. The complete PR description now records the exact base/head and validation.

The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main 3d2e5c336e at head 6a76291781. The new base preserves plain-text inputs in JSON_UNQUOTE and adds planner-function, frontend prepared-statement, and BVT coverage. It does not consume MORPCVersion55 or modify this PR's catalog, CREATE DATABASE, publication, restore, or data-branch identity contract.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Post-rebase focused JSON_UNQUOTE planner tests, the frontend prepared-statement control, compile DDL, and frontend data-branch/publication tests passed; git diff --check is clean. The complete PR description now records the exact base/head and validation.

The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main e78ff09781 at head 281af3da16. The intervening base preserves terminal dispatch results for late remote receivers, orders snapshot/view lifecycle locks across DDL transactions, and rejects non-finite linear-reference parameters.

The lifecycle change overlaps compile, publication, snapshot, and PITR paths. Semantic inspection confirms it composes with this PR, does not consume MORPCVersion55, and does not preserve mo_database.dat_type during logical restore. The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical.

Focused catalog lifecycle-lock, compile DDL/revalidation, and frontend data-branch/publication/lifecycle-context tests passed; git diff --check is clean. The complete PR description now records the exact base/head and validation.

The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r7 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain blocked on that decision.

[gouhongshen]

A final freshness check found one additional upstream commit, so I rebased again onto main 665a17da13 at head 5ff28a9b1d. That delta only adds the functional-index RFC and does not touch code, protocol versions, or this PR's files.

The rebase was conflict-free and range-diff again shows both PR commits patch-identical. Focused compile DDL/revalidation and frontend data-branch/publication/lifecycle-context tests passed on the new head; git diff --check remains clean. The complete PR description is updated to the exact base/head.

The PR remains Draft with the same mandatory blocker: design revision data-branch-database-identity-2026-09-07-r7 has no independent approval, so v55 rollout gating, logical snapshot/PITR marker preservation, and deterministic v54/v55 tests remain intentionally unimplemented.

[gouhongshen]

The latest main cc205f5f5e assigns MORPCVersion55 to session-owned temporary DDL with transactional data. I rebased at head 841bdd58e0 and advanced the still-proposed data-branch design to revision data-branch-database-identity-2026-09-07-r8, using MORPCVersion56 with a deterministic v55/v56 matrix.

The upstream change overlaps pkg/sql/compile/ddl.go, but its temporary CREATE TABLE routing is semantically independent of this PR's typed internal CREATE DATABASE path. The rebase was conflict-free; range-diff shows the implementation commit remains patch-identical, while the design commit changes only the necessary rollout version and v55 ownership record.

Focused session-temporary-DDL compile/frontend tests and this PR's compile DDL, data-branch, publication, and lifecycle-context tests passed on an isolated Go build cache; git diff --check is clean. The complete PR description now records the exact base/head and validation.

The PR remains Draft because r8 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v55/v56 tests remain blocked on that decision.

[gouhongshen]

Rebased onto latest main 39225f5fbc at head 9fd92cd87b. The new base adds MongoDB sort and unwind pipeline stages and changes only the MongoDB query translator, its tests, RFC, and E2E harness; it does not consume MORPCVersion56 or overlap this PR's catalog, CREATE DATABASE, frontend, publication, restore, or data-branch identity paths.

The rebase was conflict-free, and range-diff shows both PR commits remain patch-identical. Post-rebase focused compile DDL and frontend data-branch/publication tests passed; git diff --check is clean. The complete PR description now records the exact base/head and validation.

The PR remains Draft because design revision data-branch-database-identity-2026-09-07-r8 still lacks independent approval; production gating, logical snapshot/PITR marker preservation, and deterministic v55/v56 tests remain blocked on that decision.