Skip to content

Add optional cursor to DeleteRelationships for resumable batched deletion - #174

Merged
josephschorr merged 1 commit into
authzed:mainfrom
josephschorr:delete-relationships-cursor
Sep 10, 2026
Merged

josephschorr merged 1 commit into
authzed:mainfrom
josephschorr:delete-relationships-cursor

Conversation

@josephschorr

Copy link
Copy Markdown
Member

Summary

Adds optional_cursor to DeleteRelationshipsRequest and after_result_cursor to DeleteRelationshipsResponse, mirroring the cursor pattern already used by ReadRelationships and LookupResources. Together they let a large partial deletion run as a series of calls that each resume where the previous one left off, instead of re-examining the relationships already deleted by earlier calls.

Motivation

Today the partial-deletion loop (optional_limit + optional_allow_partial_deletions) re-issues the same filter on each call. On CockroachDB that means every batch rescans the MVCC tombstones left by prior batches: the per-batch KV scan cost grows roughly linearly with the number of already-deleted relationships (quadratic total work), which is what makes deleting a large relationship set on a big table degrade over time.

A cursor lets each batch resume after the previous one via a primary-key seek, so its scan cost stays flat regardless of accumulated churn. In a controlled tombstone-churn workload on a single-node CockroachDB, the naive loop's per-batch KV scan time climbed ~24x (10ms to 245ms) to delete the same number of rows, while a cursored delete stayed flat (~10ms). The gap widens with scale.

Semantics

  • The fields require optional_limit and optional_allow_partial_deletions.
  • Supported only by datastores whose deletion can be ordered and resumed; others return an error when a cursor is supplied.
  • after_result_cursor is populated only while deletion_progress is DELETION_PROGRESS_PARTIAL, and is unset once DELETION_PROGRESS_COMPLETE is returned.

Compatibility

Additive and backward compatible — buf breaking passes against main. docs/apidocs.swagger.json regenerated.

Status

Draft: this is the API contract. The SpiceDB server-side implementation (CockroachDB cursored delete) is a follow-up that depends on these fields.

…tion

Adds optional_cursor to DeleteRelationshipsRequest and after_result_cursor to
DeleteRelationshipsResponse, mirroring the cursor pattern already used by
ReadRelationships and LookupResources.

Together they let a large partial deletion be performed as a series of calls
that each resume where the previous one left off, rather than re-examining the
relationships already deleted by earlier calls. On datastores whose deletion
can be ordered and resumed (e.g. CockroachDB via a primary-key cursor) this
avoids the per-batch rescan of prior deletions' tombstones that makes a naive
limit loop degrade on large tables.

The fields require optional_limit and optional_allow_partial_deletions, and are
supported only by datastores that can order and resume deletion; others return
an error when a cursor is supplied. The change is additive and backward
compatible (buf breaking passes against main).

Signed-off-by: Joseph Schorr <josephschorr@users.noreply.github.com>
@josephschorr
josephschorr marked this pull request as ready for review September 10, 2026 21:15

@tstirrat15 tstirrat15 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@josephschorr
josephschorr merged commit 0c8e4ab into authzed:main Sep 10, 2026
3 checks passed
@josephschorr
josephschorr deleted the delete-relationships-cursor branch September 10, 2026 22:38
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 10, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants