Add experimental RoaringLookupResources to the Materialize API - #175
Merged
josephschorr merged 1 commit intoSep 16, 2026
Merged
josephschorr merged 1 commit into
josephschorr merged 1 commit into
Conversation
josephschorr
force-pushed
the
experimental-roaring-lookup-resources
branch
2 times, most recently
from
September 15, 2026 17:28
22115ca to
b495f75
Compare
Adds an experimental API returning the resources of a given type on which a subject has a given permission, as a roaring64 bitmap in the RoaringFormatSpec 64-bit portable format. That format is consumed directly by OpenSearch's bitmap terms filtering (value_type: bitmap on long fields), letting integrators send a compact permission set per query instead of replicating access data into the search index. The IDs in the bitmap are the resource object IDs exactly as they appear in the relationships; no surrogate ID is introduced. For the API to be usable, every resource object ID of the requested type must be a canonical decimal integer of at most 44 bits (<= 17592186044415). Any ID violating that fails the call with FAILED_PRECONDITION rather than returning a partial bitmap. The response carries the bitmap's cardinality and the ZedToken at which the lookup was performed.
josephschorr
force-pushed
the
experimental-roaring-lookup-resources
branch
from
September 15, 2026 17:30
b495f75 to
11008f5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
ExperimentalRoaringLookupResources, a new experimental RPC inauthzed.api.materialize.v0: given a resource type, a permission, and a subject, it returns a roaring64 bitmap of the IDs of every resource of that type on which the subject has the permission.The bitmap is serialized in the RoaringFormatSpec 64-bit portable format, which is what OpenSearch consumes directly via a
termsquery with"value_type": "bitmap"onlongfields (once Base64-encoded), and what Java'sRoaring64NavigableMap.deserializePortable()reads. Integrators can attach the permission set to each search query instead of replicating access data into the search index and keeping it in sync.API shape
Consistencyblock,resource_object_type,permission, and aSubjectReference— one subject per request.cardinality, and theZedToken(at_revision) at which the lookup was performed, so callers can thread the same snapshot through subsequent requests.Constraints, documented on the RPC
uint64. Any that does not fails the call withFAILED_PRECONDITIONrather than returning a partial bitmap — a bitmap that is quietly too small is a wrong authorization answer. Canonical matters:document:007anddocument:7are distinct objects that both parse to 7.group#memberlooked up for agroup#membersubject), the subject is one of its own resources, so its object ID is held to the same rule.Naming
Follows the existing convention in this package (
ExperimentalCountRelationshipsByFilterinRelationshipsService): the RPC and its messages carry theExperimentalprefix; the API is subject to change or removal.buf lint,buf format, andbuf buildare clean.