Add set-cardinality methods to Roaring64Map - #875
Merged
lemire merged 3 commits intoSep 4, 2026
Conversation
Roaring64Map had no way to ask for the size of an intersection, or whether two bitmaps intersect at all, without materializing the intersection via operator&. The 32-bit Roaring class has provided both for a long time. Walk the two ordered maps of 32-bit bitmaps in tandem and delegate to the 32-bit routines on shared high keys. intersect() stops at the first shared value. Cardinality sums go through an overflow-checked accumulator that mirrors what cardinality() does when the true result is 2^64.
Completes the set-cardinality methods that the 32-bit Roaring class already provides, so callers can size a union, symmetric difference or difference without materializing the result. Union and symmetric difference must account for keys held by only one side, so unmatched keys contribute their full cardinality and the leftover tail of either map is drained. Difference is asymmetric: only keys present in the left map contribute, and a key missing from the right map contributes in full. All sums go through the overflow-checked accumulator. The checked test harness shadows each method against the equivalent std::set algorithm.
anandheritage
marked this pull request as draft
September 2, 2026 15:27
anandheritage
marked this pull request as ready for review
September 2, 2026 15:39
Contributor
|
Looks good! If you don't mind, would you consider adding the same methods to Roaring64 as well? It has the same gap, but the C API that Roaring64 wraps provides corresponding methods, so they would be only one-line delegation. |
Contributor
Author
|
Thanks @SylvesterKwon will take that as a follow up. @lemire Let me know if this is good to be merged. |
Member
|
Running tests |
Contributor
Author
@lemire |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Category: new feature. This closes a long-standing gap between the 32-bit
Roaringclass and the 64-bitRoaring64Map, with tests included.Motivation
Roaring64Maphas no way to ask for the size of a set operation. Callers haveto build the entire result and then read its cardinality:
uint64_t n = (a & b).cardinality();That allocates a full bitmap purely to discard it. The 32-bit
Roaringclasshas offered the direct forms for a long time, so this is an existing, tested
API being extended to the 64-bit map rather than a new concept.
Added
and_cardinality,or_cardinality,xor_cardinality,andnot_cardinalityand
intersect, matching the existing 32-bit signatures and semantics.Implementation
Roaring64Mapstores an orderedstd::map<uint32_t, Roaring>keyed on thehigh 32 bits. Each method walks the two maps in tandem and delegates to the
corresponding 32-bit routine wherever both sides hold the same high key, so no
intermediate bitmap is allocated and each map is traversed once. The container
level logic is entirely reused; this is routing, not a reimplementation.
The operations differ only in how they treat keys held by a single side.
Intersection ignores them. Union and symmetric difference add their full
cardinality and then drain whichever map still has entries. Difference is
asymmetric: only left-hand keys contribute, and a key absent from the right map
contributes in full.
intersectreturns on the first shared value instead ofcounting the rest.
Cardinality sums go through a small overflow-checked helper. A 64-bit bitmap
can hold 2^64 values, which is not representable in a
uint64_t, so it throwsstd::length_errorin that case, matching existingRoaring64Map::cardinality()behaviour. There are at most 2^32 keys eachholding at most 2^32 values, so the true sum cannot exceed 2^64 and the
wraparound check is exact rather than approximate.
Tests
Every method is verified two independent ways.
In
tests/cpp_unit.cpp, each is compared against the existing materializingoperators, for example
r1.or_cardinality(r2) == (r1 | r2).cardinality(),across operands with shared keys, disjoint keys, keys on only one side, empty
bitmaps, and self-comparison.
All five are also added to
tests/roaring64map_checked.hh, the harness thatshadows a
Roaring64Mapagainststd::set<uint64_t>, so they are additionallychecked against
std::set_intersection,std::set_union,std::set_symmetric_differenceandstd::set_differenceover every pairing ofthose operands. Using two unrelated oracles means a mistake would have to be
mirrored identically in CRoaring's own operators and in libstdc++ to go
unnoticed.
I confirmed the tests actually detect regressions by mutating the
implementations: removing the tail-drain from
or_cardinality, and lettingandnot_cardinalitystop once the right-hand map is exhausted. Both stillcompiled and still passed 75 of 77 tests, and both were caught by the unit test
and the checked harness.
ctestpasses 27/27 locally. The amalgamated single-file build was regeneratedand exercised separately, since it is a distinct build path. I ran
tools/run-clangcldocker.sh, which found nothing to reformat.