Repository navigation
stack-encrypt: every index is compiled into the core crate — move Equality, Match, Ore, Ope into their own crates #1063
Description
Activity
The data form of an index has to be open, or this split does not reach its goal.
The proposal above puts
Index<S>andIndexSpecin the core crate. Today (#1069)IndexSpecis a closed enum with four variants:Equality,Match(options),Ore,Ope. An index crate outside core cannot add a variant to it, so every new index would still mean editing core for its data form, and a binding (the Go guest, a future napi shell) could never name an index the enum does not list. The trait side is already open; the data side is not.What this issue should deliver instead:
IndexSpecbecomes a record, not an enum: the kind name the index declares (eq,match,ore,ope, laterjsonand whatever a new crate adds) plus that index's own serialised options.Index<S>::spec()produces it, as it does today. The wire strings do not change.- The reverse direction, data to operation, needs the engine to resolve a kind name to the crate that implements it. That means the engine is compiled with a known set of indexes. Decision this issue must make: an explicit set handed to the lowering (
dynamic::record, stack-encrypt:dynamic::recordis a second executor that already seals differently from the derive — make it a lowering into the plan builder #1059), or compile-time registration (theinventorypattern). Either way the set is fixed at build time; a name outside it is a plan error before any key request. Indexes are cryptographic operations the engine has to know about regardless, so this is not the per-language EQL assembly question, which was settled the other way (each language assembles EQL types from standard outputs; no registry). - The two enums that exist in the meantime,
IndexSpecand the olderdynamic::TermKind, are being collapsed into one ahead of this work so the boundary has a single spelling. This issue then replaces that one enum with the open record.
Related: #1056 (introduced
Index<S>andIndexSpec), #1059 (the lowering that consumes the data form), #1060 (the first index that would be added after the split).ADR-0002 (#1139) moves order-term derivation into a new vitaminc crate,
vitaminc-ore, with block ORE, CLLW ORE and CLLW OPE as schemes over one
canonical plaintext encoding. TheOreandOpeindex crates proposed here
become thin wrappers over it.Step 3's "byte output must not change" no longer holds for three cases, all
intended and listed in the ADR:- float terms for
-0.0and negative-sign NaN, which now canonicalise; Bool, which loses its order terms;- text order terms, which normalise and fold.
None of these is deployed. The fixtures change with #1140.
- float terms for
Background
stack-encrypt (
packages/stack-encrypt) is our Rust field-level encryption library. Its indexes (Equality,Match,Orefor order-revealing encryption,Opefor order-preserving encryption) derive search terms beside a ciphertext. #1056 introduces anIndex<S>trait so the plan builder only ever sees "an index" and tuples of them (design:docs/plans/2026-10-04-plan-builder.md, PR #1052, "The index crates").Problem
All index implementations live inside stack-encrypt. An earlier effort split search-term handling into modules under
src/sem, not crates, so every index's dependencies (ORE and OPE libraries, match tokenisation) are compiled into every consumer, and adding an index means changing stack-encrypt itself.Proposal
Index<S>andIndexSpec(the data form of an index) in a small core crate the engine depends on.Equality,Match,OreandOpe(and laterJson, see stack-encrypt has no searchable JSON — add aJsonindex that seals a document's entries under one data key #1060) into their own crates, each implementingIndex<S>. Parameters stay on the struct and lower throughspec().Relationship to other work
Index<S>).docs/plans/2026-10-04-plan-builder.mdon PR docs(plans): the plan builder, one front door for stack-encrypt, the derive, the FFI and Go #1052.