feat(orderable-bytes)!: variable-length encodings for strings and byte strings

[coderdan]

Summary

orderable-bytes turns values into bytes whose order matches the values' order, and order-revealing (ORE) and order-preserving (OPE) encryption run on top of those bytes. Until now it covered fixed-size types only. Strings had no shared encoding, so each scheme encoded text its own way, with different and lossy rules. This PR adds an encoding for strings and byte strings, so vitaminc-ore can use one order encoding for every kind of value.

Breaking (0.2.0): the traits are reorganised so that fixed-length and variable-length encodings can't be confused. Nothing about the encoded bytes changes: every golden vector from #96 passes unchanged.

Stacked on #96. Merge that first.

Traits

pub trait OrderableBytes: Sealed {          // was ToOrderableBytes
    type Bytes<'a>: AsRef<[u8]> where Self: 'a;
    fn to_orderable_bytes(&self) -> Self::Bytes<'_>;
}
pub trait FixedOrderableBytes: OrderableBytes {
    const ENCODED_LEN: usize;
    type Array: AsRef<[u8]> + Copy;          // always [u8; ENCODED_LEN]
    fn to_fixed_orderable_bytes(&self) -> Self::Array;
}
pub trait VariableOrderableBytes: OrderableBytes {}

• Every type implements exactly one subtrait. Numbers, bool, char, [u8; N], Decimal and the chrono types implement FixedOrderableBytes. str, String, [u8] and Vec<u8> implement VariableOrderableBytes. &T implements the same traits as T.
• Owned fixed-length bytes: to_orderable_bytes may borrow from the value, so its output can't outlive it. to_fixed_orderable_bytes returns the same bytes as an owned array, so generic code can return or store them (fn term<T: FixedOrderableBytes>(v: T) -> T::Array). Binding Bytes<'a> to an owned type for every 'a doesn't compile: the Self: 'a bound gives E0311.
• Sealed: only this crate implements the traits. The compiler therefore enforces that each type implements exactly one subtrait, and that the fixed bytes are exactly ENCODED_LEN long, because one macro writes each type's impls from a single length. A compile_fail,E0277 doctest checks that downstream impls are rejected.
• Why the rename: in 0.1, ToOrderableBytes meant fixed-length. If that name had become the general trait, 0.1 code bounded on it would still compile and silently accept strings. Code that zero-pads to a block would then make "a" equal "a\0", and left-padding would put "b" before "aa". With the old name removed, those bounds fail to compile, and code that pads bounds on FixedOrderableBytes, which rejects strings. A compile_fail,E0277 doctest pins this.
• Bytes is a generic associated type, so variable-length impls can return a borrowed slice.

Changes

• The new variable module:
  • Encoding: the value's own bytes, borrowed, so no copy of the plaintext is made.
  • Order: byte order is already the order these types define. A prefix sorts first, and str sorts by Unicode code point.
  • No normalisation: é written as U+00E9 and as e + U+0301 encode differently. Callers normalise first.
  • Never pad, never concatenate: the bytes don't mark where they end. The module docs and the README spell this out.
• Fixed-length types:
  • One macro, impl_fixed_orderable_bytes!, writes every fixed-length type's impls. The 14 primitives are no longer copy-pasted impl pairs.
  • [u8; N] is new and encodes to itself, so hashes and UUIDs need no copy into a Vec<u8>.
  • chrono::datetime_utc::ENCODED_LEN is removed. It was the only per-module length constant; use the associated constant.
• ore-rs:
  • It reads block lengths through FixedOrderableBytes.
  • The decimal bench now derives its block length instead of hard-coding 14.
  • Its public API and ciphertexts are unchanged, and its edits are in separate refactor(ore-rs) commits.
• README:
  • New sections on variable-length encodings and on upgrading from 0.1. The upgrade guide covers T::Bytes becoming T::Array, and sealing.
  • An updated table of supported types.
  • The usage example previously called decimal::to_orderable_bytes and decimal::ENCODED_LEN, neither of which exists. It now goes through the traits and uses f64 and strings.
• README doctests: the README's Rust examples run as doctests (a cfg(doctest) item includes the README). They need no optional feature, so a plain cargo test runs them.
• Tests:
  • quickcheck properties: order and equality match String's and Vec<u8>'s own Ord/Eq;
  • a check that the encoding borrows rather than copies;
  • prefix ordering;
  • "no normalisation";
  • golden rows for str and bytes;
  • a check that each variable-length type implements VariableOrderableBytes;
  • compile_fail,E0277 doctests: String can't be passed to a pad-to-block function, and a downstream type can't implement OrderableBytes;
  • &T encodes as T under all three traits;
  • [u8; N]: golden rows and a quickcheck property that its order matches array order;
  • the golden check asserts that to_orderable_bytes and to_fixed_orderable_bytes return the same bytes.

Verification

• cargo test --all-features passes on stable and on 1.78.0 (152 tests), and cargo test -p orderable-bytes passes with default features, including the README and compile_fail doctests. One README block is rust,ignore; it is the 0.1 code in the upgrade guide.
• cargo clippy --no-deps --all-targets --all-features -- -D warnings passes on stable and 1.78. cargo fmt -- --check and cargo doc with -D warnings are clean.
• cargo bench -p ore-rs --bench decimal --no-run builds.

Related

• Closes orderable-bytes has no variable-length encoding — text and bytes can't share the order layer #94
• Design: ADR-0002 in docs(adr): one value model, three encodings in vitaminc, EQL v4 by producer stack#1139
• Downstream:
  • cllw-ore and cipherstash-client stay on 0.1 and are unaffected.
  • vitaminc-ore (New vitaminc-ore crate: one order encoding per kind, with block ORE, CLLW ORE and CLLW OPE as pluggable schemes vitaminc#374) depends on 0.2.

Review notes

• Release: release-plz should compute orderable-bytes 0.2.0 from the ! commits. The pending release PR chore: release #88 (0.1.2) predates this.
• ore-rs version: the ore-rs edits are in non-breaking refactor(ore-rs) commits, so ore-rs should get a patch release, not 0.9.0. Its orderable-bytes = "^0.1" requirement can't be raised to 0.2 in this PR, because the path crate is still 0.1.1 until release-plz bumps it. release-plz updates it at release time. Only a manual cargo publish before then would fail.
• History: the two feat(orderable-bytes)! commits don't build ore-rs on their own; the refactor(ore-rs) commit after each one fixes it. This is deliberate, so that release-plz doesn't attribute a breaking change to ore-rs.
• Design: the encoding is the identity on bytes, not an escaped and terminated format. That was decided in review.

[Copilot]

🟡 Changes recommended

The public trait documentation incorrectly claims that per-type modules expose free encoded-length constants.

1 open finding

• Documentation falsely claims modules expose free ENCODED_LEN constants · New

What changed in this PR

Adds borrowed variable-length orderable encodings while separating fixed-length metadata into a dedicated trait.

Changes:

• Adds identity encodings for strings and byte strings.
• Introduces FixedOrderableBytes and migrates fixed-width implementations.
• Expands documentation, doctests, and golden/property tests.

File	Description
packages/​ore-rs/​src/​encrypt.rs	Uses the new fixed-length trait.
packages/​ore-rs/​src/​decimal.rs	Migrates decimal length lookup.
packages/​ore-rs/​src/​chrono.rs	Migrates chrono length lookups.
packages/​orderable-bytes/​tests/​golden.rs	Adds variable-length golden vectors.
packages/​orderable-bytes/​src/​variable.rs	Implements borrowed string and byte encodings.
packages/​orderable-bytes/​src/​primitive.rs	Splits primitive trait implementations.
packages/​orderable-bytes/​src/​lib.rs	Defines the revised trait API and doctests.
packages/​orderable-bytes/​src/​decimal.rs	Implements the fixed-length trait for decimals.
packages/​orderable-bytes/​src/​chrono.rs	Implements the fixed-length trait for chrono types.
packages/​orderable-bytes/​README.md	Documents usage and variable-length constraints.
packages/​orderable-bytes/​Cargo.toml	Updates the package description.

🧠 Review effort: Balanced

---

Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.

[Copilot]

🟢 Approval recommended

The trait migration, borrowed encodings, compatibility safeguards, and fixed-length ORE boundaries are consistent and comprehensively tested.

0 open findings

1 resolved since last review

• Documentation falsely claims modules expose free ENCODED_LEN constants

🧠 Review effort: Balanced