Skip to content

Name the usage a 51Did was created for, and take the raw byte surface off FodId - #38

Open
jwrosewell wants to merge 3 commits into
mainfrom
feature/usage-accessor
Open

Name the usage a 51Did was created for, and take the raw byte surface off FodId#38
jwrosewell wants to merge 3 commits into
mainfrom
feature/usage-accessor

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Two changes to the same surface, in the order they were made.

1. Name the usage a 51Did was created for

FodId gains usage() (None, NonMarketing, Standard, Personalized) and usage_from_consent(), reading bits 0 to 2 and bit 3 of the flags byte. The usage bits are cumulative, so the accessor answers the highest usage granted rather than leaving callers to mask bits and misread every marketing identifier as non-marketing. Same accessor, same names, in all six 51Did packages. Found by the Trusted Server work, which must work only through this crate.

2. Take the raw byte surface off FodId

This is issue 39, raised on all six packages so the surface stays the same in every language. Once every bit has a name, the raw byte and the offsets are the remaining way to get it wrong, so they go.

  • flags() is removed outright rather than made pub(crate). The typed accessors read the private field directly and the new unit tests are in the same module, so no accessor is needed anywhere.
  • hash(), HASH_OFFSET and HASH_LENGTH are deleted, with the re-export block in lib.rs and the test that only proved the alias matched.
  • The layout constants are pub(crate) and their lib.rs re-exports are gone, so pub use fodid::{FodId, IdType, Usage}; is now the whole typed surface. Two of the nine, PAYLOAD_LENGTH and RANDOM_PAYLOAD_LENGTH, are deleted instead, because nothing inside the crate reads them and pub(crate) would have left dead code under -D warnings.
  • Rust never had a date minutes accessor and none is added. The other five packages are losing theirs, which brings them into line with this crate.

How the tests still build payloads

Rust integration tests are a separate crate and cannot see pub(crate), and a feature flag would have put the layout back in front of consumers. The tests therefore carry the layout themselves in fodid/tests/layout/mod.rs, taken from the specification rather than from the reader, which also removes a circularity, since a payload built from the reader's own constants agrees with the reader whatever either of them says. New unit tests in src/fodid.rs tie the reader's constants to the same published numbers. Where a test wanted the raw byte it reads fod_id.payload()[layout::FLAGS_OFFSET].

fodid-cloud

fodid-cloud/tests/cloud_fodid.rs asserted flags() and now asserts the usage, the consent bit and the type. Two doc comments there listed "flags" among the fields a caller reads and are reworded. No code in fodid-cloud/src changed, and nothing in cloud-request-engine was touched.

The specification

The byte structure is now specified once for all six languages, on specifications 25, at Identifier layout with the surface each package exposes at Package surface. The layout table was repeated in the crate documentation, the FodId doc comment and the README, and all three now name those pages as the authority. AGENTS.md still pointed at FodId::hash from before the match key rename and is corrected. Those links resolve once that pull request merges.

One thing added that is not the issue

fodid/Cargo.toml names reqwest-fetch in its checked cfg list. The OWID source is copied into this crate as a private module, feature gates and all, and SWAN-community/owid-rust 10 puts the ready made transport behind a feature of that name, so the copied mod.rs gains three gates naming it. Building this crate against that branch turns them into three errors under cargo clippy -- -D warnings, which is what continuous integration runs here, even though nothing in this crate enables the feature or reaches the network.

Checked by pointing the submodule at that branch, running ci/copy-owid-source.ps1 and building: three errors before the line, clean after it, with cargo build -p fiftyone-fodid-cloud --no-default-features --target wasm32-wasip1 still succeeding. Naming the value costs nothing while no copied file carries it, and it means the submodule pointer can move later without taking the build down.

Verification

cargo build --all-targets     exit 0
cargo test                    exit 0   67 suites, 542 passed, 0 failed, 7 ignored
cargo fmt --check             exit 0
cargo clippy --all-targets -- -D warnings   exit 0
cargo doc --no-deps -p fodid -p fiftyone-fodid-cloud   clean

The seven ignored are the pre-existing live cloud and network tests. The crate needs the OWID copy, so ci/copy-owid-source.ps1 was run first. The documentation build is included because removing public items breaks intra-doc links.

Closes #39

Produced with AI assistance and needs human review.

FodId exposed the flags byte and id_type read from bits 6-7, and nothing
for the usage in bits 0-2, so a caller deciding whether an identifier may
be passed to a demand source had to mask the byte by hand. The three
usages are cumulative in the byte, non-marketing 0b001, standard 0b011
and personalized 0b111, so a caller masking for the non-marketing bit
alone would read every marketing identifier as non-marketing, the wrong
way round for a data protection decision.

Usage names the four states, None where no bit is set, and usage()
answers with the highest granted. usage_from_consent() reads bit 3,
which records that the usage came from a consent string rather than
being stated. Usage::id_usage gives the cloud's id.usage value. The
names are the same in every 51Did package, which all gain the accessor
together.

Found by the Trusted Server work, which must work only through this
crate. fodid tests: 83 passed, two new; clippy and rustfmt clean.
…ic surface

The 51Did packages gave a caller two ways to read what an identifier says,
being a typed accessor for each field, and the raw byte with the offsets to
read the same thing by hand. The second way is the one that produces the
wrong answer, because the usage bits are cumulative, so anyone masking the
flags byte for the non-marketing bit reads every marketing identifier as
non-marketing, which is backwards for a rule that says a non-marketing
identifier must never be passed to a demand source. These crates are not
widely deployed, so the raw surface is removed rather than deprecated, and
the same removal is being made in the other five packages so that the
surface stays the same in every language.

Three things left the public API.

1. FodId::flags. The byte itself is still held in the struct and read by
   id_type, usage and usage_from_consent, which take the field directly,
   so no accessor for it remains even inside the crate. Where a test wants
   the byte it reads payload()[0] from the OWID envelope, which is public
   and unchanged.
2. FodId::hash and the HASH_OFFSET and HASH_LENGTH constants, the obsolete
   names left behind by the match key rename. Nothing called them.
3. FLAGS_OFFSET, LICENSE_ID_OFFSET, LICENSE_ID_LENGTH, MATCH_KEY_OFFSET,
   MATCH_KEY_LENGTH, HEADER_LENGTH and GUID_LENGTH are now pub(crate) and
   are no longer re-exported from lib.rs. RANDOM_PAYLOAD_LENGTH and
   PAYLOAD_LENGTH are deleted outright, because nothing inside the crate
   used either one, so making them internal would have left two constants
   no code reads.

Rust integration tests are a separate crate and cannot see pub(crate)
items, so the tests under fodid/tests that build payloads byte by byte now
carry the layout themselves in the new fodid/tests/layout/mod.rs, taken
from the published specification rather than from the reader. That is what
a test of a byte format should do anyway, because a payload built from the
reader's own constants agrees with the reader whatever either of them says.
New unit tests in fodid/src/fodid.rs tie the reader's constants to the same
published numbers, replacing the integration test that only compared the
constants with each other.

The repeated byte layout tables in the crate documentation and in
fodid/README.md are replaced by a short summary and a link to the
specification, which is now the authority.
https://github.com/51Degrees/specifications/blob/main/did-specification/identifier-layout.md
https://github.com/51Degrees/specifications/blob/main/did-specification/package-surface.md

fodid-cloud needed three small changes, being a test that asserted the raw
byte and now asserts the usage, the consent bit and the identifier type,
and two documentation comments that listed the flags among the fields a
caller reads.

Rust never had a raw date minutes accessor and none is added. The other
five packages are losing theirs, which brings them into line with this
crate.

Closes #39
@jwrosewell jwrosewell changed the title Name the usage a 51Did was created for, as the highest granted Name the usage a 51Did was created for, and take the raw byte surface off FodId Sep 6, 2026
The OWID source is copied into this crate as a private module, feature gates
and all, so a gate naming a feature this crate does not declare trips the
checked cfg lint. The lint is kept on rather than switched off, with the
names the copied files use listed, which is why endpoints and fetch are
already there.

SWAN-community/owid-rust 10 makes the key fetch asynchronous and puts the
ready made transport behind a new reqwest-fetch feature, so the copied mod.rs
gains three gates on that name. Building this crate against that branch turns
them into three errors under cargo clippy with warnings denied, which is what
CI runs, and the crate stops compiling even though nothing in it enables the
feature or reaches the network.

Naming the value now costs nothing while no copied file carries it, and means
the submodule pointer can move without taking the build down. Checked by
pointing the submodule at that branch, running the copy script and building:
three errors before, clean after, with the wasm target still building.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the raw flags accessor, the byte layout constants and the Hash aliases from FodId (breaking change)

1 participant