Skip to content

Remove the raw flags accessor, the byte layout constants and the Hash aliases from FodId (breaking change) #39

Description

@jwrosewell

The problem

The 51Did packages give a caller two ways to read what an identifier says: a typed accessor for each field, and the raw bytes and offsets to read the same thing by hand. The second way is the one a consumer is told never to use, and it is the way that produces the bug the usage accessor was added to stop: the usage bits are cumulative, non-marketing 001, standard 011, personalized 111, so a caller who masks the byte for the non-marketing bit reads every marketing identifier as non-marketing, which is exactly backwards for a rule that says a non-marketing identifier must never be passed to a demand source.

These packages are not widely deployed, so the raw surface can be removed rather than deprecated. A breaking change now costs nothing and closes the door for good. This issue is raised on all six packages so that the public surface stays the same across languages, which is the property that stops one language drifting from the rest.

What to remove, in every package

The surface of FodId was compared across all six packages on their main branches on 6 September 2026. Three things are redundant in the same way everywhere.

1. The raw flags accessor. Every bit of the byte now has a name: bits 0 to 2 are the usage, bit 3 is whether the usage came from a consent string, bits 6 and 7 are the identifier type, and bits 4 and 5 are unused. The raw byte has no remaining legitimate use and is the way back to masking.

Package Remove Already replaced by
.NET FiftyOne.Did FodId.Flags Usage, UsageFromConsent, Type
Java pipeline.did getFlags() getUsage(), isUsageFromConsent(), getType()
Node fiftyone.pipeline.did fod.flags fod.usage, fod.usageFromConsent, fod.type
Python fiftyone_pipeline_did fod.flags fod.usage, fod.usage_from_consent, fod.type
PHP fiftyone.pipeline.did getFlags() getUsage(), isUsageFromConsent(), getType()
Rust fodid flags() usage(), usage_from_consent(), id_type()

2. The deprecated aliases from the match key rename. Each package kept the old Hash names "so existing callers still compile". There are no such callers. Remove Hash/HashOffset/HashLength (.NET), getHash()/HASH_OFFSET/HASH_LENGTH (Java), hash/HASH_OFFSET/HASH_LENGTH (Node and Python), getHash()/HASH_OFFSET/HASH_LENGTH (PHP), and hash() plus the HASH_OFFSET/HASH_LENGTH re-exports in lib.rs (Rust).

3. The byte layout constants. FLAGS_OFFSET, LICENSE_ID_OFFSET, LICENSE_ID_LENGTH, MATCH_KEY_OFFSET, MATCH_KEY_LENGTH, HEADER_LENGTH, GUID_LENGTH, RANDOM_PAYLOAD_LENGTH and PAYLOAD_LENGTH are public in all six, in each language's spelling. The only reason a consumer wants an offset is to read the payload by hand, and the only reason to do that is to read a bit that now has a name, so this and item 1 are one decision. Make them internal. The packages' own tests build payloads byte by byte and need them, as do the cloud service and its tests, and both should reach them through a non-public path: internal with InternalsVisibleTo in .NET, package-private in Java, a non-exported module in Node, an underscore module in Python, a non-public class or trait in PHP, and pub(crate) in Rust with the lib.rs re-export gone.

One candidate to decide, not a definite

The raw date minutes. DateMinutes (.NET), getDateMinutes() (Java), dateMinutes (Node), date_minutes (Python) and getDateMinutes() (PHP) give the wire encoding of the date beside the typed date. Rust has no such accessor and nobody has missed it. The .NET documentation states a purpose: it is the value the OWID public-key?date= parameter takes, and a caller comparing creation times may want the integer. Against that, the only caller who builds that request by hand is one bypassing DidClient, which builds it already, and a typed date compares perfectly well. This looks like the same class, a raw encoding beside a typed accessor, and removing it would bring five packages into line with the sixth. Decide it with the rest rather than leave it.

One inconsistency, noted but separate

The static base64 helpers differ by language and are not about raw bits, so they are outside this issue but worth a separate one: .NET has public NormaliseBase64 and ToBase64Url, Node has public toStandardBase64 and toBase64Url, Python has public to_standard_base64 and to_base64_url, PHP has public toStandardBase64 only, Java has package-private toStandardBase64 and no URL form, and Rust has neither. The instance asBase64Url exists everywhere, which is the one callers use.

What must keep working

  • The package's own tests, which build payloads byte by byte, through the internal path.
  • The cloud service and its tests, which read and write the layout, through the same.
  • Every typed accessor, unchanged.

Where this came from

Found by the Trusted Server work, which must work only through these packages and found first that the usage had no name, fixed in the usage accessor pull requests (pipeline-dotnet 397, pipeline-java 127, pipeline-node 192, pipeline-python 76, pipeline-php-did 14, rust 38), and then that the raw byte and the offsets were the remaining way to get it wrong. The sweep across all six was done from this session on 6 September 2026 and every claim above was read from the packages' main branches rather than from a working copy.

Notes

Written with AI assistance on James Rosewell's instruction and needs human review. Nothing has been changed for this issue yet.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions