Carry the terms a 51Did was created under in the identifier

pipeline.did/README.md

@@ -32,9 +32,9 @@ layout out from this package. What follows is a summary of the part that
 changes how this package behaves.
 
 The identifier carries a five byte header of Flags and License Id, then the
-match key, then an optional creator context section. Bits 6-7 of Flags
-select the type, which decides how long the match key is and so what the
-least a payload can hold is.
+match key, then the Terms byte, then an optional creator context section.
+Bits 6-7 of Flags select the type, which decides how long the match key is
+and so what the least a payload can hold is.
 
 | Bits 7-6 | `IdType`        | Match key length | Minimum payload |
 |---------:|-----------------|-------------:|----------------:|
@@ -46,6 +46,29 @@ least a payload can hold is.
 Identifiers issued before the type tag existed have bits 6-7 zeroed and decode
 as `PROBABILISTIC`.
 
+The Terms byte is not counted in those minimums. An identifier whose
+payload ends at the match key has no Terms byte, and a missing byte is read
+as index zero, which says the terms are not stated in the identifier.
+Absence and zero mean the same thing, so no reader has to tell them apart
+and no presence flag exists. See the terms section below.
+
+## The payload version
+
+Bits 4 and 5 of the Flags byte say which payload layout the identifier
+follows, and this package reads version 0. A payload naming version 1, 2
+or 3 is refused with `FodIdParseStatus.UNSUPPORTED_PAYLOAD_VERSION`, and
+the throwing readers name the version they found in the message.
+
+The fields are never read under the layout this package knows once the
+version says otherwise. A later version exists precisely because a field
+moved, so reading such a payload here would answer with values that are
+wrong rather than absent, which is worse than refusing. A version that
+nothing checks protects nothing.
+
+The version is not exposed. Either this package read the layout, in which
+case the accessors are the answer, or it did not, in which case there is
+no identifier to read fields from.
+
 The minimums in that table are the only lengths this package enforces. There
 is no upper bound. An identifier carrying a creator context is longer than
 the minimum, its extra bytes have a shape only the cloud knows, and a reader
@@ -187,6 +210,10 @@ Usage   usage      = fodId.getUsage();     // what the identifier may be used fo
 boolean fromConsent = fodId.isUsageFromConsent();
 long    licenseId  = fodId.getLicenseId();
 byte[]  matchKey   = fodId.getMatchKey();  // SHA-256 or GUID bytes, see type
+String  terms      = fodId.getTerms();     // address of the terms document
+                                           // it was created under, null
+                                           // where it names none this
+                                           // package knows
 
 // Delegated OWID-level fields and operations.
 String  domain   = fodId.getDomain();
@@ -244,6 +271,60 @@ if (fodId.getUsage() == Usage.NON_MARKETING) {
 }
 ```
 
+## Which terms a 51Did was created under
+
+`getTerms()` answers with the address of the terms document the identifier
+was created under. The answer travels inside the identifier, so a receiver
+always has it, rather than depending on the surrounding protocol to carry
+the terms alongside the identifier where any hop can drop them without the
+identifier looking any different. The package turns the index into the
+address, so a caller never handles the byte.
+
+The byte after the match key is an index into a table in the specification
+and is not a version number, so that a later document can live at any
+address rather than only at one a number could compose. An index is never
+reused and never repointed once published, because repointing one would
+rewrite what an identifier already issued says it agreed to.
+
+| Index | Document | `getTerms()` |
+|---|---|---|
+| `0` | Not stated in the identifier | `null` |
+| `1` | Model Terms for Marketing, version 2 | `https://m4ow.uk/mtm/2.txt` |
+| any other | One this package cannot name | `null` |
+
+An index added to the specification after this package was released answers
+with no address, and the package never builds an address from the index,
+because that would name a document nobody wrote and a receiver would record
+having accepted terms that do not exist. A caller therefore cannot tell an
+index of zero from an index this package cannot name, which is deliberate,
+since both lead to the same place.
+
+```java
+if (fodId.getTerms() == null) {
+    // The identifier does not say which terms it was created under, so the
+    // answer has to come from the data accompanying it.
+}
+```
+
+No address does not mean the identifier is unrestricted. It means only
+that the identifier does not carry the answer, so the answer has to come
+from somewhere else, being the Terms Document Locator in an OpenRTB request
+or whatever the surrounding protocol provides. Carrying the terms in the
+identifier does not remove the need to carry a locator where a protocol has
+one, and where the two disagree the identifier's own value is the one that
+describes the identifier, because it is inside the signature and the
+accompanying data is not.
+
+The terms and the usage answer different questions and a receiver needs
+both. `getUsage()` says where an identifier may go and `getTerms()` says
+which document it was created under. An identifier created for non-marketing
+carries index zero and so answers with no address, because the Model Terms
+govern marketing use and a non-marketing identifier is not created under
+them, and it stays barred from a demand source by its usage.
+
+This package never fetches the address. It returns it and the receiver
+decides what to do with it.
+
 ## Verifying on your server
 
 `DidClient` handles every manipulation of a 51Did a server needs against the

pipeline.did/src/main/java/fiftyone/pipeline/did/FodId.java

@@ -50,12 +50,25 @@
  * Payload layout. Read a 51Did through the typed accessors below, never by
  * walking the payload bytes. The identifier carries a five byte header of
  * Flags and License Id, then the match key, whose length the identifier
- * type in bits 6-7 of Flags decides, and then an optional creator context
+ * type in bits 6-7 of Flags decides, then the Terms byte naming the terms
+ * document it was created under, and then an optional creator context
  * section that binds the identifier to the browser and connection it was
  * created on. Only 51Degrees can read that section, so this reader exposes
- * it only as the part of {@link #getPayload()} beyond the match key, its
+ * it only as the part of {@link #getPayload()} beyond the Terms, its
  * lengths belong to the cloud, and this reader therefore puts no upper
- * bound on a payload. The byte layout is specified at
+ * bound on a payload. A payload that ends at the match key has no Terms
+ * byte, and a missing byte reads as a Terms of zero, so absence and zero
+ * mean the same thing.
+ * <p>
+ * Bits 4 and 5 of the Flags byte say which payload layout the identifier
+ * follows, and this package reads version 0. A payload naming any other
+ * version is refused with
+ * {@link FodIdParseStatus#UNSUPPORTED_PAYLOAD_VERSION} rather than read
+ * under the layout this package knows, because a later version exists
+ * precisely because a field moved, so reading one here would answer with
+ * values that are wrong rather than absent. The version is not exposed,
+ * because a caller has nothing to decide with it.
+ * The byte layout is specified at
  * <a href="https://github.com/51Degrees/specifications/blob/main/did-specification/identifier-layout.md">identifier-layout.md</a>,
  * which is the authority for it, and the surface every 51Did package
  * offers is specified at
@@ -130,21 +143,42 @@ public final class FodId {
     static final int PAYLOAD_LENGTH =
         MATCH_KEY_OFFSET + MATCH_KEY_LENGTH;
 
+    /**
+     * Byte length of the Terms field, which follows the match key. It is
+     * not part of any minimum above, because a payload that ends at the
+     * match key reads as a Terms of zero.
+     */
+    static final int TERMS_LENGTH = 1;
+
+    /**
+     * The payload layout version this package reads, carried in bits 4 and
+     * 5 of the Flags byte. Any other version is refused rather than read
+     * under this layout.
+     */
+    static final int SUPPORTED_PAYLOAD_VERSION = 0;
+
     private final Owid owid;
     private final int flags;
     private final long licenseId;
     private final byte[] matchKey;
+    private final int termsIndex;
 
     /**
      * Built only by {@link #read(Owid)} once the payload has passed the
      * 51Did rules, so an instance never exists for a payload that failed
      * them.
      */
-    private FodId(Owid owid, int flags, long licenseId, byte[] matchKey) {
+    private FodId(
+            Owid owid,
+            int flags,
+            long licenseId,
+            byte[] matchKey,
+            int termsIndex) {
         this.owid = owid;
         this.flags = flags;
         this.licenseId = licenseId;
         this.matchKey = matchKey;
+        this.termsIndex = termsIndex;
     }
 
     // ----- Reading without throwing -----
@@ -206,15 +240,26 @@ private static FodIdParseResult read(OwidParseResult envelope) {
      * The rules are lower bounds only. The header must be present before the
      * type can be read, and the type then sets the least the payload can
      * hold. Anything longer is accepted as it stands, because the bytes past
-     * the match key are a creator context section whose shape the cloud
-     * judges.
+     * the match key are the Terms and then a creator context section whose
+     * shape the cloud judges. The Terms adds nothing to those bounds, since
+     * a payload that ends at the match key reads as a Terms of zero.
      */
     private static FodIdParseResult read(Owid owid) {
         byte[] payload = owid.getPayload();
         if (payload.length < HEADER_LENGTH) {
             return FodIdParseResult.failed(FodIdParseStatus.PAYLOAD_TOO_SHORT);
         }
         int flags = payload[FLAGS_OFFSET] & 0xFF;
+        // The version is read before any field, because a later version
+        // exists precisely because a field moved. Reading a payload of a
+        // version this package does not know under the layout it does know
+        // would answer with values that are wrong rather than absent,
+        // which is worse than refusing, and a version that nothing checks
+        // protects nothing.
+        int payloadVersion = payloadVersionOf(flags);
+        if (payloadVersion != SUPPORTED_PAYLOAD_VERSION) {
+            return FodIdParseResult.unsupportedPayloadVersion(payloadVersion);
+        }
         int matchKeyLength;
         switch (IdType.fromFlags(flags)) {
             case RANDOM:
@@ -246,8 +291,17 @@ private static FodIdParseResult read(Owid owid) {
         // bytes.
         byte[] matchKey = Arrays.copyOfRange(
             payload, MATCH_KEY_OFFSET, MATCH_KEY_OFFSET + matchKeyLength);
+        // The Terms byte follows the match key, wherever the type put its
+        // end. A payload that stops there has no Terms byte, and a missing
+        // byte is read as zero, which says the terms are not stated in the
+        // identifier. Absence and zero therefore mean the same thing and
+        // nothing has to tell them apart.
+        int termsOffset = MATCH_KEY_OFFSET + matchKeyLength;
+        int termsIndex = payload.length > termsOffset
+            ? payload[termsOffset] & 0xFF
+            : 0;
         return FodIdParseResult.parsed(
-            new FodId(owid, flags, licenseId, matchKey));
+            new FodId(owid, flags, licenseId, matchKey, termsIndex));
     }
 
     // ----- Reading with exceptions -----
@@ -349,6 +403,19 @@ public static FodId fromOwid(Owid owid) throws OwidException {
      * failure is an OWID one, which is the split the readers have always
      * made. The message names the status and the parameter, never the input.
      */
+    /**
+     * Bits 4 and 5 of the Flags byte, being the version of the payload
+     * layout the identifier follows. The envelope carries a version of its
+     * own at its first byte, which versions the envelope, whilst this one
+     * versions the payload.
+     *
+     * @param flags the Flags byte
+     * @return the payload layout version (0 to 3)
+     */
+    private static int payloadVersionOf(int flags) {
+        return (flags >> 4) & 0b11;
+    }
+
     private static FodId valueOrThrow(FodIdParseResult result, String paramName)
             throws OwidException {
         switch (result.getStatus()) {
@@ -362,6 +429,11 @@ private static FodId valueOrThrow(FodIdParseResult result, String paramName)
                 throw new IllegalArgumentException(
                     "51Did payload is shorter than the minimum for its "
                     + "identifier type (" + paramName + ").");
+            case UNSUPPORTED_PAYLOAD_VERSION:
+                throw new IllegalArgumentException(
+                    "51Did payload version " + result.getPayloadVersion()
+                    + " is not one this package can read ("
+                    + paramName + ").");
             default:
                 throw new OwidException(
                     "The value is not an OWID envelope: "
@@ -440,6 +512,36 @@ public byte[] getMatchKey() {
         return matchKey.clone();
     }
 
+    /**
+     * The address of the terms document this 51Did was created under, from
+     * the Terms byte that follows the match key.
+     * <p>
+     * The byte is an index into a table in the specification and this
+     * package turns the index into the address, so a caller never handles
+     * the byte. The address is answered and never fetched, and the receiver
+     * decides what to do with the document.
+     * <p>
+     * Null covers both an index of zero, which says the terms are not
+     * stated in the identifier, and an index added to the specification
+     * after this package was released, which it cannot name. A caller
+     * cannot tell those two apart, which is deliberate, because both lead
+     * to the same place, being that the identifier does not say which terms
+     * it was created under and the answer has to come from somewhere else.
+     * No package may build an address from an index it does not know, since
+     * that would name a document nobody wrote.
+     * <p>
+     * No address does not mean the identifier is unrestricted. Where an
+     * identifier may go is a separate question {@link #getUsage()} answers,
+     * which still bars a non-marketing identifier from a demand source.
+     *
+     * @return the address of the terms document, or null where the
+     *         identifier names no document this package knows, which is
+     *         never an empty string and is never built from the index
+     */
+    public String getTerms() {
+        return Terms.fromIndex(termsIndex).getUrl();
+    }
+
     /** @return the OWID version. */
     public Version getVersion() {
         return owid.getVersion();

pipeline.did/src/main/java/fiftyone/pipeline/did/FodIdParseResult.java

@@ -49,17 +49,49 @@ public final class FodIdParseResult {
 
     private final FodIdParseStatus status;
 
-    private FodIdParseResult(FodId value, FodIdParseStatus status) {
+    private final int payloadVersion;
+
+    private FodIdParseResult(
+            FodId value, FodIdParseStatus status, int payloadVersion) {
         this.value = value;
         this.status = status;
+        this.payloadVersion = payloadVersion;
     }
 
     static FodIdParseResult parsed(FodId value) {
-        return new FodIdParseResult(value, FodIdParseStatus.PARSED);
+        return new FodIdParseResult(
+            value, FodIdParseStatus.PARSED, FodId.SUPPORTED_PAYLOAD_VERSION);
     }
 
     static FodIdParseResult failed(FodIdParseStatus status) {
-        return new FodIdParseResult(null, status);
+        return new FodIdParseResult(
+            null, status, FodId.SUPPORTED_PAYLOAD_VERSION);
+    }
+
+    /**
+     * A read refused because the payload names a layout version this
+     * package does not know, carrying the version so that the throwing
+     * readers can name it in their message. The version is not public,
+     * because a caller has nothing to decide with it.
+     *
+     * @param payloadVersion the version the payload named
+     * @return the refused read
+     */
+    static FodIdParseResult unsupportedPayloadVersion(int payloadVersion) {
+        return new FodIdParseResult(
+            null,
+            FodIdParseStatus.UNSUPPORTED_PAYLOAD_VERSION,
+            payloadVersion);
+    }
+
+    /**
+     * The payload layout version a refused read found, for the message the
+     * throwing readers give. Zero for every other outcome.
+     *
+     * @return the version the payload named
+     */
+    int getPayloadVersion() {
+        return payloadVersion;
     }
 
     /**

pipeline.did/src/main/java/fiftyone/pipeline/did/FodIdParseStatus.java

@@ -114,7 +114,16 @@ public enum FodIdParseStatus {
      * anything past the match key is a creator context section whose lengths
      * belong to the cloud.
      */
-    INVALID_TYPE_PAYLOAD_LENGTH;
+    INVALID_TYPE_PAYLOAD_LENGTH,
+
+    /**
+     * Bits 4 and 5 of the Flags byte name a payload layout version this
+     * package does not know, so no field is read. A later version exists
+     * precisely because a field moved, so reading the payload under the
+     * layout this package knows would answer with values that are wrong
+     * rather than absent.
+     */
+    UNSUPPORTED_PAYLOAD_VERSION;
 
     /**
      * Carries an OWID status across unchanged.