docs(specs): the token request reveals its headers, and the verifier checks them - #31
docs(specs): the token request reveals its headers, and the verifier checks them#31xgreenx wants to merge 13 commits into
Conversation
…checks them The token request's headers were hidden. §5.2 left the request direction's headers out of its table, and §6.4 said so outright -- REQ-PLAT-43D bounded the disclosure to "the seven rows marked `yes`", and the prose named the `Host` header as one "this table hides". That leaves a region of the request the Platform Verifier cannot read, in the one direction it has to reason about. It reads `grant_type`, `client_id` and `code_verifier` out of the body with a form-encoding reading -- and common REQ-COMMON-21B fixes the media type precisely because it "selects the platform's request parser". A media type the verifier cannot see is a value the profile pins and nothing checks, and the platform may have parsed those bytes into fields other than the ones read. So the request line and every request header are revealed, and REQ-PLAT-56A has the verifier compare them against a fixed list, byte for byte and in order. Both halves are needed: revealing without comparing would leave the bytes public AND unconstrained, which is neither private nor checked. Both profiles' requests are now enumerated the way §5.3 and §6.5 already enumerate the identity request's headers, which is the precedent this follows. Nothing here is user data. The request is composed byte for byte by the implementation and driven over a raw MPC-TLS session, not the browser's HTTP stack, so there is no cookie jar and no ambient credential: the four headers are profile constants. Revealing them discloses nothing and leaves the sent direction with no region a verifier cannot read -- and concealment would not have protected a user in any case, since the implementation is what would have put anything private there. The response direction is unchanged and still hides its status line and headers: those are the platform's own bytes and nothing reads them. That asymmetry is now stated rather than left to be inferred. The `Host` argument survives and had to. A revealed `Host` header is still not the authority -- it is prover-composed text -- so the authority still reaches the verifier as the authenticated TLS server identity, and the header is compared against the profile like every other. `client_secret` is untouched: still committed, still ordered last by REQ-COMMON-22 so the revealed run stays contiguous. The linter reports what it reported before this change: 3 errors, 9 warnings, all pre-existing. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
… around it The header list in the previous commit was wrong, and wrong in the way that would have been found by a failing launch rather than by review: it enumerated four headers and required the verifier to reject any other set, but a request carrying a body carries a `Content-Length`. hyper emits one for every known-size body -- `set_length` calls `set_content_length` -- and libid sets it nowhere, so the GitHub Token Service's own exchange would have produced a head of five headers and been refused by its own profile. As written the rule rejected every genuine attestation. `Content-Length` cannot be a profile constant: its value is the body's byte count. So REQ-PLAT-56B pins the value against something the verifying side can derive rather than against a literal -- the signed transcript length of the sent direction less the head is the body, whether the body is revealed whole as in X's request or revealed up to a committed suffix as in GitHub's, because common REQ-COMMON-35 makes the direction tile exactly. That is not bookkeeping. The verifier takes the body to be everything after the sole CRLFCRLF; the platform takes it to be `Content-Length` bytes. Where those disagree the fields the verifier reads are not the fields the platform parsed, which is the same divergence REQ-PLAT-56's `grant_type` check exists to stop, reached by a different route. `Transfer-Encoding` overrides `Content-Length` outright and so is refused rather than described. REQ-PLAT-56C carries the line-ending discipline over from the identity request. Common REQ-COMMON-39 already refuses a bare line feed and an obsolete line fold there, because a parser that accepts either ends the head somewhere the verifier does not -- and the token head, which no requirement had ever covered, is the same head with the same parsers reading it. REQ-PLAT-56A now also says the match is exhaustive rather than a presence test. "These five appear" is satisfied by a request carrying a sixth. Found by an audit of the previous commit, which was written from a reading of the request that no HTTP client produces. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Order carries no meaning here. Field order is insignificant in HTTP except for repeated names, which REQ-PLAT-56A forbids outright, and the `transfer-encoding`/`content-length` precedence REQ-PLAT-56B settles is by presence rather than position. Nothing a reordering does is not already refused by requiring the exact set. What fixing it would cost is real. It binds every prover to the header order its HTTP library happens to emit: the browser reaches the wire through tlsn's wasm prover, whose `HttpRequest` carries headers in a `HashMap`, so the order is whatever that iteration gives on the day. Pinning it also pins `content-length` last, which is not a promise anyone made -- it is where hyper's `set_length` inserts it, and a patch release could move it. So the set is fixed and the order is not, and the prose says so once rather than three times. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Sections 5.3 and 6.5 said "in this order" of headers no verifier orders: the identity request is held to coverage, one line-anchored authorization and the bearer framing, and common section 6 already says header order carries no proof semantics unless a profile commits it. The token request's headers were freed of order in this branch; these two now say the same. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
…rows The tables listed each body field as its own revealed range. The verifiers, the prover and the attested record carry the request as one range -- the record cannot hold adjacent ranges apart, it merges them before signing -- so a browser planning one range per field would see its count change under it. Say so, once per platform. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
api.github.com refuses any request without one: 403, with a body that says so, where the same request with one gets the 401 an absent bearer earns. Section 6.5 listed five headers and left it out, so a prover built from the text alone could never reach the identity read. The browser draft already sends one. X's endpoint and both token endpoints do not care, checked the same way, so the token request's pinned set is unchanged. The value is the runtime's to choose: nothing verifies it, and GitHub requires only that it exist. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Both identity requests said "exactly N headers". No verifier holds them to that: the contract compares the request line and the authorization line, and the prover's layout finds the same line and nothing else. A count nothing checks is a rule nobody can rely on and a prover can only fail to meet, as GitHub's user-agent just showed. Say which lines are compared and leave the rest to the runtime. The token request is unchanged: there the verifier does hold the set. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
REQ-PLAT-56A had the verifier reject any header outside the profile's list. A header outside it changes only what the platform answers, and a wrong answer is a response the verifier cannot read, not one it can be fooled by; the rule bound every prover to one HTTP library's habits for nothing. It now requires `host` and the media type, forbids the five names that change what the platform does with the request in a way no revealed byte shows, and ignores the rest. `transfer-encoding` moves from 56B to that list, beside `content-encoding`. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The linter says so, and it is right: two MUSTs in one sentence read as one rule with a clause. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
REQ-PLAT-56A said values are compared exactly; the verifier removes the optional whitespace around them first, and lowercases names with their whitespace removed, which is the normalization REQ-COMMON-39 already gives the identity request. A verifier built from the text alone would have refused `content-type:application/...`, which the contract accepts. 56C now also names a line with no colon, which the contract refuses. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
…uests REQ-COMMON-39 counted `authorization:bearer` lines, so a second header under Basic or a platform's token scheme was never counted and the Identity Platform answered for whichever credential it honoured; the committed bearer is the one thing the cross-bind fixes. The needle is now `authorization:` under any scheme. REQ-COMMON-39A states the line-ending rule the verifier already applies to the identity request -- no bare line feed, no fold -- which nothing in this file said, and adds the bare carriage return. REQ-COMMON-39B forbids on the identity request the names that change what the platform does with it, `cookie` above all, with `_` read as `-`; the token request's REQ-PLAT-56A now refers to that one list plus `authorization`, and 56C names the bare carriage return and points at 39A. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The verifier requires exactly one empty line in the revealed token request and a declared length with no leading zero; REQ-PLAT-56B said neither. A second empty line is a second place a parser could end the head, and a second spelling of the count is a second thing to compare one spelling of. Assisted-by: Claude Opus 5 Signed-off-by: xgreenx <xgreenx9999@gmail.com>
GitHub pretty-prints `/user` for the media type the profile pins, so the compact delimiters this specification spells matched nothing it serves. REQ-COMMON-19F fixes what libid-org/libid-contracts#37 does: the Platform Verifier removes the JSON whitespace touching a structural byte and matches, counts and reads over what is left; the Implementation reveals a member as the wire carried it, whitespace inside, and never commits that whitespace with a bearer; the compact spellings name the member after removal. TEST-COMMON-10A lists the vectors. REQ-PLAT-51 judges GitHub's terminator after that removal and REQ-PLAT-60 keeps the whitespace in the reveal. Carries the vectors of #32 in the form the merged verifier implements. Co-authored-by: Wondertan <hlibwondertan@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Signed-off-by: xgreenx <xgreenx9999@gmail.com>
There was a problem hiding this comment.
I compared this revision with the ceremony implementation and current verifier. The main implementation findings are covered. These details remain:
-
Make the JSON extraction boundary explicit in
REQ-COMMON-19F.The verifier deliberately distinguishes reading from counting: it extracts a complete field from one contiguous revealed range, but counts duplicate delimiters across all revealed ranges, including delimiters split across ranges.
The updated requirement specifies normalization without retaining that distinction explicitly. Please state that normalization applies within a range for extraction, and that concatenation used for duplicate detection cannot construct a field value. This matches the existing verifier.
Restore the corresponding
TEST-COMMON-10Anegatives: disjoint fragments assembled into a field, a duplicate delimiter split across reveals, and hidden bearer-prefix whitespace. The existing HTTP-chunk-boundary case does not cover these attestation-range cases. Relevant requirements. -
Describe whitespace removal in terms of complete runs.
“Every whitespace byte that touches a structural byte” is ambiguous for several consecutive whitespace bytes. The verifier removes an entire contiguous run when either adjacent byte is structural.
Suggested wording: “Remove each maximal run of JSON whitespace when the byte immediately before or after that run is structural.” This agrees with the mixed-whitespace vectors already listed.
-
Add conformance coverage for the new header requirements.
REQ-COMMON-39A/39BandREQ-PLAT-56A/56B/56Chave no linked tests. Whole-suite lint introduces five corresponding E10 findings.Extend the existing test entries, retaining their IDs, to cover:
- Additional permitted headers and required/forbidden header normalization.
- A second Authorization header using a different scheme.
- Bare CR/LF, folded headers and malformed header lines.
- Missing, duplicate, padded or mismatched Content-Length.
- Request/body boundaries, including GitHub’s committed secret suffix.
-
Resolve the remaining header prose contradictions.
REQ-PLAT-56Asays to ignore every other header, but56Brequires checking Content-Length. Explicitly exempt that check.- The X token explanation still says its Host header is hidden, contradicting the updated disclosure table.
- The PR description and GitHub rationale still describe a fixed header list, although the final requirements permit additional headers. Relevant section.
Based on
main, deliberately — see the note at the end.The problem
The token request's headers were hidden, and for GitHub that was normative rather than ambiguous:
That leaves a region of the request the Platform Verifier cannot read, in the one direction it has to reason about. It reads
grant_type,client_idandcode_verifierout of the body with a form-encoding reading — while common REQ-COMMON-21B fixes the media type precisely because it "selects the platform's request parser".A media type the verifier cannot see is a value the profile pins and nothing checks. The platform may have parsed those bytes into fields other than the ones read, and every revealed byte would still look correct.
The change
The request line and every request header are revealed, and REQ-PLAT-56A has the verifier compare them against a fixed list, byte for byte and in order.
Both halves are load-bearing. Revealing without comparing would leave the bytes public and unconstrained — neither private nor checked, which is roughly where the contract is today. Both profiles' requests are now enumerated the way §5.3 and §6.5 already enumerate the identity request's headers; that precedent is what this follows.
Why this discloses nothing
The request is composed byte for byte by the implementation and driven over a raw MPC-TLS session — not the browser's HTTP stack, so there is no cookie jar and no ambient credential attachment. The headers are four profile constants (
host,content-type,accept,connection).Concealment would not have protected a user in any case: the implementation is what would have put anything private there, so hiding it hides it from review, not from the party that inserted it. Enumeration is the protection — nothing outside the fixed list can be present, because the verifier rejects it.
The response direction is unchanged and still hides its status line and headers: those are the platform's own bytes and nothing reads them. That asymmetry is now stated rather than inferred.
Two things deliberately preserved
Hostargument. A revealedHostheader is still not the authority — it is prover-composed text — so the authority still reaches the verifier as the authenticated TLS server identity. The prose keeps that reasoning and adds why revealing it changes nothing about it.client_secret: still committed, still ordered last by REQ-COMMON-22 so the revealed run stays contiguous.Verification
lint_spec.pyreports 3 errors, 9 warnings — identical tomain, all pre-existing. My first draft added two W4 warnings (twoMUSTs in one sentence); those are split.Base branch
Not stacked on #19 (
docs/ceremony-terminology), even though it rewrites these exact table rows to renamePlatform Verifier→LOPPV. That PR is a draft, last updated 2026-09-03, alreadyCONFLICTINGwithmain, and part of a sweeping rename (Consumer→Identity Integration,Chain Profile→Ledger Profile) that will have to redo its diff regardless. Stacking a small correctness fix on it would strand the fix behind a large pending decision. When #19 is refreshed, its rename sweep absorbs these rows like every other.In flight alongside this
feat/ceremony-rebuild-plancurrently selects per-field body ranges and commits the headers; itsselectTokenRevealsbecomes a single range under this.JSON whitespace (REQ-COMMON-19F, TEST-COMMON-10A). GitHub pretty-prints
/userfor the pinned media type. The Platform Verifier removes the JSON whitespace that touches a structural byte before it matches, counts and reads (what libid-org/libid-contracts#37 does); the Implementation reveals a member as the wire carried it, whitespace inside, and never commits it with a bearer; the compact spellings in this specification name the member after removal. REQ-PLAT-51 judges the GitHub terminator after that removal; REQ-PLAT-60 keeps the whitespace in the reveal. Carries the vectors of #32.