Skip to content

docs(specs): the token request reveals its headers, and the verifier checks them - #31

Open
xgreenx wants to merge 13 commits into
mainfrom
specs/token-request-reveals-its-headers
Open

docs(specs): the token request reveals its headers, and the verifier checks them#31
xgreenx wants to merge 13 commits into
mainfrom
specs/token-request-reveals-its-headers

Conversation

@xgreenx

@xgreenx xgreenx commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Based on main, deliberately — see the note at the end.

The problem

The token request's headers were hidden, and for GitHub that was normative rather than ambiguous:

REQ-PLAT-43D: The GitHub Token Service MUST reveal no range outside the seven rows marked yes above.

…the transcript holds the authority only in a Host header this table hides

That leaves a region of the request the Platform Verifier cannot read, in the one direction it has to reason about. It reads grant_type, client_id and code_verifier out of the body with a form-encoding reading — while common REQ-COMMON-21B fixes the media type precisely because it "selects the platform's request parser".

A media type the verifier cannot see is a value the profile pins and nothing checks. The platform may have parsed those bytes into fields other than the ones read, and every revealed byte would still look correct.

The change

The request line and every request header are revealed, and REQ-PLAT-56A has the verifier compare them against a fixed list, byte for byte and in order.

Both halves are load-bearing. Revealing without comparing would leave the bytes public and unconstrained — neither private nor checked, which is roughly where the contract is today. Both profiles' requests are now enumerated the way §5.3 and §6.5 already enumerate the identity request's headers; that precedent is what this follows.

Why this discloses nothing

The request is composed byte for byte by the implementation and driven over a raw MPC-TLS session — not the browser's HTTP stack, so there is no cookie jar and no ambient credential attachment. The headers are four profile constants (host, content-type, accept, connection).

Concealment would not have protected a user in any case: the implementation is what would have put anything private there, so hiding it hides it from review, not from the party that inserted it. Enumeration is the protection — nothing outside the fixed list can be present, because the verifier rejects it.

The response direction is unchanged and still hides its status line and headers: those are the platform's own bytes and nothing reads them. That asymmetry is now stated rather than inferred.

Two things deliberately preserved

  • The Host argument. A revealed Host header is still not the authority — it is prover-composed text — so the authority still reaches the verifier as the authenticated TLS server identity. The prose keeps that reasoning and adds why revealing it changes nothing about it.
  • client_secret: still committed, still ordered last by REQ-COMMON-22 so the revealed run stays contiguous.

Verification

lint_spec.py reports 3 errors, 9 warnings — identical to main, all pre-existing. My first draft added two W4 warnings (two MUSTs in one sentence); those are split.

Base branch

Not stacked on #19 (docs/ceremony-terminology), even though it rewrites these exact table rows to rename Platform VerifierLOPPV. That PR is a draft, last updated 2026-09-03, already CONFLICTING with main, and part of a sweeping rename (ConsumerIdentity Integration, Chain ProfileLedger Profile) that will have to redo its diff regardless. Stacking a small correctness fix on it would strand the fix behind a large pending decision. When #19 is refreshed, its rename sweep absorbs these rows like every other.

In flight alongside this

  • libid-contracts: the verifier-side header comparison REQ-PLAT-56A requires.
  • A security audit of whether comparing the header set is sufficient, or whether more about the token request needs checking. If it returns findings, this PR may need another requirement before it lands.
  • libid: the browser draft on feat/ceremony-rebuild-plan currently selects per-field body ranges and commits the headers; its selectTokenReveals becomes a single range under this.

JSON whitespace (REQ-COMMON-19F, TEST-COMMON-10A). GitHub pretty-prints /user for the pinned media type. The Platform Verifier removes the JSON whitespace that touches a structural byte before it matches, counts and reads (what libid-org/libid-contracts#37 does); the Implementation reveals a member as the wire carried it, whitespace inside, and never commits it with a bearer; the compact spellings in this specification name the member after removal. REQ-PLAT-51 judges the GitHub terminator after that removal; REQ-PLAT-60 keeps the whitespace in the reveal. Carries the vectors of #32.

…checks them

The token request's headers were hidden. §5.2 left the request direction's
headers out of its table, and §6.4 said so outright -- REQ-PLAT-43D bounded
the disclosure to "the seven rows marked `yes`", and the prose named the
`Host` header as one "this table hides".

That leaves a region of the request the Platform Verifier cannot read, in the
one direction it has to reason about. It reads `grant_type`, `client_id` and
`code_verifier` out of the body with a form-encoding reading -- and common
REQ-COMMON-21B fixes the media type precisely because it "selects the
platform's request parser". A media type the verifier cannot see is a value
the profile pins and nothing checks, and the platform may have parsed those
bytes into fields other than the ones read.

So the request line and every request header are revealed, and REQ-PLAT-56A
has the verifier compare them against a fixed list, byte for byte and in
order. Both halves are needed: revealing without comparing would leave the
bytes public AND unconstrained, which is neither private nor checked. Both
profiles' requests are now enumerated the way §5.3 and §6.5 already enumerate
the identity request's headers, which is the precedent this follows.

Nothing here is user data. The request is composed byte for byte by the
implementation and driven over a raw MPC-TLS session, not the browser's HTTP
stack, so there is no cookie jar and no ambient credential: the four headers
are profile constants. Revealing them discloses nothing and leaves the sent
direction with no region a verifier cannot read -- and concealment would not
have protected a user in any case, since the implementation is what would have
put anything private there.

The response direction is unchanged and still hides its status line and
headers: those are the platform's own bytes and nothing reads them. That
asymmetry is now stated rather than left to be inferred.

The `Host` argument survives and had to. A revealed `Host` header is still not
the authority -- it is prover-composed text -- so the authority still reaches
the verifier as the authenticated TLS server identity, and the header is
compared against the profile like every other.

`client_secret` is untouched: still committed, still ordered last by
REQ-COMMON-22 so the revealed run stays contiguous.

The linter reports what it reported before this change: 3 errors, 9 warnings,
all pre-existing.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
… around it

The header list in the previous commit was wrong, and wrong in the way that
would have been found by a failing launch rather than by review: it enumerated
four headers and required the verifier to reject any other set, but a request
carrying a body carries a `Content-Length`. hyper emits one for every
known-size body -- `set_length` calls `set_content_length` -- and libid sets it
nowhere, so the GitHub Token Service's own exchange would have produced a head
of five headers and been refused by its own profile. As written the rule
rejected every genuine attestation.

`Content-Length` cannot be a profile constant: its value is the body's byte
count. So REQ-PLAT-56B pins the value against something the verifying side can
derive rather than against a literal -- the signed transcript length of the
sent direction less the head is the body, whether the body is revealed whole
as in X's request or revealed up to a committed suffix as in GitHub's, because
common REQ-COMMON-35 makes the direction tile exactly.

That is not bookkeeping. The verifier takes the body to be everything after
the sole CRLFCRLF; the platform takes it to be `Content-Length` bytes. Where
those disagree the fields the verifier reads are not the fields the platform
parsed, which is the same divergence REQ-PLAT-56's `grant_type` check exists
to stop, reached by a different route. `Transfer-Encoding` overrides
`Content-Length` outright and so is refused rather than described.

REQ-PLAT-56C carries the line-ending discipline over from the identity
request. Common REQ-COMMON-39 already refuses a bare line feed and an obsolete
line fold there, because a parser that accepts either ends the head somewhere
the verifier does not -- and the token head, which no requirement had ever
covered, is the same head with the same parsers reading it.

REQ-PLAT-56A now also says the match is exhaustive rather than a presence
test. "These five appear" is satisfied by a request carrying a sixth.

Found by an audit of the previous commit, which was written from a reading of
the request that no HTTP client produces.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Order carries no meaning here. Field order is insignificant in HTTP except for
repeated names, which REQ-PLAT-56A forbids outright, and the
`transfer-encoding`/`content-length` precedence REQ-PLAT-56B settles is by
presence rather than position. Nothing a reordering does is not already refused
by requiring the exact set.

What fixing it would cost is real. It binds every prover to the header order
its HTTP library happens to emit: the browser reaches the wire through tlsn's
wasm prover, whose `HttpRequest` carries headers in a `HashMap`, so the order
is whatever that iteration gives on the day. Pinning it also pins
`content-length` last, which is not a promise anyone made -- it is where
hyper's `set_length` inserts it, and a patch release could move it.

So the set is fixed and the order is not, and the prose says so once rather
than three times.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Sections 5.3 and 6.5 said "in this order" of headers no verifier orders:
the identity request is held to coverage, one line-anchored
authorization and the bearer framing, and common section 6 already says
header order carries no proof semantics unless a profile commits it.
The token request's headers were freed of order in this branch; these
two now say the same.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
@xgreenx
xgreenx marked this pull request as ready for review September 9, 2026 17:20
…rows

The tables listed each body field as its own revealed range. The
verifiers, the prover and the attested record carry the request as one
range -- the record cannot hold adjacent ranges apart, it merges them
before signing -- so a browser planning one range per field would see
its count change under it. Say so, once per platform.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
api.github.com refuses any request without one: 403, with a body that
says so, where the same request with one gets the 401 an absent bearer
earns. Section 6.5 listed five headers and left it out, so a prover
built from the text alone could never reach the identity read. The
browser draft already sends one. X's endpoint and both token endpoints
do not care, checked the same way, so the token request's pinned set is
unchanged.

The value is the runtime's to choose: nothing verifies it, and GitHub
requires only that it exist.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Both identity requests said "exactly N headers". No verifier holds them
to that: the contract compares the request line and the authorization
line, and the prover's layout finds the same line and nothing else. A
count nothing checks is a rule nobody can rely on and a prover can
only fail to meet, as GitHub's user-agent just showed. Say which lines
are compared and leave the rest to the runtime.

The token request is unchanged: there the verifier does hold the set.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
REQ-PLAT-56A had the verifier reject any header outside the profile's
list. A header outside it changes only what the platform answers, and a
wrong answer is a response the verifier cannot read, not one it can be
fooled by; the rule bound every prover to one HTTP library's habits for
nothing. It now requires `host` and the media type, forbids the five
names that change what the platform does with the request in a way no
revealed byte shows, and ignores the rest. `transfer-encoding` moves
from 56B to that list, beside `content-encoding`.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The linter says so, and it is right: two MUSTs in one sentence read as
one rule with a clause.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
REQ-PLAT-56A said values are compared exactly; the verifier removes the
optional whitespace around them first, and lowercases names with their
whitespace removed, which is the normalization REQ-COMMON-39 already
gives the identity request. A verifier built from the text alone would
have refused `content-type:application/...`, which the contract
accepts. 56C now also names a line with no colon, which the contract
refuses.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
…uests

REQ-COMMON-39 counted `authorization:bearer` lines, so a second header
under Basic or a platform's token scheme was never counted and the
Identity Platform answered for whichever credential it honoured; the
committed bearer is the one thing the cross-bind fixes. The needle is
now `authorization:` under any scheme.

REQ-COMMON-39A states the line-ending rule the verifier already applies
to the identity request -- no bare line feed, no fold -- which nothing
in this file said, and adds the bare carriage return. REQ-COMMON-39B
forbids on the identity request the names that change what the platform
does with it, `cookie` above all, with `_` read as `-`; the token
request's REQ-PLAT-56A now refers to that one list plus `authorization`,
and 56C names the bare carriage return and points at 39A.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The verifier requires exactly one empty line in the revealed token
request and a declared length with no leading zero; REQ-PLAT-56B said
neither. A second empty line is a second place a parser could end the
head, and a second spelling of the count is a second thing to compare
one spelling of.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
GitHub pretty-prints `/user` for the media type the profile pins, so the
compact delimiters this specification spells matched nothing it serves.
REQ-COMMON-19F fixes what libid-org/libid-contracts#37 does: the Platform
Verifier removes the JSON whitespace touching a structural byte and matches,
counts and reads over what is left; the Implementation reveals a member as
the wire carried it, whitespace inside, and never commits that whitespace
with a bearer; the compact spellings name the member after removal.
TEST-COMMON-10A lists the vectors. REQ-PLAT-51 judges GitHub's terminator
after that removal and REQ-PLAT-60 keeps the whitespace in the reveal.

Carries the vectors of #32 in the form the merged verifier
implements.

Co-authored-by: Wondertan <hlibwondertan@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: xgreenx <xgreenx9999@gmail.com>

@Wondertan Wondertan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I compared this revision with the ceremony implementation and current verifier. The main implementation findings are covered. These details remain:

  1. Make the JSON extraction boundary explicit in REQ-COMMON-19F.

    The verifier deliberately distinguishes reading from counting: it extracts a complete field from one contiguous revealed range, but counts duplicate delimiters across all revealed ranges, including delimiters split across ranges.

    The updated requirement specifies normalization without retaining that distinction explicitly. Please state that normalization applies within a range for extraction, and that concatenation used for duplicate detection cannot construct a field value. This matches the existing verifier.

    Restore the corresponding TEST-COMMON-10A negatives: disjoint fragments assembled into a field, a duplicate delimiter split across reveals, and hidden bearer-prefix whitespace. The existing HTTP-chunk-boundary case does not cover these attestation-range cases. Relevant requirements.

  2. Describe whitespace removal in terms of complete runs.

    “Every whitespace byte that touches a structural byte” is ambiguous for several consecutive whitespace bytes. The verifier removes an entire contiguous run when either adjacent byte is structural.

    Suggested wording: “Remove each maximal run of JSON whitespace when the byte immediately before or after that run is structural.” This agrees with the mixed-whitespace vectors already listed.

  3. Add conformance coverage for the new header requirements.

    REQ-COMMON-39A/39B and REQ-PLAT-56A/56B/56C have no linked tests. Whole-suite lint introduces five corresponding E10 findings.

    Extend the existing test entries, retaining their IDs, to cover:

    • Additional permitted headers and required/forbidden header normalization.
    • A second Authorization header using a different scheme.
    • Bare CR/LF, folded headers and malformed header lines.
    • Missing, duplicate, padded or mismatched Content-Length.
    • Request/body boundaries, including GitHub’s committed secret suffix.
  4. Resolve the remaining header prose contradictions.

    • REQ-PLAT-56A says to ignore every other header, but 56B requires checking Content-Length. Explicitly exempt that check.
    • The X token explanation still says its Host header is hidden, contradicting the updated disclosure table.
    • The PR description and GitHub rationale still describe a fixed header list, although the final requirements permit additional headers. Relevant section.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants