Skip to content

feat!: rotate Google's JWKS roots through the Notary Service - #9

Open
xgreenx wants to merge 35 commits into
mainfrom
ci/libid-deploy-0.10
Open

xgreenx wants to merge 35 commits into
mainfrom
ci/libid-deploy-0.10

Conversation

@xgreenx

@xgreenx xgreenx commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Rotation through the Notary Service, proven against a real notary and live Google; multi-arch image. Supersedes #5, #7 and #8.

Changes

  • Rotation through the Notary Service: rotate takes a notarized reading verified by NotaryService; Merkle proofs gone; refusable moduli never reach the decision.
  • Prover lives here: src/jwks/, on libid_tlsn; no notary crate dependency.
  • No mock prover: it shipped in the release binary; deleted with its config, tests and sole dependencies.
  • network_file reads the real schema: [contracts].google_jwt_roots and legacy [identity].identity_jwks_roots.
  • Startup check: run/once refuse a config lacking notary_url or a gas signer.
  • Multi-arch image: amd64 + arm64, non-root, bases pinned by digest.
  • E2E stack is compose.yaml: anvil, the pinned notary, one-shot libid-deploy 0.11.0; health checks replace poll loops; CI and laptops run the same commands (README).
  • The e2e reads the published local-dev.toml (e2e/, byte-identical, checked at image build) through the keeper's own network_file. No YAML address extraction, no --rpc-url; its anvil hostname is one /etc/hosts line.

Verified

  • 25 unit + 4 integration tests; clippy -D warnings, nightly fmt clean.
  • Local: up -d --wait --build 14s, e2e 7s, a second up is a no-op; also up under Compose 2.15.1.
  • ci.yml: 103 lines deleted, 11 added.

Not verified / known issues

  • Hosts line not applied locally (no sudo): the local e2e used a 127.0.0.1 copy via KEEPER_E2E_NETWORK_FILE; CI runs the verbatim file.
  • No fast rotation test: only the e2e, asserting !google_keys.is_empty().
  • Breaking config: [mock_notary] fails the load (unknown field).
  • Notary pinned at custom-c957999, in compose.yaml only, until notary releases.

IdentityJwksRoots no longer verifies a Merkle transcript through the
legacy Notary contract: a rotation is the ordinary notarized session
every other consumer submits -- the ceremony section 9.1 record of the
JWKS reading plus the notary's EIP-191 signature over it -- verified by
the Notary Service, which charges the Notary Fee for it. The keeper
follows the contract.

* proof: `ProofSource::obtain` hands back the notary's `NotarizedSession`
  as it came off the wire (`notarize_jwks` for MPC-TLS, `build_session`
  for the mock). The record's bytes are what the signature covers, so
  nothing re-encodes them on the way to the chain.
* chain: `rotate_calldata` encodes `rotate(attestedData, proof)`;
  `submit_rotation` reads `quoteRotation()` right before sending and
  attaches it as value -- the contract forwards it whole and the service
  refuses anything but the exact fee, so a fee changed between decision
  and submission would otherwise waste an MPC-TLS session on a
  `WrongValue` revert. Trust is read from `trustedHashExpiresAt` only.
* config: one JWKS contract per network. `GoogleOidcVerifier` is
  archived with the wallet product, so `google_oidc_verifier`, the
  `ContractKind`/`Target` pair and the `[contracts]` mirror of the
  chain-configurations schema go; a network resolves to its
  `identity_jwks_roots` address or fails to load, which is what the
  legacy eden-testnet fixture now does (kept verbatim).
* e2e: NotaryService (fee 1000 wei) + IdentityJwksRoots from the embedded
  artifacts, Google's real pretty-printed body as the fixture (chunked by
  the mock, so the on-chain de-chunker runs), dry-run -> rotation ->
  steady state, and the fee lands in the service exactly once.

Pins: notary at the head of feat/jwks-ceremony-attestation and the
libid-rs crates at the head of fix/origin-form-request-target (the
commit that puts `GET /oauth2/v3/certs HTTP/1.1` on the wire, the line
the contract pins), both revs until the stack is tagged; the notary's
`[patch]` of tlsnotary/tlsn is repeated here because a `[patch]` is
read from the root manifest only. libid-contracts = "0.7.0" is NOT
PUBLISHED YET: the crate embeds forge output that is not in git, so it
cannot be a git dependency. Verified against a local checkout of that
release candidate through an uncommitted `[patch.crates-io]`; the
Cargo.lock entry for libid-contracts therefore has no source or
checksum, and `cargo build --locked` (the Dockerfile) fails until the
release is on crates.io and `cargo update -p libid-contracts` records
it.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
libid-contracts 0.7.0 renames IdentityJwksRoots to GoogleJwtRoots and moves
it beside the verifier it serves: the list is the google/v1 Platform
Verifier's trusted signing keys and nothing else, so "identity" said
nothing about it. The keeper follows the contract, on every surface that
spelled the old name:

* config: the inline key and the `[identity]` mirror of a
  chain-configurations network file are `google_jwt_roots`;
  `ResolvedNetwork.google_jwt_roots`; the load error and the address
  error name the new key.
* chain: the binding is `libid_contracts::bindings::ceremony::GoogleJwtRoots`
  (out of `identity`), calldata and the ABI round-trip test with it.
* run/proof/lib docs, log labels and the status table; the crate
  description says what the keeper keeps: Google's JWT signing keys,
  trusted on chain.
* e2e: deploys the `GoogleJwtRoots` artifact and configures it under the
  new key; the eden-testnet fixture's comment names the new key (the file
  is otherwise verbatim and still resolves to no contract).

Breaking for keeper.toml and for referenced network files: a
`identity_jwks_roots` key is now unknown and refused at load, which is the
right failure for a contract that no longer exists under that name.

Verified against a scratchpad copy of libid-contracts/rust/contracts at
feat/jwks-roots-notary-service (4d97ac3) with the version bumped to 0.7.0
through an uncommitted `[patch.crates-io]`, removed before this commit;
Cargo.lock is untouched.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The contract installs 2048-bit moduli only; a key of any other size
reverts InvalidModulusLength. The keeper hashed whatever decoded, so such
a key would read as untrusted and send a rotation that reverts, every
tick, for as long as Google published it. It is skipped with a warning
now, and the live keys keep rotating; a set with no acceptable key at all
is still an error.

Found by the key-insertion audit of GoogleJwtRoots (2026-09-03).

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The contract keeps the latest reading's key set and the one before it, and
nothing per kid: no modulusOfKid, no currentRoots. The end-to-end test
asserts the reading became the current generation whole, the kid hash the
keeper computed for a storage key nothing keys by any more is gone, and
the comments say lifetime where they said TTL.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
libid-contracts v0.7.0 is published, so the lock entry that was resolved
under a local path patch gains its registry source and checksum. Nothing
else moves: the manifest already asked for 0.7.0, and the notary and
libid-rs pins stay on their revs until those stacks release.

Verified against the published crate: nightly fmt, cargo build --locked,
clippy -D warnings, cargo test --all (18 unit, 4 integration incl. the
Anvil end-to-end rotation).

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The keeper obtained its notarized reading through the notary crate's
`jwks` helpers, which made it depend on the whole notary server and pin
the notary's exact libid-rs revision so cargo kept one copy of the types.
The notary is client-agnostic -- it signs the same record for every
session -- so those helpers were always keeper code.

Move them in: `jwks::layout` (reveal both directions whole, commit
nothing), `jwks::prover::{run_jwks_prover, notarize_jwks}` over
`libid_tlsn::prover_generic`, and `jwks::mock::MockProver`, which
synthesizes a byte-faithful record without MPC for the end-to-end test;
plus `examples/notarize_jwks`, which captures a real reading as a fixture.
`NotarizedSession` is `libid_transcript::AttestationWire`, so the keeper
and the notary agree on the frame by construction. The module carries its
own error type, the notary's minus the server-only variants.

Manifest: drop `notary`; libid-crypto and libid-signer move from rev
391078f to tag v0.3.0, joined by libid-tlsn, libid-transcript and
libid-ceremony at the same tag. Running the MPC prover means carrying the
TLSNotary fork pin (libid-org/tlsn 550220e) and, because cargo ignores
dependency-workspace patches, the same 24-entry libid-org/mpz patch block
the notary carries; the two move together.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Review of the move found leftovers from the module's notary origins, none
functional:

  * the module doc pinned [`NotarizedSession`] to `crate::NotarizedSession`,
    where the notary re-exported it; here the alias lives in this module,
    so the bare link resolves and the definition line goes (cargo doc was
    warning);
  * `jwks::Error` carried `Io`, `Json` and `Base64` variants nothing in the
    module converts into -- `Io`/`Json` served the notary's server side,
    `Base64` never existed there. The type now names exactly the sources
    the module converts: Jwks, Tlsn, Transcript, Crypto, Http;
  * three manifest comments still described a `notary` dependency: the
    rust-version floor, the publish = false rationale, and the libid-rs pin
    block (which also narrated a branch rev now replaced by the tag). The
    tlsn-patch comment likewise; and a stray blank line where the notary
    block was;
  * the e2e doc and the module doc still spoke from the notary's point of
    view;
  * the mock's hyper test uses `tokio::io::duplex`/`AsyncReadExt`, which
    compiled only through feature unification -- declare `io-util`.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
ae8d1c5 replaced a single line of a multi-line comment and left its
neighbour dangling mid-sentence. Rewrite the block under the tlsn `[patch]`
header as one paragraph. Comment only.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
… example

`examples/notarize_jwks` was a one-off from 2026-09-03: it captured a real
notarized reading so libid-contracts#21 could test `GoogleJwtRoots` against
a record Google actually served. It came along with the prover helpers when
they moved here, but it is not keeper functionality -- the keeper's own
`proof.rs` -> `notarize_jwks` -> `rotate` path is that flow with the chain
step on the end.

Replace it with the test that flow deserves. `e2e.rs` proves everything but
MPC-TLS, with the mock as the proof source; `tests/e2e_real.rs` proves the
rest: a running notary the keeper shares no crate with, the real contracts
on Anvil, Google's live key set, and one genuine MPC-TLS rotation -- dry run,
the rotation with the fee landing in the service, steady state. It is
`#[ignore]`d because it needs a notary (`KEEPER_E2E_NOTARY`, local binary or
the ghcr.io image) and network to Google; the module docs say how to run it.
`KEEPER_E2E_CAPTURE` writes the record it obtained, which is what the
example used to do, with the signature checked against the trusted address
first.

`chrono` leaves the dev-dependencies with the example.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
`cargo test --all` covers everything but MPC-TLS, with the mock prover
standing in. The ignored e2e covers the rest, and nothing ran it.

The notary is built from source at a pinned rev rather than pulled. The
published image predates the ceremony-attestation TCP path this test
drives. Bump NOTARY_REV once an image carries it.

The job reaches www.googleapis.com on purpose. GoogleJwtRoots pins that
authority, so a local TLS fixture cannot stand in.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Latest of both. libid-rs v0.4.0 brings libid-profiles and libid-identity
0.10 and two transcript fixes the JWKS layout depends on: a JSON member
is found with the whitespace JSON allows, and one scan finds the key for
both member shapes.

libid-contracts 0.10.0 is the version the ceremony contracts ship in.
The anvil rotation e2e deploys NotaryService and GoogleJwtRoots from its
artifacts and still reaches steady state.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
reqwest 0.13 is the version alloy's HTTP transport already pulls, so the
build now carries one reqwest and one TLS stack instead of two; `rustls`
is that release's name for the `rustls-tls` backend. toml 1, base64 0.23
and num-bigint 0.5 are the current majors, and nothing here exchanges
their types with another crate.

alloy and k256 stay: libid-contracts hands back alloy 1 types and
libid-crypto takes a k256 0.13 key, so a newer major would be a second,
incompatible type rather than an upgrade.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
1.98.1 is the current stable. The pin sits above the crate's inherited
1.95 floor, which no job exercises -- so say that, rather than repeating
a claim about building on the floor that stopped being true at 1.97.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
CI no longer deploys the contracts from inside this test: libid-deploy
puts them on the chain, so the addresses have to come in from outside.
KEEPER_E2E_GOOGLE_JWT_ROOTS selects that stack, KEEPER_E2E_RPC names its
chain, KEEPER_E2E_NOTARY_SERVICE overrides the service the fee is paid
to -- and with none of them set the test still spawns its own anvil and
deploys, so a local run stays one command.

The fee now comes from quoteRotation() rather than the constant, and two
wirings an external stack can get wrong are asserted before the session.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The job built the notary from source and deployed the contracts from
inside the test, so it proved the rotation against two things no
deployment ever runs. It now pulls the published notary image and lays
the stack down with libid-deploy on a throwaway anvil, which is exactly
what a deployment does; the image, the tool and its network file are one
env var each at the top of the file.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The graph has no openssl-sys -- ldd on the binary lists libc, libm and
libgcc_s -- so libssl-dev and libssl3 were claiming a dependency that
does not exist. The keeper writes nothing and listens on nothing, so it
runs as uid 10001. KEEPER_CONFIG now points at /etc/keeper/keeper.toml,
because --config defaults to a relative path that resolves against / in
a container, which makes a mounted config the whole run recipe. The
OCI labels link the ghcr package to this repo, as the notary's do.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
A release candidate could not be published at all, and every release
moved :latest, so pulling it could land on an RC. Both now follow the
notary's rules, which downstream consumers already read. The header also
described a wasm bundle this repo does not ship.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The head of the notary's release branch, and the tag its own image
workflow published last. A `custom-` tag is a build artefact: this
becomes a version tag once notary#10 merges and a release is cut.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The notary pull is the slowest step and depends on nothing else in the
job, so running it first both overlaps the download with the deploy and
proves the image starts even on a run that fails further down.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The step looked only under [identity]. chain-configurations declares the
ceremony contracts under [contracts], so the lookup found nothing and the
job failed after a successful deploy.

Read both sections. The pre-0.10 schema carried the roots under
[identity] as identity_jwks_roots, and the new one is still settling
upstream, so accepting either keeps this from breaking on a rename.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
custom-1f54c19 was amd64 only, so the job worked on GitHub runners but
forced emulation for anyone running it on an arm64 machine.

custom-c957999 is a manifest list covering linux/amd64 and linux/arm64.
Verified: the arm64 variant runs natively here and the rotation passes
against it.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
A tag alone lets the builder and the runtime drift under a rebuild; the
digest of each multi-platform index fixes every architecture at once.
The header now carries the docker run recipe the image expects.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
One tag must pull on GitHub runners, EKS nodes of either architecture
and Apple Silicon. Each platform compiles natively on a runner of its
own architecture and pushes by digest; a final job joins the digests
under the tag. QEMU is not used: an emulated tlsn/mpz build costs
hours, while ubuntu-24.04-arm is native and free for a public repo.
Same shape as libid-org/notary#10.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
A missing notary_url or gas signer used to surface only when a
rotation was due, weeks after the deployment looked healthy. `run`
and `once` without --dry-run now check both at startup, and a config
file that does not exist says where its path came from.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
The image needs Cargo.toml, Cargo.lock and src/ and nothing else. Without
a .dockerignore a local build ships target/ to the daemon as well -- 3.6 GB
against 265 kB here -- for files no stage copies.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
A `network_file` reference looked for `[identity].google_jwt_roots`, a key
that repo has never written. It spells the address `[contracts].google_jwt_roots`
since 0.10, and `[identity].identity_jwks_roots` before that, so every
network_file reference failed with "names no JWKS contract". Read both,
newest first.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The pinned v0.7.0 release does not exist; the tree is 0.10.0 and asset names
come from the tag. local-dev.toml names the compose service `anvil`, but this
job runs anvil on the host, so --rpc-url supplies the endpoint and the pinned
file stays byte-identical to what was released.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The CI header said two jobs; e2e made it three. The Dockerfile claimed
`run --dry-run` needs a notary and a signer; main.rs exempts it. The
e2e_real recipe pointed at a released notary tag, but every tag through
0.3.0-rc.3 still answers googleapis.com with the old JWKS response. And
decision.rs cited `libid-oidc-core`, a crate that exists nowhere.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
v0.11.0 takes the ceremony verifiers from libid-contracts 0.12.0. The
canonical addresses the keeper reads are unchanged; the circuit verifiers
moved, which this job does not touch.

Assisted-by: Claude Opus 5
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
@xgreenx xgreenx changed the title fix(config): read the roots where chain-configurations writes it; take 0.10.0 feat!: rotate Google's JWKS roots through the Notary Service Sep 14, 2026
It shipped in the production binary: `pub mod mock` had no cfg or
feature gate, and a `[mock_notary]` block in keeper.toml made a release
build sign records with a local key. The only guard was the deployment's
trust list. Green chose deletion over a feature gate.

Rotation is now proven only by tests/e2e_real.rs, in the E2E CI job. A
keeper.toml that still carries `[mock_notary]` fails at load: the schema
denies unknown fields and names the key.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The deleted rotation test was the only one that parsed the body Google
serves, and the deleted config test the only one that reached
deny_unknown_fields. Both checks are cheap and deterministic, so they stay
as plain tests, and tests/fixtures/certs.json keeps a reader.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Nothing `cargo test --all` runs spawns a chain any more: the rotation
test that did is gone, and e2e_real.rs is ignored there. The e2e job
still installs foundry for its own anvil.

Assisted-by: Claude Fable 5.1
Signed-off-by: xgreenx <xgreenx9999@gmail.com>
The e2e's stack was assembled by CI-only YAML: a notary container, a
bare anvil process, two poll loops and a curl of the deploy tool. One
compose file brings the same stack up anywhere with `up -d --wait`,
with health checks in place of the loops. The deploy step runs as a
one-shot service from the pinned chain-configurations 0.11.0 release,
whose local-dev.toml is committed verbatim and proven byte-identical
at image build, so the test can read the published file unmodified.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
The test took the roots address from an env var that CI extracted from
local-dev.toml with an embedded Python script, then wrote an inline
network: the file's schema was understood twice, and the keeper's own
`network_file` path, the one a deployment uses, was never exercised,
which is how its wrong key name survived. Now the test references the
published file and drives whatever the keeper resolves from it. The
CI job becomes the commands a developer runs; the file's compose-only
`anvil` hostname is mapped on the host, so nothing passes --rpc-url.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
The README did not exist; the e2e was runnable only by reading CI.

Signed-off-by: xgreenx <xgreenx9999@gmail.com>
Assisted-by: Claude Fable 5.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant