Skip to content

Add 03-use-cases/keyless-agent-credentials.md (keyless AWS and tailnet access for off-cloud agents) - #3

Open
jonatw wants to merge 6 commits into
shaun-agent:mainfrom
jonatw:keyless-clearance-usecase
Open

Add 03-use-cases/keyless-agent-credentials.md (keyless AWS and tailnet access for off-cloud agents)#3
jonatw wants to merge 6 commits into
shaun-agent:mainfrom
jonatw:keyless-clearance-usecase

Conversation

@jonatw

@jonatw jonatw commented Jul 25, 2026

Copy link
Copy Markdown

Adds a new 03-use-cases/ page: Keyless Access for Off-Cloud Agents — how an openab agent running outside AWS obtains scoped, short-lived credentials for AWS, a Tailscale tailnet, and GitHub with no long-lived secret in the container (IAM Roles Anywhere → STS Outbound Identity Federation → Tailscale WIF).

Source

Converted from an existing writeup — keyless-clearance.md — and rewritten to fit openab-map's use-case conventions:

  • Neutral, third-person technical voice matching the other 03-use-cases pages (schedule-agent-tasks, deploy-multi-agent) — no first-person or personal attribution.
  • Dropped the source's extended metaphor; kept the mechanics, the [Today] / [Proposed] / [Vision] maturity tags, a reference config, an adversarial risk section, and a component table.
  • Framed explicitly as a community-contributed pattern, not an openab-official standard.
  • Cross-repo references use full URLs; a Related footer links the sibling use-cases and the core "own the layers above" doc.

Review ask

Please review for fit and accuracy — happy to adjust voice, depth, or placement.

@shaun-agent

Copy link
Copy Markdown
Owner

@jonatw thanks for this — the three-leg chain (X.509 → Roles Anywhere → STS outbound OIDC → Tailscale WIF) is exactly the kind of field-tested pattern the map wants, and the [Today]/[Proposed]/[Vision] tagging is great. We ran an independent review against the current octobroker source and AWS/Tailscale docs before merging, and there are five things we'd like to fix first. Proposed fixes below — tell us if you agree (or push back where we're wrong):

1. The [Today] GitHub leg contradicts the "keyless" scope statement.
The scope note says "No long-lived shared secret lives in the runtime", but the octobroker leg has the agent holding a static, reusable X-Octobroker-Key. That's a real trade (one revocable broker key instead of N provider secrets), just not the stated property.
Proposed fix: add one honest sentence to the scope note — "the GitHub leg still holds one long-lived broker key; 'keyless' applies to the AWS/tailnet/GitHub upstream credentials, not the broker hop" — and mention key rotation/revocation for it in Blast Radius.

2. octobroker attribution.
The doc calls it "openab's broker (octobroker)". octobroker is a separate community project (openabdev/octobroker), not an OpenAB component — OpenAB itself neither ships nor requires it. Since the map just gained 01-core-concepts/mcp-facade.md, the accurate framing is: octobroker is a standalone credential broker that an OpenAB deployment can consume (e.g. as an mcp.json downstream of the MCP Facade, whose native tools/call dispatch keeps OBK's per-tool allowlists effective).
Proposed fix: reword the [Today] bullet accordingly + cross-link the facade page.

3. Trust-anchor revocation is overstated.
"Disabling or deleting the trust anchor invalidates every leaf at once" — it blocks new CreateSession calls; already-issued STS credentials stay valid until they expire (up to your duration_seconds).
Proposed fix: "…prevents new sessions; in-flight credentials live out their ≤1h TTL — keep profile duration short."

4. The SSM section can't execute under its own reference policy.
The reference role explicitly denies ssm:GetParameter* and kms:Decrypt, yet the persistence section has the agent reading SecureStrings. Also, the two values being protected (Tailscale client-id, audience) are documented by Tailscale as non-secret.
Proposed fix: either (a) drop SecureString for these two values (plain String parameter or config file suffices), or (b) introduce a separate retrieval principal and say so explicitly. Either resolves the contradiction.

5. Runbook + citations need a validation pass.
The Tailscale KB link (/kb/1317/workload-identity-federation) now redirects to an unrelated page — current doc is https://tailscale.com/docs/features/workload-identity-federation. Please re-run the runbook end-to-end on a clean host and paste the (redacted) outputs; and for the "audience/duration condition keys absent from request context" note, add the date + region of that test or an AWS doc citation, since the whole risk-bounding argument rests on it.

Also one small convention nit: consolidate Related/Sources into a ## Further Reading section like the other use-case pages, and cross-link 04-decision-trees/secrets-strategy.md with a line distinguishing OpenAB-owned secrets (resolved at boot, never passed to agents) from agent workload credentials (what this page provides).

None of this touches your architecture — legs 1–3 and the pluggable-source table survive as-is. If you agree, push the edits and we'll merge; if you'd rather we handle it, say the word and we'll apply them on a branch with full attribution (the map now documents exactly that take-over convention 🙂). 你若覺得哪點不對,直接反駁沒問題!

🤖 Generated with Claude Code

jonatw and others added 4 commits August 4, 2026 16:02
…ationale

Addresses all five review points, plus three errors found while verifying.

Review points:
1. "Keyless" scope now excludes the GitHub leg explicitly. That leg is
   rewritten from scratch: it records what is actually deployed (App private
   key in the agent runtime, self-minted installation tokens, installation-wide
   scope, one shared cache, static PAT fallback) rather than claiming a broker.
2. octobroker attribution corrected. It is a standalone community project,
   not an OpenAB component, and the reference deployment does not run it.
   Cross-linked to the MCP Facade as the composition point.
3. Trust-anchor revocation no longer overstated. Replaced with a layered
   model: anchor and CRL act at CreateSession only; aws:TokenIssueTime is the
   sole lever reaching already-issued sessions; static secrets are outside all
   of it. Notes that CRL import works with a self-managed CA.
4. SSM contradiction resolved. Tailscale client-id and audience are documented
   non-secrets and are plain String parameters; storing them as SecureString
   would be unreadable under the reference role's own deny. Real secrets are a
   separate class with a separate reader principal.
5. Citations re-verified. Tailscale KB link replaced (the old one redirects to
   an unrelated page). Condition-key claim carries its test date and region.

Additional corrections found during verification:
- The "condition keys absent from request context" note was a misdiagnosis and
  is withdrawn. Both keys exist and are enforced. Audience is multi-valued so
  it needs ForAllValues:StringEquals; DurationSeconds is optional so it needs
  NumericLessThanEqualsIfExists. A bare operator on either fails closed, which
  looks identical to the key being absent.
- Runbook: get-web-identity-token requires --signing-algorithm; the
  token-exchange endpoint takes client_id + jwt, not RFC-8693 parameters; and
  Tailscale v1.94.0+ can skip the JWT entirely via --client-id + --audience.
- Reference config gains the aws:SourceArn confused-deputy guard, wildcarded
  denies, and sts:AssumeRoot as its own entry.

Also merges main so the MCP Facade cross-link resolves on this branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ulary

- Use OpenAB in prose to match every sibling page (the lowercase form was
  inherited from the source write-up).
- Rewrite the sts:AssumeRoot note without the AssumeRo-ot hyphenation, which
  read like a literal action name inside backticks. State the prefix-expansion
  rule directly instead.
- Component reference: Needs a broker -> Proposed (needs a broker), so every
  row uses the Today/Proposed/Vision vocabulary declared in the intro.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e tailnet

The old filename was residue from the source write-up's aviation/border
metaphor, which this page deliberately dropped — "clearance" appears nowhere
in the body, and the name matched neither of the directory's naming patterns.

The title now names its own scope. "Keyless" is accurate and searchable for the
AWS and tailnet legs, which is what this page delivers; stating those two in the
title means a reader meets the GitHub boundary as a stated limit rather than
discovering it mid-page. The scope note shortens accordingly — it no longer has
to establish what the title already says, so it goes straight to why GitHub is
excluded.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonatw jonatw changed the title Add 03-use-cases/keyless-clearance.md (keyless access for off-cloud a… Add 03-use-cases/keyless-agent-credentials.md (keyless AWS and tailnet access for off-cloud agents) Aug 4, 2026
Three of the six entries (governed community bot, deploy multiple agents,
hook into the lifecycle) had no topical relation to credentials and no
annotation. Sibling pages only carry lateral use-case links when they are
related and explained, so those three are dropped.

What is OpenAB is now bound to the 'thin broker' keyword in the intro rather
than sitting in a list. The remaining two entries are also inline where they
are relevant, and stay listed with the annotations the review asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jonatw

jonatw commented Aug 4, 2026

Copy link
Copy Markdown
Author

Thank you @shaun-agent — this was a genuinely careful review, and it deserves a straight answer first: all five points were right, and I should have caught most of them myself.

The previous revision was assembled by pulling material out of our codebase and internal notes and stitching it into the use-case format. I checked that the architecture was right; I did not go back and re-verify the prose against the live environment or the current upstream docs. Several claims were true when first written and had quietly stopped being true. Please pass on my thanks to whoever (and whatever) did the independent pass — one of the things you flagged led me to withdraw a load-bearing claim that had been wrong for a while.

Everything below is pushed. I re-verified each item against the live deployment or primary docs rather than against my own earlier text.


1. The [Today] GitHub leg contradicts the "keyless" scope — agreed, and it is worse than you found

We do not run octobroker at all. That bullet was wrong in a way your fix would have papered over, so I rewrote the whole leg to record what actually runs:

  • Containerised agents fetch the GitHub App private key at use time (ssm get-parameter --with-decryption, via the task role) and sign their own RS256 JWT with it. The off-cloud host keeps the same key as a PEM on local disk.
  • Minted tokens carry the whole installation's scope — we pass neither repositories nor permissions.
  • One cache serves every purpose, so a git push leaves a token on disk that can also open issues, merge PRs, and drive Actions.
  • There is a static PAT fallback for when minting fails.

So it is not "one long-lived broker key" — it is a key that mints unlimited tokens, which is strictly worse. Fetching it through a role does not help: the role controls who may download the key, not the key's lifetime, scope, or auditability.

The section now says this plainly, and the title has been scoped to match (see the rename note at the bottom).

2. octobroker attribution — agreed, and the original contradicted itself

Worse than a mis-attribution: [Today] claimed octobroker ran our GitHub border while [Vision] said openab had not adopted it. Both in the same document. The source write-up this was converted from carried an explicit "openab does not currently run on octobroker" note; it was lost in the conversion.

Adopted your framing verbatim in substance: standalone community project, not an OpenAB component, and this page's reference deployment does not run it. Cross-linked the MCP Facade as the composition point, including the reason it works — native tools/call dispatch keeps per-tool allowlists effective rather than collapsing everything into execute_capability.

I also restructured the section around why a broker has to exist at all: Tailscale accepts a federated OIDC identity, GitHub does not, so the chain has a hard discontinuity and something must hold the App key. Today that something is every agent. The design question is not whether to have a broker but whether there are N unaudited ones or one audited one.

3. Trust-anchor revocation overstated — agreed, and expanded

Your correction is in, and pulling on it turned up more than one sentence's worth. It is now a four-layer model:

Revoking Lever Reaches already-issued credentials?
Future identity issuance Disable/delete trust anchor No — new CreateSession only
One compromised leaf rolesanywhere:ImportCrl No — checked at CreateSession
STS sessions already issued aws:TokenIssueTime deny Yes
A static secret already read Nothing in AWS — rotate at issuer No

Two things worth adding for anyone reading this page as a runbook:

  • CRL import works with a self-managed CA. Paying for a managed private CA buys automated CRL generation, distribution, and OCSP — not the ability to revoke at all. And no CRL at any price reaches credentials already issued.
  • aws:TokenIssueTime is the only instant lever, and it should be the first step in an incident, not trust-anchor deletion. Noted that AWS's own console sets the cutoff at roughly now + 30 seconds to absorb policy propagation.

4. The SSM section cannot execute under its own reference policy — agreed; reality picks your option (a)

Both halves of your fix turned out to be true simultaneously:

  • (a) Tailscale documents the client-id and audience as non-secret ("These are not secrets and will be visible in the admin console"), and our live parameters for the paths in use are plain String — read without --with-decryption, producing no kms:Decrypt. Storing them as SecureString would not just be unnecessary; the reference role's own deny would make them unreadable.
  • (b) The genuinely secret class is read by a different principal — the container's task role — never the read-only minting role. That is now stated explicitly, along with the limit of what a scoped vault buys: it controls who may retrieve a secret, never what happens to it afterwards.

5. Runbook and citations need a validation pass — agreed, and the condition-key note was flatly wrong

The Tailscale link is confirmed dead: /kb/1317/workload-identity-federation now 302s to /docs/integrations/google-sync, an unrelated page. Replaced with /docs/features/workload-identity-federation.

On the audience/duration note you asked me to date — I withdrew it instead. It was a misdiagnosis of our own operator error, and both keys are enforced in our deployed policy today (verified 2026-08-04, ap-northeast-1, by reading the live inline policy back from IAM). The root cause is visible in the API contract:

  • Audience is Audience.member.N — an array of 1–10 strings, so sts:IdentityTokenAudience is multi-valued. A bare StringEquals against a multi-valued key fails closed, which presents exactly like "the key is absent from the request context." Needs ForAllValues:StringEquals.
  • DurationSeconds is optional (60–3600, default 300) and is only injected when the caller passes it, so a bare NumericLessThanEquals denies every call that omits it. Needs NumericLessThanEqualsIfExists.

You were right that the whole risk-bounding argument rested on that claim. It did, and the argument is now the opposite one: audience lock and duration cap are enforced, not procedural.


Three more the validation pass turned up

Re-running the runbook against primary docs found errors nobody had flagged:

  • aws sts get-web-identity-token requires --signing-algorithm (RS256 or ES384). The published command omitted it and would fail outright.
  • The token-exchange call used RFC-8693 parameters (grant_type / subject_token / subject_token_type). Tailscale's documented endpoint takes form-encoded client_id + jwt.
  • Tailscale client v1.94.0+ can skip the JWT entirely--client-id plus --audience, auto-detecting AWS/GCP/GitHub Actions. That is the path our own containerised agents use, and the doc only showed the older manual flow. All three paths are now listed.

The reference config also gained things the deployed policy has and the page did not: the aws:SourceArn confused-deputy guard, wildcarded denies (an enumerated list had missed ssm:GetParameterHistory), and sts:AssumeRoot as its own entry — sts:AssumeRole* expands from the literal prefix AssumeRole and never reaches it.

Convention nit

Done — Related and Sources are consolidated into ## Further Reading, with the secrets-strategy cross-link drawing exactly the distinction you described: OpenAB-owned secrets resolved once at boot and never handed to an agent, versus the agent workload credentials this page covers.

While there I also pruned three lateral use-case links inherited from the old Related block — governed community bot, deploy multiple agents, hook into the lifecycle. They carried no annotation and no topical relation to credentials, which is not how the sibling pages use lateral links; yours explain why the reader should follow them. What is OpenAB moved inline onto the "thin broker" keyword rather than sitting in a list. Say the word if you would rather have the fuller list back.

One heads-up on your end

The mcp-facade.md link you pointed me at 404s from this branch — the branch was cut before that file landed on main, so the relative link resolved to nothing. Merged main in, and it resolves now. Worth knowing if other in-flight PRs are cross-linking recently-added pages.

Rename

The file is now 03-use-cases/keyless-agent-credentials.md, titled Keyless AWS and Tailnet Access for Off-Cloud Agents. The old name was residue from the source write-up's aviation metaphor, which this page deliberately dropped — "clearance" appeared nowhere in the body. Scoping the title means a reader meets the GitHub boundary as a stated limit rather than discovering it mid-page. Happy to revert either if it cuts against how you organise the map — that is your call, not mine.


Thanks again. 你們這輪抓得很準,尤其第 5 點逼我回頭實測,才發現那條 note 早就過期了 —— 這種錯自己是照不出來的。

@shaun-agent

Copy link
Copy Markdown
Owner

@jonatw the revision is a big step up — 4 of the 5 original points are fully resolved (keyless scoping, octobroker attribution + facade cross-link, revocation semantics, conventions), and the dated condition-key evidence is exactly what we asked for. Three small items still block merge:

  1. The SSM-reader contradiction survived the rewrite. Plain String correctly drops the KMS requirement, but reading it still needs ssm:GetParameter — and the reference minting role explicitly denies ssm:GetParameter* while the text says the reference deployment reads these parameters. One sentence fixes it: name a separate reader (task role / bootstrap step), or use a local config file for the off-cloud host.
  2. GitHub key paragraph is internally contradictory. "GitHub logs every issuance" has no supporting source in GitHub's installation-token docs, and "rotation is a single operation at the App" conflicts with your own later (correct) analysis that a distributed private key means updating every host and cache. Drop or substantiate the logging claim; align the rotation sentence with the later section.
  3. Stale citation: the IAM Outbound Federation link redirects to the IAM guide root — current page is https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_outbound.html.

Two non-blocking nits while you're in there: octobroker#54 is now closed-completed (and the settled design split PolicyClassifier into WriteClassifier + ResourceResolver), and S2's "Fargate/EC2/Lambda/SSO" grouping — Identity Center SSO is interactive/cache-backed, not an unattended workload role, so it deserves its own line.

Same offer as before: push these three and we merge, or say the word and we'll apply them with attribution per the take-over convention.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants