Skip to content

feat: add Bedrock IAM authentication with automatic credential renewal - #452

Open
JWThewes wants to merge 2 commits into
mainfrom
feature/bedrock-iam
Open

JWThewes wants to merge 2 commits into
mainfrom
feature/bedrock-iam

Conversation

@JWThewes

Copy link
Copy Markdown
Contributor

What changes

AWS deployments can now use IAM roles for Bedrock inference instead of distributing Bedrock API keys. Platform admins choose the authentication mode and can point the platform or an individual space at a dedicated inference role, including a role in another AWS account and region.

  • One explicit Bedrock mode: IAM supports Claude Code, OpenCode, and Codex. New runs use the space role or inherit platform IAM; personal and space Bedrock key overrides are disabled in IAM mode. Kiro keeps its separate API key.
  • Existing runs keep their credentials: switching modes or changing a role affects future runs. Running and paused intents retain their pinned key or IAM binding.
  • Admin setup wizard: generates CloudShell commands, trust policies, inference policies, and the broker's permission to assume the selected role. Supports new/existing roles, optional external IDs, copy/download, and a connection check before activation. Only platform admins can configure, verify, or clear space roles; the API enforces this.
  • Shared inference paths: stages, Composer, Quorum, model discovery, and graph enrichment use the same credential resolution. Built-in MCP access to application data retains the application runtime identity and region.

Credential lifetime and isolation

IAM defines access; the credential broker uses STS AssumeRole to obtain one-hour temporary credentials. A signed handoff binds each invocation to its execution, provider, role, and region. The runtime cannot assume roles directly, and issued inference sessions explicitly deny further role assumption.

Each invocation exposes an authenticated loopback AWS credential endpoint to its CLI. Credentials and renewal authorization stay in memory. Refresh begins five minutes before expiry, retries transient failures while the current credentials remain valid, and keeps the same CLI process alive. Answering a parked stage after several hours obtains a fresh invocation grant and fresh credentials for its pinned role.

If credentials expire, an independent watchdog stops that invocation's process group, attempts the normal session persistence and Git checkpoint, and reports bedrock_credentials_expired through the existing durable stage callback. The separate, non-sliding eight-hour invocation authorization limit reports bedrock_authorization_expired. Other invocations continue independently; there is no key fallback or automatic restart.

Related fixes found during verification

  • Create the CLI working directory before launching one-shot commands. A missing directory caused Codex/OpenCode Quorum calls to fail with cli_failed even though STS role assumption succeeded. The regression test launches real child processes for all four CLIs.
  • Allow dynamodb:ConditionCheckItem on the execution table for workflow checkpoints. Live investigation found this application-role permission missing independently of inference IAM.

Screenshots and live verification

Captured from the existing dev deployment on September 10, 2026. Account identifiers are redacted; browser chrome and unrelated content are cropped. These screenshots show the live configuration and observed results, not proof that every change at this PR head is deployed. No configuration or IAM permissions were changed while preparing this PR.

Platform authentication selection

Platform IAM mode, disabled Bedrock key overrides, and separate Kiro credentials

Setup wizard: connection details, generated policies, and platform verification

Connection details and existing-role option

Generated CloudShell commands and trust policy

The live platform check succeeded in us-east-1 and discovered 27 Claude inference profiles.

Successful platform connection check

Space inheritance and cross-account/cross-region configuration

flappy-bird inherits the platform role in us-east-1.

flappy-bird inherits platform IAM

casita-thewes uses a space role in a different AWS account and eu-north-1. Its live check successfully assumed the role and discovered 21 Claude inference profiles.

casita-thewes space role in eu-north-1

Successful casita-thewes cross-account and cross-region check

Connection checks verify STS and model discovery; they do not invoke models or prove every selected model's inference permissions or Codex availability.

Existing flappy-bird inference and graph enrichment evidence

The inspected Codex intent is pinned to platform IAM. Its intent-capture and reverse-engineering stages produced 12 artifacts; persisted metrics show 12 successful graph-enrichment inference calls. The screenshot shows reverse-engineering awaiting human review. Its separate Quorum failures motivated the working-directory fix above; this is not a claim that those failures have already been fixed in the deployment.

Existing IAM-backed run with generated artifacts awaiting review

Validation

  • Affected backend suite via the repository commit hook: 162 files, 2,885 tests passed (Node 24; Gremlin and DynamoDB Local containers).
  • Full frontend suite: 78 files, 545 tests passed (Node 24).
  • Frontend production build and TypeScript checking passed.
  • Repository lint and formatter checks, staged secret scanning, AWS SDK alignment, and Terraform formatting passed. Terraform validation passed with existing DynamoDB deprecation warnings.
  • Dependency audit: frontend has no reported production vulnerabilities. The root audit reports four existing advisories (two high, two moderate) in brace-expansion, js-yaml, hono, and qs; their lockfile entries are identical to main. The local hook treats these as advisory; this PR does not resolve them.

Coverage includes mode selection and key-run compatibility, platform-admin authorization, signed grant/renewal binding, role isolation, AWS credential endpoint authentication, all three Bedrock CLI configurations, application MCP identity, three hours of simulated uninterrupted renewal without replacing the child process, transient/hung renewal failures, process-group termination, typed callbacks, independent invocation cancellation, and paused-stage resumption.

The space-deletion integration test now verifies that IAM configuration is removed with the keys. DynamoDB Local test startup disables telemetry so it does not depend on an outbound telemetry request completing.

Long-duration behavior is tested with a simulated clock rather than a three-hour live soak. Composer uses the shared tested inference path but was not independently exercised in the live space verification. Live screenshots are not an end-to-end validation of all three CLIs.

Rollout

Deploy the application, broker, runtime, and Terraform changes together. Rebuild/publish managed environments from the updated base before assigning them to IAM spaces. An absent platform setting preserves API-key mode for existing deployments. AWS admins apply the wizard's generated policies, verify the connection, enable IAM, and select models available in the target account/region. The application generates IAM setup instructions; it does not provision roles or attach policies itself.

See the setup and lifecycle guide for configuration, revocation behavior, and operational limits.

@jeromevdl

Copy link
Copy Markdown
Contributor

@JWThewes thanks for reducing the scope of the PR. I still think we have a design issue and would suggest a refactoring before landing Bedrock IAM.

TL;DR This PR touches 71 files not because the feature is large, but because "agent credentials" is a transversal concern with no owning abstraction. Every new auth mode has to be threaded by hand through every layer. I think we should pause this PR, do a small behavior-preserving refactor first, then rebuild the IAM feature on top of the clean base. The expensive/security-sensitive work here is not wasted — it gets re-homed, not thrown away.

OK, only ~20 of the 71 are hand-written product code (the rest is ~13 test files, 7 Terraform, 8 zero-line PNG screenshots, 3 dependency/lockfile, and docs). The problem is those ~20: roughly half are edits to existing files just to thread a new authType through them. That churn is the smell, and it traces to four pre-existing design issues in the credential subsystem:

  1. Grant issuance is fanned out to 4 entrypoints. issueAgentCredentialGrant has 8 callers across lambda/agents, lambda/discussions, lambda/v2-orchestrator, and lambda/intents. Each one independently assembles bindings + purpose and mints a grant. No single owner — which is why a Bedrock PR ends up editing intents/index.js.
  2. The domain is hardcoded string unions branched everywhere. AGENT_CREDENTIAL_PROVIDERS = ['bedrock','kiro'], SOURCES = ['user','space','platform'], and now authType. Every layer does if (provider === …) / if (authType === 'iam') instead of dispatching. PROVIDER_CONFIG is a data table, so IAM — which isn't "a token in SSM" — didn't fit and spilled branches into agent-credentials.js, auth-resolver.js, intents, and two frontend files.
  3. The integration contract is an SSM parameter-path naming convention, not a typed boundary. The PR couldn't extend agentCredentialPath, so it added a parallel bedrockAuthPath/readBedrockAuth/writeBedrockAuth scheme alongside it. Two conventions now.
  4. Cross-process handoff via env vars leaks credential logic into launcher code. Because bindings/creds flow through process env (PLATFORM_CREDENTIAL_BINDINGS, BEDROCK_IAM_CREDENTIALS_URI, AIDLC_RUNTIME_AWS_ENV), the feature had to touch spawn, drivers, mcp-secret-resolver, and clients. This is also the root of a HIGH security finding in review — the task-role env snapshot is visible to the agent CLI. A cleaner env boundary makes that finding largely dissolve rather than get patched.

One more risk signal: the shared seams everything depends on — issueAgentCredentialGrant, verifyIssuedAgentCredentialGrant, loadAgentCredentialGrantSecret — have no direct test coverage today.

Target arcihtecture

Introduce one new module (lambda/shared/agent-credential-provider/) that holds a provider registry. Each credential provider is a strategy object implementing a small fixed interface. Nothing else in the codebase branches on provider or auth type anymore; it asks the registry.

  lambda/shared/agent-credential-provider/                                                                                                                                                                                                                                                                                                
    registry.js              # the interface + the provider map + get()/keys()                                                                                                                                                                                                                                              
    providers/                                                                                                                                                                                                                                                                                                              
      bedrock-apikey.js      # existing bearer-token-in-SSM behavior                                                                                                                                                                                                                                                        
      kiro.js                # existing bearer-token-in-SSM behavior                                                                                                                                                                                                                                                        
      bedrock-iam.js         # (PR2) STS assume-role + renewal + inference session policy

The interface every provider implements

  CredentialProvider {                                                                                                                                                                                                                                                                                                      
    id                         // 'bedrock' | 'kiro' | 'bedrock-iam'                                                                                                                                                                                                                                                        
    parameterName, inputField, setField   // storage descriptor (was PROVIDER_CONFIG)                                                                                                                                                                                                                                       
    normalizeBinding(raw)      // replaces: normalizeCredentialBinding + the IAM authType branch                                                                                                                                                                                                                            
    prepareEnv(cred, ctx)      // replaces: auth-resolver + spawn + mcp-secret-resolver + clients env branches                                                                                                                                                                                                              
    display(cred)              // replaces: intents/index.js + 2 frontend display branches                                                                                                                                                                                                                                  
    redeem(claims, ctx)        // replaces: credential-broker's STS/IAM branch                                                                                                                                                                                                                                              
  } 

AGENT_CREDENTIAL_PROVIDERS = ['bedrock','kiro'] becomes registry.keys(). The four scattered decision points collapse into the four methods above.

PR 1 — Prep refactor (behavior-preserving, backend-only)

Migrates the two existing homogeneous providers behind the registry. No behavior change; existing tests stay green. Small diffs.

New (3 files)

File Purpose
shared/agent-credential-providers/registry.js interface + provider map + get/keys
shared/agent-credential-providers/providers/bedrock-apikey.js wraps today's bedrock bearer-token behavior
shared/agent-credential-providers/providers/kiro.js wraps today's kiro key behavior

Edited (3–4 files, mostly shrinking)

File Change
shared/agent-credentials.js PROVIDER_CONFIG moves into the strategies; loops iterate registry.keys(); file gets smaller
shared/agent-credential-grants.js pull the provider list from the registry (small)
agentcore/auth-resolver.js replace per-provider env logic with provider.prepareEnv(...)
credential-broker/index.js replace inline read with provider.redeem(...) (trivial today — both just return a token)

Tests (1–2 files) — add coverage on the currently-untested seams (issue/verifyAgentCredentialGrant, registry dispatch); move existing provider tests.

≈ 7–9 files, small diffs, no frontend / Terraform / deps.

Note on the interface design. The two current providers are homogeneous (both are a bearer token in SSM injected as one env var), so there is little branching to unwind and the registry looks almost trivial. The abstraction is only stress-tested by the heterogeneous IAM case (STS instead of a token, a credential URI instead of a bearer var, a renewal lifecycle). Sketch the bedrock-iam provider's shape before finalizing the PR 1 interface so it isn't designed blind. If that feels too speculative, the alternative is to co-design — introduce the registry and add IAM as the second real provider in a single PR; still far smaller than 71 files, at the cost of mixing the refactor with the security-sensitive IAM diff.

PR 2 — Bedrock IAM feature (on the clean base)

The hard security logic from the current #452 lifts over almost verbatim, but now lands as one new provider file instead of being threaded through ~15 existing files.

New files (mostly lifted from #452 as-is)

File Origin Purpose
shared/agent-credential-providers/providers/bedrock-iam.js re-homed from #452's shared/bedrock-iam.js + agentcore/bedrock-iam.js + the branches currently scattered across auth-resolver / broker the whole IAM strategy in one place: role/region validate, SSM config read/write, prepareEnv (credential URI, no task-role leak), redeem (STS assume-role + inference session policy), renewal, display
frontend/.../BedrockIamWizard.tsx (+ test) from #452, unchanged setup UI
docs/using-the-platform/bedrock-iam.md (+ assets) from #452, unchanged feature docs
terraform/.../bedrock-iam roles (1–2 files) from #452, unchanged broker role + inference-only role
provider test file reworked from #452's IAM tests IAM strategy coverage

Edited (small)

File Change
shared/agent-credential-providers/registry.js register bedrock-iam — one line
shared/agent-credential-grants.js add the renewal-token audience/TTL (grant-level, genuinely new)
credential-broker/index.js wire the STS client into the redeem context (near-zero if the provider owns the call)
frontend settings page mount the wizard — one edit

≈ 12–14 files, but the core code delta is one new provider file — versus ~15 edited files today. auth-resolver, intents, spawn, drivers, mcp-secret-resolver, clients, and the two frontend display files are no longer touched. That is where the 71→14 reduction comes from, and it is also why the HIGH env-leak finding stops being a whack-a-mole patch: the env boundary lives in exactly one place (prepareEnv).


Expected outcome

  • A future backend auth mode becomes one new provider file + one registration line, not a thread-through of every dispatch site. (Frontend UI work per mode stays irreducible.)
  • The HIGH security finding is addressed structurally, not patched — the env boundary lives in one place.
  • The untested credential seams get coverage.
  • Review is safer, not shorter: a mechanical behavior-preserving refactor (PR 1) is reviewed separately from the security-sensitive feature (PR 2), instead of both mixed into one 71-file diff.

Note: this refactor does not shrink the diff. The total across PR 1 + PR 2 is about the same as #452's source footprint (~20 files) — the 71 was always inflated by tests, screenshots, Terraform, and the lockfile, none of which change. So the pitch is not "fewer files." It is that the same work becomes safer today and cheaper tomorrow: a mechanical refactor reviewed apart from the security-sensitive feature, a structural fix for the HIGH env-leak finding, coverage on untested seams, and a next backend auth mode that costs ~one file.

@JWThewes What do you think ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants