Repository navigation
Commit bf575fe
authored
feat(credentials): saved credentials and credential-only block versions, with keyless AWS/Databricks OIDC (#9763)
* feat(credentials): saved AWS and Databricks credentials with keyless OIDC
Adds an optional saved credential to every AWS block and the Databricks
block, alongside the existing inline key fields, plus Sim's own
workload-identity (OIDC) issuer so a credential can assume a customer's
AWS role or federate into a Databricks service principal without any
stored secret.
Existing workflows are unchanged: the inline fields stay, show, and
validate exactly as before until someone selects a saved credential.
- Issuer at /oidc/workload (discovery + JWKS), signing key from
WORKLOAD_IDENTITY_SIGNING_KEY; 404 and keyless option off when unset
- Credential-scoped subjects, HMAC-bound to the preparing user and
workspace, so a member cannot reuse another credential's subject
- aws-service-account: access keys, keys + AssumeRole (external ID), or
keyless AssumeRoleWithWebIdentity; resolved server-side per call
- databricks-service-account: PAT, service principal M2M secret, or
keyless RFC 8693 token federation
- AWS families accept the executor-authorized credential through trusted
operation context; clients add the session token only for that pair
- S3 get_object signs x-amz-security-token for temporary credentials;
inline signatures are byte-identical to before
- Tool credential binding honors optional (required: false) policies
* fix(credentials): close review findings on saved AWS/Databricks credentials
- Models can no longer inject a credential selector (or the token, session
token, or host derived from one) into a hybrid tool call, which would
have swapped an agent tool's inline keys for another workspace credential
- Databricks connections own the workspace host (authoritative `domain`)
- Drop the advanced credential-ID input: with a single picker, "is a
credential selected" can no longer disagree with the active canonical mode
- OIDC subject proofs are bound to their provider
- Databricks token and identity fetches refuse redirects
- Correct the AWS module doc on presigned URLs carrying a session token
* chore(credentials): satisfy test-pattern, unused-export, and test-type audits
* fix(credentials): keyless reconnect, availability, and shared credential plumbing
- Reconnecting a keyless AWS role or Databricks principal keeps its trusted
subject by default instead of issuing a new one that breaks the
customer's trust policy
- The connect dialogs offer the keyless mode only where the deployment can
sign tokens; the prepare endpoint reports availability instead of failing
- Register AWS (all 19 block types) and Databricks as stored-credential
integrations, so org settings, the v2 credentials API, and Copilot can
create and see them
- Optional STS region for GovCloud and China access keys
- One subject binder and config parser shared by both providers (neutral
copy), one modal core for submit/errors/trust display
- Generic `authorizedServiceAccountCredential` operation context instead of
an AWS-specific field; Databricks resolver checks credential type and
honors cancellation
- Model-blocked params derive from each tool's authoritative params
* test(credentials): type the handler mock in the AWS credential seam test
* feat(credentials): credential-only block versions instead of hybrid inline keys
Existing AWS and Databricks blocks are left exactly as on staging, only
marked legacy and hidden from the toolbar; placed instances keep running
and show an upgrade badge. Each gets a credential-only successor
(s3_v2 ... databricks_v2, textract_v3) built by spreading the legacy block
and swapping its secret fields for a required saved-credential picker.
- withSavedCredential derives the successor's fields, inputs, and params
builder; S3 and Textract split their builders so legacy key checks stay
- tools declare credentialSuppliedParams so serializer validation and the
canonical-id check stop asking credential-only blocks for inline keys
- CloudWatch gets saved-credential log group/stream selectors that sign
inside the credential's STS session
- docs generator resolves multi-level spreads and same-file param builders
- the AWS/Databricks providers are derived from the catalog again, so the
hand-maintained token-credential mapping is reverted
* docs(credentials): document credential-only blocks by their credential, not inline keys
The generator omits a tool's credential-supplied params on pages for
blocks that sign in with a saved credential, as it already omits
accessToken. Intros now point at the AWS/Databricks credential guides.
Regenerates OpenAPI, CLI API, and CLI docs for the new selector keys.
* fix(blocks): never replace trigger fields that share a secret's id
A webhook trigger can register with a field named like the block's
secret (PagerDuty, Fathom, EmailBison, GitLab). withSavedCredential now
leaves trigger-mode fields, and their dependencies, exactly as they are.
* feat(credentials): generic saved-token plumbing for credential-only blocks
Wave 0 of moving inline-secret blocks onto saved credentials.
- tokenCredentialOAuth(provider, credentialParams) declares which tool
params a saved token fills; credentialSuppliedParams is derived from it
- the executor fills those params after authorization, overriding any
caller value, and fails closed when the credential lacks a field
- hosted keys are never injected when a saved credential supplies the key
- v2 tool execute no longer demands credential-supplied params when a
credentialId is passed (fixes AWS/Databricks direct calls)
- withSavedCredential accepts an optional/conditional requirement
- check-block-registry verifies every credential-only successor's tools
authorize the picker's provider and actually fill what they declare
- redact secretAccessKey/awsSecretAccessKey keys in logs
- serializer contract test covers every operation of each successor
* feat(credentials): Tavily saved credential and credential-only tavily_v2
Reference migration for wave 1. A generic tokenCheckValidator verifies a
token with one documented authenticated request (no redirects); Tavily
uses GET /usage. Legacy tavily is unchanged apart from its legacy marker.
* fix(credentials): close wave-0 audit findings
- agent tool-input no longer synthesizes inline secret fields a block's
saved-credential picker supplies; one shared rule
(credentialSuppliedParamsFor) now drives tool-input, serializer
validation, and the canonical-id check
- the picker inherits the replaced secret's condition and requirement, so
sts_v2's unsigned calls (web identity, SAML) need no credential
- withSavedCredential replaces both halves of a canonical secret pair and
refuses options that would silently do nothing
- Databricks fills its inline apiKey/host through credentialParams, so its
tools are byte-identical to staging apart from the policy line
- AWS operation families record their tool ids; a test and an executor
guard ensure every AWS-credential tool resolves the credential
- the registry contract check fails on unpinned host params, optional
credentials over required params, malformed pickers, and kind mismatch
- hosted-key admission (v2) and pre-resolution validation honor a selected
saved credential; credential errors name fields in plain words
* feat(credentials): wave 1 — saved token credentials for 48 single-token blocks
Each block keeps its legacy definition (only hideFromToolbar + sunset) and
gains a credential-only <type>_v2. Every service gets a token-paste
provider with a one-request validator against a documented, read-only,
credit-free endpoint, and its tools fill the inline key from the saved
credential via tokenCredentialOAuth.
Blocks: affinity, agentmail, agentphone, ahrefs, apify, apollo, brex,
brightdata, browser_use, buffer, circleback, clerk, cloudflare,
crunchbase, daytona, elevenlabs, enrich, gamma, hex, huggingface,
incidentio, instantly, langsmith, launchdarkly, linq, loops, luma,
mailchimp, otter, persona, pinecone, profound, quartr, rb2b, rippling,
rocketlane, semrush, sendgrid, similarweb, sixtyfour, smartlead,
sportmonks, square, tailscale, trigger_dev, uptimerobot, youcom, youtube.
The registry contract check now probes each request tool's real URL
builder to find params that steer the host, and requires the saved
credential to fill them.
* fix(credentials): close wave-1 audit findings
- Pinecone and Semrush leave wave 1: Pinecone's per-call indexHost would
let a shared key reach any host, and Semrush has no credit-free check
that rejects a wrong key. Both are back to staging exactly.
- tokenCheckValidator: forbiddenMeansScoped accepts a valid key that lacks
the check endpoint's scope (Apollo, Brex, Bright Data, Apify, Rippling,
Persona); acceptsBody proves keys whose rejection arrives in a 200 body
(Buffer GraphQL)
- the host-steering probe lives in scripts/saved-credential-contract.ts:
typed placeholders for required params only, id/host shapes, bare and
scheme probes, and a broad destination-name fallback where the URL
cannot be built; covered by a fixture test
- S3 refuses a bucket or region that would move the signed request's host
- spread-inheriting docs keep their base block's outputs (restores the
Smartlead, Trigger.dev, and Instantly output tables)
- incident.io's credential is optional for alert events, which use their
own alert source token
- withSavedCredential refuses replaced secrets that show or require
differently; copy, intro, and hint fixes from the audit
* fix(credentials): close wave-1 re-audit findings
- host probe also defeats unparsed suffix checks and finds hosts behind
optional switches, probing every placeholder shape; broader destination
names where a URL cannot be built; the contract check fails when a
successor's tool is missing from the executable registry
- the optional-credential rule evaluates a conditional picker per
operation; incident.io hides its picker on alert events instead
- withSavedCredential takes an explicit picker condition and refuses one
that depends on a replaced secret at any nesting depth
- a scoped 403 counts only when it is the vendor's JSON response, never a
firewall or CDN page
- S3 names a bad bucket or region in its error
- pinecone/semrush catalog dates restored to staging
* feat(credentials): wave 1b — Mem0, Zep, and Discord saved credentials
Their legacy params builders throw without an inline secret, so each is
extracted into a module function with a requireKey flag: the legacy block
keeps identical validation and messages, the credential-only v2 skips only
the key check. discord_v2 hides its picker for webhook posts, which never
use the bot token. BlockToolParams is now a shared type for extracted
builders (S3, Textract, Discord).
* feat(credentials): wave 1b — v3 credential-only versions and Flint
Blocks that already had a v2 get a credential-only v3 that spreads it:
cursor, dub, extend, fireflies, intercom, pulse, quiver, reducto. The v2
becomes legacy and every older version's replacedBy points at v3. Extend,
Pulse, and Reducto's v2 builders trimmed the key inline, so each is
extracted into a builder that takes the key as an argument; the legacy v2
passes the trimmed key exactly as before.
Flint gets flint_v2 with a dedicated validator: its API has no read
endpoint, so an unknown task id answering 404 proves the key.
tokenCheckValidator gains rejectsKey for vendors that answer a bad key
with a server error (Fireflies returns 500 auth_failed), so the user is
told the key is wrong rather than the vendor down.
* fix(credentials): close wave-1b audit findings and CI boundary failures
- token checks for the 59 new providers live in one module
(validators/token-checks.ts) and their tool policies in
tools/shared/saved-credentials.ts, so the tool-registry boundary stays
within its baselines instead of adding ~60 modules to every graph
- the host-steering probe builds URLs through the shared transport
(new prepareToolRequestUrl) and also flags a host validated against a
vendor allowlist when two accepted values reach different tenants
- the AWS routing test reads typed metadata through getToolMetadata
- Flint accepts only its documented JSON 'Task not found' 404; Intercom no
longer treats a 403 (suspended workspace, plan without API) as valid;
Fireflies keeps its error text in logs when a 5xx isn't auth_failed
- Cursor help text, placeholder wording, Pulse intro, sorted secret fields
* fix(credentials): address pre-ship verification findings
- Refuse a keyless AWS role whose trust policy doesn't pin the credential's
subject, so no other workspace can assume it
- Treat a malformed workload-identity signing key as not configured
- Brex: reject an unknown token's scope-style 403; the token-check factory
now runs rejectsKey before accepting a scoped 403
- Flint: probe with a docs-shaped task id and a size-capped body read
- Drop the versioned BlockMeta entries that doubled templates; alias bare
block types in the app icon map so template icon clusters resolve
- Widen the host-probe fallback names and rotate digits in alternates
- Copy: picker placeholder casing, Databricks v2 description, Pulse and
Tailscale skills, AWS/Persona/Rippling intros, AWS credential docs
- Tests: AWS executor signing, AWS/Databricks credential setup, legacy
params builders, serializer coverage for blocks without operations
* refactor(credentials): one saved-credential table and shared helpers
- withSavedCredential reads the replaced fields from SAVED_CREDENTIAL_OAUTH,
so blocks and tools share one table instead of two hand-kept copies
- Reuse isTriggerModeSubBlock, getDependsOnFields, and a shared
isSavedCredentialPicker; build the replacedBy map once in the registry check
- newWorkloadIdentitySubject owns the subject format; the AWS trust-pin probe
runs alongside verification
- The issuer parses a bad signing key once instead of per request
- Token checks share the accepted result and the capped JSON reader
* chore(credentials): cleanup pass
- Keyless setup query gets its own key root, so broad credential
invalidations can't refetch an open setup and mint a new subject
- Named staleTime constant for the setup query
- Correct stale comments (relying parties, Databricks param names)
- Tighten tests: assert the refusal reason, make the BYOK guard reachable,
drop a duplicate and a test that could not fail
* fix(credentials): address review round 1
- A scoped 403 whose body can't be read is provider_unavailable, never accepted
- Provider error snippets read through the bounded reader
- Props interfaces for KeylessTrustFields and useWorkloadIdentitySetup; TSDoc on issuer responses
* fix(credentials): never cache the issuer's not-configured 4041 parent 61843fe commit bf575fe
2,165 files changed
Lines changed: 16517 additions & 3926 deletions
File tree
- apps
- docs
- components/ui
- content
- docs
- cli
- integrations
- platform/self-hosting
- sim
- app
- api/credentials/workload-identity
- oidc/workload
- .well-known/openid-configuration
- jwks
- workspace/[workspaceId]/integrations/components/connect-service-account-modal
- blocks
- blocks
- hooks/queries
- lib
- api/contracts
- block-metadata
- core
- config
- security
- credentials
- application
- orchestration
- token-service-accounts
- validators
- workload-identity
- integrations
- internal
- appconfig
- athena
- aws
- cloudformation
- cloudtrail
- cloudwatch
- codepipeline
- dynamodb
- ec2
- iam
- identity-center
- lambda
- rds
- s3
- secrets-manager
- ses
- sqs
- ssm
- sts
- textract
- tool-operations
- mothership/generated
- oauth
- permission-groups
- selectors
- server/providers
- tool-execution/application
- scripts
- serializer
- tools
- affinity
- agentmail
- agentphone
- ahrefs
- apify
- apollo
- appconfig
- athena
- brex
- brightdata
- browser_use
- buffer
- circleback
- clerk
- cloudflare
- cloudformation
- cloudtrail
- cloudwatch
- codepipeline
- crunchbase
- cursor
- databricks
- daytona
- discord
- dub
- dynamodb
- ec2
- elevenlabs
- enrich
- extend
- fireflies
- flint
- gamma
- generated
- hex
- huggingface
- iam
- identity_center
- incidentio
- instantly
- intercom
- lambda
- langsmith
- launchdarkly
- linq
- loops
- luma
- mailchimp
- mem0
- otter
- persona
- profound
- pulse
- quartr
- quiver
- rb2b
- rds
- reducto
- rippling
- rocketlane
- s3
- secrets_manager
- sendgrid
- ses
- similarweb
- sixtyfour
- smartlead
- sportmonks_core
- sportmonks_football
- sportmonks_motorsport
- sportmonks_odds
- sqs
- square
- ssm
- sts
- tailscale
- tavily
- textract
- trigger_dev
- uptimerobot
- youcom
- youtube
- zep
- packages
- deployment-config/src
- sim-cli/src/generated
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
| 335 | + | |
335 | 336 | | |
336 | 337 | | |
| 338 | + | |
337 | 339 | | |
| 340 | + | |
338 | 341 | | |
339 | 342 | | |
| 343 | + | |
340 | 344 | | |
341 | 345 | | |
342 | 346 | | |
343 | 347 | | |
344 | 348 | | |
345 | 349 | | |
| 350 | + | |
346 | 351 | | |
| 352 | + | |
347 | 353 | | |
| 354 | + | |
348 | 355 | | |
349 | 356 | | |
350 | 357 | | |
351 | 358 | | |
| 359 | + | |
352 | 360 | | |
353 | 361 | | |
354 | 362 | | |
| |||
357 | 365 | | |
358 | 366 | | |
359 | 367 | | |
| 368 | + | |
360 | 369 | | |
| 370 | + | |
361 | 371 | | |
| 372 | + | |
362 | 373 | | |
| 374 | + | |
363 | 375 | | |
364 | 376 | | |
365 | 377 | | |
366 | 378 | | |
367 | 379 | | |
368 | 380 | | |
| 381 | + | |
369 | 382 | | |
370 | 383 | | |
| 384 | + | |
371 | 385 | | |
372 | 386 | | |
373 | 387 | | |
| 388 | + | |
374 | 389 | | |
| 390 | + | |
375 | 391 | | |
| 392 | + | |
376 | 393 | | |
| 394 | + | |
377 | 395 | | |
378 | 396 | | |
| 397 | + | |
379 | 398 | | |
380 | 399 | | |
381 | 400 | | |
| |||
385 | 404 | | |
386 | 405 | | |
387 | 406 | | |
| 407 | + | |
388 | 408 | | |
389 | 409 | | |
| 410 | + | |
390 | 411 | | |
391 | 412 | | |
| 413 | + | |
392 | 414 | | |
393 | 415 | | |
394 | 416 | | |
| 417 | + | |
395 | 418 | | |
396 | 419 | | |
397 | 420 | | |
| 421 | + | |
398 | 422 | | |
399 | 423 | | |
400 | 424 | | |
| |||
403 | 427 | | |
404 | 428 | | |
405 | 429 | | |
| 430 | + | |
406 | 431 | | |
407 | 432 | | |
| 433 | + | |
408 | 434 | | |
409 | 435 | | |
| 436 | + | |
410 | 437 | | |
411 | 438 | | |
| 439 | + | |
412 | 440 | | |
413 | 441 | | |
414 | 442 | | |
| 443 | + | |
415 | 444 | | |
416 | 445 | | |
417 | 446 | | |
418 | 447 | | |
419 | 448 | | |
420 | 449 | | |
| 450 | + | |
421 | 451 | | |
422 | 452 | | |
423 | 453 | | |
| |||
428 | 458 | | |
429 | 459 | | |
430 | 460 | | |
| 461 | + | |
431 | 462 | | |
432 | 463 | | |
| 464 | + | |
433 | 465 | | |
434 | 466 | | |
435 | 467 | | |
| 468 | + | |
436 | 469 | | |
437 | 470 | | |
438 | 471 | | |
| |||
478 | 511 | | |
479 | 512 | | |
480 | 513 | | |
| 514 | + | |
481 | 515 | | |
482 | 516 | | |
483 | 517 | | |
| 518 | + | |
484 | 519 | | |
485 | 520 | | |
486 | 521 | | |
487 | 522 | | |
| 523 | + | |
488 | 524 | | |
489 | 525 | | |
| 526 | + | |
490 | 527 | | |
491 | 528 | | |
492 | 529 | | |
493 | 530 | | |
| 531 | + | |
494 | 532 | | |
495 | 533 | | |
496 | 534 | | |
| 535 | + | |
497 | 536 | | |
498 | 537 | | |
| 538 | + | |
499 | 539 | | |
500 | 540 | | |
501 | 541 | | |
| |||
510 | 550 | | |
511 | 551 | | |
512 | 552 | | |
| 553 | + | |
513 | 554 | | |
| 555 | + | |
514 | 556 | | |
515 | 557 | | |
| 558 | + | |
516 | 559 | | |
517 | 560 | | |
518 | 561 | | |
519 | 562 | | |
520 | 563 | | |
521 | 564 | | |
522 | 565 | | |
| 566 | + | |
523 | 567 | | |
524 | 568 | | |
525 | 569 | | |
526 | 570 | | |
527 | 571 | | |
| 572 | + | |
528 | 573 | | |
| 574 | + | |
529 | 575 | | |
| 576 | + | |
530 | 577 | | |
531 | 578 | | |
532 | 579 | | |
533 | 580 | | |
534 | 581 | | |
535 | 582 | | |
| 583 | + | |
536 | 584 | | |
537 | 585 | | |
538 | 586 | | |
| |||
571 | 619 | | |
572 | 620 | | |
573 | 621 | | |
| 622 | + | |
574 | 623 | | |
575 | 624 | | |
576 | 625 | | |
577 | 626 | | |
578 | 627 | | |
579 | 628 | | |
| 629 | + | |
580 | 630 | | |
581 | 631 | | |
582 | 632 | | |
| |||
588 | 638 | | |
589 | 639 | | |
590 | 640 | | |
| 641 | + | |
591 | 642 | | |
592 | 643 | | |
593 | 644 | | |
| 645 | + | |
594 | 646 | | |
595 | 647 | | |
| 648 | + | |
596 | 649 | | |
597 | 650 | | |
598 | 651 | | |
| 652 | + | |
599 | 653 | | |
600 | 654 | | |
601 | 655 | | |
602 | 656 | | |
| 657 | + | |
603 | 658 | | |
| 659 | + | |
604 | 660 | | |
605 | 661 | | |
606 | 662 | | |
607 | 663 | | |
| 664 | + | |
608 | 665 | | |
609 | 666 | | |
610 | 667 | | |
611 | 668 | | |
| 669 | + | |
612 | 670 | | |
| 671 | + | |
613 | 672 | | |
614 | 673 | | |
615 | 674 | | |
616 | 675 | | |
617 | 676 | | |
| 677 | + | |
618 | 678 | | |
619 | 679 | | |
620 | 680 | | |
621 | 681 | | |
622 | 682 | | |
623 | 683 | | |
624 | 684 | | |
| 685 | + | |
625 | 686 | | |
626 | 687 | | |
627 | 688 | | |
| 689 | + | |
628 | 690 | | |
629 | 691 | | |
630 | 692 | | |
631 | 693 | | |
632 | 694 | | |
| 695 | + | |
633 | 696 | | |
634 | 697 | | |
635 | 698 | | |
636 | 699 | | |
637 | 700 | | |
638 | 701 | | |
639 | 702 | | |
| 703 | + | |
640 | 704 | | |
| 705 | + | |
641 | 706 | | |
642 | 707 | | |
643 | 708 | | |
644 | 709 | | |
| 710 | + | |
645 | 711 | | |
646 | 712 | | |
647 | 713 | | |
648 | 714 | | |
| 715 | + | |
649 | 716 | | |
| 717 | + | |
650 | 718 | | |
| 719 | + | |
651 | 720 | | |
652 | 721 | | |
653 | 722 | | |
| 723 | + | |
654 | 724 | | |
655 | 725 | | |
656 | 726 | | |
657 | 727 | | |
658 | 728 | | |
| 729 | + | |
659 | 730 | | |
660 | 731 | | |
661 | 732 | | |
662 | 733 | | |
663 | 734 | | |
664 | 735 | | |
| 736 | + | |
665 | 737 | | |
| 738 | + | |
666 | 739 | | |
667 | 740 | | |
668 | 741 | | |
669 | 742 | | |
| 743 | + | |
670 | 744 | | |
671 | 745 | | |
672 | 746 | | |
| |||
675 | 749 | | |
676 | 750 | | |
677 | 751 | | |
| 752 | + | |
678 | 753 | | |
679 | 754 | | |
680 | 755 | | |
681 | 756 | | |
682 | 757 | | |
683 | 758 | | |
684 | 759 | | |
| 760 | + | |
685 | 761 | | |
686 | 762 | | |
687 | 763 | | |
| |||
704 | 780 | | |
705 | 781 | | |
706 | 782 | | |
| 783 | + | |
707 | 784 | | |
| 785 | + | |
708 | 786 | | |
709 | 787 | | |
| 788 | + | |
710 | 789 | | |
711 | 790 | | |
712 | 791 | | |
| |||
0 commit comments