diff --git a/docs/production-deployment/worker-deployments/serverless-workers/aws-lambda.mdx b/docs/production-deployment/worker-deployments/serverless-workers/aws-lambda.mdx index 505e274207..fd2f2aa44a 100644 --- a/docs/production-deployment/worker-deployments/serverless-workers/aws-lambda.mdx +++ b/docs/production-deployment/worker-deployments/serverless-workers/aws-lambda.mdx @@ -27,7 +27,8 @@ This guide walks through deploying a Temporal [Serverless Worker](/serverless-wo ## Prerequisites {#prerequisites} - A Temporal Cloud account or a self-hosted Temporal Service vx.xx.x or later. - - Your Temporal Service frontend must be reachable from the Lambda execution environment. For Temporal Cloud, no additional configuration is needed. For self-hosted deployments on a private network, configure the Lambda function with [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html) to reach the Temporal frontend. + - For self-hosted deployments, complete the [self-hosted setup](/production-deployment/worker-deployments/serverless-workers/self-hosted-setup) before following this guide. + - The Temporal Service frontend must be reachable from the Lambda execution environment. For Temporal Cloud, no additional configuration is needed. For self-hosted deployments on a private network, configure the Lambda function with [VPC access](https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html) to reach the Temporal frontend. - An AWS account with permissions to create and invoke Lambda functions and create IAM roles. - The AWS-specific steps in this guide require the [`aws` CLI](https://aws.amazon.com/cli/) installed and configured with your AWS credentials. You may use other tools to perform the steps, such as the AWS Console or the AWS SDKs. @@ -150,7 +151,10 @@ aws lambda update-function-code \ --zip-file fileb://function.zip ``` -## 3. Configure IAM for Temporal invocation {#configure-iam} +## 3. Configure IAM for Temporal invocation (Cloud only) {#configure-iam} + +This section applies to Temporal Cloud. +For self-hosted Temporal Service deployments, see [Self-hosted setup](/production-deployment/worker-deployments/serverless-workers/self-hosted-setup#create-invocation-role) for IAM configuration with a different CloudFormation template. Temporal needs permission to invoke your Lambda function. The Temporal server assumes an IAM role in your AWS account to call `lambda:InvokeFunction`. @@ -158,12 +162,6 @@ The trust policy on the role includes an External ID condition to prevent [confu Deploy the following CloudFormation template to create the invocation role and its permissions. [Download the template](/files/temporal-cloud-serverless-worker-role.yaml). -:::note - -This template is scoped to Temporal Cloud. Self-hosted configuration guidance is in progress. - -::: - | Parameter | Description | |---|---| | `AssumeRoleExternalId` | A unique identifier that Temporal Cloud presents when assuming the role. Provided in your Namespace configuration. | @@ -369,4 +367,4 @@ The Worker starts, connects to Temporal, picks up the task, and processes it. You can verify the invocation by checking: - **Temporal UI:** The Workflow execution should show task completions in the event history. -- **AWS CloudWatch Logs:** The Lambda function's log group (`/aws/lambda/my-temporal-worker`) should show invocation logs with the Worker startup, task processing, and graceful shutdown. +- **AWS CloudWatch Logs:** The Lambda function's log group (`/aws/lambda/my-temporal-worker`) should show invocation logs with the Worker startup, task processing, and graceful shutdown. \ No newline at end of file diff --git a/docs/production-deployment/worker-deployments/serverless-workers/self-hosted-setup.mdx b/docs/production-deployment/worker-deployments/serverless-workers/self-hosted-setup.mdx new file mode 100644 index 0000000000..2b13a63efa --- /dev/null +++ b/docs/production-deployment/worker-deployments/serverless-workers/self-hosted-setup.mdx @@ -0,0 +1,199 @@ +--- +id: self-hosted-setup +title: Self-hosted setup for Serverless Workers +sidebar_label: Self-hosted setup +description: Configure a self-hosted Temporal Service to use Serverless Workers with AWS Lambda. +slug: /production-deployment/worker-deployments/serverless-workers/self-hosted-setup +toc_max_heading_level: 4 +keywords: + - serverless + - self-hosted + - lambda + - aws + - worker + - deployment +tags: + - Workers + - Deploy + - Serverless + - Self-hosted +--- + +This page covers the prerequisites for running [Serverless Workers](/serverless-workers) on a self-hosted Temporal +Service with AWS Lambda: + +1. Enable the Worker Controller Instance (WCI) on the server through dynamic configuration. +2. Provide the server with AWS credentials to assume IAM roles. +3. Create an IAM role in your AWS account that grants Temporal permission to invoke Lambda functions. + +Once setup is complete, follow the +[AWS Lambda deployment guide](/production-deployment/worker-deployments/serverless-workers/aws-lambda) to deploy your +Worker. + +## Enable the Worker Controller {#enable-worker-controller} + +The Worker Controller Instance (WCI) is the server component that monitors Task Queues and invokes compute providers. It +is disabled by default and must be enabled through [dynamic configuration](/references/dynamic-configuration). + +Add the following keys to your dynamic config file: + +```yaml +workercontroller.enabled: + - value: true + +workercontroller.compute_providers.enabled: + - value: + - aws-lambda + +workercontroller.scaling_algorithms.enabled: + - value: + - no-sync +``` + +To enable the Worker Controller for specific Namespaces instead of globally: + +```yaml +workercontroller.enabled: + - value: true + constraints: + namespace: 'your-namespace' +``` + +The Temporal Service watches the dynamic config file for changes and applies updates without a restart. + +## Configure AWS credentials {#configure-aws-credentials} + +The Temporal Service needs AWS credentials to assume an IAM role that invokes Lambda functions. How you provide +credentials depends on where the Temporal Service runs. + +**On AWS infrastructure (EC2, ECS, EKS):** The server uses the attached instance role, task role, or pod role +automatically. No additional credential configuration is needed. The attached role must have `sts:AssumeRole` permission +for the Lambda invocation role created in the next step. + +**Outside AWS:** Use [IAM Roles Anywhere](https://aws.amazon.com/iam/roles-anywhere/), or configure static AWS credentials in the server's environment (not recommended): + +``` +AWS_ACCESS_KEY_ID= +AWS_SECRET_ACCESS_KEY= +AWS_REGION= +``` + +These credentials must belong to an IAM user or role that has `sts:AssumeRole` permission for the Lambda invocation +role. + +## Create the Lambda invocation role {#create-invocation-role} + +Temporal invokes Lambda functions by assuming an IAM role in your AWS account. This role needs `lambda:GetFunction` and `lambda:InvokeFunction` +permission on your Worker Lambda functions, and a trust policy that allows the Temporal server's identity to assume it. + +Deploy the following CloudFormation template to create the role. +[Download the template](/files/temporal-self-hosted-serverless-worker-role.yaml). + +| Parameter | Description | +| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `TemporalIamRoleArn` | The ARN of the IAM role or user that the Temporal Service runs as. This is the identity the server uses to call `sts:AssumeRole`. To find the ARN, run `aws sts get-caller-identity` in the server's environment. | +| `AssumeRoleExternalId` | A unique string to prevent [confused deputy](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html) attacks. Choose any value and pass the same value when creating the Worker Deployment Version. | +| `LambdaFunctionARNs` | Comma-separated list of Lambda function ARNs that Temporal may invoke. | +| `RoleName` | Base name for the created IAM role. Defaults to `Temporal-Serverless-Worker`. | + +
+CloudFormation template + +```yaml +AWSTemplateFormatVersion: '2010-09-09' +Description: + Creates an IAM role that a self-hosted Temporal Service can assume to invoke Lambda functions for Serverless Workers. + +Parameters: + TemporalIamRoleArn: + Type: String + Description: The ARN of the IAM role or user that the Temporal Service runs as. + + AssumeRoleExternalId: + Type: String + Description: A unique identifier to prevent confused deputy attacks. + AllowedPattern: '[a-zA-Z0-9_+=,.@-]*' + MinLength: 5 + MaxLength: 45 + + LambdaFunctionARNs: + Type: CommaDelimitedList + Description: >- + Comma-separated list of Lambda function ARNs to invoke (e.g., + arn:aws:lambda:us-west-2:123456789012:function:worker-1,arn:aws:lambda:us-west-2:123456789012:function:worker-2) + + RoleName: + Type: String + Default: 'Temporal-Serverless-Worker' + +Resources: + TemporalServerlessWorker: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub '${RoleName}-${AWS::StackName}' + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + AWS: [!Ref TemporalIamRoleArn] + Action: sts:AssumeRole + Condition: + StringEquals: + 'sts:ExternalId': [!Ref AssumeRoleExternalId] + Description: 'The role the Temporal Service uses to invoke Lambda functions for Serverless Workers' + MaxSessionDuration: 3600 + + TemporalLambdaInvokePermissions: + Type: AWS::IAM::Policy + DependsOn: TemporalServerlessWorker + Properties: + PolicyName: 'Temporal-Lambda-Invoke-Permissions' + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - lambda:InvokeFunction + - lambda:GetFunction + Resource: !Ref LambdaFunctionARNs + Roles: + - !Sub '${RoleName}-${AWS::StackName}' + +Outputs: + RoleARN: + Description: The ARN of the IAM role created for the Temporal Service + Value: !GetAtt TemporalServerlessWorker.Arn + Export: + Name: !Sub '${AWS::StackName}-RoleARN' + + RoleName: + Description: The name of the IAM role + Value: !Ref RoleName + + LambdaFunctionARNs: + Description: The Lambda function ARNs that can be invoked + Value: !Join [', ', !Ref LambdaFunctionARNs] +``` + +
+ +Deploy the template: + +```bash +aws cloudformation create-stack --stack-name temporal-serverless-worker --template-body file://temporal-self-hosted-serverless-worker-role.yaml --parameters ParameterKey=TemporalIamRoleArn,ParameterValue= ParameterKey=AssumeRoleExternalId,ParameterValue= ParameterKey=LambdaFunctionARNs,ParameterValue='""' --capabilities CAPABILITY_NAMED_IAM --region +``` + +After the stack finishes creating, retrieve the IAM role ARN from the stack outputs: + +```bash +aws cloudformation describe-stacks --stack-name temporal-serverless-worker --query 'Stacks[0].Outputs[?OutputKey==`RoleARN`].OutputValue' --output text --region +``` + +Use this role ARN when creating the Worker Deployment Version. + +## Next steps {#next-steps} + +Follow the [AWS Lambda deployment guide](/production-deployment/worker-deployments/serverless-workers/aws-lambda) to +write your Worker code, deploy it to Lambda, and create a Worker Deployment Version with the IAM role from the previous +step. diff --git a/sidebars.js b/sidebars.js index 995dd27f9a..39fdc7a644 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1219,6 +1219,7 @@ module.exports = { }, items: [ 'production-deployment/worker-deployments/serverless-workers/aws-lambda', + 'production-deployment/worker-deployments/serverless-workers/self-hosted-setup', ], }, ], diff --git a/static/files/temporal-self-hosted-serverless-worker-role.yaml b/static/files/temporal-self-hosted-serverless-worker-role.yaml new file mode 100644 index 0000000000..488fa1ad96 --- /dev/null +++ b/static/files/temporal-self-hosted-serverless-worker-role.yaml @@ -0,0 +1,74 @@ +AWSTemplateFormatVersion: '2010-09-09' +Description: Creates an IAM role that a self-hosted Temporal Service can assume to invoke Lambda functions for Serverless Workers. + +Parameters: + TemporalIamRoleArn: + Type: String + Description: The ARN of the IAM role or user that the Temporal Service runs as. + + AssumeRoleExternalId: + Type: String + Description: A unique identifier to prevent confused deputy attacks. + AllowedPattern: '[a-zA-Z0-9_+=,.@-]*' + MinLength: 5 + MaxLength: 45 + + LambdaFunctionARNs: + Type: CommaDelimitedList + Description: >- + Comma-separated list of Lambda function ARNs to invoke + (e.g., arn:aws:lambda:us-west-2:123456789012:function:worker-1,arn:aws:lambda:us-west-2:123456789012:function:worker-2) + + RoleName: + Type: String + Default: 'Temporal-Serverless-Worker' + +Resources: + TemporalServerlessWorker: + Type: AWS::IAM::Role + Properties: + RoleName: !Sub '${RoleName}-${AWS::StackName}' + AssumeRolePolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Principal: + AWS: + [!Ref TemporalIamRoleArn] + Action: sts:AssumeRole + Condition: + StringEquals: + 'sts:ExternalId': [!Ref AssumeRoleExternalId] + Description: "The role the Temporal Service uses to invoke Lambda functions for Serverless Workers" + MaxSessionDuration: 3600 + + TemporalLambdaInvokePermissions: + Type: AWS::IAM::Policy + DependsOn: TemporalServerlessWorker + Properties: + PolicyName: 'Temporal-Lambda-Invoke-Permissions' + PolicyDocument: + Version: '2012-10-17' + Statement: + - Effect: Allow + Action: + - lambda:InvokeFunction + - lambda:GetFunction + Resource: !Ref LambdaFunctionARNs + Roles: + - !Sub '${RoleName}-${AWS::StackName}' + +Outputs: + RoleARN: + Description: The ARN of the IAM role created for the Temporal Service + Value: !GetAtt TemporalServerlessWorker.Arn + Export: + Name: !Sub "${AWS::StackName}-RoleARN" + + RoleName: + Description: The name of the IAM role + Value: !Ref RoleName + + LambdaFunctionARNs: + Description: The Lambda function ARNs that can be invoked + Value: !Join [", ", !Ref LambdaFunctionARNs]