diff --git a/docs/deploy/configuration/backups.md b/docs/deploy/configuration/backups.md index de8ac6b11..55cfaf732 100644 --- a/docs/deploy/configuration/backups.md +++ b/docs/deploy/configuration/backups.md @@ -126,7 +126,7 @@ Note multiple providers can be enabled simultaneously. ### S3 (AWS or S3-compatible) -- Works with AWS S3 and S3-compatible services (e.g., MinIO) +- Works with Amazon S3 and S3-compatible object stores (for example, MinIO) - Supports multi-node deployments - Recommended for production use @@ -140,7 +140,7 @@ Add `backup-s3` to the `ENABLE_MODULES` environment variable. For example, to en ENABLE_MODULES=backup-s3,text2vec-cohere ``` -#### S3 Configuration (vendor-agnostic) +#### S3 configuration (vendor-agnostic) This configuration applies to any S3-compatible backend. @@ -148,25 +148,51 @@ This configuration applies to any S3-compatible backend. | --- | --- | --- | | `BACKUP_S3_BUCKET` | yes | The name of the S3 bucket for all backups. | | `BACKUP_S3_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. | -| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint to be used.

Optional, defaults to `"s3.amazonaws.com"`. | -| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Optional, defaults to `"true"`. | +| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`. If the value includes a scheme, the `backup-s3` module fails to initialize and Weaviate does not start.

Optional, defaults to `"s3.amazonaws.com"`. | +| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Only the exact value `false` (in any capitalization) disables TLS. Any other value — including `0`, `off`, `no`, or a typo — leaves TLS enabled.

Optional, defaults to `"true"`. | +| `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete probe that Weaviate runs against the bucket before each backup. Useful for least-privilege credentials that can write objects but cannot delete them.

Optional, defaults to `false`. See [Skip the storage access check](#skip-the-storage-access-check). | -#### S3 Configuration (AWS-specific) +#### S3 authentication -For AWS, provide Weaviate with authentication details. You can choose between access-key or ARN-based authentication: +For Amazon S3, provide Weaviate with authentication details. You can choose between AWS IAM/ARN-based authentication or access-key authentication. For S3-compatible object stores, use access-key authentication. -#### Option 1: With IAM and ARN roles +The `backup-s3` module resolves credentials with its own logic rather than the AWS shared configuration chain. Profiles in `~/.aws/credentials` and the `AWS_PROFILE` variable are not read. Credentials are resolved in this order: -The backup module will first try to authenticate itself using AWS IAM. If the authentication fails then it will try to authenticate with `Option 2`. +1. An external authentication broker, if `BACKUP_S3_AUTH_PROXY_ENDPOINT` is set. This advanced option takes precedence over everything else, including access keys. +1. The access key and secret access key, if both are set in Weaviate's environment. +1. AWS IAM (an IRSA or EC2 instance role), used only when no access key and secret access key are set. -#### Option 2: With access key and secret access key +:::caution Access keys shadow an attached IAM role +If both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are present in Weaviate's environment, Weaviate uses them and never contacts IAM — even when an instance role is correctly attached. Stale or leftover keys therefore take priority over the role, silently. Unset both variables to authenticate with IAM. +::: + +##### Option 1: With AWS IAM and ARN roles + +Weaviate uses AWS IAM only when no access key and secret access key are set in its environment. No additional variables are required; the module uses the IRSA or EC2 instance role attached to the workload. + +##### Option 2: With access key and secret access key + +These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. Both the key and the secret must be set; if only one is present, Weaviate falls back to IAM. | Environment variable | Description | | --- | --- | -| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. | -| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. | -| `AWS_REGION` | (Optional) The AWS Region. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | - +| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. The legacy name `AWS_ACCESS_KEY` is also accepted. | +| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. The legacy name `AWS_SECRET_KEY` is also accepted. | +| `AWS_REGION` | Optional for Amazon S3. Set it for S3-compatible object stores, some of which require a specific region value. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | + +#### S3-compatible endpoints + +To use an S3-compatible object store, set `BACKUP_S3_ENDPOINT` to the provider's S3 endpoint host, optionally including a port, and authenticate with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Set `AWS_REGION` if your provider requires one. + +```bash +BACKUP_S3_BUCKET=weaviate-backups +BACKUP_S3_ENDPOINT=your-s3-endpoint.example.com # host[:port], no scheme +BACKUP_S3_USE_SSL=true +AWS_ACCESS_KEY_ID= +AWS_SECRET_ACCESS_KEY= +# Optional: set if your provider requires a region. +# AWS_REGION= +``` ### GCS (Google Cloud Storage)