Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 39 additions & 13 deletions docs/deploy/configuration/backups.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ Note multiple providers can be enabled simultaneously.

### S3 (AWS or S3-compatible)

- Works with AWS S3 and S3-compatible services (e.g., MinIO)
- Works with Amazon S3 and S3-compatible object stores (for example, MinIO)
- Supports multi-node deployments
- Recommended for production use

Expand All @@ -140,33 +140,59 @@ Add `backup-s3` to the `ENABLE_MODULES` environment variable. For example, to en
ENABLE_MODULES=backup-s3,text2vec-cohere
```

#### S3 Configuration (vendor-agnostic)
#### S3 configuration (vendor-agnostic)

This configuration applies to any S3-compatible backend.

| Environment variable | Required | Description |
| --- | --- | --- |
| `BACKUP_S3_BUCKET` | yes | The name of the S3 bucket for all backups. |
| `BACKUP_S3_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from. <br/><br/>Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. |
| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint to be used. <br/><br/>Optional, defaults to `"s3.amazonaws.com"`. |
| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS. <br/><br/>Optional, defaults to `"true"`. |
| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`. If the value includes a scheme, the `backup-s3` module fails to initialize and Weaviate does not start. <br/><br/>Optional, defaults to `"s3.amazonaws.com"`. |
| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS. <br/><br/>Only the exact value `false` (in any capitalization) disables TLS. Any other value — including `0`, `off`, `no`, or a typo — leaves TLS enabled. <br/><br/>Optional, defaults to `"true"`. |
| `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete probe that Weaviate runs against the bucket before each backup. Useful for least-privilege credentials that can write objects but cannot delete them. <br/><br/>Optional, defaults to `false`. See [Skip the storage access check](#skip-the-storage-access-check). |

#### S3 Configuration (AWS-specific)
#### S3 authentication

For AWS, provide Weaviate with authentication details. You can choose between access-key or ARN-based authentication:
For Amazon S3, provide Weaviate with authentication details. You can choose between AWS IAM/ARN-based authentication or access-key authentication. For S3-compatible object stores, use access-key authentication.

#### Option 1: With IAM and ARN roles
The `backup-s3` module resolves credentials with its own logic rather than the AWS shared configuration chain. Profiles in `~/.aws/credentials` and the `AWS_PROFILE` variable are not read. Credentials are resolved in this order:

The backup module will first try to authenticate itself using AWS IAM. If the authentication fails then it will try to authenticate with `Option 2`.
1. An external authentication broker, if `BACKUP_S3_AUTH_PROXY_ENDPOINT` is set. This advanced option takes precedence over everything else, including access keys.
1. The access key and secret access key, if both are set in Weaviate's environment.
1. AWS IAM (an IRSA or EC2 instance role), used only when no access key and secret access key are set.

#### Option 2: With access key and secret access key
:::caution Access keys shadow an attached IAM role
If both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are present in Weaviate's environment, Weaviate uses them and never contacts IAM — even when an instance role is correctly attached. Stale or leftover keys therefore take priority over the role, silently. Unset both variables to authenticate with IAM.
:::

##### Option 1: With AWS IAM and ARN roles

Weaviate uses AWS IAM only when no access key and secret access key are set in its environment. No additional variables are required; the module uses the IRSA or EC2 instance role attached to the workload.

##### Option 2: With access key and secret access key

These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. Both the key and the secret must be set; if only one is present, Weaviate falls back to IAM.

| Environment variable | Description |
| --- | --- |
| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. |
| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. |
| `AWS_REGION` | (Optional) The AWS Region. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. |

| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. The legacy name `AWS_ACCESS_KEY` is also accepted. |
| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. The legacy name `AWS_SECRET_KEY` is also accepted. |
| `AWS_REGION` | Optional for Amazon S3. Set it for S3-compatible object stores, some of which require a specific region value. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. |

#### S3-compatible endpoints

To use an S3-compatible object store, set `BACKUP_S3_ENDPOINT` to the provider's S3 endpoint host, optionally including a port, and authenticate with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Set `AWS_REGION` if your provider requires one.

```bash
BACKUP_S3_BUCKET=weaviate-backups
BACKUP_S3_ENDPOINT=your-s3-endpoint.example.com # host[:port], no scheme
BACKUP_S3_USE_SSL=true
AWS_ACCESS_KEY_ID=<your-access-key-id>
AWS_SECRET_ACCESS_KEY=<your-secret-access-key>
# Optional: set if your provider requires a region.
# AWS_REGION=<your-region>
```

### GCS (Google Cloud Storage)

Expand Down
Loading