From cecf15c2496a620ec52807ca8b412e9dd2451706 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gonzalo=20Pe=C3=B1a-Castellanos?= Date: Wed, 8 Jul 2026 15:12:18 -0500 Subject: [PATCH 1/2] docs(backups): clarify S3-compatible endpoint configuration --- docs/deploy/configuration/backups.md | 29 ++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/deploy/configuration/backups.md b/docs/deploy/configuration/backups.md index de8ac6b11..20789579d 100644 --- a/docs/deploy/configuration/backups.md +++ b/docs/deploy/configuration/backups.md @@ -126,7 +126,7 @@ Note multiple providers can be enabled simultaneously. ### S3 (AWS or S3-compatible) -- Works with AWS S3 and S3-compatible services (e.g., MinIO) +- Works with Amazon S3 and any S3-compatible object store (for example, Backblaze B2, Cloudflare R2, or MinIO) - Supports multi-node deployments - Recommended for production use @@ -148,25 +148,42 @@ This configuration applies to any S3-compatible backend. | --- | --- | --- | | `BACKUP_S3_BUCKET` | yes | The name of the S3 bucket for all backups. | | `BACKUP_S3_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. | -| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint to be used.

Optional, defaults to `"s3.amazonaws.com"`. | +| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`.

Optional, defaults to `"s3.amazonaws.com"`. | | `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Optional, defaults to `"true"`. | -#### S3 Configuration (AWS-specific) +#### S3 authentication -For AWS, provide Weaviate with authentication details. You can choose between access-key or ARN-based authentication: +For Amazon S3, provide Weaviate with authentication details. You can choose between AWS IAM/ARN-based authentication or access-key authentication. For S3-compatible object stores, use access-key authentication. -#### Option 1: With IAM and ARN roles +#### Option 1: With AWS IAM and ARN roles -The backup module will first try to authenticate itself using AWS IAM. If the authentication fails then it will try to authenticate with `Option 2`. +The backup module first tries to authenticate using AWS IAM. If IAM authentication fails, it falls back to the configured access key and secret access key. #### Option 2: With access key and secret access key +These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. + | Environment variable | Description | | --- | --- | | `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. | | `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. | | `AWS_REGION` | (Optional) The AWS Region. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | +#### S3-compatible endpoints + +To use an S3-compatible object store, set `BACKUP_S3_ENDPOINT` to the provider's S3 endpoint host, optionally including a port, and authenticate with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Set `AWS_REGION` if your provider requires one. + +``` +ENABLE_MODULES=backup-s3 +BACKUP_S3_BUCKET=weaviate-backups +BACKUP_S3_ENDPOINT=your-s3-endpoint.example.com # host[:port], no scheme +BACKUP_S3_USE_SSL=true +AWS_ACCESS_KEY_ID= +AWS_SECRET_ACCESS_KEY= +# Optional: set if your provider requires a region. +# AWS_REGION= +``` + ### GCS (Google Cloud Storage) From a9d3ac3c6852051a5be03eaa6060edcc6a16c296 Mon Sep 17 00:00:00 2001 From: Ivan Despot <66276597+g-despot@users.noreply.github.com> Date: Fri, 14 Aug 2026 10:59:04 +0200 Subject: [PATCH 2/2] docs(backups): correct S3 credential precedence, tighten S3-compatible guidance The page said the backup module tries IAM first and falls back to access keys. resolveCredentials does the opposite: an auth proxy wins, then static env keys, and IAM is reached only when no keys are set. Stale keys therefore shadow an attached instance role silently, which is the failure users hit. Also: - keep MinIO as the only named S3-compatible store; drop the "any store" claim, since core requires multipart upload, ListObjectsV2 and Content-MD5 and exposes no addressing-style or CA setting - document the legacy AWS_ACCESS_KEY / AWS_SECRET_KEY aliases - note that a scheme in BACKUP_S3_ENDPOINT stops Weaviate from starting - note that only the literal "false" disables BACKUP_S3_USE_SSL - link BACKUP_SKIP_ACCESS_CHECK from the S3 table - nest Option 1 and Option 2 under S3 authentication - drop ENABLE_MODULES from the example; it contradicted the one above it - tag the environment block as bash --- docs/deploy/configuration/backups.md | 37 +++++++++++++++++----------- 1 file changed, 23 insertions(+), 14 deletions(-) diff --git a/docs/deploy/configuration/backups.md b/docs/deploy/configuration/backups.md index 20789579d..55cfaf732 100644 --- a/docs/deploy/configuration/backups.md +++ b/docs/deploy/configuration/backups.md @@ -126,7 +126,7 @@ Note multiple providers can be enabled simultaneously. ### S3 (AWS or S3-compatible) -- Works with Amazon S3 and any S3-compatible object store (for example, Backblaze B2, Cloudflare R2, or MinIO) +- Works with Amazon S3 and S3-compatible object stores (for example, MinIO) - Supports multi-node deployments - Recommended for production use @@ -140,7 +140,7 @@ Add `backup-s3` to the `ENABLE_MODULES` environment variable. For example, to en ENABLE_MODULES=backup-s3,text2vec-cohere ``` -#### S3 Configuration (vendor-agnostic) +#### S3 configuration (vendor-agnostic) This configuration applies to any S3-compatible backend. @@ -148,33 +148,43 @@ This configuration applies to any S3-compatible backend. | --- | --- | --- | | `BACKUP_S3_BUCKET` | yes | The name of the S3 bucket for all backups. | | `BACKUP_S3_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. | -| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`.

Optional, defaults to `"s3.amazonaws.com"`. | -| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Optional, defaults to `"true"`. | +| `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`. If the value includes a scheme, the `backup-s3` module fails to initialize and Weaviate does not start.

Optional, defaults to `"s3.amazonaws.com"`. | +| `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Only the exact value `false` (in any capitalization) disables TLS. Any other value — including `0`, `off`, `no`, or a typo — leaves TLS enabled.

Optional, defaults to `"true"`. | +| `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete probe that Weaviate runs against the bucket before each backup. Useful for least-privilege credentials that can write objects but cannot delete them.

Optional, defaults to `false`. See [Skip the storage access check](#skip-the-storage-access-check). | #### S3 authentication For Amazon S3, provide Weaviate with authentication details. You can choose between AWS IAM/ARN-based authentication or access-key authentication. For S3-compatible object stores, use access-key authentication. -#### Option 1: With AWS IAM and ARN roles +The `backup-s3` module resolves credentials with its own logic rather than the AWS shared configuration chain. Profiles in `~/.aws/credentials` and the `AWS_PROFILE` variable are not read. Credentials are resolved in this order: -The backup module first tries to authenticate using AWS IAM. If IAM authentication fails, it falls back to the configured access key and secret access key. +1. An external authentication broker, if `BACKUP_S3_AUTH_PROXY_ENDPOINT` is set. This advanced option takes precedence over everything else, including access keys. +1. The access key and secret access key, if both are set in Weaviate's environment. +1. AWS IAM (an IRSA or EC2 instance role), used only when no access key and secret access key are set. -#### Option 2: With access key and secret access key +:::caution Access keys shadow an attached IAM role +If both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are present in Weaviate's environment, Weaviate uses them and never contacts IAM — even when an instance role is correctly attached. Stale or leftover keys therefore take priority over the role, silently. Unset both variables to authenticate with IAM. +::: + +##### Option 1: With AWS IAM and ARN roles + +Weaviate uses AWS IAM only when no access key and secret access key are set in its environment. No additional variables are required; the module uses the IRSA or EC2 instance role attached to the workload. -These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. +##### Option 2: With access key and secret access key + +These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. Both the key and the secret must be set; if only one is present, Weaviate falls back to IAM. | Environment variable | Description | | --- | --- | -| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. | -| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. | -| `AWS_REGION` | (Optional) The AWS Region. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | +| `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. The legacy name `AWS_ACCESS_KEY` is also accepted. | +| `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. The legacy name `AWS_SECRET_KEY` is also accepted. | +| `AWS_REGION` | Optional for Amazon S3. Set it for S3-compatible object stores, some of which require a specific region value. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | #### S3-compatible endpoints To use an S3-compatible object store, set `BACKUP_S3_ENDPOINT` to the provider's S3 endpoint host, optionally including a port, and authenticate with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Set `AWS_REGION` if your provider requires one. -``` -ENABLE_MODULES=backup-s3 +```bash BACKUP_S3_BUCKET=weaviate-backups BACKUP_S3_ENDPOINT=your-s3-endpoint.example.com # host[:port], no scheme BACKUP_S3_USE_SSL=true @@ -184,7 +194,6 @@ AWS_SECRET_ACCESS_KEY= # AWS_REGION= ``` - ### GCS (Google Cloud Storage) - Works with Google Cloud Storage