Skip to content

Feature Request: Support S3-compatible object storage for attachments #295

Open

Description

@hemantprasad

Description

Windshift currently stores uploaded attachments only on the local filesystem, configured via ATTACHMENT_PATH. There is no option to use S3 or an S3-compatible object store (RustFS, MinIO, Wasabi, AWS S3, etc.) as the attachment backend.

This is a real limitation for production self-hosted deployments:

  • No horizontal scaling. If Windshift ever runs multiple replicas (behind a load balancer, in Kubernetes, etc.), every replica needs access to the same filesystem path. Today that means NFS/EFS or similar shared storage, extra infrastructure that S3 support would remove entirely.
  • No offloaded backups/lifecycle management. With local storage, attachments have to be backed up alongside the application data volume manually. S3-compatible storage gives teams native versioning, lifecycle policies (auto-expiry, tiering to cold storage), and independent backup/retention from the app itself.
  • Harder Kubernetes deployments. A RWX (ReadWriteMany) PVC is either unavailable or awkward on many clusters. Object storage is the standard pattern for stateless container workloads and would make Helm/K8s deployments much simpler.
  • Ecosystem expectation. S3-compatible storage is a fairly standard option in comparable self-hosted tools; its absence is one of the more surprising gaps for teams evaluating Windshift for anything beyond a single-node install.

Use Cases

  • Kubernetes / multi-replica deployment: A team runs Windshift with 2+ replicas behind a load balancer for availability. Attachments need to be readable from any replica, which requires shared object storage rather than local disk.
  • Cost-effective offloading: A self-hoster wants attachments on cheap, redundant storage (e.g. RustFS or AWS S3) instead of consuming primary server/VM disk space.
  • Compliance/retention policies: An organization needs bucket-level lifecycle rules (e.g., auto-delete attachments after N years) enforced at the storage layer rather than built into the app.
  • Disaster recovery: Object storage's built-in replication/versioning is used as the attachment backup strategy instead of a separate file-level backup job.

Proposed Solution

Add an S3-compatible storage backend as an alternative to ATTACHMENT_PATH, configured via environment variables (consistent with how POSTGRES_CONNECTION_STRING is offered alongside SQLite today — i.e., local disk stays the zero-config default, S3 is opt-in):

```
STORAGE_BACKEND=s3 # default: "local"
S3_ENDPOINT= # required for non-AWS (RustFS, MinIO etc.)
S3_BUCKET=
S3_REGION=
S3_ACCESS_KEY_ID=
S3_SECRET_ACCESS_KEY=
S3_FORCE_PATH_STYLE=false # true for RustFS/older S3-compatible servers
```

Suggested behavior:

  • local (default) preserves current behavior — no breaking change for existing installs.
  • s3 mode streams uploads/downloads directly to/from the bucket rather than buffering the whole file through local disk where avoidable.
  • Existing local attachments are not auto-migrated; a one-time migration script (similar in spirit to deploy/backup-restore.md) could be a separate follow-up rather than blocking this issue.
  • Presigned URLs (rather than proxying file bytes through the Go server) would be ideal for performance/scalability, if that fits the existing attachment-serving architecture.

Acceptance Criteria

  • Attachments can be uploaded/downloaded when STORAGE_BACKEND=s3 is set
  • Works against AWS S3 and at least one self-hosted S3-compatible target (e.g. RustFS, MinIO) for self-hosters who don't want a cloud dependency
  • ATTACHMENT_PATH/local mode continues to work unchanged when S3 is not configured
  • Documented in the self-hosting configuration reference alongside the existing storage & TLS settings
  • Startup fails fast with a clear error if STORAGE_BACKEND=s3 is set but required S3 variables are missing

Additional context

Related to the existing "Storage & TLS" configuration section in the self-hosting docs — this would sit alongside ATTACHMENT_PATH as an alternative rather than a replacement.

Activity

  1. valentimarco commented on Sep 28, 2026

    @valentimarco
    Contributor

    @stefan-ernst i would like to propose a PR for this, can i?

  2. stefan-ernst commented on Sep 29, 2026

    @stefan-ernst
    Contributor

    hey @valentimarco yes, PRs welcome!

  3. valentimarco commented on Oct 15, 2026

    @valentimarco
    Contributor

    Sorry for the long wait, i have the PR but i tested only on seaweedfs project, i will setup a aws one.
    I have a question regarding connection managment: If the connection goes down, how the recovery should work? an health endpoint is enough to manage the problem in clusters? (Never handle this problem my self, i usually implement above a microservice or a base microservice)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions