Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
version: 2
updates:
- package-ecosystem: gomod
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
groups:
go-deps:
patterns:
- "*"

- package-ecosystem: gomod
directory: /examples/entgo
schedule:
interval: weekly
cooldown:
default-days: 7
groups:
example-go-deps:
patterns:
- "*"

- package-ecosystem: github-actions
directory: /
schedule:
interval: weekly
cooldown:
default-days: 7
groups:
actions-deps:
patterns:
- "*"
17 changes: 17 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
on: pull_request

permissions:
contents: read

jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version-file: go.mod
- run: make lint
- run: make test
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.PHONY: test lint

test:
go test -v -race ./...

lint:
go vet ./...
go fmt ./...
61 changes: 61 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# sqlitestream

[![Go Reference](https://pkg.go.dev/badge/github.com/luzilla/sqlitestream.svg)](https://pkg.go.dev/github.com/luzilla/sqlitestream)

Open a SQLite database with WAL PRAGMAs suitable for [Litestream](https://litestream.io). When a replica URL is set, the file is restored from the replica if it is missing locally, and continuous replication runs in the background until `Close`.

Pure-Go: uses the `modernc.org/sqlite` driver, no cgo.

## Install

```sh
go get github.com/luzilla/sqlitestream
```

## API

```go
func Open(ctx context.Context, path string, opts ...Option) (*DB, error)
func Restore(ctx context.Context, outputPath string, force bool, opts ...Option) error

func (*DB) DB() *sql.DB // underlying handle
func (*DB) Close(ctx context.Context) error // flushes final WAL, stops replication

func WithReplicaURL(url string) Option
func WithAWSCredentials(accessKeyID, secretAccessKey string) Option
func WithLogger(logger *slog.Logger) Option
```

Without `WithReplicaURL`, `Open` just opens the file in WAL mode and does no
replication.

## Replica URL

An S3 replica URL looks like:

```
s3://bucket/path?region=eu-central-1
```

For an S3-compatible endpoint add `endpoint` and `forcePathStyle`:

```
s3://bucket/path?endpoint=http://127.0.0.1:9000&region=eu-central-1&forcePathStyle=true
```

> [!IMPORTANT]
> Pass credentials with `WithAWSCredentials`. Without them, Litestream falls back to the AWS default chain (env vars, shared config, instance role). Credentials in the URL userinfo (`s3://key:secret@…`) are **not** read.

## Examples

- [`examples/plain`](examples/plain) — `database/sql`
- [`examples/entgo`](examples/entgo) — [Ent](https://entgo.io)

## Development

```sh
make test # go test -race
make lint # go vet + go fmt
```

The suite includes an S3 round-trip test that starts an S3 mock server (`adobe/s3mock`); it needs a running Docker daemon.
5 changes: 5 additions & 0 deletions docs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// Package sqlitestream opens a SQLite database with PRAGMAs suitable for
// Litestream and, when a replica URL is configured via WithReplicaURL,
// restores the file from the replica if it's missing locally and runs
// continuous replication in the background.
package sqlitestream
4 changes: 4 additions & 0 deletions examples/entgo/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Ent generated code — recreate with `go generate ./...`.
# Keep the schema (source of truth), ignore everything else under ent/.
/ent/*
!/ent/schema/
17 changes: 17 additions & 0 deletions examples/entgo/ent/schema/user.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
package schema

import (
"entgo.io/ent"
"entgo.io/ent/schema/field"
)

// User is a minimal Ent schema for the example.
type User struct {
ent.Schema
}

func (User) Fields() []ent.Field {
return []ent.Field{
field.String("name").NotEmpty(),
}
}
3 changes: 3 additions & 0 deletions examples/entgo/generate.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
package main

//go:generate go run entgo.io/ent/cmd/ent generate ./ent/schema
79 changes: 79 additions & 0 deletions examples/entgo/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
module github.com/luzilla/sqlitestream/examples/entgo

go 1.26.1

replace github.com/luzilla/sqlitestream => ../..

require (
entgo.io/ent v0.14.6
github.com/luzilla/sqlitestream v0.0.0
)

require (
ariga.io/atlas v0.36.2-0.20250730182955-2c6300d0a3e1 // indirect
github.com/agext/levenshtein v1.2.3 // indirect
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.41.0 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 // indirect
github.com/aws/aws-sdk-go-v2/config v1.32.6 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.19.6 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.16 // indirect
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.20.18 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.16 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.16 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.4 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.7 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.16 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.16 // indirect
github.com/aws/aws-sdk-go-v2/service/s3 v1.95.0 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.8 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.12 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.41.5 // indirect
github.com/aws/smithy-go v1.24.0 // indirect
github.com/benbjohnson/litestream v0.5.11 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmatcuk/doublestar v1.3.4 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-openapi/inflect v0.19.0 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hablullah/go-hijri v1.0.2 // indirect
github.com/hablullah/go-juliandays v1.0.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hashicorp/hcl/v2 v2.18.1 // indirect
github.com/jalaali/go-jalaali v0.0.0-20210801064154-80525e88d958 // indirect
github.com/lmittmann/tint v1.1.3 // indirect
github.com/magefile/mage v1.14.0 // indirect
github.com/markusmobius/go-dateparser v1.2.4 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/pierrec/lz4/v4 v4.1.22 // indirect
github.com/prometheus/client_golang v1.17.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/psanford/sqlite3vfs v0.0.0-20251127171934-4e34e03a991a // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/superfly/ltx v0.5.1 // indirect
github.com/tetratelabs/wazero v1.2.1 // indirect
github.com/wasilibs/go-re2 v1.3.0 // indirect
github.com/zclconf/go-cty v1.14.4 // indirect
github.com/zclconf/go-cty-yaml v1.1.0 // indirect
golang.org/x/exp v0.0.0-20251023183803-a4bb9ffd2546 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/text v0.31.0 // indirect
google.golang.org/protobuf v1.35.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/libc v1.67.6 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.44.3 // indirect
)
Loading