Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,10 @@ ui/host/node_modules/
ui/host/dist/

# Dev files
.env
.env.*
.env*
**/.env*
secrets/
.sops.yaml
.tmp/
tmp/

Expand Down
265 changes: 265 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,265 @@
# Generated by cmd/fanout-docgen from internal/config/config.go.
# Regenerate with just docs-generate; verify with just docs-generate-check.
# Fanout does not load dotenv files. Inject through your launcher or deployment.
# Precedence: built-in defaults < nonempty environment values.
# All settings require restart. Empty environment values mean no override.
# Every entry is commented out so copying this template sets nothing, preserving image ENV.
# Uncomment only settings you want to supply; use runtime secret injection for secrets.
# Local authentication requires FANOUT_AUTH_CODE_SECRET (32+ characters).
# Chat is disabled unless FANOUT_AI_PROVIDER selects anthropic or openai; its key is required.

# ANTHROPIC_API_KEY; string; secret
# Required when FANOUT_AI_PROVIDER is anthropic.
# Default: (empty)
# ANTHROPIC_API_KEY=
# Alternatively set ANTHROPIC_API_KEY_FILE to a secret file path; do not set both.

# ANTHROPIC_BASE_URL; string
# Defaults to https://api.anthropic.com. Fanout appends /v1/messages, preserving any gateway path prefix. Use an HTTP(S) base URL without embedded credentials, a query, or a fragment.
# Default: (empty)
# ANTHROPIC_BASE_URL=

# FANOUT_ADDR; string
# Binds the browser, API, MCP, OTLP/gRPC and OTLP/HTTP listener.
# Default: 127.0.0.1:7520
# FANOUT_ADDR=

# FANOUT_AI_MODEL; string
# Overrides the provider's default: claude-sonnet-5-5 for Anthropic and gpt-6.1-sol for OpenAI. The OpenAI provider expects a reasoning model supporting reasoning.encrypted_content for stateless Responses replay. A 400 rejecting encrypted content is reported as a server-side configuration error; choose a compatible reasoning model with FANOUT_AI_MODEL.
# Default: (empty)
# FANOUT_AI_MODEL=

# FANOUT_AI_PROVIDER; string
# Enables chat by selecting exactly anthropic or openai. Empty disables chat.
# Default: (empty)
# FANOUT_AI_PROVIDER=

# FANOUT_ALERTS_ENABLED; boolean
# Runs the alert evaluation engine.
# Default: true
# FANOUT_ALERTS_ENABLED=

# FANOUT_ALERTS_EVALUATION_INTERVAL; duration
# Schedules alert evaluation; use a Go duration of at least 1s.
# Default: 30s
# FANOUT_ALERTS_EVALUATION_INTERVAL=

# FANOUT_ALERTS_HISTORY_DAYS; integer
# Sets alert history retention in days.
# Default: 7
# FANOUT_ALERTS_HISTORY_DAYS=

# FANOUT_AUTH_CODE_SECRET; string; secret
# Signs local sign-in codes; required in local mode with at least 32 characters.
# Default: (empty)
# FANOUT_AUTH_CODE_SECRET=
# Alternatively set FANOUT_AUTH_CODE_SECRET_FILE to a secret file path; do not set both.

# FANOUT_AUTH_MODE; string
# Selects local sign-in or oidc; each mode validates its prerequisites.
# Default: local
# FANOUT_AUTH_MODE=

# FANOUT_DATA_DIR; string
# Stores telemetry, query state, and the control database.
# Default: ./data
# FANOUT_DATA_DIR=

# FANOUT_DEFAULT_NAMESPACE; string
# Supplies the namespace for telemetry without one.
# Default: default
# FANOUT_DEFAULT_NAMESPACE=

# FANOUT_DUCKDB_MAX_CONNECTIONS; integer
# Caps the DuckDB connection pool. Reads scan immutable Parquet concurrently; rollup-cache writes are serialized by the query write gate. Zero means "size it from the machine" — the same spelling DuckDBThreads uses for deferring to a default. Resolution happens in resolveSizing and is reported in the startup configuration log.
# Default: 0
# FANOUT_DUCKDB_MAX_CONNECTIONS=

# FANOUT_DUCKDB_MEMORY; string
# Caps DuckDB's memory (e.g. "8GB"). Empty means Fanout sizes it from detected memory, reserving headroom for the Go runtime; see resolveSizing. DuckDB's own default is 80% of detected RAM, which is calculated as though DuckDB owned the machine and has been observed to get the process OOM-killed once the Go heap is added on top.
# Default: (empty)
# FANOUT_DUCKDB_MEMORY=

# FANOUT_DUCKDB_THREADS; integer
# Caps DuckDB's global query worker pool. Zero leaves DuckDB's own default in place (one worker per core). Set it to leave cores free for ingest on a query-heavy co-tenant host.
# Default: (empty)
# FANOUT_DUCKDB_THREADS=

# FANOUT_INGEST_BATCH_SIZE; integer
# Limits telemetry rows per batch; valid values are 1–50000.
# Default: 50000
# FANOUT_INGEST_BATCH_SIZE=

# FANOUT_INGEST_MAX_IN_FLIGHT_BYTES; integer
# Caps the decoded telemetry the process will hold across concurrent requests. Concurrency is HTTP/2 streams times connections and each handler holds its batch until the commit is durable, so without a ceiling a burst is bounded only by memory. Zero is unbounded, which is the behaviour that predates this setting.
# Default: 268435456
# FANOUT_INGEST_MAX_IN_FLIGHT_BYTES=

# FANOUT_MAINTENANCE_INTERVAL; duration
# Controls Parquet retention and compaction, and query-cache checkpointing.
# Default: 1h
# FANOUT_MAINTENANCE_INTERVAL=

# FANOUT_MCP_ENABLED; boolean
# Serves the MCP endpoint at /mcp.
# Default: true
# FANOUT_MCP_ENABLED=

# FANOUT_METRICS_PUBLIC; boolean
# Allows unauthenticated Prometheus scrapes.
# Default: false
# FANOUT_METRICS_PUBLIC=

# FANOUT_METRICS_TOKEN; string; secret
# Authorizes Prometheus scrapes with a bearer token; optional when using an admin session.
# Default: (empty)
# FANOUT_METRICS_TOKEN=
# Alternatively set FANOUT_METRICS_TOKEN_FILE to a secret file path; do not set both.

# FANOUT_OIDC_ADMIN_GROUPS; string
# Lists identity-provider groups mapped to admin, separated by commas.
# Default: (empty)
# FANOUT_OIDC_ADMIN_GROUPS=

# FANOUT_OIDC_ALLOWED_DOMAINS; string
# Lists allowed email domains, separated by commas; wildcard * requires verified email.
# Default: (empty)
# FANOUT_OIDC_ALLOWED_DOMAINS=

# FANOUT_OIDC_ALLOWED_GROUPS; string
# Lists allowed identity-provider groups, separated by commas.
# Default: (empty)
# FANOUT_OIDC_ALLOWED_GROUPS=

# FANOUT_OIDC_AUTO_PROVISION; boolean
# Creates accounts on OIDC sign-in; requires an allowed group or domain.
# Default: false
# FANOUT_OIDC_AUTO_PROVISION=

# FANOUT_OIDC_CLIENT_ID; string
# Identifies Fanout's client registration; required in OIDC mode.
# Default: (empty)
# FANOUT_OIDC_CLIENT_ID=

# FANOUT_OIDC_CLIENT_SECRET; string; secret
# Authenticates Fanout's client registration; required in OIDC mode.
# Default: (empty)
# FANOUT_OIDC_CLIENT_SECRET=
# Alternatively set FANOUT_OIDC_CLIENT_SECRET_FILE to a secret file path; do not set both.

# FANOUT_OIDC_DEFAULT_ROLE; string
# Assigns viewer, operator, or admin to users without a group role mapping.
# Default: viewer
# FANOUT_OIDC_DEFAULT_ROLE=

# FANOUT_OIDC_EMAIL_CLAIM; string
# Names the claim containing the user's email address.
# Default: email
# FANOUT_OIDC_EMAIL_CLAIM=

# FANOUT_OIDC_EMAIL_VERIFICATION; string
# Selects required verified-email claims or issuer trust; issuer trust requires an allowlist.
# Default: required
# FANOUT_OIDC_EMAIL_VERIFICATION=

# FANOUT_OIDC_ISSUER_URL; string
# Selects the identity provider issuer; required in OIDC mode.
# Default: (empty)
# FANOUT_OIDC_ISSUER_URL=

# FANOUT_OIDC_OPERATOR_GROUPS; string
# Lists identity-provider groups mapped to operator, separated by commas.
# Default: (empty)
# FANOUT_OIDC_OPERATOR_GROUPS=

# FANOUT_PPROF_ENABLED; boolean
# Exposes Go's net/http/pprof handlers at /debug/pprof/* for CPU/heap/mutex/goroutine profiling under load. Off by default; the routes require an admin browser session with operations:read.
# Default: false
# FANOUT_PPROF_ENABLED=

# FANOUT_PUBLIC_URL; string
# Sets the external browser origin. Required for OIDC; HTTPS is required with MCP.
# Default: (empty)
# FANOUT_PUBLIC_URL=

# FANOUT_RETENTION_DAYS; integer
# Sets telemetry retention in days; zero disables expiry.
# Default: 30
# FANOUT_RETENTION_DAYS=

# FANOUT_ROLLUP_INTERVAL; duration
# Schedules analytical rollups; use a Go duration of at least 1s.
# Default: 1m
# FANOUT_ROLLUP_INTERVAL=

# FANOUT_ROLLUP_SKIP_TO_LATEST; boolean
# RollupSkipToLatest, set once at boot, advances the service and edge rollup watermarks to the current max ingested timestamp. Existing history is not aggregated into those analytical rollups. Completed-batch trace caches still rebuild all retained history in bounded passes; uncached files remain immediately queryable. Useful for large pre-seeded datasets (benchmarks, restores); off in normal operation.
# Default: false
# FANOUT_ROLLUP_SKIP_TO_LATEST=

# FANOUT_SELF_SIGNUP; boolean
# Allows verified email addresses to create viewer accounts; requires local auth and SMTP.
# Default: false
# FANOUT_SELF_SIGNUP=

# FANOUT_SESSION_ABSOLUTE_TTL; duration
# Limits total session lifetime; it must be at least FANOUT_SESSION_IDLE_TTL.
# Default: 168h
# FANOUT_SESSION_ABSOLUTE_TTL=

# FANOUT_SESSION_IDLE_TTL; duration
# Expires inactive sessions; use a Go duration of at least 5m.
# Default: 12h
# FANOUT_SESSION_IDLE_TTL=

# FANOUT_SMTP_FROM; string
# Sets the email sender address; required with SMTP.
# Default: (empty)
# FANOUT_SMTP_FROM=

# FANOUT_SMTP_HOST; string
# Selects the mail server. Host, username, password, and from must all be set to enable SMTP.
# Default: (empty)
# FANOUT_SMTP_HOST=

# FANOUT_SMTP_PASSWORD; string; secret
# Authenticates to the mail server; required with SMTP.
# Default: (empty)
# FANOUT_SMTP_PASSWORD=
# Alternatively set FANOUT_SMTP_PASSWORD_FILE to a secret file path; do not set both.

# FANOUT_SMTP_PORT; integer
# Sets the mail server's positive TCP port.
# Default: 587
# FANOUT_SMTP_PORT=

# FANOUT_SMTP_USERNAME; string
# Authenticates to the mail server; required with SMTP.
# Default: (empty)
# FANOUT_SMTP_USERNAME=

# FANOUT_TLS_CERT_FILE; string
# Selects the TLS certificate file; requires FANOUT_TLS_KEY_FILE.
# Default: (empty)
# FANOUT_TLS_CERT_FILE=

# FANOUT_TLS_KEY_FILE; string
# Selects the TLS private key file; requires FANOUT_TLS_CERT_FILE.
# Default: (empty)
# FANOUT_TLS_KEY_FILE=

# FANOUT_TRUSTED_PROXY_CIDRS; string
# Lists trusted proxy networks, separated by commas; empty trusts no forwarded client addresses.
# Default: (empty)
# FANOUT_TRUSTED_PROXY_CIDRS=

# OPENAI_API_KEY; string; secret
# Required when FANOUT_AI_PROVIDER is openai.
# Default: (empty)
# OPENAI_API_KEY=
# Alternatively set OPENAI_API_KEY_FILE to a secret file path; do not set both.

# OPENAI_BASE_URL; string
# Defaults to https://api.openai.com/v1. Fanout appends /responses, preserving the API version and any gateway path prefix. Use an HTTP(S) base URL without embedded credentials, a query, or a fragment.
# Default: (empty)
# OPENAI_BASE_URL=
10 changes: 5 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,11 @@ coverage.out
coverage.html
cover.out

# Local environment scratch files. Fanout runtime configuration uses explicit
# YAML documents and process environment; it does not load dotenv files.
.env
.env.*
!.env.example
# Personal environment files and operator secret profiles remain private.
.env*
!/.env.example
/secrets/
/.sops.yaml

# Root-scoped for the same reason as testdata/: a bare `*.png` would silently
# swallow committed image assets anywhere in the tree.
Expand Down
34 changes: 34 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,39 @@
# Agent instructions

## Configuration conventions

- Configuration is environment-only. `internal/config/config.go` is the
authoritative schema: built-in defaults, then nonempty process environment.
Custom names use `FANOUT_` plus a clear name, adding the area only when it
clarifies; vendor credentials and endpoints keep `ANTHROPIC_API_KEY`,
`ANTHROPIC_BASE_URL`, `OPENAI_API_KEY`, and `OPENAI_BASE_URL`.
Provider and model selection remain `FANOUT_AI_PROVIDER` and `FANOUT_AI_MODEL`.
An empty provider disables chat; exactly `anthropic` or `openai` requires
that provider's key. Ignore AI credentials and endpoints when chat is disabled;
validate only the selected provider's endpoint.
- Every `secret:"true"` field accepts `<ENV>_FILE`. Both forms nonempty fail;
secret files lose exactly one trailing LF or CRLF and must remain nonempty.
Errors name variables and paths, never secret contents. No file-mode check.
- `group` tags serve docgen only, using ai, alerts, auth, ingest, mcp, metrics,
server, smtp, and storage. Run `just docs-generate` after schema changes and
verify `just docs-generate-check`. The generator owns `.env.example` and the
site reference; every template value is empty, with defaults and file forms
in comments. Secret defaults must never appear in public artifacts.
- The binary does not load dotenv. `just run` uses Node's native `parseEnv` for
optional ignored `./.env`, then inherited environment values win. Personal
development credentials may live there. Never shell-source dotenv. Only
Fanout's namespace rejects unknown names; tolerate platform service variables
and ignore unrelated names in shared vendor namespaces.
- Native listener default is `127.0.0.1:7520`. The image supplies
`FANOUT_ADDR=:7520` and `FANOUT_DATA_DIR=/var/lib/fanout/data`.
- Repository-managed secret profiles adopt SOPS authority through a verified
migration, with key recovery in 1Password. Existing vault/platform delivery
remains authoritative until all consumers and recovery are verified. Follow
`docs/operations.md` for delivery; never commit credential profiles in plaintext.
- This public product repository ships no operator secrets or recipients. Keep
operator `secrets/`, personal environment files, and private migration sources
ignored and outside Docker inputs; select private SOPS profiles explicitly.

## Public repository

- Keep credentials, private telemetry, local workstation paths, and private AI
Expand Down
27 changes: 21 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,15 @@ You need Go 1.27.2 and a C/C++ toolchain on native Linux or macOS (amd64 or arm6
[Bun](https://bun.sh), [just](https://just.systems),
[golangci-lint](https://golangci-lint.run/), and
[Lefthook](https://github.com/evilmartians/lefthook). Running in local auth mode
also requires a 32-character authentication code secret; SMTP and an AI key are
optional — see [README](README.md#requirements).
also requires a 32-character authentication code secret. SMTP and AI chat are
optional; selecting an AI provider requires its key — see [README](README.md#requirements).

Copy `.env.example` to ignored `.env` for personal development settings and
credentials. Every entry is commented out; uncomment only values you want to
supply. Copying the template sets nothing. After building, `just run` loads it with Node's native dotenv
parser; inherited environment values win. The binary validates the settings.
Keep this file out of Git and Docker inputs. See
[runtime secret delivery](docs/operations.md#runtime-secret-delivery).

The DuckDB wrapper downloads the pinned, checksum-verified engine from
[Fanout's dependency artifact mirror](https://github.com/labstack/fanout/releases/tag/duckdb-v2.0.0-alpha43763)
Expand Down Expand Up @@ -83,10 +90,18 @@ PNG in the same change. The recipe fails when the
font is missing rather than shipping a card in a substitute face, which typst
would otherwise do without complaint.

**Runtime configuration is strict.** [`fanout.example.yaml`](fanout.example.yaml)
is the complete schema. New settings need a YAML key, a `FANOUT_` environment
name, a test, and matching documentation; unknown names deliberately fail
startup.
**Runtime configuration is strict.** `internal/config/config.go` owns the
schema. New settings need an environment name, a documentation `group` tag,
a purpose comment, a test, and matching documentation. Custom settings use
`FANOUT_` plus a clear name, adding the area only when it clarifies; vendor
credentials and endpoints keep the vendor's documented names. Unknown owned
names fail startup, while unrelated shared-vendor/tool variables are ignored.
Secrets use `secret:"true"` and automatically accept an `_FILE` form; do not
set both forms nonempty. Run `just docs-generate` to regenerate the settings
reference and `.env.example`; `just docs-generate-check` verifies them. The
template leaves all values empty and documents defaults in comments.
Personal environment files and operator profiles stay ignored and outside
Docker. Never shell-source dotenv.

**SQLite migrations own the control schema.** Write forward SQL changes in
`internal/db/migrations` using Goose annotations. `just db-migrate-create NAME`
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ LABEL org.opencontainers.image.title="Fanout" \
org.opencontainers.image.version="${VERSION}"

COPY --from=build /app/fanout /usr/local/bin/fanout
COPY fanout.docker.yaml /etc/fanout/fanout.yaml
COPY LICENSE NOTICE THIRD_PARTY_NOTICES TRADEMARK.md /usr/share/licenses/fanout/
COPY --from=build --chown=nonroot:nonroot /runtime/var/lib/fanout /var/lib/fanout

Expand All @@ -55,6 +54,8 @@ COPY --from=build --chown=nonroot:nonroot /runtime/var/lib/fanout /var/lib/fanou
USER nonroot:nonroot
WORKDIR /var/lib/fanout

ENV FANOUT_ADDR=:7520 FANOUT_DATA_DIR=/var/lib/fanout/data

EXPOSE 7520

# Startup does DuckDB catalog attachment and maintenance, so the grace period
Expand All @@ -64,4 +65,3 @@ HEALTHCHECK --interval=30s --timeout=3s --start-period=40s --retries=3 \
CMD ["fanout", "healthcheck"]

ENTRYPOINT ["fanout"]
CMD ["--config", "/etc/fanout/fanout.yaml"]
Loading
Loading