diff --git a/.env.example b/.env.example index f3c44e18..35443f4b 100644 --- a/.env.example +++ b/.env.example @@ -82,16 +82,16 @@ # FANOUT_DUCKDB_THREADS; integer # Caps DuckDB's global query worker pool. Zero leaves DuckDB's own default in place (one worker per core). Set it to leave cores free for ingest on a query-heavy co-tenant host. -# Default: (empty) +# Default: 0 # FANOUT_DUCKDB_THREADS= # FANOUT_INGEST_BATCH_SIZE; integer -# Limits telemetry rows per batch; valid values are 1–50000. +# Sets the group-commit row target; valid values are 1–50000. Individual requests remain atomic and can exceed this target. # Default: 50000 # FANOUT_INGEST_BATCH_SIZE= # FANOUT_INGEST_MAX_IN_FLIGHT_BYTES; integer -# Caps the decoded telemetry the process will hold across concurrent requests. Concurrency is HTTP/2 streams times connections and each handler holds its batch until the commit is durable, so without a ceiling a burst is bounded only by memory. Zero is unbounded, which is the behaviour that predates this setting. +# Caps the decoded telemetry the process will hold across concurrent requests. Concurrency is HTTP/2 streams times connections and each handler holds its batch until the commit is durable, so without a ceiling a burst is bounded only by memory. Zero is unbounded. A single request is admitted when no other request holds budget, even if it exceeds the limit. # Default: 268435456 # FANOUT_INGEST_MAX_IN_FLIGHT_BYTES= diff --git a/AGENTS.md b/AGENTS.md index ac4ba3fe..71d54049 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,6 +13,7 @@ validate only the selected provider's endpoint. - Every `secret:"true"` field accepts `_FILE`. Both forms nonempty fail; secret files lose exactly one trailing LF or CRLF and must remain nonempty. + Unselected AI providers' value and file forms are ignored. Errors name variables and paths, never secret contents. No file-mode check. - `group` tags serve docgen only, using ai, alerts, auth, ingest, mcp, metrics, server, smtp, and storage. Run `just docs-generate` after schema changes and diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e8bbb304..f0d9e5f1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,21 +6,25 @@ Thanks for your interest. Issues and pull requests are both welcome. ```sh just install # browser dependencies and git hooks -just build # browser assets, then the binaries +just build # browser assets, then the binary just check # the full gate ``` You need Go 1.27.2 and a C/C++ toolchain on native Linux or macOS (amd64 or arm64), -[Bun](https://bun.sh), [just](https://just.systems), +[Bun](https://bun.sh), Node.js 24.19.0, [just](https://just.systems), [golangci-lint](https://golangci-lint.run/), and [Lefthook](https://github.com/evilmartians/lefthook). Running in local auth mode also requires a 32-character authentication code secret. SMTP and AI chat are -optional; selecting an AI provider requires its key — see [README](README.md#requirements). +optional; selecting an AI provider requires its key — see [README](README.md#build-from-source). Copy `.env.example` to ignored `.env` for personal development settings and -credentials. Every entry is commented out; uncomment only values you want to -supply. Copying the template sets nothing. After building, `just run` loads it with Node's native dotenv -parser; inherited environment values win. The binary validates the settings. +credentials. Every entry is commented out, with types, defaults, and secret file +forms in comments; uncomment only values you want to supply. Copying the template +sets nothing and preserves the image's environment defaults. After building, +`just run` loads optional `./.env` with Node's native dotenv +parser (`parseEnv`); inherited environment values win, including empty values. +The binary itself reads only process environment and validates the settings; +production launchers and deployments supply their own values. Keep this file out of Git and Docker inputs. See [runtime secret delivery](docs/operations.md#runtime-secret-delivery). @@ -57,9 +61,11 @@ and browser tests, then the Go suite under the race detector. Run `just e2e` with Google Chrome installed for the separate CI browser smoke suite. See [browser smoke testing](docs/testing.md) for coverage and failure artifacts. -`just install` also installs [Lefthook](https://lefthook.dev), which runs +`just install` also installs [Lefthook](https://lefthook.dev)'s Git hooks, which run formatting and linting on commit and the full gate on push. That is the fastest way to avoid a red CI run. +The pre-push hook may be skipped when Lefthook detects no changed files; +CI runs the same `just check` unconditionally and enforces the gate. The browser workspace uses `package.json` overrides for transitive security fixes that their direct dependencies have not locked yet. Keep those overrides @@ -72,8 +78,9 @@ only the existing build-only braces advisory exception documented in the justfil and its MCP app builds into `internal/mcp/apps`; both outputs are tracked in git because `go:embed` needs them present in a source checkout. If you change anything under `ui/`, run `just ui` and commit the regenerated output. +The binary is only as fresh as the last UI build. `just ui-check` fails when the embedded bytes do not match a fresh build, so -CI catches this, but it is friendlier to catch it yourself. +CI catches this; it is part of `just check`. **Diagrams are generated.** `docs/diagrams/*.svg` is rendered from the `.d2` source beside it. Edit the `.d2`, run `just diagrams`, and commit both. The @@ -98,8 +105,8 @@ credentials and endpoints keep the vendor's documented names. Unknown owned names fail startup, while unrelated shared-vendor/tool variables are ignored. Secrets use `secret:"true"` and automatically accept an `_FILE` form; do not set both forms nonempty. Run `just docs-generate` to regenerate the settings -reference and `.env.example`; `just docs-generate-check` verifies them. The -template leaves all values empty and documents defaults in comments. +reference and `.env.example`; `just docs-generate-check` verifies them. +The template leaves all values empty. Personal environment files and operator profiles stay ignored and outside Docker. Never shell-source dotenv. @@ -116,6 +123,15 @@ them manually with the same database guard used at startup. A version table without a positive applied version does not make an existing schema managed. DuckDB maintains its own query schema. +## Project layout + +```text +cmd/fanout/ process composition and the single entry point +internal/ ingest, storage, query, agent, MCP, auth, alerts +ui/host/ React browser host and shared MCP app renderer (build-time) +docs/diagrams/ d2 sources and rendered SVG +``` + ## Commits and pull requests Write commit messages that explain why the change is needed, not only what it diff --git a/README.md b/README.md index 9bfefa7b..57cb6885 100644 --- a/README.md +++ b/README.md @@ -6,126 +6,63 @@ [![License: Apache 2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE) [![Go Reference](https://pkg.go.dev/badge/github.com/labstack/fanout.svg)](https://pkg.go.dev/github.com/labstack/fanout) -Fanout ingests OpenTelemetry data, durably publishes it as atomic Parquet -batches, and puts an AI agent in front of it — in one Go process with no -external dependencies to operate. Persistent trace indexes serve targeted -reads; embedded DuckDB handles SQL, broad scans, and rebuildable rollups. Point -an SDK or Collector at it, open the browser, and ask questions about your -telemetry in plain language. + + + Fanout System overview dashboard showing request rates, errors, latency, and service charts. + + +Fanout keeps OpenTelemetry traces, logs, and metrics in one binary and one data directory, with no +external services to operate. Fanout is pre-release and changes without migration paths; the browser +client, chat investigator, MCP tools, dashboards, and alerts are preview capabilities +([status](https://fanout.run/status/capabilities)). Start with the +[documentation](https://fanout.run) or run it below. + +- OTLP ingest over gRPC and HTTP, with local Parquet storage and DuckDB analytics. +- An investigation agent that answers questions about your telemetry in plain language. +- An MCP server for external agents using the same typed query tools. +- Dashboards across traces, logs, and metrics, and alerts on service error rate, latency, + throughput, and log volume. -There is no separate ingester, query service, metadata database, object store, -or dashboard server to deploy. One binary, one data directory. - -## Architecture - -One process owns ingest, storage, query, alerting, the agent runtime, and an -MCP server. Everything below the dashed boundary is compiled into a single -executable, including the React client. - -![Fanout architecture](docs/diagrams/architecture.svg) - -Telemetry lands over OTLP/gRPC or OTLP/HTTP. Concurrent small requests may -share a group-commit batch, while up to four workers independently encode and -durably publish atomic Parquet directories with persistent trace indexes. -Targeted trace reads go through those indexes; DuckDB scans the same Parquet -for SQL and maintains rebuildable service, endpoint, and edge rollups. The -browser client, an in-process agent, and any -external MCP host all reach the same typed observability contract rather than -issuing raw SQL. - -Attribute and resource values support up to 128 nested arrays or objects. -An export that exceeds this limit is rejected in full with OTLP -`INVALID_ARGUMENT` (HTTP 400); accepted values retain their original types. - -Parquet is authoritative telemetry, DuckDB query state is rebuildable, and -SQLite is reserved for transactional product state. Native compaction -prepares replacements while reads continue and briefly gates readers only for -the crash-safe namespace swap: - -![Fanout persistence](docs/diagrams/persistence.svg) - -Application state (users, sessions, dashboards, alert rules, agent threads) -lives in the control SQLite database and never sits on the telemetry write -path. The published Parquet directories are self-describing, so startup can -discover the authoritative batch set directly from the filesystem. - -## Performance - -The independent [Fanout Bench](https://github.com/labstack/fanout-bench) -project measures authenticated ingest and optional dashboard read load against -your hardware. It uses the official OpenTelemetry generator and publishes raw, -reproducible evidence separately from the production binary. Ingest, indexed -reads, DuckDB analytics, and native Parquet maintenance have separate -coordination paths but still compete for the same CPU, memory bandwidth, -filesystem cache, and disk. - -The current publication candidate is **296,196 accepted OpenTelemetry items per -second** sustained for five minutes with traces, logs, and metrics arriving -together on a machine with eight logical CPUs and 15.6 GiB of memory. It is a -single run, and the benchmark harness that produced it carried uncommitted -local changes, so it is not Fanout's official headline yet. The [performance methodology](https://fanout.run/explanation/performance) -shows the signal breakdown, quality gates, limitations, and publication bar. - -## How it compares - -Fanout is a single node holding traces, logs, and metrics for a system you can -reason about from one place. That premise, rather than any single feature, is -what separates it from its neighbours. - -| If you use | Where Fanout differs | -| --- | --- | -| **Grafana with Loki, Tempo, and Mimir** | That stack keeps a service and a query language per signal, plus object storage underneath. Fanout keeps one process, one data directory, and one typed contract across all three signals, at the cost of the horizontal scale those components are built for. | -| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, authoritative Parquet storage, indexed trace reads, DuckDB analytics, alerting, and the browser client into one binary. | -| **Jaeger** | Jaeger covers traces and expects a storage backend you run separately. Fanout ingests traces, logs, and metrics into the same store, with nothing else to deploy. | -| **Prometheus with Grafana** | Prometheus pulls metrics and is excellent at them. Fanout accepts pushed OTLP for all three signals and is built around investigating a specific incident rather than maintaining long-range metric series. | -| **Datadog**, **Honeycomb**, **Grafana Cloud** | Those are managed services: someone else runs the storage, the scaling, and the upgrades, and your telemetry leaves your network to get there. Fanout is a binary you run, on data that stays on your disk. | -| **An OpenTelemetry Collector piped into ClickHouse** | The same shape, assembled by hand: collector, database, dashboards, and the glue between them. Fanout is that assembly as one program, with an agent and an MCP server already wired to the same query contract. | +## Quick start -Fanout is a single node. It has no clustering, no replication, and no object -tier; a deployment that outgrows one machine's disk and CPU has outgrown -Fanout. +### Docker -## Requirements +```sh +docker run --name fanout -p 7520:7520 \ + -v fanout-data:/var/lib/fanout/data \ + -e FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) \ + labstack/fanout:latest +``` -- **Go 1.27.2 and a C/C++ toolchain** on native Linux or macOS, amd64 or arm64. - Build with `just build` or `bash scripts/with-duckdb.sh go build ./cmd/fanout`; - the wrapper supplies DuckDB's pinned headers, static libraries, and CGO flags. - The first build needs network access to download the checksum-verified engine. -- **[Bun](https://bun.sh)** — compiles the browser assets -- **[just](https://just.systems)** — task runner -- A 32-character **authentication code secret** +Open the one-time setup URL printed by the container and create the first administrator. Fanout +displays the ingest token exactly once; save it with your collector secrets. For later sign-in +without SMTP, mint a 15-minute, single-use link against the running container's control database: -SMTP and an AI provider are optional. Without SMTP, an operator can mint a -short-lived login link from the local Fanout binary. With no AI provider selected, ingest, -dashboards, traces, logs, metrics, and MCP continue to work; only investigation -chat and AI-assisted controls are hidden. +```sh +docker exec fanout fanout login-link admin@example.com +``` -## Quick start +The UI, API, MCP, and both OTLP transports share port `7520`. See +[installation](https://fanout.run/start/install) for image defaults and bind-mount ownership, and +[first boot](https://fanout.run/start/first-boot) for setup and SMTP login. ### Native binary -Release archives support Linux and macOS on amd64 and arm64. The installer -verifies the selected archive against the release checksum before extracting: +Install a checksum-verified release for Linux or macOS, amd64 or arm64: ```sh curl -fsSL https://raw.githubusercontent.com/labstack/fanout/main/scripts/install.sh | sh +export FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) +fanout ``` -Set `FANOUT_VERSION=v{YYYY.M}.{N}` to pin a release and `FANOUT_PREFIX` to -choose the installation directory. +Set `FANOUT_VERSION=v{YYYY.M}.{N}` to pin a release and `FANOUT_PREFIX` to choose the installation +directory. Native installs bind `127.0.0.1:7520`; set `FANOUT_ADDR=:7520` for remote access, then +follow the same first-boot setup. -### Docker - -```sh -docker run --name fanout -p 7520:7520 \ - -v fanout-data:/var/lib/fanout/data \ - -e FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) \ - labstack/fanout:latest -``` +### Send telemetry -Open the one-time setup URL printed by the container and create the first -administrator. Fanout displays the ingest token exactly once; save it with -your collector secrets. A standard OTLP/HTTP exporter can then use: +A standard OTLP/HTTP exporter can use the saved ingest token: ```sh export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf @@ -149,98 +86,56 @@ service: logs: { receivers: [otlp], exporters: [otlp_http/fanout] } ``` -This assumes the Collector's existing `otlp` receiver and an environment -variable containing the one-time token. Replace `fanout` with the private -hostname reachable from that Collector. +Use an existing `otlp` receiver, set `INGEST_TOKEN`, and replace `fanout` with the Collector's +reachable private hostname. OTLP/gRPC uses `localhost:7520` locally; see [send +telemetry](https://fanout.run/start/send-telemetry) for TLS and HTTP/2 setup. -The equivalent OTLP/gRPC endpoint is `localhost:7520` (plaintext locally). -Behind a TLS proxy, both transports use the public origin; gRPC requires -HTTP/2 to Fanout. -For later sign-in -without SMTP, mint a 15-minute, single-use link against the running -container's control database: - -```sh -docker exec fanout fanout login-link admin@example.com -``` +### Enable chat Select `FANOUT_AI_PROVIDER=anthropic` and inject `ANTHROPIC_API_KEY`, or select -`FANOUT_AI_PROVIDER=openai` and inject `OPENAI_API_KEY`, to enable chat. The -selected provider requires its key at startup. Configure all four SMTP settings -(`FANOUT_SMTP_HOST`, `FANOUT_SMTP_USERNAME`, `FANOUT_SMTP_PASSWORD`, and -`FANOUT_SMTP_FROM`) to enable email-code login. +`FANOUT_AI_PROVIDER=openai` and inject `OPENAI_API_KEY`, then restart Fanout; the selected provider +requires its key at startup. With no provider, chat is disabled and the other features still work. +See [AI settings](https://fanout.run/reference/settings/ai). -The distroless image runs unprivileged as UID 65532. A bind-mounted host directory at -`/var/lib/fanout/data` must be writable by that user; a named volume, as above, -needs no such handling. +### Connect an MCP client -The image sets `FANOUT_ADDR=:7520` and -`FANOUT_DATA_DIR=/var/lib/fanout/data`. Override settings with deployment -environment values or mounted secret files. Publish port `7520`; the UI, API, -MCP, OTLP/HTTP, and OTLP/gRPC share this listener. +Set `FANOUT_PUBLIC_URL` to your stable, externally reachable HTTPS origin, then point a remote MCP +client at that origin's `/mcp` endpoint. The client discovers and registers through OAuth; you +authorize its scopes in the browser. See [connect over +MCP](https://fanout.run/guides/connect-over-mcp) for setup and permissions. -### From source +## How it compares -```sh -git clone https://github.com/labstack/fanout.git -cd fanout -just install # browser dependencies and git hooks -just build # browser assets, then the binaries -``` +| If you use | Where Fanout differs | +| --- | --- | +| **Grafana with Loki, Tempo, and Mimir** | That stack keeps a service and a query language per signal, plus object storage underneath. Fanout keeps one process, one data directory, and one typed contract across all three signals, at the cost of the horizontal scale those components are built for. | +| **SigNoz** | Both are OTLP-native and self-hosted. SigNoz composes a collector, ClickHouse, and query services; Fanout compiles ingest, authoritative Parquet storage, indexed trace reads, DuckDB analytics, alerting, and the browser client into one binary. | +| **Jaeger** | Jaeger covers traces and expects a storage backend you run separately. Fanout ingests traces, logs, and metrics into the same store, with nothing else to deploy. | +| **Prometheus with Grafana** | Prometheus pulls metrics and is excellent at them. Fanout accepts pushed OTLP for all three signals and is built around investigating a specific incident rather than maintaining long-range metric series. | +| **Datadog**, **Honeycomb**, **Grafana Cloud** | Those are managed services: someone else runs the storage, the scaling, and the upgrades, and your telemetry leaves your network to get there. Fanout is a binary you run, on data that stays on your disk. | +| **An OpenTelemetry Collector piped into ClickHouse** | The same shape, assembled by hand: collector, database, dashboards, and the glue between them. Fanout is that assembly as one program, with an agent and an MCP server already wired to the same query contract. | -Run it with the minimum configuration: +Fanout is a single node. It has no clustering, no replication, and no object tier; a deployment that +outgrows one machine's disk and CPU has outgrown Fanout. -```sh -export FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) # must be 32+ characters +## Architecture -./bin/fanout -``` +One process contains ingest, storage, query, alerts, the agent, MCP, and the React client. Browser +and agent clients share a typed observability contract. -Optionally select `FANOUT_AI_PROVIDER=anthropic` with `ANTHROPIC_API_KEY`, or -`FANOUT_AI_PROVIDER=openai` with `OPENAI_API_KEY`, for chat. Set the SMTP variables shown above for -email-code login. Without SMTP, run `./bin/fanout login-link -admin@example.com` from the same configuration and data directory. +![Fanout architecture](docs/diagrams/architecture.svg) -Fanout serves the UI, API, MCP, OTLP/gRPC and OTLP/HTTP on -. The first account -created becomes the administrator and receives the ingest token once. +Parquet holds telemetry, DuckDB query state is rebuildable, and SQLite holds transactional product +state. See [how storage works](https://fanout.run/explanation/storage-model) for durable ingest, +indexed reads, and compaction while reads continue. -Point any OpenTelemetry collector or SDK at either OTLP endpoint with the -ingest token. Use the separate -[Fanout Bench](https://github.com/labstack/fanout-bench) project for controlled -capacity tests. +![Fanout persistence](docs/diagrams/persistence.svg) ## Configuration -Configuration is resolved once at startup from built-in defaults followed by -nonempty process environment values, and validated before data files or -listeners open. Every change requires a restart. Native installs bind -`127.0.0.1:7520` by default; set `FANOUT_ADDR=:7520` for other hosts to reach it. - -Custom settings use `FANOUT_` plus a clear name, adding the area only when it -clarifies. Vendor credentials and endpoints keep their documented names: -`ANTHROPIC_API_KEY`, `ANTHROPIC_BASE_URL`, `OPENAI_API_KEY`, and -`OPENAI_BASE_URL`. Unknown `FANOUT_` variables fail startup, except for -platform-injected service-discovery names. Unrelated vendor/tool variables are -ignored. Empty values mean unset. Bools, integers, and Go durations are parsed -as their declared types. - -[`.env.example`](.env.example) is generated from -[`internal/config/config.go`](internal/config/config.go), with every value -empty and every entry commented out. Types, defaults, and secret file forms -are documented in comments. Copying it sets nothing and preserves image ENV. -Copy it to ignored `.env` and uncomment personal settings and development -credentials as needed. After building, `just run` uses Node's -`parseEnv` to load optional `./.env`, then lets inherited environment values -win, including empty values. The binary itself reads only process environment; -production launchers and deployments supply their own values. - -Every secret also accepts a file path through `_FILE`: -`FANOUT_AUTH_CODE_SECRET_FILE`, `FANOUT_OIDC_CLIENT_SECRET_FILE`, -`FANOUT_SMTP_PASSWORD_FILE`, `FANOUT_METRICS_TOKEN_FILE`, -`ANTHROPIC_API_KEY_FILE`, and `OPENAI_API_KEY_FILE`. Set either the value or -its file path. Setting both nonempty fails startup. Fanout strips exactly one -trailing LF or CRLF from a secret file and rejects an empty result. +Configuration is environment-only: built-in defaults followed by nonempty process environment +values, validated before data files or listeners open. Settings are read once at startup; every +change requires a restart. | Environment variable | Default | Purpose | | --- | --- | --- | @@ -254,78 +149,65 @@ trailing LF or CRLF from a secret file and rejects an empty result. | `FANOUT_RETENTION_DAYS` | `30` | Telemetry retention window | | `FANOUT_MCP_ENABLED` | `true` | Serve the MCP endpoint at `/mcp` | -Chat requires an explicit provider and its credential. Inherited keys never -enable it. Only the selected provider's endpoint is validated and used. -`FANOUT_AI_MODEL` selects the model. `ANTHROPIC_BASE_URL` defaults to -`https://api.anthropic.com`; Fanout appends `/v1/messages`. `OPENAI_BASE_URL` -defaults to `https://api.openai.com/v1`; Fanout appends `/responses`. -Both preserve gateway prefixes and accept a trailing slash. The selected URL -must use HTTP(S) without credentials, a query, or a fragment. - -See [runtime secret delivery](docs/operations.md#runtime-secret-delivery) for -deployment examples. This public repository excludes operator secret profiles -and private recipient policy. Run `just docs-generate` after changing the -schema; `just docs-generate-check` verifies `.env.example` and the reference. +Every secret accepts `_FILE` instead of its value; see the [settings +reference](https://fanout.run/reference/configuration) for validation and [runtime secret +delivery](docs/operations.md#runtime-secret-delivery) for deployment examples. ### Advanced DuckDB sizing -Most deployments should not set DuckDB variables. Give the process or -container the CPU and memory limits it may use; at startup Fanout reserves -headroom for Go and sizes the DuckDB connection pool from available CPUs. The -resolved values and whether Fanout chose them are available in the -`runtime_sizing` block returned by `/readyz`. +Fanout sizes DuckDB automatically; measured workloads can override memory, connections, and threads +as described in [DuckDB sizing](https://fanout.run/reference/configuration#duckdb-sizing). -These variables are escape hatches for measured, unusual workloads: +For TLS, health checks, backups, upgrades, retention, and recovery, see the [operator +runbook](docs/operations.md). -| Variable | Automatic behavior | When to override | -| --- | --- | --- | -| `FANOUT_DUCKDB_MEMORY` | 60% of the container or host memory available to Fanout | A measured co-tenant workload needs a different Go/DuckDB split | -| `FANOUT_DUCKDB_MAX_CONNECTIONS` | Available Go CPUs, bounded to 2–16 connections | Query concurrency has been benchmarked for this machine | -| `FANOUT_DUCKDB_THREADS` | DuckDB chooses its own query worker count | Query-heavy work must leave specific cores free for ingest | +## Performance -An explicit value always wins. Startup logs and `/readyz` report the detected -host and cgroup limits, the selected source, and whether detection was -incomplete. If Fanout cannot conclusively inspect a container limit, it warns; -set `FANOUT_DUCKDB_MEMORY` for a guaranteed bound. +[Fanout Bench](https://github.com/labstack/fanout-bench) measures authenticated ingest and optional +dashboard read load on your hardware; the [performance +methodology](https://fanout.run/explanation/performance) covers reproducible evidence and +publication requirements. -For TLS and reverse proxies, health checks, backups and restores, upgrades, -retention, and recovery, see the [operator runbook](docs/operations.md). +## Build from source -## Development +- **Go 1.27.2 and a C/C++ toolchain** on native Linux or macOS, amd64 or arm64. + Build with `just build` or `bash scripts/with-duckdb.sh go build ./cmd/fanout`; + the wrapper supplies DuckDB's pinned headers, static libraries, and CGO flags. + The first build needs network access to download the checksum-verified engine. +- **[Bun](https://bun.sh)** — compiles the browser assets +- **Node.js 24.19.0** — runs development helpers and builds the documentation site +- **[just](https://just.systems)** — task runner +- A 32-character **authentication code secret** ```sh -just # list every recipe -just check # the full gate: format, lint, asset freshness, and tests -just test # Go tests -just ui # rebuild the embedded browser assets +git clone https://github.com/labstack/fanout.git +cd fanout +just install # browser dependencies and git hooks +just build # browser assets, then the binary ``` -The `ui/host` workspace builds the SPA and MCP app **into** `internal/ui/dist` and -`internal/mcp/apps`, and those outputs are committed because `go:embed` needs -them present in a source checkout. The binary is therefore only ever as fresh -as the last UI build. `just ui-check` compares temporary builds of both outputs -and fails if the embedded bytes no longer match. It is part of `just check` and runs in CI. - -[Lefthook](https://lefthook.dev) runs formatting and linting on commit and the -full gate on push; `just install` wires it up. The pre-push hook is a -convenience and lefthook may skip it when it detects no changed files — CI runs -the same `just check` unconditionally, and that is what actually enforces it. +Run it with the minimum configuration: -## Project layout +```sh +export FANOUT_AUTH_CODE_SECRET=$(openssl rand -hex 32) # must be 32+ characters -```text -cmd/fanout/ process composition and the single entry point -internal/ ingest, storage, query, agent, MCP, auth, alerts -ui/host/ React browser host and shared MCP app renderer (build-time) -docs/diagrams/ d2 sources and rendered SVG +./bin/fanout ``` +Open and follow first-boot setup. SMTP and chat are optional; without SMTP, +run `./bin/fanout login-link admin@example.com` with the same configuration and data directory. + +## Development + +Run `just` to list recipes, `just check` for the full gate, `just test` for Go tests, and `just ui` +to rebuild embedded browser assets. See [CONTRIBUTING.md](CONTRIBUTING.md) for environment setup, +generated assets, hooks, and project layout. + ## Releases -Versions are CalVer — `v{YYYY.M}.{N}`, numbered from 0 within each month, so -`v2026.8.1` is the second release of August 2026. Pushing a tag publishes the -same release manifest to Docker Hub and GHCR and moves both `latest` tags. -GHCR remains the canonical registry and also carries development images: +Versions are CalVer — `v{YYYY.M}.{N}`, numbered from 0 within each month. Tagged releases publish +identical manifests to Docker Hub (`labstack/fanout`) and GHCR (`ghcr.io/labstack/fanout`); GHCR is +canonical and also carries development images. | Image tag | Points at | | --- | --- | @@ -336,33 +218,20 @@ GHCR remains the canonical registry and also carries development images: | `ghcr.io/labstack/fanout:main` | the tip of `main` | | `ghcr.io/labstack/fanout:sha-` | one specific commit | -Release images are multi-architecture for `linux/amd64` and `linux/arm64`. -Release archives provide Linux and macOS binaries for amd64 and arm64; every -artifact is built on a native runner because DuckDB requires cgo. -The complete release and verification contract is in -[docs/release.md](docs/release.md). +See [docs/release.md](docs/release.md) for platforms, checksums, provenance, and verification. ## Contributing -Issues and pull requests are welcome — see [CONTRIBUTING.md](CONTRIBUTING.md). -Run `just check` and `just test-race` before opening a pull request; together -they match CI. For anything security-related, follow [SECURITY.md](SECURITY.md) -instead of opening an issue. +Issues and pull requests are welcome; follow [CONTRIBUTING.md](CONTRIBUTING.md) and run `just check` +and `just test-race` before opening a pull request. Report security issues through +[SECURITY.md](SECURITY.md). ## Scope -This repository is Fanout itself, and it builds to a working binary with no -other repository involved. Not included: LabStack's own deployment -configuration, uptime monitoring, and public demo instance. Those describe how -we operate Fanout, not what it does. - -The documentation site in [`site/`](site) is included, and is the exception that -proves the rule: it is not marketing copy but the product's own reference, and -part of it is generated. `cmd/fanout-docgen` writes every settings page from the -same `internal/config` type the loader binds, and `just check` fails when a -committed page no longer matches it. Documentation that can drift from the -binary is documentation that eventually lies about it, and the only place that -check can run is next to the code it checks. +This repository builds Fanout without another repository; LabStack's deployment configuration, +uptime monitoring, and public demo instance are not included. The documentation site in +[`site/`](site) is included, with settings pages generated from the same `internal/config` type the +loader binds and checked by `just check`. ## License diff --git a/SECURITY.md b/SECURITY.md index 7eb0bcfa..f952a29d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -32,7 +32,7 @@ interchangeable. A finding that crosses those boundaries is in scope. ## Supported versions -Fanout is pre-1.0 and moves quickly. Fixes land on `main`, and only the latest +Fanout uses CalVer and moves quickly. Fixes land on `main`, and only the latest release is supported. There are no backports to earlier tags. ## Operating Fanout safely diff --git a/cmd/fanout-docgen/main.go b/cmd/fanout-docgen/main.go index 5ece1e13..df805d14 100644 --- a/cmd/fanout-docgen/main.go +++ b/cmd/fanout-docgen/main.go @@ -247,9 +247,9 @@ func renderAlertEnv(vars []field) []byte { b.WriteString("against one service's rollup for the current window. These are the values it\n") b.WriteString("can read — the names below match the variables declared and bound by the\n") b.WriteString("evaluator.\n\n") - b.WriteString("Rules are type-checked against these declared variables when saved through the\n") - b.WriteString("API. Unknown variables, type mistakes, and non-boolean results are rejected\n") - b.WriteString("before the rule is saved.\n\n") + b.WriteString("Nonempty expressions supplied when creating or replacing rules through the\n") + b.WriteString("API are type-checked against these declared variables. Unknown variables,\n") + b.WriteString("type mistakes, and non-boolean results are rejected before the rule is saved.\n\n") b.WriteString("Use CEL's standard operators and functions, such as `&&`, `||`, `!`, and\n") b.WriteString("`service.startsWith(\"checkout\")`. Numeric variables are doubles; cross-type\n") b.WriteString("numeric comparisons let you compare them with integer literals, as in\n") diff --git a/cmd/fanout-docgen/routes.go b/cmd/fanout-docgen/routes.go index c71121aa..8181912a 100644 --- a/cmd/fanout-docgen/routes.go +++ b/cmd/fanout-docgen/routes.go @@ -244,7 +244,8 @@ func renderRoutes(routes []api.RouteDoc) []byte { b.WriteString("{/* Generated by cmd/fanout-docgen from internal/api, internal/agent and cmd/fanout. Edit the generator, not this page. */}\n\n") - b.WriteString("Every route below is served on `FANOUT_ADDR` (`:7520` by default).\n") + b.WriteString("Every route below is served on `FANOUT_ADDR` (`127.0.0.1:7520` by default;\n") + b.WriteString("the image sets `:7520`).\n") b.WriteString("OTLP shares this listener but uses its own handlers and ingest credential; see\n") b.WriteString("[send your first telemetry](/start/send-telemetry).\n\n") @@ -260,8 +261,10 @@ func renderRoutes(routes []api.RouteDoc) []byte { b.WriteString(" **not** the default. Enabling it also turns on mutex and block sampling.\n") b.WriteString("- `/mcp`, `/api/mcp`, `/oauth/*` and `/.well-known/*` — only when\n") b.WriteString(" `FANOUT_MCP_ENABLED` is true, which **is** the default.\n") - b.WriteString("- `/api/agent/*` — only when an AI provider key is\n") - b.WriteString(" configured. Without one the investigator is not registered at all.\n\n") + b.WriteString("- `/api/agent/*` — only when `FANOUT_AI_PROVIDER` selects `anthropic` or\n") + b.WriteString(" `openai`, with that provider's required key. An empty provider disables chat.\n") + b.WriteString("- `/api/auth/oidc/start` and `/api/auth/oidc/callback` — only in\n") + b.WriteString(" `FANOUT_AUTH_MODE=oidc`.\n\n") b.WriteString("Everything else is always registered. A path shown with `:name` or `*` is the\n") b.WriteString("pattern Echo matches on, not a literal URL.\n") b.WriteString(":::\n\n") @@ -274,12 +277,12 @@ func renderRoutes(routes []api.RouteDoc) []byte { b.WriteString("The supported programmatic interface is **MCP**, at `/mcp` — see\n") b.WriteString("[MCP tools](/reference/mcp-tools). The stable non-MCP surfaces are OTLP\n") b.WriteString("ingest, and the operational endpoints in [endpoints](/reference/endpoints).\n\n") - b.WriteString("Two exceptions worth knowing, because HTTP is currently the only way to reach\n") - b.WriteString("them: alert rules (`/api/alerting/rules`, `/api/alerts`) and ingest-token rotation\n") - b.WriteString("(`/api/settings/ingest`). Neither has a browser page yet.\n") + b.WriteString("Alert rules (`/api/alerting/rules`, `/api/alerts`) currently have no browser\n") + b.WriteString("page. Ingest-token rotation uses `POST /api/settings/ingest/token/rotate`\n") + b.WriteString("and is also available to administrators on **Connect telemetry**.\n") b.WriteString(":::\n\n") - b.WriteString("## Errors\n\nApplication API errors, including unknown routes and wrong methods, return exactly\n`{\"code\":\"bad_request\",\"message\":\"Safe explanation\"}`. Codes are stable\nsnake_case identifiers; clients branch on `code` rather than message text.\nInvalid dashboard or panel specs also include `problems` with field paths,\nmessages and optional hints. Every error response uses `Cache-Control: no-store`;\nHEAD errors have no body. Server failures return a short, safe message; the\nunderlying error is logged and never returned.\n\nDefault codes by HTTP status:\n\n| Status | Code |\n|---|---|\n| 400 | `bad_request` |\n| 401 | `unauthorized` |\n| 403 | `forbidden` |\n| 404 | `not_found` |\n| 405 | `method_not_allowed` |\n| 409 | `conflict` |\n| 413 | `payload_too_large` |\n| 415 | `unsupported_media_type` |\n| 422 | `unprocessable` |\n| 429 | `rate_limited` |\n| 500 | `internal` |\n| 502 | `bad_gateway` |\n| 503 | `unavailable` |\n| 504 | `timeout` |\n\nOther 4xx statuses use `bad_request`; other 5xx statuses use `internal`.\nSpecific conditions use `already_current`, `dashboard_version_not_found`,\n`dashboard_version_conflict`, `invalid_spec` and `last_active_administrator`.\nTool failures carry `error: {code, message}` inside the tool result.\nPanel execution results keep their own status and error fields.\nOAuth, OTLP/gRPC, MCP JSON-RPC and AG-UI run errors keep their protocol spelling.\nOperational responses at `/healthz`, `/readyz` and `/metrics` keep their bodies.\n\n") + b.WriteString("## Errors\n\nApplication API errors, including unknown routes and wrong methods, return objects shaped like\n`{\"code\":\"bad_request\",\"message\":\"Safe explanation\"}`. Codes are stable\nsnake_case identifiers; clients branch on `code` rather than message text.\nInvalid dashboard or panel specs also include `problems` with field paths,\nmessages and optional hints. Every error response uses `Cache-Control: no-store`;\nHEAD errors have no body. Server failures return a short, safe message; the\nunderlying error is logged and never returned.\n\nDefault codes by HTTP status:\n\n| Status | Code |\n|---|---|\n| 400 | `bad_request` |\n| 401 | `unauthorized` |\n| 403 | `forbidden` |\n| 404 | `not_found` |\n| 405 | `method_not_allowed` |\n| 409 | `conflict` |\n| 413 | `payload_too_large` |\n| 415 | `unsupported_media_type` |\n| 422 | `unprocessable` |\n| 429 | `rate_limited` |\n| 500 | `internal` |\n| 502 | `bad_gateway` |\n| 503 | `unavailable` |\n| 504 | `timeout` |\n\nOther 4xx statuses use `bad_request`; other 5xx statuses use `internal`.\nSpecific conditions use `already_current`, `dashboard_version_not_found`,\n`dashboard_version_conflict`, `invalid_spec` and `last_active_administrator`.\nTool failures carry `error: {code, message}` inside the tool result.\nPanel execution results keep their own status and error fields.\nOAuth, OTLP/gRPC, MCP JSON-RPC and AG-UI run errors keep their protocol spelling.\nOperational responses at `/healthz`, `/readyz` and `/metrics` keep their bodies.\n\n") b.WriteString("The **Requires** column is not a description of the rule. It is the answer the\n") b.WriteString("authorization middleware gives for that exact method and path, so a route\n") @@ -302,10 +305,10 @@ func renderRoutes(routes []api.RouteDoc) []byte { b.WriteString("- **a named capability** — the caller's role must carry it.\n") b.WriteString("- **or a service credential** — additionally reachable without a browser\n") b.WriteString(" session, which is how a scraper reaches the metrics endpoint.\n\n") - b.WriteString("A method of `Any` means the route answers every verb, and the middleware\n") + b.WriteString("A method of `Any` means the route is registered for every verb, and the middleware\n") b.WriteString("gives the same answer for all of them — where it would not, this page fails\n") - b.WriteString("to build rather than pick one. `GET /*` is the single-page application:\n") - b.WriteString("anything not matched above is served the client, which is why it is public.\n") + b.WriteString("to build rather than pick one. `GET /*` serves the single-page application\n") + b.WriteString("for browser paths. Unknown `/api/*` and OTLP paths return errors instead.\n") return []byte(b.String()) } diff --git a/docs/authentication.md b/docs/authentication.md index 9308ab86..5d977091 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -12,7 +12,7 @@ stderr containing a **setup URL**: ============================================================ FANOUT SETUP - Open: http://127.0.0.1:8080/login?setup_token= + Open: http://127.0.0.1:7520/login?setup_token= Valid: one-time use, expires in 1 hour Note: this URL disappears after the first admin is created Warn: it contains an administrator credential — this output may @@ -27,8 +27,9 @@ Properties worth knowing: - It is consumed the moment the first administrator is committed, before any other setup work. A later failure in that request does not leave a live credential behind. -- It creates exactly **one** administrator. Presenting it again after setup is - complete returns `403` and establishes no session. +- It creates exactly **one** administrator. A consumed credential returns `410` + and establishes no session; a concurrent request reaching account creation + after the administrator already exists returns `403`. - Ingest stays closed until setup succeeds. - The banner is printed once. Restarting only reopens setup while the user count is still zero. @@ -47,7 +48,8 @@ without them, an operator with access to the control database can mint a login link. OIDC mode instead requires an HTTPS issuer, a client, and an HTTPS `FANOUT_PUBLIC_URL`. -If you configure SMTP, send yourself a code and confirm it arrives. Credentials +After setup in local mode, send yourself a code if SMTP is configured and +confirm it arrives. Credentials can be wrong, the relay can refuse the sender, and mail can land in spam. A login link remains available when delivery fails. @@ -167,11 +169,11 @@ audit attribution isolated without shared credentials or an anonymous authorization bypass: ```sh -FANOUT_AUTH_MODE=oidc -FANOUT_OIDC_EMAIL_VERIFICATION=required -FANOUT_OIDC_AUTO_PROVISION=true -FANOUT_OIDC_ALLOWED_DOMAINS="*" -FANOUT_OIDC_DEFAULT_ROLE=viewer +export FANOUT_AUTH_MODE=oidc +export FANOUT_OIDC_EMAIL_VERIFICATION=required +export FANOUT_OIDC_AUTO_PROVISION=true +export FANOUT_OIDC_ALLOWED_DOMAINS="*" +export FANOUT_OIDC_DEFAULT_ROLE=viewer ``` The wildcard is deliberately explicit and is rejected with diff --git a/docs/images/system-overview-dark.png b/docs/images/system-overview-dark.png new file mode 100644 index 00000000..13efb960 Binary files /dev/null and b/docs/images/system-overview-dark.png differ diff --git a/docs/images/system-overview-light.png b/docs/images/system-overview-light.png new file mode 100644 index 00000000..88273477 Binary files /dev/null and b/docs/images/system-overview-light.png differ diff --git a/docs/operations.md b/docs/operations.md index 55f78a0d..29419cb6 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -39,6 +39,7 @@ Both forms nonempty fail startup. Fanout strips exactly one trailing LF or CRLF from file contents, then rejects an empty result. Errors identify the variable and path, never the contents. Fanout performs no file-mode check; the deployment controls mount access. Do not bake credentials into images or frontend bundles. +Unselected AI providers' value and file forms are ignored. For Docker, mount a provisioned secret and inject its path: diff --git a/docs/testing.md b/docs/testing.md index f509fa54..751a34f6 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -29,7 +29,7 @@ pixels to match the host, and messages must stop for five seconds after the firs settled frame. CI pins Node 24.19.0 and limits the job to 15 minutes. It runs the same recipe on pull requests and pushes. Failures write `ui/host/playwright-report/index.html` and retain traces/screenshots in `ui/host/test-results/`; both directories are -ignored by git and uploaded only on CI failure. +ignored by git. CI uploads available browser evidence after every run. For browser-free verification, run `bunx playwright test --list` in `ui/host`. `bun run test` runs unit tests and excludes Playwright specs. @@ -37,8 +37,11 @@ For browser-free verification, run `bunx playwright test --list` in `ui/host`. ## Dashboard performance measurements Correct rendering and fast rendering need separate evidence. The checked-in -smoke suite verifies populated panels, geometry and bounded fragments; it does -not currently enforce a dashboard latency ceiling. A passing native-engine +smoke suite verifies populated panels, geometry and bounded fragments. A +separate performance spec records twenty navigation-to-paint samples +per theme for a twelve-panel, 24-hour fixture and enforces a calibrated +1800 ms p95 ceiling in CI. The local representative-data target remains +1500 ms. A passing native-engine correctness test is not a timing measurement either. For a performance run, use a disposable instance with a recorded telemetry @@ -93,8 +96,8 @@ The existing theme, chart-unit and row-formatting tests check semantic text and mark contrast in both schemes. Keyboard tests cover dashboard focus, chart navigation, range cancellation, full-screen and shortcut preferences. The exact palette-order test preserves six categorical colour slots; its name alone does -not prove colour-vision-deficiency separation. There is currently no automated -axe scan or CVD validator in the checked-in smoke suite. +not prove colour-vision-deficiency separation. The checked-in suite runs axe-core +WCAG AA scans; it has no automated CVD validator. For an accessibility review, inspect dashboard and chat panels in both themes at both smoke widths. Tab through controls and charts; exercise arrows, Enter, diff --git a/internal/alert/types.go b/internal/alert/types.go index 3ce91cef..f791c904 100644 --- a/internal/alert/types.go +++ b/internal/alert/types.go @@ -45,16 +45,17 @@ type Alert struct { // field here adds a row to the reference; describe it in a sentence a rule // author would find useful, not in terms of how it is computed. type AlertEnv struct { - // Errors as a proportion of requests over the evaluation window, from 0 to 1. + // Mean of minute-rollup span error proportions over the evaluation window, + // from 0 to 1; not a request-weighted whole-window error rate. ErrorRate float64 `expr:"error_rate"` - // Median request latency in milliseconds. + // Mean of approximate minute-rollup median span latencies in milliseconds. P50 float64 `expr:"p50"` - // 95th-percentile request latency in milliseconds. The usual latency signal: - // it moves when a meaningful share of requests slow down, where the median - // does not. + // Mean of approximate minute-rollup 95th-percentile span latencies in + // milliseconds; not the whole-window p95. Latency prefers served work, + // falling back to outbound spans when no served work is available. P95 float64 `expr:"p95"` - // Requests per second. Worth pairing with any rate threshold — a service - // handling two requests a minute reaches a 50% error rate on one failure. + // Total span count over the current rollup window, not requests per second. + // Pair it with an error threshold to avoid firing on very little traffic. Throughput float64 `expr:"throughput"` // Log records recorded for the service over the window. LogCount float64 `expr:"log_count"` @@ -62,16 +63,18 @@ type AlertEnv struct { // in standard deviations. Use it to catch a service that has changed without // committing to an absolute threshold that fits every service. ZScore float64 `expr:"z_score"` - // A composite 0-100 score combining errors, latency and traffic. Convenient - // for a single catch-all rule; too coarse to explain why it moved. + // The detector's instance-wide composite 0-100 health score. Every service + // receives the same score; it is not a per-service value. HealthScore float64 `expr:"health_score"` - // Change in error rate against the baseline, as a proportion. + // Relative change in error rate against the preceding rollup window, as a + // percentage; zero when the baseline is missing or zero. ErrorRateDelta float64 `expr:"error_rate_delta"` - // Change in p95 latency against the baseline, in milliseconds. + // Relative change in p95 latency against the preceding rollup window, as a + // percentage; zero when the baseline is missing or zero. P95Delta float64 `expr:"p95_delta"` - // Change in throughput against the baseline, in requests per second. Negative - // when traffic drops, which is how a rule catches a service that has stopped - // receiving requests rather than started failing them. + // Relative change in span count against the preceding rollup window, as a + // percentage. Negative when traffic drops; zero when the baseline is missing + // or zero. ThroughputDelta float64 `expr:"throughput_delta"` // The service name being evaluated, as a string. Compare it when one rule // needs to behave differently for one service. diff --git a/internal/config/config.go b/internal/config/config.go index 818cd965..f423e59f 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -53,13 +53,15 @@ type Config struct { Addr string `group:"server" env:"FANOUT_ADDR" default:"127.0.0.1:7520"` // DataDir stores telemetry, query state, and the control database. DataDir string `group:"storage" env:"FANOUT_DATA_DIR" default:"./data"` - // IngestBatchSize limits telemetry rows per batch; valid values are 1–50000. + // IngestBatchSize sets the group-commit row target; valid values are 1–50000. + // Individual requests remain atomic and can exceed this target. IngestBatchSize int `group:"ingest" env:"FANOUT_INGEST_BATCH_SIZE" default:"50000"` // IngestMaxInFlightBytes caps the decoded telemetry the process will hold // across concurrent requests. Concurrency is HTTP/2 streams times // connections and each handler holds its batch until the commit is // durable, so without a ceiling a burst is bounded only by memory. Zero is - // unbounded, which is the behaviour that predates this setting. + // unbounded. A single request is admitted when no other request holds budget, + // even if it exceeds the limit. IngestMaxInFlightBytes int `group:"ingest" env:"FANOUT_INGEST_MAX_IN_FLIGHT_BYTES" default:"268435456"` // RollupInterval schedules analytical rollups; use a Go duration of at least 1s. RollupInterval time.Duration `group:"storage" env:"FANOUT_ROLLUP_INTERVAL" default:"1m"` @@ -106,7 +108,7 @@ type Config struct { // DuckDBThreads caps DuckDB's global query worker pool. Zero leaves // DuckDB's own default in place (one worker per core). Set it to leave // cores free for ingest on a query-heavy co-tenant host. - DuckDBThreads int `group:"storage" env:"FANOUT_DUCKDB_THREADS"` + DuckDBThreads int `group:"storage" env:"FANOUT_DUCKDB_THREADS" default:"0"` // DuckDBMaxConns caps the DuckDB connection pool. Reads scan immutable Parquet // concurrently; rollup-cache writes are serialized by the query write gate. // Zero means "size it from the machine" — the same spelling DuckDBThreads diff --git a/site/src/content/docs/explanation/authentication-modes.mdx b/site/src/content/docs/explanation/authentication-modes.mdx index da11af7c..6e6dd94b 100644 --- a/site/src/content/docs/explanation/authentication-modes.mdx +++ b/site/src/content/docs/explanation/authentication-modes.mdx @@ -62,8 +62,8 @@ Two conditions guard it, and both matter: Self-provisioned users are `viewer`. Be clear about what that grants before turning this on: a viewer can read **all** telemetry the instance holds, keep their own dashboards, and **run the chat investigator** — which calls your model -provider and spends money. What it cannot do is change anything: no alert rules, -no ingest token, no user administration. +provider and spends money. It can change its own dashboards, but cannot manage +alert rules, rotate the ingest token, or administer users. If that is more than you want to hand to anyone who can receive an email, leave self-signup off and create accounts deliberately. The full matrix is in @@ -87,10 +87,14 @@ locally instead. ### Verified identities can be auto-provisioned as viewers -`FANOUT_OIDC_AUTO_PROVISION` admits a verified identity as a `viewer` on first -login, with no shared credential and no account created ahead of time. Chat, MCP -tools and owned dashboards stay available to them; everything an operator does -stays behind the capabilities that require it. +`FANOUT_OIDC_AUTO_PROVISION` creates an eligible identity's account on first +login, after administrator setup. Its role comes from the configured admin or +operator group mapping, falling back to `FANOUT_OIDC_DEFAULT_ROLE` (`viewer`). +With the default `FANOUT_OIDC_EMAIL_VERIFICATION=required`, the identity must +have a verified email. The `issuer` policy trusts the configured issuer instead +and requires an allowlist. Auto-provisioning always requires an allowed group +or domain. Chat, MCP tools and owned dashboards are available to viewers; +operator actions still require the corresponding capabilities. This is the OIDC counterpart to local self-signup above, and it obeys the same allow policy: if `FANOUT_OIDC_ALLOWED_GROUPS` or `FANOUT_OIDC_ALLOWED_DOMAINS` diff --git a/site/src/content/docs/explanation/performance.mdx b/site/src/content/docs/explanation/performance.mdx index 4adbeca8..dbe2b62c 100644 --- a/site/src/content/docs/explanation/performance.mdx +++ b/site/src/content/docs/explanation/performance.mdx @@ -99,7 +99,7 @@ filesystem cache, and disk. Maintenance adds native Parquet merge work and short reader-exclusive namespace changes. An ingest-only number therefore still overstates what an instance does while anyone is looking at it. -Create a short-lived ingest token through first-admin setup, configure Fanout +Obtain the ingest token through first-admin setup, configure Fanout with a dedicated `FANOUT_METRICS_TOKEN`, then expose that same metrics token to the harness as `FANOUT_BENCH_METRICS_TOKEN`. Metrics are private by default. A local run starts with: diff --git a/site/src/content/docs/explanation/storage-model.mdx b/site/src/content/docs/explanation/storage-model.mdx index 526d7d7d..bf6a8a83 100644 --- a/site/src/content/docs/explanation/storage-model.mdx +++ b/site/src/content/docs/explanation/storage-model.mdx @@ -56,11 +56,17 @@ Trace lookup binary-searches a fixed-width sidecar on disk, seeks directly to th matching Parquet row ranges, and verifies the complete trace ID after hashing. The sidecar is an index, not another telemetry representation. -Dashboard rollups are rebuildable DuckDB caches. SQLite remains independent and -stores only transactional product state. +Dashboard rollups are rebuildable DuckDB caches. SQLite stores only +transactional product state and never holds telemetry rows. Ingest +authentication still reads the ingest token hash from it on every export, so an +unreadable control database rejects OTLP requests. ## Attributes retain their types +Attribute and resource values support up to 128 nested arrays or objects. +An export that exceeds this limit is rejected in full with OTLP +`INVALID_ARGUMENT` (HTTP 400); accepted values retain their original types. + Span, log, and metric attributes and resource attributes use Parquet VARIANT columns. Frequently queried resource and messaging fields also have typed Parquet columns inside the VARIANT encoding. The span read view exposes diff --git a/site/src/content/docs/explanation/why-one-binary.mdx b/site/src/content/docs/explanation/why-one-binary.mdx index c638d9a5..ab4f1f88 100644 --- a/site/src/content/docs/explanation/why-one-binary.mdx +++ b/site/src/content/docs/explanation/why-one-binary.mdx @@ -28,9 +28,9 @@ investigator calls the same typed query tools the API exposes, which is why what it tells you matches what the dashboards show — there is no second implementation to disagree. -**Configuration in one place.** Every setting is one `FANOUT_` variable, -[generated into one reference](/reference/configuration) from the type the -loader binds. +**Configuration in one place.** Settings use `FANOUT_` variables, with vendor +names for AI credentials and endpoints, [generated into one +reference](/reference/configuration) from the type the loader binds. ## What it costs diff --git a/site/src/content/docs/guides/connect-over-mcp.mdx b/site/src/content/docs/guides/connect-over-mcp.mdx index e88eda6f..cae68495 100644 --- a/site/src/content/docs/guides/connect-over-mcp.mdx +++ b/site/src/content/docs/guides/connect-over-mcp.mdx @@ -19,7 +19,7 @@ panel previews, targeted edits and saved-version restore through chat or MCP. ## Set the public URL first ```sh -FANOUT_PUBLIC_URL=https://fanout.example.com +export FANOUT_PUBLIC_URL=https://fanout.example.com ``` Fanout derives the MCP resource URI from this — the canonical, externally @@ -35,7 +35,8 @@ The resource is derived from `FANOUT_PUBLIC_URL`; there is no separate MCP public-origin setting. An unrecognised `FANOUT_`-prefixed variable is a startup error rather than something ignored. -`FANOUT_MCP_ENABLED` (default `true`) turns the endpoint off entirely. +Set `FANOUT_MCP_ENABLED=false` to turn the endpoint off entirely; it defaults +to `true`. ## What the client does diff --git a/site/src/content/docs/guides/set-up-alerts.mdx b/site/src/content/docs/guides/set-up-alerts.mdx index 157d453d..d212f2e1 100644 --- a/site/src/content/docs/guides/set-up-alerts.mdx +++ b/site/src/content/docs/guides/set-up-alerts.mdx @@ -21,7 +21,7 @@ Rules are managed over HTTP. There is no browser page for them yet — see | `GET /api/alerts/summary` | Counts by state | | `GET /api/alerting/rules` | List rules | | `POST /api/alerting/rules` | Create and compile a rule | -| `PUT /api/alerting/rules/:id` | Replace and recompile a rule | +| `PUT /api/alerting/rules/:id` | Replace a rule; recompile it when enabled | Reading takes `telemetry:read`; creating and replacing takes `alerts:manage`, which starts at the `operator` role. See [roles](/reference/roles). @@ -32,6 +32,8 @@ An expression, an optional service to scope it to, timing, and delivery: | Field | Means | |---|---| +| `name` | Required rule name | +| `enabled` | Set `true` to evaluate the rule; omitted means disabled | | `expression` | The condition, in [CEL](https://cel.dev) | | `service` | Evaluate only for this service; omit for all | | `for_seconds` | How long the condition must hold before firing | @@ -42,9 +44,10 @@ An expression, an optional service to scope it to, timing, and delivery: | `webhook_template` | Custom payload body; omit for the default | | `notify_on_resolve` | Also deliver when the alert clears | -When you create or replace a rule through the API, its expression is type-checked -against the declared variables and must return a boolean. An unknown variable or -a type mistake is rejected at save time. +Nonempty expressions supplied when creating or replacing a rule are type-checked +against the declared variables and must return a boolean. An unknown variable +or a type mistake is rejected at save time. Supply a complete rule body when +replacing it; omitted fields take their zero values. ## Variables available to an expression @@ -60,9 +63,9 @@ So a rule body reads like the thing it means: error_rate > 0.02 && throughput > 10 ``` -The throughput clause matters more than it looks. A service handling two -requests a minute reaches a 50% error rate on one failure, and without a floor -every quiet service pages you. +The throughput clause matters more than it looks. Here `throughput` is the +total span count in the current rollup window, not requests per second. Without +a traffic floor, a single failure on a quiet service can trigger the rule. ## The lifecycle diff --git a/site/src/content/docs/guides/terminate-tls.mdx b/site/src/content/docs/guides/terminate-tls.mdx index 6e150168..1caaaa5b 100644 --- a/site/src/content/docs/guides/terminate-tls.mdx +++ b/site/src/content/docs/guides/terminate-tls.mdx @@ -17,8 +17,8 @@ Set both files. Setting one without the other is a startup error rather than a half-configured listener: ```sh -FANOUT_TLS_CERT_FILE=/etc/fanout/tls/cert.pem -FANOUT_TLS_KEY_FILE=/etc/fanout/tls/key.pem +export FANOUT_TLS_CERT_FILE=/etc/fanout/tls/cert.pem +export FANOUT_TLS_KEY_FILE=/etc/fanout/tls/key.pem ``` That enables TLS 1.3 on the shared listener. Browser traffic and OTLP/HTTP @@ -30,8 +30,8 @@ Terminate at the proxy and forward to Fanout's shared port (`7520` by default). come with it: ```sh -FANOUT_PUBLIC_URL=https://fanout.example.com -FANOUT_TRUSTED_PROXY_CIDRS=10.0.1.0/24 +export FANOUT_PUBLIC_URL=https://fanout.example.com +export FANOUT_TRUSTED_PROXY_CIDRS=10.0.1.0/24 ``` `FANOUT_PUBLIC_URL` is the externally reachable HTTPS origin. Fanout uses it as diff --git a/site/src/content/docs/guides/troubleshoot.mdx b/site/src/content/docs/guides/troubleshoot.mdx index 3dd0405d..d256abd3 100644 --- a/site/src/content/docs/guides/troubleshoot.mdx +++ b/site/src/content/docs/guides/troubleshoot.mdx @@ -11,12 +11,14 @@ status: preview ## The process will not start Fanout validates its whole configuration before opening any listener, so a -startup refusal is a configuration message rather than a crash. Read the error: -it names the setting. - -The one that surprises people is an unrecognised variable. Fanout rejects any -`FANOUT_`-prefixed name it does not know, so a typo — or a setting renamed in a -release — stops the process instead of being ignored. That is deliberate; see +configuration refusal names the setting. Other startup errors can report +storage, listener, or identity-provider failures; read the error before changing +settings. + +The one that surprises people is an unrecognised variable. Fanout rejects +unknown `FANOUT_`-prefixed setting names, except platform-injected service-discovery +names, so a typo stops the process instead of being ignored. That is deliberate; +see [configuration](/reference/configuration). ## An unresolved compaction blocks startup @@ -83,12 +85,15 @@ trace index. First stop Fanout and back up `telemetry/`, then verify the full authoritative set: ```sh +# Use the same environment and data directory as the server. fanout repair verify ``` -The command is read-only and names every unreadable batch. Restore each named -batch from backup when possible. If one cannot be recovered and discarding -only its telemetry is preferable to leaving the instance offline, set it aside +The command is read-only and reports confirmed corrupt batches. Unsupported +formats and operational errors stop verification; they do not authorize +quarantine. Restore each reported corrupt batch from backup when possible. If +one cannot be recovered and discarding only its telemetry is preferable to +leaving the instance offline, set it aside explicitly: ```sh @@ -119,7 +124,7 @@ Work down this list in order: `fo_`-prefixed and goes in `Authorization: Bearer`. Browser sessions and MCP tokens are not ingest credentials. 3. **Is the listener reachable?** Both OTLP transports use `FANOUT_ADDR` - (`:7520` by default). + (`127.0.0.1:7520` by default; the image sets `:7520`). Check container port mappings, bind address and firewall rules. 4. **Right protocol?** Both use port `7520`. gRPC requires HTTP/2 through any proxy. Use `grpc` or `http/protobuf` consistently with the exporter diff --git a/site/src/content/docs/guides/tune-retention.mdx b/site/src/content/docs/guides/tune-retention.mdx index a7878a73..5c6576d9 100644 --- a/site/src/content/docs/guides/tune-retention.mdx +++ b/site/src/content/docs/guides/tune-retention.mdx @@ -14,7 +14,7 @@ jobs and are worth separating before changing any of them. ## How long data is kept ```sh -FANOUT_RETENTION_DAYS=30 +export FANOUT_RETENTION_DAYS=30 ``` Zero means keep everything. Changing it does not delete anything immediately — @@ -28,7 +28,7 @@ automatically uses wider timeline buckets for windows longer than 24 hours. ## The maintenance cycle ```sh -FANOUT_MAINTENANCE_INTERVAL=1h +export FANOUT_MAINTENANCE_INTERVAL=1h ``` This is the expensive pass: retention deletes plus full compaction. Lower it to @@ -40,7 +40,7 @@ The target number of telemetry rows that concurrent small requests may share in one atomic Parquet batch: ```sh -FANOUT_INGEST_BATCH_SIZE=50000 +export FANOUT_INGEST_BATCH_SIZE=50000 ``` A larger target can improve sustained write throughput and create fewer files diff --git a/site/src/content/docs/index.mdx b/site/src/content/docs/index.mdx index 920dd3c6..7049b5d8 100644 --- a/site/src/content/docs/index.mdx +++ b/site/src/content/docs/index.mdx @@ -136,7 +136,7 @@ None of them is a roadmap item phrased as a virtue. - **Not multi-tenant:** namespaces scope queries, not access. Anyone who can query the instance can query every namespace in it; an instance is one trust domain. - **No horizontal scale:** ingest, query and maintenance share a process and a machine. There is no scaling one without the others, because they are the same thing. - **No universal capacity number:** the benchmark above describes one release, host and workload. Your payload cardinality, query mix and disk decide the capacity of your instance. [Measure your own hardware](/explanation/performance). -- **No migration paths:** Fanout is pre-release. A renamed setting is renamed, not aliased, which is why an unknown variable stops the process rather than being ignored. +- **No migration paths:** Fanout is pre-release. Storage-format changes may require a fresh data directory; preserve backups before upgrading. diff --git a/site/src/content/docs/reference/alert-expressions.mdx b/site/src/content/docs/reference/alert-expressions.mdx index 364a73c1..e9ce03fb 100644 --- a/site/src/content/docs/reference/alert-expressions.mdx +++ b/site/src/content/docs/reference/alert-expressions.mdx @@ -16,9 +16,9 @@ against one service's rollup for the current window. These are the values it can read — the names below match the variables declared and bound by the evaluator. -Rules are type-checked against these declared variables when saved through the -API. Unknown variables, type mistakes, and non-boolean results are rejected -before the rule is saved. +Nonempty expressions supplied when creating or replacing rules through the +API are type-checked against these declared variables. Unknown variables, +type mistakes, and non-boolean results are rejected before the rule is saved. Use CEL's standard operators and functions, such as `&&`, `||`, `!`, and `service.startsWith("checkout")`. Numeric variables are doubles; cross-type @@ -27,16 +27,16 @@ numeric comparisons let you compare them with integer literals, as in | Variable | Type | Means | |---|---|---| -| `error_rate` | number | Errors as a proportion of requests over the evaluation window, from 0 to 1. | -| `error_rate_delta` | number | Change in error rate against the baseline, as a proportion. | -| `health_score` | number | A composite 0-100 score combining errors, latency and traffic. Convenient for a single catch-all rule; too coarse to explain why it moved. | +| `error_rate` | number | Mean of minute-rollup span error proportions over the evaluation window, from 0 to 1; not a request-weighted whole-window error rate. | +| `error_rate_delta` | number | Relative change in error rate against the preceding rollup window, as a percentage; zero when the baseline is missing or zero. | +| `health_score` | number | The detector's instance-wide composite 0-100 health score. Every service receives the same score; it is not a per-service value. | | `log_count` | number | Log records recorded for the service over the window. | -| `p50` | number | Median request latency in milliseconds. | -| `p95` | number | 95th-percentile request latency in milliseconds. The usual latency signal: it moves when a meaningful share of requests slow down, where the median does not. | -| `p95_delta` | number | Change in p95 latency against the baseline, in milliseconds. | +| `p50` | number | Mean of approximate minute-rollup median span latencies in milliseconds. | +| `p95` | number | Mean of approximate minute-rollup 95th-percentile span latencies in milliseconds; not the whole-window p95. Latency prefers served work, falling back to outbound spans when no served work is available. | +| `p95_delta` | number | Relative change in p95 latency against the preceding rollup window, as a percentage; zero when the baseline is missing or zero. | | `service` | string | The service name being evaluated, as a string. Compare it when one rule needs to behave differently for one service. | -| `throughput` | number | Requests per second. Worth pairing with any rate threshold — a service handling two requests a minute reaches a 50% error rate on one failure. | -| `throughput_delta` | number | Change in throughput against the baseline, in requests per second. Negative when traffic drops, which is how a rule catches a service that has stopped receiving requests rather than started failing them. | +| `throughput` | number | Total span count over the current rollup window, not requests per second. Pair it with an error threshold to avoid firing on very little traffic. | +| `throughput_delta` | number | Relative change in span count against the preceding rollup window, as a percentage. Negative when traffic drops; zero when the baseline is missing or zero. | | `z_score` | number | How far the service's current behaviour sits from its own recent baseline, in standard deviations. Use it to catch a service that has changed without committing to an absolute threshold that fits every service. | See [Set up alerts](/guides/set-up-alerts) for the rule model these sit in, diff --git a/site/src/content/docs/reference/configuration.mdx b/site/src/content/docs/reference/configuration.mdx index 9804d1f1..afe5e887 100644 --- a/site/src/content/docs/reference/configuration.mdx +++ b/site/src/content/docs/reference/configuration.mdx @@ -45,6 +45,7 @@ provider's endpoint is validated and used. Inherited credentials never select a provider. `FANOUT_AI_MODEL` selects the model; unrelated vendor/tool settings such as `ANTHROPIC_MODEL` and `OPENAI_MODEL` are ignored. See [AI settings](/reference/settings/ai) for base URL meanings. +Both provider base URLs accept a trailing slash. ## Unknown names @@ -70,6 +71,7 @@ Setting both forms nonempty fails startup. File contents lose exactly one trailing LF or CRLF; an empty result fails. Errors name variables and paths, never contents. The deployment controls file access; Fanout performs no file-mode check. Secrets are redacted from startup logs. +Unselected AI providers' value and file forms are ignored. Use deployment environment injection, Docker/Compose secrets, Kubernetes Secrets, or a private selected SOPS profile. This public repository ships no @@ -79,7 +81,29 @@ for examples. ## Validation -The whole configuration is validated before any listener opens. A value the +The whole configuration is validated before data files or listeners open. A value the binary cannot accept stops startup. [First boot](/start/first-boot) lists what is checked. The healthcheck command derives its probe from the listener and TLS environment settings without validating unrelated settings or secrets. + +## DuckDB sizing + +Most deployments should not set DuckDB variables. Give the process or +container the CPU and memory limits it may use; at startup Fanout reserves +headroom for Go and sizes the DuckDB connection pool from available CPUs. The +resolved values and whether Fanout chose them are available in the +`runtime_sizing` block returned by `/readyz`. + +These variables are escape hatches for measured, unusual workloads: + +| Variable | Automatic behavior | When to override | +|---|---|---| +| `FANOUT_DUCKDB_MEMORY` | 60% of the container or host memory available to Fanout | A measured co-tenant workload needs a different Go/DuckDB split | +| `FANOUT_DUCKDB_MAX_CONNECTIONS` | Available Go CPUs, bounded to 2–16 connections | Query concurrency has been benchmarked for this machine | +| `FANOUT_DUCKDB_THREADS` | DuckDB chooses its own query worker count | Query-heavy work must leave specific cores free for ingest | + +An explicit value always wins. Startup logs and `/readyz` report the detected +host and cgroup limits, the selected source, and whether detection was +incomplete. If Fanout cannot conclusively inspect a container limit, it warns; +set `FANOUT_DUCKDB_MEMORY` for a guaranteed bound. See +[storage settings](/reference/settings/storage) for types and defaults. diff --git a/site/src/content/docs/reference/endpoints.mdx b/site/src/content/docs/reference/endpoints.mdx index 82500214..f953be9f 100644 --- a/site/src/content/docs/reference/endpoints.mdx +++ b/site/src/content/docs/reference/endpoints.mdx @@ -9,7 +9,7 @@ status: preview --- Everything below is served by the same HTTP listener as the browser client, on -`FANOUT_ADDR` (`:7520` by default). +`FANOUT_ADDR` (`127.0.0.1:7520` by default; the image sets `:7520`). This page covers the operational endpoints and what they are for. For the complete surface — every route with the exact credential it requires, taken from diff --git a/site/src/content/docs/reference/http-routes.mdx b/site/src/content/docs/reference/http-routes.mdx index c3d21f6e..1a9f1162 100644 --- a/site/src/content/docs/reference/http-routes.mdx +++ b/site/src/content/docs/reference/http-routes.mdx @@ -11,7 +11,8 @@ generated: true {/* Generated by cmd/fanout-docgen from internal/api, internal/agent and cmd/fanout. Edit the generator, not this page. */} -Every route below is served on `FANOUT_ADDR` (`:7520` by default). +Every route below is served on `FANOUT_ADDR` (`127.0.0.1:7520` by default; +the image sets `:7520`). OTLP shares this listener but uses its own handlers and ingest credential; see [send your first telemetry](/start/send-telemetry). @@ -29,8 +30,10 @@ concluding a route is reachable — or that it is not: **not** the default. Enabling it also turns on mutex and block sampling. - `/mcp`, `/api/mcp`, `/oauth/*` and `/.well-known/*` — only when `FANOUT_MCP_ENABLED` is true, which **is** the default. -- `/api/agent/*` — only when an AI provider key is - configured. Without one the investigator is not registered at all. +- `/api/agent/*` — only when `FANOUT_AI_PROVIDER` selects `anthropic` or + `openai`, with that provider's required key. An empty provider disables chat. +- `/api/auth/oidc/start` and `/api/auth/oidc/callback` — only in + `FANOUT_AUTH_MODE=oidc`. Everything else is always registered. A path shown with `:name` or `*` is the pattern Echo matches on, not a literal URL. @@ -46,14 +49,14 @@ The supported programmatic interface is **MCP**, at `/mcp` — see [MCP tools](/reference/mcp-tools). The stable non-MCP surfaces are OTLP ingest, and the operational endpoints in [endpoints](/reference/endpoints). -Two exceptions worth knowing, because HTTP is currently the only way to reach -them: alert rules (`/api/alerting/rules`, `/api/alerts`) and ingest-token rotation -(`/api/settings/ingest`). Neither has a browser page yet. +Alert rules (`/api/alerting/rules`, `/api/alerts`) currently have no browser +page. Ingest-token rotation uses `POST /api/settings/ingest/token/rotate` +and is also available to administrators on **Connect telemetry**. ::: ## Errors -Application API errors, including unknown routes and wrong methods, return exactly +Application API errors, including unknown routes and wrong methods, return objects shaped like `{"code":"bad_request","message":"Safe explanation"}`. Codes are stable snake_case identifiers; clients branch on `code` rather than message text. Invalid dashboard or panel specs also include `problems` with field paths, @@ -174,7 +177,7 @@ Capabilities map to roles in [roles](/reference/roles). - **or a service credential** — additionally reachable without a browser session, which is how a scraper reaches the metrics endpoint. -A method of `Any` means the route answers every verb, and the middleware +A method of `Any` means the route is registered for every verb, and the middleware gives the same answer for all of them — where it would not, this page fails -to build rather than pick one. `GET /*` is the single-page application: -anything not matched above is served the client, which is why it is public. +to build rather than pick one. `GET /*` serves the single-page application +for browser paths. Unknown `/api/*` and OTLP paths return errors instead. diff --git a/site/src/content/docs/reference/settings/ingest.mdx b/site/src/content/docs/reference/settings/ingest.mdx index 02271e66..53b8042b 100644 --- a/site/src/content/docs/reference/settings/ingest.mdx +++ b/site/src/content/docs/reference/settings/ingest.mdx @@ -31,8 +31,8 @@ Supplies the namespace for telemetry without one. ### `FANOUT_INGEST_BATCH_SIZE` -Limits telemetry rows per batch; valid values are 1–50000. +Sets the group-commit row target; valid values are 1–50000. Individual requests remain atomic and can exceed this target. ### `FANOUT_INGEST_MAX_IN_FLIGHT_BYTES` -Caps the decoded telemetry the process will hold across concurrent requests. Concurrency is HTTP/2 streams times connections and each handler holds its batch until the commit is durable, so without a ceiling a burst is bounded only by memory. Zero is unbounded, which is the behaviour that predates this setting. +Caps the decoded telemetry the process will hold across concurrent requests. Concurrency is HTTP/2 streams times connections and each handler holds its batch until the commit is durable, so without a ceiling a burst is bounded only by memory. Zero is unbounded. A single request is admitted when no other request holds budget, even if it exceeds the limit. diff --git a/site/src/content/docs/reference/settings/storage.mdx b/site/src/content/docs/reference/settings/storage.mdx index b852f70d..a7937158 100644 --- a/site/src/content/docs/reference/settings/storage.mdx +++ b/site/src/content/docs/reference/settings/storage.mdx @@ -22,7 +22,7 @@ Vendor credentials and endpoints keep the vendor's documented names. | `FANOUT_DATA_DIR` | string | `./data` | | `FANOUT_DUCKDB_MAX_CONNECTIONS` | integer | `0` | | `FANOUT_DUCKDB_MEMORY` | string | — | -| `FANOUT_DUCKDB_THREADS` | integer | — | +| `FANOUT_DUCKDB_THREADS` | integer | `0` | | `FANOUT_MAINTENANCE_INTERVAL` | duration | `1h` | | `FANOUT_RETENTION_DAYS` | integer | `30` | | `FANOUT_ROLLUP_INTERVAL` | duration | `1m` | diff --git a/site/src/content/docs/start/first-boot.mdx b/site/src/content/docs/start/first-boot.mdx index 1d7de056..8c16a2f7 100644 --- a/site/src/content/docs/start/first-boot.mdx +++ b/site/src/content/docs/start/first-boot.mdx @@ -15,8 +15,7 @@ acceptable. ## What it checks -Every duration and size has a floor, and the auth settings are checked as a -group rather than individually: +Timing, batch sizing and auth settings are checked together: - Addresses and the data directory must not be empty. - `FANOUT_ROLLUP_INTERVAL` must be at least `1s`; @@ -31,7 +30,8 @@ group rather than individually: - In `oidc` mode the issuer URL and `FANOUT_PUBLIC_URL` must both be HTTPS, and a client ID and secret are required. -An unrecognised `FANOUT_`-prefixed variable is an error too. A typo in a setting +An unknown `FANOUT_` setting name is an error too, except platform-injected +service-discovery names. A typo in a setting name is otherwise indistinguishable from a setting you never made, and the failure surfaces months later as a default nobody chose. @@ -73,7 +73,9 @@ fresh data directory if you cannot account for who has seen it. ## Signing in afterwards Local mode has no passwords. Once SMTP is configured, active users receive -short-lived codes by email. OIDC deployments use the provider's flow instead. +short-lived codes by email. Without SMTP, run `fanout login-link ` with +the server's environment and data directory to mint a 15-minute, single-use +link for an existing active user. OIDC deployments use the provider's flow instead. ## Next diff --git a/site/src/content/docs/start/install.mdx b/site/src/content/docs/start/install.mdx index 264f4751..a1f96561 100644 --- a/site/src/content/docs/start/install.mdx +++ b/site/src/content/docs/start/install.mdx @@ -37,6 +37,8 @@ or `FANOUT_AI_PROVIDER=openai` with `OPENAI_API_KEY`, to enable the chat investigator. Chat is optional and disabled when no provider is selected, even with inherited vendor credentials or endpoints. Selecting a provider requires its key at startup and validates only its endpoint. +Without a selected provider, ingest, dashboards, traces, logs, metrics, and MCP +continue to work; only investigation chat and AI-assisted controls are hidden. :::caution[Use a named volume, or fix the ownership yourself] The image is distroless and runs unprivileged as UID/GID 65532. A **named @@ -106,10 +108,11 @@ runner, because DuckDB needs cgo. |---|---| | `labstack/fanout:latest` | the newest release | | `labstack/fanout:2026.8.0` | that exact release | -| `labstack/fanout:main` | the tip of `main` | -| `labstack/fanout:sha-` | one specific commit | +| `ghcr.io/labstack/fanout:main` | the tip of `main` | +| `ghcr.io/labstack/fanout:sha-` | one specific commit | -Every tag above also exists under `ghcr.io/labstack/fanout`. +Release and `latest` tags also exist under `ghcr.io/labstack/fanout`. +Development tags are GHCR-only. Standalone signatures and SBOMs are deliberately not published yet — the reasoning is in the repository's release contract. diff --git a/site/src/content/docs/start/send-telemetry.mdx b/site/src/content/docs/start/send-telemetry.mdx index 47f2724a..5c633f40 100644 --- a/site/src/content/docs/start/send-telemetry.mdx +++ b/site/src/content/docs/start/send-telemetry.mdx @@ -53,6 +53,14 @@ variables such as `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` must include their path ## Configure a Collector ```yaml +receivers: + otlp: + protocols: + grpc: + endpoint: 0.0.0.0:4317 + http: + endpoint: 0.0.0.0:4318 + exporters: otlp/fanout: endpoint: "https://fanout.example.com:443" @@ -62,10 +70,13 @@ exporters: service: pipelines: traces: + receivers: [otlp] exporters: [otlp/fanout] logs: + receivers: [otlp] exporters: [otlp/fanout] metrics: + receivers: [otlp] exporters: [otlp/fanout] ``` @@ -202,9 +213,10 @@ namespace in it. An instance is the trust domain. ## One listener -`FANOUT_ADDR` (`:7520`) controls the shared listener in both the native -binary and container. Use `127.0.0.1:7520` to restrict it to local clients, or -publish `7520:7520` in Docker. There are no separate ingest address settings. +`FANOUT_ADDR` controls the shared listener. The native default is +`127.0.0.1:7520`; the image sets `:7520`. Set `FANOUT_ADDR=:7520` for remote +clients, or publish `7520:7520` in Docker. There are no separate ingest address +settings. When using a reverse proxy, enable HTTP/2 to the backend for gRPC. Plaintext Fanout accepts prior-knowledge HTTP/2 (h2c); a TLS listener negotiates HTTP/2 diff --git a/site/src/content/docs/status/capabilities.mdx b/site/src/content/docs/status/capabilities.mdx index 13d35718..a9fa214c 100644 --- a/site/src/content/docs/status/capabilities.mdx +++ b/site/src/content/docs/status/capabilities.mdx @@ -44,7 +44,7 @@ directory. | OTLP ingest over gRPC and HTTP | shipped | Both transports, one port and one ingest token. | | Parquet storage, trace indexes, and DuckDB analytics | shipped | Authoritative local Parquet; rebuildable DuckDB query state. | | Browser client | preview | Ships in the binary; surfaces still moving. | -| Chat investigator | preview | Requires an AI provider key; disabled and logged without one. | +| Chat investigator | preview | Disabled unless `FANOUT_AI_PROVIDER` selects `anthropic` or `openai`; the selected provider's key is required at startup. | | Agent-built dashboards | preview | [Build and refine from chat](/guides/build-dashboards): fifteen panel types, variables, comparison, saved grids and immutable versions with restore. View links retain filters and time; dashboard access remains owner-scoped. | | MCP tools | preview | Served at `/mcp`; the tool set is not frozen. | | Alerts | preview | Rules evaluate on an interval against rollups. | @@ -62,9 +62,8 @@ directory. ## What "no migration path" means in practice It means what it says. A storage-format change may require starting from an -empty data directory, and a renamed setting is renamed rather than aliased. The -configuration loader rejects unrecognised `FANOUT_`-prefixed variables, so a -setting that has moved surfaces as a refusal to start rather than as a silently -ignored value — which is the one guarantee that makes the rest survivable. +empty data directory. The configuration loader rejects unknown `FANOUT_` +setting names, except platform-injected service-discovery names, so a typo +surfaces as a refusal to start rather than as a silently ignored value. Do not run a Fanout instance as the only copy of data you need to keep. diff --git a/site/src/pages/llms.txt.ts b/site/src/pages/llms.txt.ts index 3e0a5d8f..ea21257b 100644 --- a/site/src/pages/llms.txt.ts +++ b/site/src/pages/llms.txt.ts @@ -65,10 +65,10 @@ export const GET: APIRoute = async ({ site }) => { "", "- Fanout reads built-in defaults and process environment. Native installs listen on 127.0.0.1:7520; the image supplies FANOUT_ADDR=:7520. Secret variables accept _FILE paths. Commands: `fanout [flags]`, `fanout version`, `fanout login-link ` (local recovery with host access), `fanout healthcheck [url]`, `fanout repair verify`, and `fanout repair quarantine --batch `. Product operations use HTTP or MCP.", "- MCP is the intended agent interface. The server exposes its tools at `/mcp`.", - "- MCP and the HTTP API authenticate with an API key, which is not the same credential as the ingest token. Both are `fo_`-prefixed; the routes tell them apart by prefix.", + "- External MCP clients authenticate with resource-bound OAuth tokens. The browser API uses browser sessions; `/api/mcp` uses the same session. These credentials do not authorize OTLP ingest.", "- Telemetry goes in over OTLP only: gRPC and HTTP on the same port as the application (7520 by default). There is no bespoke ingest API to script against.", "- Ingest always requires the ingest token, and rejects everything until the first administrator exists — a collector started before setup will fail until setup completes.", - "- Configuration is environment-first, and every variable is `FANOUT_`-prefixed. An unrecognised `FANOUT_` variable is a startup error rather than a silently ignored one.", + "- Configuration is environment-only: built-in defaults followed by nonempty process environment values. Custom settings use FANOUT_; AI credentials and endpoints use ANTHROPIC_API_KEY, ANTHROPIC_BASE_URL, OPENAI_API_KEY and OPENAI_BASE_URL. Unknown FANOUT_ setting names fail startup, except platform-injected service-discovery names. Chat is off unless FANOUT_AI_PROVIDER selects anthropic or openai and its required key is supplied; OPENAI_BASE_URL includes /v1.", "", ];