diff --git a/README.md b/README.md index 723e544..043f0cc 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,9 @@ Since we are using [Open Policy Agent](https://www.openpolicyagent.org/) (aka `O ### Log Format -The docker image provides default log format (`/etc/nginx/log_format`). It's not possible to extend the log format, so if you'd want to add/remove certain fields you have to override it. +The docker image provides a default JSON log format (`/etc/nginx/log_format.conf`). It can't be extended in place, so adding or removing fields means overriding the whole file. + +When the Fluent Bit sidecar is enabled, the chart renders a second, human-readable format alongside it: `kubectl logs` gets the readable one while the JSON format goes to the sidecar. Controlled by `fluentbit.accessLog.stdoutReadable`. ## Helm Chart @@ -46,6 +48,8 @@ There's an option to dynamically add annotations to the pod. You might find it u There's support for instrumenting NGINX with OpenTelemetry (currently only for tracing). Simply the relevant parameters in the `values.yaml` file. +There's also an optional Fluent Bit log-processing sidecar, off by default. When enabled it forwards only the selected access-log statuses and error-log severities to a central OTLP/HTTP endpoint instead of shipping every line, and derives Prometheus metrics from the access log — merged with the NGINX exporter's series onto one `/metrics` endpoint. See the `fluentbit` parameters in [values.md](./helm/values.md). + #### Overriding NGINX configuration files If you wish to override the default configuration files, you can do it by providing an external ConfigMap and supplying Volumes & VolumeMounts that'll be added to the Deployment. diff --git a/helm/config/fluent-bit.yaml b/helm/config/fluent-bit.yaml new file mode 100644 index 0000000..13869a6 --- /dev/null +++ b/helm/config/fluent-bit.yaml @@ -0,0 +1,180 @@ +# Fluent Bit configuration, rendered through Helm `tpl` from the chart ConfigMap. +# +# YAML, not classic `.conf`: only YAML config supports `processors`, the one way to reach an +# outgoing record's OTLP resource attributes. Classic config can set log attributes but never +# resource ones, so every forwarded record would arrive at Loki as `unknown_service`. +# +# Fluent Bit's own `fluentbit_metrics` input is deliberately not collected — too noisy. +{{- $debug := .Values.fluentbit.debug }} +{{- $stages := ternary $debug.stages (dict) $debug.enabled }} + +service: + flush: 1 + daemon: off + log_level: {{ $debug.enabled | ternary $debug.logLevel "info" }} + # Relative to this file, i.e. the image's own /fluent-bit/etc/parsers.conf (json, syslog, …). + parsers_file: parsers.conf +{{- if .Values.fluentbit.errorLog.enabled }} + +# Parses the standard open-source nginx error-log line into structured fields. Format: +# 2024/01/15 10:23:45 [error] 1234#5678: *90 , client: …, server: …, request: "…" +# The leading timestamp is optional (some syslog paths strip it), the connection id (*cid) is +# optional (worker/startup messages omit it), and the client/server/request tail is optional +# and matched as a unit so a comma inside does not truncate it. +parsers: + - name: nginx_error + format: regex + regex: '^(?:(?