Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 85 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,58 +7,112 @@
[![GitHub release](https://img.shields.io/github/release/hamba/statter.svg)](https://github.com/hamba/statter/releases)
[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/hamba/statter/master/LICENSE)

Go stats clients
`statter` is a Go library for recording counters, gauges, histograms, and
timings through interchangeable reporters.

## Overview

Install with:
## Install

```shell
go get github.com/hamba/statter/v2
```

#### Supported stats clients
* **L2met** Writes l2met to a `Logger` interface
* **Statsd** Writes statsd to `UDP`
* **Prometheus** Exposes stats via `HTTP`
* **VictoriaMetrics** Exposes stats via `HTTP`

**Note:** This project has renamed the default branch from `master` to `main`. You will need to update your local environment.
## Quick start

## Usage
Create a reporter, pass it to `statter.New` with a positive flush interval, and
close the root statter when the application is shutting down. Closing flushes
pending values and closes the reporter if it implements `io.Closer`.

```go
reporter := statsd.New(statsdAddr, "")
stats := statter.New(reporter, 10*time.Second).With("my-prefix")
package main

import (
"time"

"github.com/hamba/statter/v2"
"github.com/hamba/statter/v2/reporter/statsd"
"github.com/hamba/statter/v2/tags"
)

func main() {
reporter, err := statsd.New("127.0.0.1:8125", "")
if err != nil {
panic(err)
}

root := statter.New(reporter, 10*time.Second)
defer root.Close()

stats := root.With("my-service")
stats.Counter("requests", tags.Str("route", "/")).Inc(1)
stats.Gauge("queue-depth").Set(3)
stats.Histogram("response-size").Observe(512)
stats.Timing("request-duration").Observe(25 * time.Millisecond)
}
```

Counters report the accumulated increment since the previous flush and reset
after each flush. Gauges retain their last value and are reported on every
flush. Histograms and timings are sent directly when the reporter supports
them; otherwise, statter aggregates observations between flushes and reports
count, sum, mean, standard deviation, minimum, maximum, and configured
percentiles.

## Names and tags

Use `With` to apply a shared name prefix and tags to metrics. Prefixes are joined
using `.` by default, and tags with the same key are overridden by the newer
value. The separator can be changed with `WithSeparator`.

stats.Counter("my-counter", tags.Str("tag", "value")).Inc(1)
```go
serviceStats := stats.With("worker", tags.Str("region", "west"))
serviceStats.Counter("jobs", tags.Str("queue", "email")).Inc(1)
```

### Scopes
`WithContext` attaches tags to a context, and `FromContext` adds those tags to a
statter. This is useful for carrying request-specific tags through functions
without passing a scoped statter explicitly.

A `Scope` is a sub-statter identified by an id, which tracks the metrics created
through it.
## Scopes

Requesting a scope with an existing id but different tags removes the previous
scope and every metric created through it. This keeps a metric unique for a set
of tags, such as a gauge carrying a revision that changes over time:
A `Scope` tracks the metrics created through it so that they can be removed as
a group. Scope IDs are unique across the statter tree. Requesting an existing
ID with different tags or a different parent deletes its previous metrics and
replaces the scope. This is useful for metrics that represent a replaceable
set of labels, such as build or revision information:

```go
stats.Scope("build-info", tags.Str("revision", rev)).Gauge("info").Set(1)
stats.Scope("build-info", tags.Str("revision", revision)).
Gauge("info").
Set(1)
```

The tracked metrics can also be removed together as a batch:
Delete a scope explicitly when its metrics are no longer relevant:

```go
scope := stats.Scope("tenant:"+id, tags.Str("tenant", id))
scope := stats.Scope("tenant:"+tenantID, tags.Str("tenant", tenantID))
scope.Counter("requests").Inc(1)
scope.Timing("latency").Observe(d)

scope.Delete() // Both metrics are removed.
scope.Timing("latency").Observe(25 * time.Millisecond)
scope.Delete()
```

Use `HasScope` to determine if a scope currently exists.
Metrics are removed from the backend only when the reporter supports removal.
Otherwise deletion stops local aggregation, but the reporter may retain the
last values it received.

Only metrics first registered through the Scope are tracked by it. If the same
metric was registered before it was requested through a Scope, deleting the
Scope does not remove that pre-existing metric.

## Reporters

The base `Reporter` interface handles counters and gauges. A reporter may also
implement histogram and timing interfaces to receive those observations
directly, and removable interfaces to remove backend metrics when they are
deleted.

**Note:** Metrics are only removed from the backend if the reporter supports
removal, which the Prometheus and VictoriaMetrics reporters do. Otherwise
deletion only stops local aggregation.
| Reporter | Backend |
|-------------------------------------------------------------------------------------------------------|--------------------------------------------------------------|
| [`reporter/l2met`](https://pkg.go.dev/github.com/hamba/statter/v2/reporter/l2met) | Writes metrics to a `hamba/logger` logger |
| [`reporter/statsd`](https://pkg.go.dev/github.com/hamba/statter/v2/reporter/statsd) | Sends metrics to StatsD over UDP |
| [`reporter/prometheus`](https://pkg.go.dev/github.com/hamba/statter/v2/reporter/prometheus) | Exposes metrics through an HTTP handler |
| [`reporter/victoriametrics`](https://pkg.go.dev/github.com/hamba/statter/v2/reporter/victoriametrics) | Exposes metrics in Prometheus format through an HTTP handler |
11 changes: 6 additions & 5 deletions context.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,10 @@ type ctxTags struct {
tags []Tag
}

// WithContext returns a new context with the given tags attached. If tags are
// already present in ctx, the new tags are merged with them; a new tag whose
// key already exists overrides the stored value.
// WithContext returns a context derived from ctx with tags attached. If ctx
// already carries tags added by WithContext, the new tags are merged and a
// value with the same key replaces the existing value. With no tags, it
// returns ctx unchanged.
func WithContext(ctx context.Context, tags ...Tag) context.Context {
if len(tags) == 0 {
return ctx
Expand All @@ -31,8 +32,8 @@ func WithContext(ctx context.Context, tags ...Tag) context.Context {
return context.WithValue(ctx, contextKey{}, &ctxTags{tags: merged})
}

// FromContext returns a sub-statter whose tags include those stored in ctx by
// WithContext. If no tags are found in ctx, s is returned unchanged.
// FromContext returns a sub-statter with the tags stored in ctx by
// WithContext. If ctx has no such tags, it returns s unchanged.
func (s *Statter) FromContext(ctx context.Context) *Statter {
ct, _ := ctx.Value(contextKey{}).(*ctxTags)
if ct == nil || len(ct.tags) == 0 {
Expand Down
31 changes: 16 additions & 15 deletions doc.go
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
// Package statter collects and reports statistics via a pluggable [Reporter].
//
// Stats are aggregated in memory and flushed to the reporter on a fixed
// interval. Counters accumulate deltas between flushes; gauges hold their
// last-set value. Histograms and timings are either delegated directly to
// the reporter (when it implements [HistogramReporter] / [TimingReporter])
// or aggregated locally and emitted as a set of derived gauges and a counter.
// Stats are collected in memory and flushed to the reporter on a fixed
// interval. Counters report their accumulated delta and reset after each
// flush; gauges retain and report their last-set value. Histograms and timings
// are either delegated directly to the reporter (when it implements
// [HistogramReporter] or [TimingReporter]) or aggregated locally and emitted
// as derived gauges and a count counter.
//
// Metrics are identified by a name and an optional set of key/value [Tag]
// pairs. The [Statter.With] method creates a scoped sub-statter that
Expand All @@ -13,15 +14,15 @@
// instance.
//
// The [Statter.Scope] method returns a [Scope], a sub-statter identified by a
// caller supplied id. A Scope tracks the metrics created through it, so they
// can be removed together with [Scope.Delete]. Requesting a Scope with an
// existing id but different tags deletes the previous Scope and its metrics,
// which keeps metrics unique for a set of tags, such as a gauge carrying a
// revision that changes over time.
// caller-supplied ID. Newly registered metrics created through a Scope are
// tracked and can be removed together with [Scope.Delete]. Requesting a Scope
// with an existing ID but different tags or a different parent deletes the
// previous Scope and its metrics. This keeps metrics unique for a set of tags,
// such as a gauge carrying a revision that changes over time.
//
// The [Reporter] interface is the only contract that backend adapters must
// satisfy. Richer adapters may additionally implement [HistogramReporter],
// [TimingReporter], and the corresponding Removable* interfaces. Metrics are
// only removed from the backend if the reporter implements the relevant
// Removable* interface, otherwise deletion only stops local aggregation.
// Backend adapters must satisfy [Reporter]. They may additionally implement
// [HistogramReporter] and [TimingReporter] to receive observations directly,
// or the corresponding Removable* interfaces to remove metrics from the
// backend. Without a removable interface, deletion stops local aggregation
// but cannot remove values already reported to the backend.
package statter
7 changes: 4 additions & 3 deletions reporter/l2met/l2met.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Package l2met implements a l2met stats client.
// Package l2met reports counters and gauges as structured log fields using
// hamba/logger.
package l2met

import (
Expand All @@ -7,15 +8,15 @@ import (
"github.com/hamba/statter/v2/internal/bytes"
)

// L2met is a l2met client.
// L2met reports metrics to a logger.
type L2met struct {
log *logger.Logger
prefix string

pool bytes.Pool
}

// New returns a l2met reporter.
// New returns an L2met reporter that prepends prefix to metric names.
func New(log *logger.Logger, prefix string) *L2met {
if len(prefix) > 0 {
prefix += "."
Expand Down
17 changes: 10 additions & 7 deletions reporter/prometheus/prometheus.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Package prometheus implements a prometheus stats client.
// Package prometheus exposes metrics through a Prometheus HTTP handler.
package prometheus

import (
Expand All @@ -15,17 +15,19 @@ import (
"github.com/prometheus/client_golang/prometheus/promhttp"
)

// Option represents a prometheus option function.
// Option configures a Prometheus reporter created by New.
type Option func(p *Prometheus)

// WithBuckets sets the buckets to use with histograms.
// WithBuckets sets the default bucket boundaries used by histograms and
// timings unless a metric is registered with explicit buckets.
func WithBuckets(buckets []float64) Option {
return func(p *Prometheus) {
p.defBuckets = buckets
}
}

// WithErrorLog sets the error log function for logging errors when registering metrics.
// WithErrorLog sets the function used to report errors while registering
// metrics. If fn is nil, the default error logger is retained.
func WithErrorLog(fn func(string)) Option {
return func(p *Prometheus) {
if fn == nil {
Expand All @@ -35,7 +37,7 @@ func WithErrorLog(fn func(string)) Option {
}
}

// Prometheus is a prometheus stats collector.
// Prometheus collects metrics for scraping in Prometheus format.
type Prometheus struct {
namespace string

Expand All @@ -53,7 +55,7 @@ type Prometheus struct {
errLog func(string)
}

// New returns a new prometheus reporter.
// New returns a Prometheus reporter with the given namespace and options.
func New(namespace string, opts ...Option) *Prometheus {
fqn := newFQN()

Expand All @@ -72,7 +74,8 @@ func New(namespace string, opts ...Option) *Prometheus {
return p
}

// Handler returns the prometheus HTTP handler for scraping metrics.
// Handler returns an HTTP handler that exposes the collected metrics for
// Prometheus scraping.
func (p *Prometheus) Handler() http.Handler {
return promhttp.HandlerFor(p.reg, promhttp.HandlerOpts{})
}
Expand Down
16 changes: 9 additions & 7 deletions reporter/statsd/statsd.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Package statsd implements a statsd client.
// Package statsd reports metrics to a StatsD server over UDP.
package statsd

import (
Expand All @@ -21,19 +21,19 @@ func defaultConfig() config {
}
}

// Option represents a statsd option function.
// Option configures a Statsd reporter created by New.
type Option func(*config)

// WithFlushInterval sets the maximum flushInterval for packet sending.
// Defaults to 300ms.
// WithFlushInterval sets the maximum interval between UDP flushes. The
// default is 300 milliseconds.
func WithFlushInterval(interval time.Duration) Option {
return func(c *config) {
c.flushInterval = interval
}
}

// WithFlushBytes sets the maximum UDP packet size in bytes that will be sent.
// Defaults to 1432 bytes.
// WithFlushBytes sets the maximum UDP packet size in bytes. The default is
// 1432 bytes.
func WithFlushBytes(bytes int) Option {
return func(c *config) {
c.flushBytes = bytes
Expand All @@ -47,7 +47,9 @@ type Statsd struct {
es statsd.ExtendedStatSender
}

// New returns a statsd reporter.
// New returns a StatsD reporter configured with the server address, metric
// prefix, and options. It returns an error if the underlying client cannot be
// created.
func New(addr, prefix string, opts ...Option) (*Statsd, error) {
cfg := defaultConfig()
for _, o := range opts {
Expand Down
7 changes: 4 additions & 3 deletions reporter/victoriametrics/victoria_metrics.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
// Package victoriametrics implements a VictoriaMetrics stats reporter.
// Package victoriametrics exposes metrics in Prometheus format through an
// HTTP handler backed by VictoriaMetrics metrics.
package victoriametrics

import (
Expand All @@ -11,14 +12,14 @@ import (
"github.com/hamba/statter/v2/internal/bytes"
)

// VictoriaMetrics is a victoria metrics stats reporter.
// VictoriaMetrics collects metrics for HTTP scraping.
type VictoriaMetrics struct {
fqn *fqn

set *metrics.Set
}

// New returns a new victoria metrics reporter.
// New returns an empty VictoriaMetrics reporter.
func New() *VictoriaMetrics {
fqn := newFQN()

Expand Down
Loading
Loading