Skip to content

feat: sanitize "." in descriptor keys/values before publishing metrics - #1210

Open
yusofg2 wants to merge 1 commit into
envoyproxy:mainfrom
yusofg2:feature/metric-sanitization
Open

feat: sanitize "." in descriptor keys/values before publishing metrics#1210
yusofg2 wants to merge 1 commit into
envoyproxy:mainfrom
yusofg2:feature/metric-sanitization

Conversation

@yusofg2

@yusofg2 yusofg2 commented Aug 13, 2026

Copy link
Copy Markdown

Summary

In statsd/gostats, . is the metric hierarchy separator. Rate limit builds a metric name by joining the domain and descriptor entries with dots (e.g. domain.source_ip_<value>.total_hits). When a descriptor value or key contains a . (a hostname like foo.bar, a gRPC path like /helloworld.Greeter/SayHello, etc.), it injects unintended extra dot-segments into the metric name.

This primarily breaks the Prometheus sink: names are piped through the statsd_exporter mapper (examples/prom-statsd-exporter/conf.yaml), whose match rules use dots as the field separator (ratelimit.service.rate_limit.*.*.total_hits). An extra dot makes a different rule match, so the value gets split across the wrong labels (or the series is dropped entirely).

The only existing sanitization, utils.SanitizeStatName, replaces :_, |_, and dots only inside IPv4 addresses — a bare non-IPv4 dot in a value slips through.

What this changes

Adds an opt-in setting SANITIZE_DESCRIPTOR_METRIC_DOTS (default false, so existing behavior is unchanged). When enabled, . in a descriptor key or value is replaced with _ as the metric name is built.

Key correctness point: sanitization is applied only to the fragments written into metric names in the config key-builders (descriptorKey, and the detailed_metric, value_to_metric, and share_threshold paths). The descriptor-map lookup key and the rate-limit cache key are left verbatim, so rate-limit matching is unaffected — this changes metric names only, not limiting behavior.

The flag is threaded through the stats Manager (which already receives Settings), avoiding churn in the config loader chain and its call sites.

Before / after

With a descriptor value foo.bar sent through a real server + the real statsd_exporter mapper conf:

Flag off (today) — the dot creates an extra segment, so the value is split across two labels:

ratelimit_service_rate_limit_total_hits{domain="sanitize",key1="source_ip_foo",key2="bar"} 1

Flag on — the value stays intact in a single label:

ratelimit_service_rate_limit_total_hits{domain="sanitize",key1="source_ip_foo_bar"} 1

Testing

  • Config-layer unit tests for flag on/off across: dotted value, non-IPv4 value (foo.bar), dotted key, detailed_metric, value_to_metric, and a "matching still works with a dotted value" case (guards the lookup-vs-metric split).
  • A SanitizeStatKeyValue helper unit test and a manager accessor test.
  • An end-to-end integration test that drives a dotted value through a real running gRPC server and asserts the sanitized metric name for both the value_to_metric and detailed_metric builders.

Docs

README.md documents SANITIZE_DESCRIPTOR_METRIC_DOTS near the other stats env vars.

Add an opt-in SANITIZE_DESCRIPTOR_METRIC_DOTS setting (default false, so
behavior is unchanged for existing deployments). When enabled, "." in a
descriptor key or value is replaced with "_" as the metric name is built,
so dotted values (hostnames, gRPC paths, etc.) don't inject extra statsd
hierarchy segments that break the Prometheus statsd_exporter metric-name
-> label mapping.

Sanitization is applied only to the fragments written into metric names in
the config key-builders (descriptorKey, detailed_metric, value_to_metric,
and share_threshold paths); the descriptor-map lookup key and the rate-limit
cache key are left verbatim, so rate-limit matching is unaffected. This is
distinct from the pre-existing SanitizeStatName, which only strips dots
inside IPv4 addresses and runs unconditionally.

The flag is threaded through the stats Manager (which already receives
Settings), avoiding churn in the config loader chain and its ~40 call sites.

Tests: config-layer unit tests for flag on/off across the value, non-IPv4
value, key, detailed_metric, value_to_metric, and matching-still-works
cases; a SanitizeStatKeyValue helper unit test; a manager accessor test;
and an end-to-end integration test that drives a dotted value through a
real running server and asserts the sanitized metric name for both the
value_to_metric and detailed_metric builders.

Signed-off-by: Yusof Ganji <yganji@salesforce.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant