OM 2.0: Should stateset be represented in a Complex/Composite format?

[bwplotka]

See https://github.com/prometheus/docs/pull/2679/changes#r2627250204

We are debating if stateset, represented in OM 1.0 as:

# TYPE foo stateset
foo{entity="controller",foo="a"} 1.0
foo{entity="controller",foo="bb"} 0.0
foo{entity="controller",foo="ccc"} 0.0
foo{entity="replica",foo="a"} 1.0
foo{entity="replica",foo="bb"} 0.0
foo{entity="replica",foo="ccc"} 1.0

Should be composite as histograms and summaries in 2.0 e.g.

# TYPE foo stateset
foo{entity="controller"} {states:["a":1, "bb":0, "ccc":0]}
foo{entity="replica"} {states:["a":1, "bb":0, "ccc":1]}

or

# TYPE foo stateset
foo{entity="controller"} {true:["a"], false:["bb", "ccc"]}
foo{entity="replica"} {true:["a", "ccc"], false:["bb"]}

Discussion notes

• Otel does not have it, but some prev discussions happened to add statesets
• ENUM is subcase
• We don't know much about UX and usage to make clear decisions
• Ensure integer not float for composite
• for stateset ST not relevant/useful here as those are gauges

Pros & Cons for making it composite

Pros

• We make it efficiency/reliable to parse (label matching to realize if it's same "sample"). Efficiency of stateset is actually a blocker for using them instead of gauges. There's a potential to reduce cardinality for large number of states.
• Consistency with summaries and histograms
• We don't need to have an extra rule around interleaving.

Cons

• Readability maybe impaired (long lines) - but it's the same as histograms/summaries.
• What you see what you query is violated unless we plan to implement composite/native stateset eventually (and change PromQL). Even if we plan, we don't know how it will look like, so it's unlikely we will figure out the right format, unless we do UX research on potential PromQL functions like state(metric, stateName) or true_states(metric)`

[bwplotka]

From WG:

A) Keep as-is

Pros:

• Kind of useful?

Cons:

• The only multi line type.
• Discussion: We need 3.0 for composite type?
  • Mitigation A: Ignoring clause for {} instead of float
  • Mitigation B: Adding StateSet2 later on
  • Mitigation C: Just do 3.0 in some time.

B) Play with the idea of composite value

Pros:

• We want native Native Stateset in Prometheus prometheus#17914 eventually

Cons:

• We don't have enough feedback or resources to properly

C) Drop it

Pros:

• Only KSM would need to migrate to gauge. Also they implement it wrongly (?). Should be trivial to convert by simply changing type to gauge.
• We can reintroduce it in 2.1 easily (or when we are rdy for native type). It's easier to make sure spec supports adding new types vs flexibility of a single type to be composite or float.
• Putting Gauge vs StateSet on type, it does not matter? (Only metadata breaking change).

Cons:

• OM2 wouldn't support all types. Mitigation: We can share it's just didn't get wide adoption and is inconsitent with composite, single-line samples - so we wait for native stateset to be implement in PromQL/storage.

[bwplotka]

RC.0 keep as-is?

[bwplotka]

Just to connect discussions:

• I added note about label issue on KSM: Exposed StateSet metrics "state" label do not follow OpenMetrics protocol  kubernetes/kube-state-metrics#2879 (and ask if they need this/have feedback).
  • I noticed they are not alone - it's common to miss this rule about "state" label MUST be metric name (e.g. https://metricshub.com/docs/latest/#!?search=hw.status) - I think this is one learning (: to make it easier to make this label flexible or something (e.g. via metadata)
• Otel restarted discussion Does OpenTelemetry need "enum" metric type? open-telemetry/opentelemetry-specification#1711
• Some new activity and ideas on Native Stateset in Prometheus prometheus#17914

Kudos to @ringerc who progressed on this topic a lot!

[ringerc]

I've prepared a crude proof of concept implementation of a native state-set in Prometheus here to help explore how a practical state-set representation might work. prometheus/prometheus#17914 (comment)