proposal: add info-metric label discovery APIs for info() autocomplete

[aknuds1]

Add a proposal for two top-level info-metric label discovery endpoints that support autocomplete for the second argument of the PromQL info() function:

• GET|POST /api/v1/info_labels searches info data-label names.
• GET|POST /api/v1/info_label_values searches values for one exact data-label name.

Both endpoints share the same scope: an optional instant-vector expr, repeated full metric_match[] matchers on __name__, and repeated full data_match[] matchers. Expression-derived storage selection follows the same lookback, offset, and @ semantics as an instant info() evaluation.

The proposal reuses the experimental search API storage and NDJSON contracts while keeping these function-specific operations at top-level paths. It specifies a per-endpoint result limit, pre-search bounds on expression-derived matcher construction, strict stream completion, and the dual search-api,promql-experimental-functions feature gate. Composite storage search is fail-closed: if any participating non-noop backend lacks storage.Searcher, setup returns a non-streaming errorType: unavailable response rather than silently omitting that backend.

The contract has been validated in both the Prometheus UI implementation and the Grafana Prometheus datasource integration.

[colega]

This works for the existing autocomplete implementations for PromQL, why doesn't it work for the info function?

[aknuds1]

Existing label search API doesn't work for info function autocomplete because it can't derive info series scope. I.e. the info function autocomplete API has to implement info semantics, in order to return data labels matching the expression.

[tcp13equals2]

Given that this new endpoint leverages params and NDJSON from the search endpoint, perhaps this should be /api/v1/search/info_labels

[aknuds1]

I thought about this. The (subjective) reason for not going that route was that info function autocomplete endpoint is not in the same logical family. I'm open to rethinking this, however.

[aknuds1]

After revising the proposal in the meantime, I kept these as top-level info_* endpoints. The reason being that the endpoints (now split into two) are specific to info(). Please let me know if you still think nesting them under /api/v1/search/ would be preferable.

[tcp13equals2]

I would not dismiss this option so quickly. Part of the reason for the search results format being objects/maps per record was so that additional data could be augmented with metric names, label names or label values.

The idea of extending the response format is noted https://github.com/prometheus/proposals/blob/main/proposals/0074-new-labels-values-api.md#extensibility-for-mimir-thanos-cortex.

This endpoint could be extended to include a "include_values=true&values_limit=10" and the values collections could be decorated into each label record.

We already have the "only useful in a certain context" problem on the metric_names endpoint. It supports the include_metadata=true to decorate in metric metadata records for each metric name. But this only can decorate metric names where we have metadata. It's acceptable that this part of the response may not be there.

[aknuds1]

Thanks for the feedback @tcp13equals2, will consider.

[aknuds1]

The proposal has been significantly revised in the meantime. Are you suggesting though to use /api/v1/search/label_names for info function autocomplete? I'm ruling that out for the reason that the autocomplete API needs info() specific semantics.

[itsmylife]

I've implemented a PoC in grafana-prometheus-datasource using your PoC in arve/info-autocomplete branch.

From PoC stand point it's working nicely. But we should challenge the return value data type (instead of returning all potential labels/values we might return labels and then values for the selected label. In simple terms I'd like to follow current labels and and label/<label-key>/values style. I think this'll make auto complete easier. It'll also help client to fetch less data.

[aknuds1]

But we should challenge the return value data type (instead of returning all potential labels/values we might return labels and then values for the selected label.

@itsmylife This dual-endpoint scheme has been implemented. Thanks for the feedback!