Skip to content

Run database-scoped collectors against every discovered database - #1378

Open
mwimpelberg28 wants to merge 3 commits into
prometheus-community:masterfrom
mwimpelberg28:run-database-scoped-collectors-per-discovered-database
Open

Run database-scoped collectors against every discovered database#1378
mwimpelberg28 wants to merge 3 commits into
prometheus-community:masterfrom
mwimpelberg28:run-database-scoped-collectors-per-discovered-database

Conversation

@mwimpelberg28

Copy link
Copy Markdown

collector.NewRuntime only ever built one PostgresCollector, bound to DataSourceNames[0]. This meant per-database collectors like stat_user_tables, statio_user_tables and statio_user_indexes silently never ran for any database beyond the first configured DSN, whether that list came from a static, comma-separated DATA_SOURCE_NAME or from --auto-discover-databases.

Naively running every collector against every discovered database isn't safe either: most collectors (bgwriter, wal, stat_activity, stat_statements, locks, ...) read catalog views or functions that already report on the whole instance from any single connection, so running them again from a second database would re-report the same rows and the registry would panic on the duplicate metric (as already happened once, for stat_activity, in #533).

This adds an explicit scope to each registered collector (clusterScope vs databaseScope) and a multiInstanceCollector that fans the database-scoped ones out across every DSN exporter.Exporter currently targets (static list or, with autodiscovery enabled, freshly enumerated databases), while cluster-scoped collectors still only ever run once, against the originally configured DSN. The target list is re-evaluated on every scrape, so databases created or dropped after startup are picked up without restarting the exporter, matching the autodiscovery behavior already used by the legacy exporter.Exporter metrics.

Also gives statio_user_indexes a datname label, matching its sibling per-database collectors: without it, index metrics from two databases with a same-named index would collide once run concurrently.

@mwimpelberg28
mwimpelberg28 force-pushed the run-database-scoped-collectors-per-discovered-database branch from 24d4d3d to 747fa8b Compare August 25, 2026 12:54
collector.NewRuntime only ever built one PostgresCollector, bound to
DataSourceNames[0]. This meant per-database collectors like
stat_user_tables, statio_user_tables and statio_user_indexes silently
never ran for any database beyond the first configured DSN, whether
that list came from a static, comma-separated DATA_SOURCE_NAME or
from --auto-discover-databases.

Naively running every collector against every discovered database
isn't safe either: most collectors (bgwriter, wal, stat_activity,
stat_statements, locks, ...) read catalog views or functions that
already report on the whole instance from any single connection, so
running them again from a second database would re-report the same
rows and the registry would panic on the duplicate metric (as already
happened once, for stat_activity, in prometheus-community#533).

This adds an explicit scope to each registered collector (clusterScope
vs databaseScope) and a multiInstanceCollector that fans the
database-scoped ones out across every DSN exporter.Exporter currently
targets (static list or, with autodiscovery enabled, freshly
enumerated databases), while cluster-scoped collectors still only ever
run once, against the originally configured DSN. The target list is
re-evaluated on every scrape, so databases created or dropped after
startup are picked up without restarting the exporter, matching the
autodiscovery behavior already used by the legacy exporter.Exporter
metrics.

Also gives statio_user_indexes a datname label, matching its sibling
per-database collectors: without it, index metrics from two databases
with a same-named index would collide once run concurrently.

Signed-off-by: Matthew Wimpelberg <120263653+mwimpelberg28@users.noreply.github.com>
@mwimpelberg28
mwimpelberg28 force-pushed the run-database-scoped-collectors-per-discovered-database branch from 747fa8b to 22e8e25 Compare August 29, 2026 22:04
…-collectors-per-discovered-database

Signed-off-by: Matthew Wimpelberg <120263653+mwimpelberg28@users.noreply.github.com>

# Conflicts:
#	collector/collector.go
#	collector/runtime.go
@mwimpelberg28

Copy link
Copy Markdown
Author

Hi @SuperQ @wrouesnel @sysadmind — this has been open for a bit with no review yet. Also flagging that the CI and golangci-lint workflow runs are stuck in action_required since this is from a fork, so they need a maintainer approval to actually run.

Summary of the change: collector.NewRuntime previously only ever built one PostgresCollector bound to DataSourceNames[0], so per-database collectors (stat_user_tables, statio_user_tables, statio_user_indexes) silently never ran for any database beyond the first configured DSN. This PR adds an explicit scope to each registered collector (clusterScope vs databaseScope) and a multiInstanceCollector that fans database-scoped collectors out across every DSN the exporter currently targets (static list, or freshly enumerated databases when --auto-discover-databases is set), while cluster-scoped collectors still run once against the originally configured DSN. It also adds a datname label to statio_user_indexes to avoid collisions across databases.

Happy to address any feedback. Thanks for taking a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant