collector: shared registry for collector enable/disable state (PoC) - #429
Draft
nicolastakashi wants to merge 1 commit into
Draft
collector: shared registry for collector enable/disable state (PoC)#429nicolastakashi wants to merge 1 commit into
nicolastakashi wants to merge 1 commit into
Conversation
Exporters currently reimplement the same collector registry: the --collector.<name> / --no-collector.<name> flag pair, a map of enabled state, tracking which collectors the operator named explicitly, scrape filters, and lazy instantiation with caching. node_exporter and postgres_exporter carry near-identical copies of this code, and postgres_exporter has a third partial copy in its probe handler. None of that logic depends on what a collector actually does, so it can be shared. Registry is generic over the exporter's collector interface and its factory config, which lets each exporter keep its own Collector type unchanged: node_exporter: Update(ch) error postgres_exporter: Update(ctx, instance, ch) error Sharing the state also makes it introspectable in one place. Descriptors carry the metadata operators care about (default state, required extensions or server versions), so Report/WriteJSON/WriteMarkdown can answer "what collectors does this exporter have" in a form that is the same across exporters -- usable for generating documentation tables that cannot drift from the code. EnabledMetrics exposes the same state at runtime as <namespace>_collector_enabled, so the question is also answerable for a whole fleet rather than only by shelling into a host. ScrapeReporter covers the per-collector duration and success metrics that exporters define identically today. Collect loops stay in the exporters, since only they know how to call their own Update. Signed-off-by: Nicolas Takashi <nicolastakashicavalcante@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft / proof of concept — opening this to discuss the idea, not to merge as-is.
This came out of a review thread on postgres_exporter#1361, where a hand-maintained table of collectors and their default states prompted the question: could exporter-toolkit own collector enable/disable state, and could documentation be generated from it?
The duplication
node_exporter/collector/collector.goandpostgres_exporter/collector/collector.goare near-verbatim copies of each other:factories,collectorState,forcedCollectors,initiatedCollectors,registerCollector,collectorFlagAction, and the filter-and-instantiate loop. postgres_exporter has a third partial copy inprobe.go, whose// TODO: Handle filtersnode_exporter's copy had already solved — the kind of drift that duplication produces.None of that depends on what a collector does, which is what makes it shareable.
What this adds
Registry[C, Cfg]is generic over the exporter's collector interface and its factory config, so no exporter changes itsCollectortype:Plus the introspection this enables, which is the part that isn't just deduplication:
Report()/WriteJSON()— one schema, same across exporters, for docs pipelines and tooling.WriteMarkdown()— the default table rendering, for mdox or a CI diff check.EnabledMetrics()—<namespace>_collector_enabled{collector="..."}. A flag answers "what's enabled" for whoever can reach the process; a metric answers it for a fleet, and lets you alert on an exporter that shipped with a collector unintentionally off.ScrapeReporter— the per-collector duration/success metrics both exporters define identically. Collect loops stay in the exporters, since only they know how to call their ownUpdate.Prototyped against two exporters
Both wired up locally, building and passing their suites, keeping
registerCollector's signature so none of the ~60 node_exporter or ~26 postgres_exporter collector files changed:--collector.disable-defaultsinteraction with explicit flagsprobe.gocopyBoth net negative while gaining
--collector.listand the enabled metric. The postgres_exporter side is at prometheus-community/postgres_exporter#TBD.The generated postgres_exporter table reproduces all 26 rows of the hand-written one in #1361, defaults included.
Open questions
Cfgsignature acceptable, or is it worth constraining?Buildtakesfunc(name string) Cfgrather than a plainCfgbecause both exporters specialize per collector (logger.With("collector", name));StaticConfigcovers the simple case.WriteMarkdownlive here at all, or should the toolkit only emit JSON and leave rendering to each exporter's docs pipeline?collectorcollides with the exporters' owncollectorpackages, so both prototypes import it aliased.Requiresis free-form[]stringtoday. Worth making structured (extension vs. version), or is prose right for something purely informational?go.mod only promotes
client_golangandclient_modelfrom indirect to direct.