feat: enable Calcite engine for Prometheus datasources with filter pushdown

[robertpaschedag]

Description

Enables all Calcite-only PPL commands (join, lookup, flatten, expand, eventstats, etc.) to work with Prometheus datasources by making PrometheusMetricTable implement Calcite's TranslatableTable interface with a custom logical/physical scan operator and filter pushdown.

Key changes:

• PrometheusMetricTable now extends AbstractTable and implements both TranslatableTable (Calcite) and Table (V2), preserving backward compatibility
• CalciteLogicalPrometheusScan — custom logical scan node (Convention.NONE) with filter pushdown support for label matchers and time ranges
• CalciteEnumerablePrometheusScan — physical scan (EnumerableConvention) that executes PromQL via PrometheusClient.queryRange()
• PrometheusFilterPushDownRule — Calcite optimizer rule that pushes label equality and time range filters into the scan
• OpenSearchSchema — dynamic DataSourceSubSchema resolution for multi-part table references (e.g., source = prometheus.metric_name)
• CalciteRelNodeVisitor.visitRelation() — relaxed datasource guard to allow non-default datasources when their table implements Calcite's Table interface

Filter pushdown behavior:

Filter	Pushed?	Target
service = 'frontend'	Yes	PromQL label selector {service="frontend"}
@timestamp >= X AND @timestamp <= Y	Yes	Query range start/end parameters
@value > 0.5	No	In-memory EnumerableCalc
service = 'frontend' AND @value > 0.5	Partial	Label pushed, @value remains in-memory

Explain output examples:

Full pushdown:
CalciteEnumerablePrometheusScan(table=[OpenSearch, prometheus, request], PushDownContext=[LABELS->{service=frontend}])

Partial pushdown:
EnumerableCalc(condition=>($t4, 0.5))
CalciteEnumerablePrometheusScan(table=[OpenSearch, prometheus, request], PushDownContext=[LABELS->{service=frontend}])

Related Issues

Resolves #5705

Check List

• New functionality includes testing.
• New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
• New PPL command checklist all confirmed.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff or -s.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Yes..... the code got generated with assistence of AI. I was able to do some example PPL queries in local dev environment, using otel-demo sendings logs to opensearch, while sending metrics to prometheus (victoriametrics).

Screenshots

Example 1:

Example 2:

Example 3:

Example 4:

Example 5:

I generally hope, that this might be useful.

[github-actions]

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit ce1362b.

Path	Line	Severity	Description
prometheus/build.gradle	21	high	New dependency added: 'org.immutables:value-annotations:2.8.8'. Per mandatory review policy, all dependency changes must be flagged regardless of apparent legitimacy. Maintainers should verify this artifact's authenticity and necessity before merging.
prometheus/src/main/java/org/opensearch/sql/prometheus/planner/logical/rules/PrometheusPushDownContext.java	132	medium	PromQL injection risk: label values extracted from SQL conditions are directly interpolated into the PromQL query string without escaping (entry.getValue() placed inside double quotes with no sanitization). A label value containing characters like '"', '}', or ',' could break the PromQL selector syntax or allow injection of arbitrary PromQL label matchers, potentially bypassing intended data access scoping on the Prometheus server.
prometheus/build.gradle	71	low	Code coverage enforcement is disabled for the newly added Prometheus planner rules and scan implementation ('org.opensearch.sql.prometheus.planner.logical.rules.*' and 'org.opensearch.sql.prometheus.storage.scan.*'). This reduces test coverage requirements for the new code paths that handle external data retrieval and query construction, which are higher-risk areas.

The table above displays the top 10 most important findings.

Total: 3 | Critical: 0 | High: 1 | Medium: 1 | Low: 1

---

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

---

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.