Skip to content

test(integ-test): exclude with gates for analytics-engine and Prometheus suites from integTestRemote - #5709

Open
mengweieric wants to merge 1 commit into
opensearch-project:mainfrom
mengweieric:fix/integtestremote-mirror-integtest-exclusions
Open

test(integ-test): exclude with gates for analytics-engine and Prometheus suites from integTestRemote#5709
mengweieric wants to merge 1 commit into
opensearch-project:mainfrom
mengweieric:fix/integtestremote-mirror-integtest-exclusions

Conversation

@mengweieric

@mengweieric mengweieric commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Description

integTestRemote runs against an externally provided cluster (-Dtests.rest.cluster), so it can neither install plugins nor set node-level settings. Two groups of suites depend on one of those and are still selected by the task, so they fail during setup rather than being skipped.

Group Gate Suites Why they cannot run here
analytics-engine else branch on the existing analyticsEnabled gate AnalyticsEngineCompatIT, AnalyticsEngineSecurityIT, AnalyticsEngineProfileIT create composite (parquet-backed) indices and need the node settings opensearch.experimental.feature.pluggable.dataformat.enabled and ...transport.stream.enabled; each has a dedicated task whose testClusters block provisions that stack
Prometheus if (windows || ignorePrometheus), copied from integTest PrometheusDataSourceCommandsIT, ShowDataSourcesCommandIT, InformationSchemaCommandIT need a running Prometheus instance to query metrics

Observed when running integTestRemote against a cluster without the analytics-engine stack:

  • AnalyticsEngineSecurityIT — all 21 tests fail in setup, and the suite is by a wide margin the slowest in the run. Its setup polls the security configuration API 60 times at one-second intervals and only sets its initialized flag once the poll succeeds, so every test method repeats the full wait before failing.
  • AnalyticsEngineProfileIT — its index.pluggable.dataformat index creation returns 400, which the test treats as "index already exists", so later tests report a misleading no such index 404.
  • AnalyticsEngineCompatIT — skips cleanly; it already self-guards with a @Before probe of _cat/plugins ([Backport 3.7] Skip AnalyticsEngineCompatIT when analytics-engine plugin is absent #5511). It is listed so the analytics-engine group stays complete if that guard is ever removed.
  • The Prometheus suites fail creating their datasource, then fail again during @After cleanup.

Change

Both groups stay gated, so default behaviour does not change. The else branch is only taken when analyticsEnabled is false, the Prometheus group is opt-in through the existing ignorePrometheus flag, and the dedicated :analyticsEngineCompatIT, :analyticsEngineSecurityIT and :analyticsEngineProfileIT tasks are untouched and continue to run all three suites against clusters that provision what they need.

The analytics-engine exclusions are gated rather than added to the class-file exclusion list below, so that all analytics-engine filtering for this task stays together with the existing route-divergence exclusions.

Not included: the security package

integTest also excludes org/opensearch/sql/security/**, on the grounds that those suites run in :integTestWithSecurity. That reasoning does not carry over here, and this PR deliberately leaves them in place.

integTestWithSecurity runs against its own configureSecurityPlugin() test cluster, so it does not cover the remote case. integTestRemote forwards https, user and password specifically so that it can be pointed at a secured cluster, and against an FGAC-enabled cluster the security suites are meaningful and should run. A build-time exclusion would remove that supported configuration.

Whether those suites can run is a property of the cluster the task is pointed at — on a cluster without the security configuration API every PUT /_plugins/_security/api/roles/* returns 400 {"error":"no handler found for uri ..."} — which Gradle cannot determine at configuration time. That case needs a runtime capability check in the tests, not a static exclusion, and is left for separate work.

Related Issues

No separate tracking issue.

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

The unchecked items are not applicable: this changes a test task's exclusion list and adds no functionality, API surface or user-facing behaviour.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@mengweieric mengweieric added the infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc. label Aug 20, 2026
@mengweieric
mengweieric force-pushed the fix/integtestremote-mirror-integtest-exclusions branch from 54f7af5 to 0210c53 Compare August 20, 2026 04:01
@mengweieric mengweieric changed the title Exclude analytics-engine suites from integTestRemote Mirror integTest's test exclusions in integTestRemote Aug 20, 2026
@mengweieric
mengweieric force-pushed the fix/integtestremote-mirror-integtest-exclusions branch from 0210c53 to 6289059 Compare August 20, 2026 04:05
@mengweieric mengweieric changed the title Mirror integTest's test exclusions in integTestRemote Exclude suites integTestRemote cannot provide a cluster for Aug 20, 2026
@mengweieric
mengweieric force-pushed the fix/integtestremote-mirror-integtest-exclusions branch from 2ce0abe to f8658e8 Compare August 20, 2026 04:15
@mengweieric mengweieric changed the title Exclude suites integTestRemote cannot provide a cluster for test(integ-test): exclude analytics-engine and Prometheus suites from integTestRemote Aug 20, 2026
… integTestRemote

integTestRemote runs against an externally provided cluster, so it can
neither install plugins nor set node-level settings. Two groups of suites
depend on one of those and are still selected by it, so they fail during
setup instead of being skipped.

Analytics-engine suites. AnalyticsEngineCompatIT, AnalyticsEngineSecurityIT
and AnalyticsEngineProfileIT create composite (parquet-backed) indices and
depend on opensearch.experimental.feature.pluggable.dataformat.enabled and
opensearch.experimental.feature.transport.stream.enabled. Each has a
dedicated task whose testClusters block provisions that stack. These are
added as an else branch on the existing analyticsEnabled gate so that all
analytics-engine filtering for this task stays in one place.

AnalyticsEngineSecurityIT is also the slowest suite in the run: its setup
polls the security configuration API 60 times at one second intervals and
only marks itself initialised on success, so every test method repeats the
full wait before failing.

Prometheus suites. integTest already skips PrometheusDataSourceCommandsIT,
ShowDataSourcesCommandIT and InformationSchemaCommandIT when the caller
passes -DignorePrometheus, and on Windows where Prometheus is not started.
integTestRemote now honours the same condition; the block and its comment
are taken from integTest unchanged.

Both groups stay gated, so default behaviour does not change: the else
branch is only taken when analyticsEnabled is false, the Prometheus group
is opt-in through the existing flag, and the dedicated analyticsEngine*IT
tasks are untouched.

The security package is deliberately not excluded. Whether those suites can
run depends on the cluster integTestRemote is pointed at -- it forwards
https, user and password precisely so it can target a secured cluster --
so a build-time exclusion would remove a supported configuration. That
case needs a runtime capability check instead.

Signed-off-by: Eric Wei <mengwei.eric@gmail.com>
@mengweieric
mengweieric force-pushed the fix/integtestremote-mirror-integtest-exclusions branch from f8658e8 to b53453a Compare August 20, 2026 04:16
@mengweieric
mengweieric marked this pull request as ready for review August 20, 2026 04:52
@mengweieric mengweieric changed the title test(integ-test): exclude analytics-engine and Prometheus suites from integTestRemote test(integ-test): exclude with gates for analytics-engine and Prometheus suites from integTestRemote Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infrastructure Changes to infrastructure, testing, CI/CD, pipelines, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant