[Enhancement]: Speed up elasticsearch module CI (22 min vs ~1 min for other modules)

[dadoonet]

Module

Elasticsearch

Proposal

Summary

:testcontainers-elasticsearch:check is the CI critical path. On main (GitHub Actions run 34364782867, 9 Sep 2026) that job took 22 min 26 s (Gradle step alone: 21 min 50 s). Typical sibling module jobs finish in ~1–2 minutes.

The workflow is core → all modules in parallel → examples. Examples need the entire module matrix, so this one job delays the whole pipeline by ~20 minutes on every PR, even when PostgreSQL or Kafka finished in ~90 s.

This is a proposal only — no code in this issue.

Why it is slow

33 @Test methods, almost all sequential, each using try-with-resources (no shared @Container, no maxParallelForks). Elasticsearch takes ~30–60 s to reach yellow; Kibana ~40–90 s. We pay that on every method.

Cost	Count	Detail
Elasticsearch starts	~28	16 in ElasticsearchContainerTest + 12 from Kibana tests (including one throwaway elasticsearch-certutil container and 2 negative tests that still dependsOn(es))
Kibana starts	9	reusesContainerWhenReuseIsEnabled is skipped in CI (testcontainers.reuse.enable=false)
Distinct Docker images	7	ES 9.5.2 (docker.elastic.co), ES 8.19.20, ES 7.17.29, OSS 7.10.2, ES 9.5.2 (Docker Hub), Kibana 9.5.2, Kibana 9.5.0

CI setup-build runs docker image prune -af before tests, so none of those image layers are reused between jobs. Extra images are especially expensive.

Per-test timings below are estimates from test structure (one ES start ≈ 30–60 s, ES+Kibana ≈ 1.5–3 min, plus a cold pull). Individual JUnit XML times were not available from that run.

Proposals (by ROI)

Minutes are job wall clock, not total compute, unless noted.

1. Split the Gradle job in two — ~22 min → ~11–12 min wall clock

Today a single matrix entry :testcontainers-elasticsearch:check runs both test classes in series. CI already parallelizes by module. Two jobs would keep the same coverage and the same compute:

--tests org.testcontainers.elasticsearch.ElasticsearchContainerTest
--tests org.testcontainers.elasticsearch.KibanaContainerTest

Safest change: no tests removed.

2. Drop or nightly five tests / three extra images — ~22 → ~14–16 min

Good candidates to remove from the PR critical path (or move to a scheduled job):

Test	Why it is low value on every PR	Est. save
clusterHealthIsAtLeastYellowAfterStart	Wait strategy already waits for yellow/green; also covered by latestStartsWithTlsAndPassword	~45 s
v8StartsWithDefaults	8.x and 9.x share isAtLeastMajorVersion8. This is an 8.x boot canary, not distinct module behaviour — plus a full 8.x image pull	~1–2 min
ossImageHasNoXpackEndpoint	OSS is deprecated. ossImageRejectsPassword already covers isOss without starting a container	~1–2 min
v7TransportClientCanQueryClusterHealth	Transport client was removed in ES 8; getTcpHost() is @Deprecated. Full start for a dead API (docs snippet can stay as a static excerpt)	~45 s
externalModeAcceptsOlderKibanaPatchThanElasticsearch	Elastic product patch compatibility, not Testcontainers logic. Pulls a second Kibana image (9.5.0)	~2–3 min

Avoiding the ES 8, OSS, and Kibana 9.5.0 pulls is a large part of the win because of the prune.

latestWorksWithDockerHubImage is similar (second copy of 9.5.2 from Docker Hub). assertCompatibleWith is already tested; a nightly start would be enough.

3. Stop starting Elasticsearch for expected exceptions — ~1.5–2 min

managedModeRejectsWhenOnlyElasticsearchHasExplicitNetwork and managedModeRejectsWhenNetworksDiffer call kibana.start() with dependsOn(es). Elasticsearch reaches yellow, then Kibana throws IllegalStateException.

Extract ensureCorrectNetworkSetupForManagedMode() (or equivalent) to package-private and unit-test it without Docker.

4. Share one default Elasticsearch container — ~2–3 min

latestStartsWithTlsAndPassword, latestDefaultHeapIsTwoGb, and the yellow health check (if kept) use the same default config. A static @Container avoids 2–3 starts.

Do not share once env differs (TLS off, custom heap, custom wait strategy).

5. Smaller test heap, and fixture certs (follow-up)

The module defaults to 2g heap. Tests could use -Xms512m -Xmx512m to start a bit faster and maybe allow maxParallelForks=2 on a 7 GB runner. Flake risk — measure first.

externalModeReachesElasticsearchWithTlsAndServiceToken starts a throwaway ES only to run elasticsearch-certutil. Commit PEMs under src/test/resources (http_ca.crt already exists for another test).

Combined reasonable target

Split CI (1) + drop extra images (2) + negative tests without Docker (3) → about 8–10 min wall clock, without touching 9.x / 7.x / Kibana managed+external happy paths. Heap, forks, and a static container can wait until that lands.

Tests to cheapen (keep coverage, cut cost)

Test	Today	Suggestion
latestDefaultHeapIsTwoGb	Extra ES 9 with the same config as the TLS smoke	Merge into latestStartsWithTlsAndPassword
latestHeapCanBeSetViaJvmOptionsFile	Extra ES 9	Keep latestHeapCanBeSetViaEnv only on PRs
managedModeUsesCustomElasticsearchNetworkAlias	Full ES + Kibana wait, then greps env	Only needs ELASTICSEARCH_HOSTS; skip /api/status / use a shorter wait
latestHonorsCustomHttpsWaitStrategy	Full ES 9	Keep — regression for configureWaitStrategy() not overwriting waitingFor()

reusesContainerWhenReuseIsEnabled is already skipped in CI. Keep for local/reuse coverage; cost in CI is 0.

Do not cut

ElasticsearchContainer has two real branches: isAtLeastMajorVersion8 and isOss. 8.x and 9.x share the first; 7.x (and OSS) are the actual divergences. On Kibana, managed vs external and security/TLS on/off are real configure() paths.

Keep:

• latestStartsWithTlsAndPassword — 9.x TLS / auth / version / xpack / anonymous
• latestCanDisableTls — HTTP when SSL is off
• latestRejectsMismatchedCa — withCertPath + handshake failure
• one heap override path (viaEnv)
• v7UsesHttpWithoutSecurityByDefault + v7EnablesSecurityWithPassword — 7.x behaves differently
• v7SupportsCustomTlsCertificates — withCertPath + custom entrypoint (the expensive 7.x test that is actually unique)
• nonSemanticLatestTagStillStarts — :latest vs the >= 8 heuristic
• existing unit tests (no container): ossImageRejectsPassword, managedModeRejectsExplicitElasticsearchUrl, externalModeRejectsConflictingCredentials, rejectsKibanaVersionBelow8, startFailsWhenElasticsearchIsNotConfigured
• Kibana happy paths: shared network, ad-hoc network, security off, TLS off, external user/password, external without credentials, external TLS + service token

7.x note

Four 7.17.29 starts today. Useful PR minimum: HTTP without security + withPassword. Keep custom TLS 7.x. Transport client and OSS start can leave the critical path (ossImageRejectsPassword stays).

Happy to hear your feedbacks on it.

cc @pioorg