Skip to content

Split Lettuce tests across CI shards - #12835

Draft
cgcote wants to merge 2 commits into
codex/apmci-50from
codex/apmci-51
Draft

cgcote wants to merge 2 commits into
codex/apmci-50from
codex/apmci-51

Conversation

@cgcote

@cgcote cgcote commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Splits the Lettuce 5 instrumentation's six main test_inst tasks into independently scheduled virtual CI projects:

  • base tests;
  • forked tests;
  • Lettuce 5.1 tests;
  • Lettuce 6.0 tests;
  • Lettuce 6.1 tests;
  • Lettuce 6.2 tests.

The virtual projects delegate to the existing tasks and retain testcontainersLimit. The separate test_inst_latest path remains on the parent project, so latestDepTest and latestDepForkedTest continue unchanged.

Motivation

lettuce-5.0 contributed 978.1 seconds of test execution per JVM 11 test_inst pipeline, creating a 16.3-minute scheduling floor.

Pipeline 144154452 completed successfully and reported 35 Lettuce suites and 314 tests, all passing. Its direct JVM 11 measurements are:

  • Lettuce 6.2: 318.271 seconds;
  • Lettuce 5.1: 290.357 seconds;
  • Lettuce 6.0: 253.057 seconds;
  • base: 244.848 seconds;
  • Lettuce 6.1: 222.911 seconds;
  • forked: 180.254 seconds.

The largest independently scheduled Lettuce unit is now 318.271 seconds, a 67.5% reduction from 978.1 seconds. These measurements replace the provisional weights. With the refined JDBC split in #12830, the modeled eight-shard load ratio is 1.0025.

Additional Notes

Contributor Checklist

  • Preserve all Lettuce 5.0–6.2 compatibility coverage
  • Preserve both latest dependency suites
  • Retain Redis container concurrency limits
  • Tune weights from the first coverage-complete CI run

Work item: APMCI-51

@cgcote cgcote added comp: tooling Build & Tooling inst: lettuce Lettuce instrumentation type: refactoring tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes labels Oct 12, 2026
@datadog-datadog-us1-prod

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Java Benchmark SLOs — Performance SLO warning (near threshold)

Suite Status
Startup 🟡 warning

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.00 s 13.94 s [-0.2%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent 12.90 s 12.99 s [-1.4%; +0.1%] (no difference)
startup:petclinic:appsec:Agent 17.84 s 17.64 s [+0.0%; +2.3%] (maybe worse)
startup:petclinic:iast:Agent 17.24 s 17.56 s [-2.7%; -0.9%] (maybe better)
startup:petclinic:profiling:Agent 17.50 s 17.24 s [+0.1%; +2.9%] (maybe worse)
startup:petclinic:sca:Agent 17.72 s 17.60 s [-0.3%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 16.56 s 16.65 s [-1.5%; +0.5%] (no difference)

Commit: 3f933c0e · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@cgcote
cgcote force-pushed the codex/apmci-51 branch 3 times, most recently from 7168ab8 to 352b86d Compare October 12, 2026 18:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: tooling Build & Tooling inst: lettuce Lettuce instrumentation tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant