Skip to content

Disable CDS in nested JVMs of CI Visibility smoke tests on Linux arm64 - #12906

Open
daniel-mohedano wants to merge 2 commits into
masterfrom
daniel.mohedano/ci-smoke-arm64-cds-off
Open

daniel-mohedano wants to merge 2 commits into
masterfrom
daniel.mohedano/ci-smoke-arm64-cds-off

Conversation

@daniel-mohedano

@daniel-mohedano daniel-mohedano commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

CI Visibility smoke tests now disable CDS (-Xshare:off) on Linux arm64 for every JVM they start, including nested ones:

  • JVMs started with the agent arguments from CiVisibilitySmokeTest.buildJvmArguments get -Xshare:off directly. This covers the Gradle TestKit daemon (through org.gradle.jvmargs), Maven, and the JUnit Console launcher.
  • Nested JVMs get -Xshare:off through JAVA_TOOL_OPTIONS in the environment of the started processes:
    • GradleLauncherSmokeTest: Gradle wrapper and launcher, and the daemon it starts
    • GradleDaemonSmokeTest: test workers forked by the TestKit daemon
    • MavenSmokeTest: Surefire forks

The change does nothing on other platforms.

Motivation

The other smoke tests already disable CDS on Linux arm64 to avoid JVM crashes (see AbstractSmokeTest and AbstractSmokeApp). The CI Visibility smoke tests did not, and they start several levels of JVMs.

On master, these tests had about 109 failures over the last 3 weeks from JVM crashes on arm64:

  • GradleLauncherSmokeTest: 102 (launcher exits with code 134)
  • GradleDaemonSmokeTest: 4 (SIGSEGV in ClassLoaderData::add_handle)
  • MavenSmokeTest: 3

That makes this the main source of flakiness for these suites.

Additional Notes

I could not reproduce the crash locally. It happens in about 0.7% of the Gradle launcher runs on arm64 CI. CI runs on this branch should confirm whether it fixes the issue.

test-environment-trigger: skip

@daniel-mohedano daniel-mohedano added comp: testing Testing tag: no release notes Changes to exclude from release notes tag: flaky test Flaky tests comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM type: bug and removed tag: flaky test Flaky tests labels Oct 16, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 57.59 57.69 $\color{green}{\blacktriangledown}$ -0.10 56.55 $\color{red}{\blacktriangle}$ +1.04 70/316
agentEvpProxy 57.04 n/a n/a n/a n/a -

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.71% (-0.10%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 86ea008 | Docs | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 37.37 37.15 $\color{red}{\blacktriangle}$ +0.22 37.15 $\color{red}{\blacktriangle}$ +0.22 36/172
agentless 37.58 37.15 $\color{red}{\blacktriangle}$ +0.43 36.42 $\color{red}{\blacktriangle}$ +1.16 36/145
agentlessCodeCoverage 46.19 45.38 $\color{red}{\blacktriangle}$ +0.81 45.38 $\color{red}{\blacktriangle}$ +0.81 36/144
agentlessLineCoverage 57.19 56.55 $\color{red}{\blacktriangle}$ +0.64 56.55 $\color{red}{\blacktriangle}$ +0.64 36/144

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 9.73 9.92 $\color{green}{\blacktriangledown}$ -0.19 9.92 $\color{green}{\blacktriangledown}$ -0.19 35/141

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 10.93 10.75 $\color{red}{\blacktriangle}$ +0.18 10.13 $\color{red}{\blacktriangle}$ +0.80 37/167
agentless 8.43 9.92 $\color{green}{\blacktriangledown}$ -1.49 9.92 $\color{green}{\blacktriangledown}$ -1.49 36/137
agentlessCodeCoverage 15.08 18.08 $\color{green}{\blacktriangledown}$ -3.00 17.73 $\color{green}{\blacktriangledown}$ -2.65 31/124

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 90.59 91.39 $\color{green}{\blacktriangledown}$ -0.80 89.58 $\color{red}{\blacktriangle}$ +1.01 36/173
agentless 82.88 82.69 $\color{red}{\blacktriangle}$ +0.19 82.69 $\color{red}{\blacktriangle}$ +0.19 36/143
agentlessCodeCoverage 98.51 97.04 $\color{red}{\blacktriangle}$ +1.47 97.04 $\color{red}{\blacktriangle}$ +1.47 36/143
agentlessLineCoverage 114.75 113.88 $\color{red}{\blacktriangle}$ +0.87 113.88 $\color{red}{\blacktriangle}$ +0.87 36/144

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 22.77 22.54 $\color{red}{\blacktriangle}$ +0.23 22.09 $\color{red}{\blacktriangle}$ +0.68 37/172
agentless 19.93 19.59 $\color{red}{\blacktriangle}$ +0.34 19.59 $\color{red}{\blacktriangle}$ +0.34 36/139
agentlessCodeCoverage 20.11 20.39 $\color{green}{\blacktriangledown}$ -0.28 19.99 $\color{red}{\blacktriangle}$ +0.12 36/139
agentlessLineCoverage 27.32 27.52 $\color{green}{\blacktriangledown}$ -0.20 27.52 $\color{green}{\blacktriangledown}$ -0.20 22/94

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@dd-octo-sts

dd-octo-sts Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.80 s [+0.4%; +2.0%] (maybe worse)
startup:insecure-bank:tracing:Agent 12.87 s 12.91 s [-0.8%; +0.2%] (no difference)
startup:petclinic:appsec:Agent 17.14 s 16.25 s [+1.1%; +9.9%] (significantly worse)
startup:petclinic:iast:Agent 16.89 s 17.07 s [-1.8%; -0.3%] (maybe better)
startup:petclinic:profiling:Agent 16.87 s 16.82 s [-0.7%; +1.3%] (no difference)
startup:petclinic:sca:Agent 16.98 s 17.00 s [-1.0%; +0.7%] (no difference)
startup:petclinic:tracing:Agent 16.09 s 16.09 s [-0.7%; +0.8%] (no difference)

Commit: 86ea008d · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 13.50 13.67 $\color{green}{\blacktriangledown}$ -0.17 13.40 $\color{red}{\blacktriangle}$ +0.10 36/171
agentless 13.05 12.12 $\color{red}{\blacktriangle}$ +0.93 12.12 $\color{red}{\blacktriangle}$ +0.93 36/141
agentlessCodeCoverage 16.64 15.72 $\color{red}{\blacktriangle}$ +0.92 15.41 $\color{red}{\blacktriangle}$ +1.23 36/141
agentlessLineCoverage 18.49 17.38 $\color{red}{\blacktriangle}$ +1.11 17.38 $\color{red}{\blacktriangle}$ +1.11 35/141

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 96.72 95.12 $\color{red}{\blacktriangle}$ +1.60 95.12 $\color{red}{\blacktriangle}$ +1.60 37/172
agentless 92.33 91.39 $\color{red}{\blacktriangle}$ +0.94 89.58 $\color{red}{\blacktriangle}$ +2.75 36/143
agentlessCodeCoverage 101.59 101.00 $\color{red}{\blacktriangle}$ +0.59 99.00 $\color{red}{\blacktriangle}$ +2.59 36/143
agentlessLineCoverage 104.23 103.04 $\color{red}{\blacktriangle}$ +1.19 101.00 $\color{red}{\blacktriangle}$ +3.23 42/152

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 18.14 18.82 $\color{green}{\blacktriangledown}$ -0.68 19.99 $\color{green}{\blacktriangledown}$ -1.85 36/169
agentless 18.67 18.45 $\color{red}{\blacktriangle}$ +0.22 19.20 $\color{green}{\blacktriangledown}$ -0.53 37/145
agentlessCodeCoverage 21.55 21.65 $\color{green}{\blacktriangledown}$ -0.10 22.99 $\color{green}{\blacktriangledown}$ -1.44 38/147
agentlessLineCoverage 39.95 37.91 $\color{red}{\blacktriangle}$ +2.04 39.45 $\color{red}{\blacktriangle}$ +0.50 35/142

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@daniel-mohedano
daniel-mohedano marked this pull request as ready for review October 16, 2026 10:51
@daniel-mohedano
daniel-mohedano requested a review from a team as a code owner October 16, 2026 10:51
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T10:54:29.730104Z 326d588 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-16T10:54:44.025774Z 326d588 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 17.22 16.36 $\color{red}{\blacktriangle}$ +0.86 16.36 $\color{red}{\blacktriangle}$ +0.86 36/168
agentless 10.60 9.92 $\color{red}{\blacktriangle}$ +0.68 9.92 $\color{red}{\blacktriangle}$ +0.68 36/140
agentlessCodeCoverage 14.44 13.67 $\color{red}{\blacktriangle}$ +0.77 13.67 $\color{red}{\blacktriangle}$ +0.77 36/142
agentlessLineCoverage 23.40 22.54 $\color{red}{\blacktriangle}$ +0.86 22.54 $\color{red}{\blacktriangle}$ +0.86 35/140

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@datadog-datadog-us1-prod datadog-datadog-us1-prod Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

The TestKit daemon still starts with CDS enabled because its build environment is applied after JVM startup. Its daemon JVM arguments also need the Linux arm64 workaround.

Open Bits AI session

🤖 Bits Code Review · Commit 326d588

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 31.83 17.38 $\color{red}{\blacktriangle}$ +14.45 15.41 $\color{red}{\blacktriangle}$ +16.42 36/170
agentless 32.88 13.94 $\color{red}{\blacktriangle}$ +18.94 13.67 $\color{red}{\blacktriangle}$ +19.21 36/142
agentlessCodeCoverage 76.67 76.33 $\color{red}{\blacktriangle}$ +0.34 79.45 $\color{green}{\blacktriangledown}$ -2.78 36/142
agentlessLineCoverage 97.50 118.53 $\color{green}{\blacktriangledown}$ -21.03 123.36 $\color{green}{\blacktriangledown}$ -25.86 36/145

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility comp: testing Testing tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes type: bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant