Enforce continuation diagnostics on smoke tests

[amarziali]

What Does This Do

• Add a test-only companion agent that installs scope and continuation lifecycle probes in smoke-test applications and collects diagnostic reports.
• Enforce diagnostics in the shared JUnit and Spock smoke harnesses, recording each server test’s work and teardown, and CLI applications from before main through exit.
• Fail smoke tests when diagnostics cannot be installed or reports are missing, malformed, or show lifecycle problems.
• Support reasoned exclusions for specific checks while retaining their findings in reports, and document opt-outs for incompatible test modes and custom launchers.
• Update smoke-test launchers, fixtures, and documentation to support the diagnostic agent and its reporting protocol.

Issues fixed thanks to this enforcement:

• Finish HTTP client spans and close scopes when AppSec blocks requests #12611
• Prevent AI Guard HTTP pool workers from retaining request context #12612
• Fix continuation cleanup when Apache async HTTP submission fails #12613
• Prevent Akka HTTP timeout cleanup from retaining request context #12615
• Release Akka message continuations when discarded to dead letters #12616
• Prevent context retention in OpenJDK virtual pollers and HttpClient selector #12626
• Avoid capturing request context in OpenJDK 25 virtual-thread timeout tasks #12851

Motivation

Additional Notes

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: [PROJ-IDENT]

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.90 s	13.88 s	[-0.5%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent	12.91 s	12.96 s	[-1.1%; +0.3%] (no difference)
startup:petclinic:appsec:Agent	17.10 s	16.97 s	[-0.0%; +1.6%] (no difference)
startup:petclinic:iast:Agent	16.97 s	17.03 s	[-1.3%; +0.6%] (no difference)
startup:petclinic:profiling:Agent	16.44 s	16.81 s	[-3.1%; -1.2%] (significantly better)
startup:petclinic:sca:Agent	17.11 s	16.21 s	[+1.3%; +9.9%] (significantly worse)
startup:petclinic:tracing:Agent	16.06 s	16.07 s	[-0.9%; +0.8%] (no difference)

Commit: 0578df0b · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[pr-commenter]

Kafka / producer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	andrea.marziali/diag2	andrea.marziali/smoke-continuation-diagnostics
git_commit_date	1790164547	1790237122
git_commit_sha	9b23b48	7cc7f3f

See matching parameters

	Baseline	Candidate
ci_job_date	1790238398	1790238398
ci_job_id	2075168828	2075168828
ci_pipeline_id	139719425	139719425
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.31	11.0.31
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost	*.localhost
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-bp2s9q7t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-bp2s9q7t 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce	same

[pr-commenter]

Kafka / consumer-benchmark

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	andrea.marziali/diag2	andrea.marziali/smoke-continuation-diagnostics
git_commit_date	1790164547	1790180406
git_commit_sha	9b23b48	be38218

See matching parameters

	Baseline	Candidate
ci_job_date	1790181655	1790181655
ci_job_id	2073022568	2073022568
ci_pipeline_id	139553485	139553485
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion	11.0.31	11.0.31
jmhVersion	1.36	1.36
jvm	/usr/lib/jvm/java-11-openjdk-amd64/bin/java	/usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs	-Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost	*.localhost
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-d2lfy03r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-d2lfy03r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName	OpenJDK 64-Bit Server VM	OpenJDK 64-Bit Server VM
vmVersion	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu	11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results

scenario	Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume	same

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-16T12:28:45.986950Z	0578df0	Manual request
🔒 Security Review	✅ Completed 2026-10-16T12:29:48.132646Z	0578df0	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4a69c5dbd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[datadog-datadog-prod-us1]

The most critical production constraint is that unescaped diagnostic arguments injected into shell environment variables (CATALINA_OPTS, JAVA_OPTS) in ProcessManager.groovy will split on spaces, preventing child JVMs from starting in affected smoke tests. A separate issue in SmokeDiagnosticsBridge.java hardcodes a relocated class namespace, causing ClassNotFoundException and application abort when built with -PdisableShadowRelocate=true.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit e4a69c5}

[amarziali]

@DataDog review

[datadog-datadog-prod-us1]

More details

Companion-agent installation and end-to-end diagnostic enforcement remain unverified at runtime; the assessment covers source-level handling of smoke-test recording windows, process exits, and custom launchers.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0578df0}

[amarziali]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 0578df0b16

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

🛡️ Codex Security Review · Automatically triggered

Security review completed. No security issues were found in this pull request.

Reviewed commit: 0578df0b16

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.