Skip to content

Support agentless log submission for JBoss LogManager - #12726

Open
daniel-mohedano wants to merge 6 commits into
daniel.mohedano/agentless-logs-logbackfrom
daniel.mohedano/agentless-logs-jboss-logmanager
Open

daniel-mohedano wants to merge 6 commits into
daniel.mohedano/agentless-logs-logbackfrom
daniel.mohedano/agentless-logs-jboss-logmanager

Conversation

@daniel-mohedano

@daniel-mohedano daniel-mohedano commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Second PR of the agentless log submission stack. It adds a shared helper and uses it for JBoss LogManager.

  • AgentlessLogSubmission (agent-bootstrap) builds the submitted event, using the same payload shape as the Log4j2 agentless appender:
    • level filtering against DD_AGENTLESS_LOG_SUBMISSION_LEVEL, with level names normalized to TRACE ... FATAL
    • a contextMap with the producer correlation IDs and the dd.service, dd.env and dd.version values (when log injection is enabled), overlaid with the user context. Logback and Log4j2 events already carry these values.
    • the exception message, class name and stack trace, truncated to 16 KB like Log4j2
  • JBoss LogManager: when agentless log submission is enabled, a second advice on LoggerNode.publish(ExtLogRecord) submits each record once.
    • It runs after the logger level and filter checks, and before any handler, including async ones.
    • A call-depth guard sends each record once, even when it reaches parent handlers. As a side effect, a record that a handler logs while another record is being published is not submitted. The existing correlation advice uses the same guard.
    • The user context comes from the JBoss MDC.
  • The JBoss correlation modules are now context tracking modules, so MDC injection also works with CI Visibility when tracing is disabled.
  • The log injection smoke tests now assert direct submission for JBoss backends, including JUL routed to JBoss LogManager.
  • The logs intake now creates its HTTP client when the first log is sent, instead of when its thread starts. Creating the client initializes TLS, and the JDK security event logging can then initialize the JUL LogManager on the intake thread. JBoss LogManager 2.1+ (through JBossLoggerFinder) then reads its configuration with the wrong context class loader and ignores the application's logging.properties.

Stack:

  1. Logback (Support agentless log submission for Logback #12725)
  2. Shared submission helper and JBoss LogManager (this PR) (Support agentless log submission for JBoss LogManager #12726)
  3. tinylog 2 (Support agentless log submission for tinylog #12727)
  4. Log4j 1 (Support agentless log submission for Log4j 1 #12728)

Motivation

See the first PR of the stack. JBoss LogManager already had correlation, but nothing submitted its records with agentless submission.

Additional Notes

  • Supported from 1.2.0.GA. Version 1.1.x has no LoggerNode.publish, so neither the existing correlation hook nor submission applies there. The Muzzle range is unchanged.
  • Nothing changes when agentless log submission is disabled, except the MDC injection fix above.

Testing:

  • AgentlessLogSubmission unit tests.
  • Forked JUnit 5 instrumentation tests on 1.2.0.GA and latest (3.x, JDK 11+). They cover:
    • submission and level filtering, JUL levels, exceptions
    • user MDC, child spans, no active span
    • parent handlers, AsyncHandler
    • injection disabled, CI Visibility with tracing disabled
  • :dd-smoke-tests:log-injection:test and Muzzle pass.

Jira ticket: SDTEST-3805

@daniel-mohedano daniel-mohedano added type: feature Enhancements and improvements comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM comp: logs Logs Product labels Oct 2, 2026
@daniel-mohedano
daniel-mohedano added this pull request to stack #12729 October 2, 2026 09:20
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 57.76 57.69 $\color{red}{\blacktriangle}$ +0.07 56.55 $\color{red}{\blacktriangle}$ +1.21 54/308
agentEvpProxy 58.12 n/a n/a n/a n/a -

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

🎯 Code Coverage (details)
• Patch Coverage: 65.22%
• Overall Coverage: 59.70% (-9.13%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: bef2ad6 | Docs | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 37.58 37.15 $\color{red}{\blacktriangle}$ +0.43 37.15 $\color{red}{\blacktriangle}$ +0.43 28/168
agentless 36.27 37.15 $\color{green}{\blacktriangledown}$ -0.88 36.42 $\color{green}{\blacktriangledown}$ -0.15 28/141
agentlessCodeCoverage 46.70 45.38 $\color{red}{\blacktriangle}$ +1.32 45.38 $\color{red}{\blacktriangle}$ +1.32 28/140
agentlessLineCoverage 55.33 56.55 $\color{green}{\blacktriangledown}$ -1.22 56.55 $\color{green}{\blacktriangledown}$ -1.22 27/139

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 91.37 91.39 $\color{green}{\blacktriangledown}$ -0.02 89.58 $\color{red}{\blacktriangle}$ +1.79 29/170
agentless 82.83 82.69 $\color{red}{\blacktriangle}$ +0.14 82.69 $\color{red}{\blacktriangle}$ +0.14 28/139
agentlessCodeCoverage 99.36 97.04 $\color{red}{\blacktriangle}$ +2.32 97.04 $\color{red}{\blacktriangle}$ +2.32 28/139
agentlessLineCoverage 114.71 113.88 $\color{red}{\blacktriangle}$ +0.83 113.88 $\color{red}{\blacktriangle}$ +0.83 27/139

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 13.45 10.54 $\color{red}{\blacktriangle}$ +2.91 10.13 $\color{red}{\blacktriangle}$ +3.32 30/164
agentless 10.36 10.33 $\color{red}{\blacktriangle}$ +0.03 9.92 $\color{red}{\blacktriangle}$ +0.44 28/133
agentlessCodeCoverage 18.42 18.08 $\color{red}{\blacktriangle}$ +0.34 17.73 $\color{red}{\blacktriangle}$ +0.69 24/122

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 22.31 22.54 $\color{green}{\blacktriangledown}$ -0.23 22.09 $\color{red}{\blacktriangle}$ +0.22 29/168
agentless 20.07 19.59 $\color{red}{\blacktriangle}$ +0.48 19.59 $\color{red}{\blacktriangle}$ +0.48 28/135
agentlessCodeCoverage 20.28 20.39 $\color{green}{\blacktriangledown}$ -0.11 19.99 $\color{red}{\blacktriangle}$ +0.29 27/134
agentlessLineCoverage 27.90 26.98 $\color{red}{\blacktriangle}$ +0.92 26.98 $\color{red}{\blacktriangle}$ +0.92 18/89

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 9.85 9.92 $\color{green}{\blacktriangledown}$ -0.07 9.92 $\color{green}{\blacktriangledown}$ -0.07 28/138

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 13.07 13.67 $\color{green}{\blacktriangledown}$ -0.60 13.40 $\color{green}{\blacktriangledown}$ -0.33 29/168
agentless 11.85 12.37 $\color{green}{\blacktriangledown}$ -0.52 12.12 $\color{green}{\blacktriangledown}$ -0.27 29/138
agentlessCodeCoverage 15.60 15.72 $\color{green}{\blacktriangledown}$ -0.12 15.41 $\color{red}{\blacktriangle}$ +0.19 28/137
agentlessLineCoverage 17.38 17.73 $\color{green}{\blacktriangledown}$ -0.35 17.38 $\color{gray}{\bullet}$ 0.00 27/136

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@dd-octo-sts

dd-octo-sts Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.03 s 13.98 s [-0.6%; +1.3%] (no difference)
startup:insecure-bank:tracing:Agent 12.92 s 12.98 s [-1.5%; +0.6%] (no difference)
startup:petclinic:appsec:Agent 17.16 s 16.97 s [+0.2%; +2.0%] (maybe worse)
startup:petclinic:iast:Agent 16.93 s 17.09 s [-1.8%; -0.1%] (maybe better)
startup:petclinic:profiling:Agent 16.58 s 16.85 s [-2.7%; -0.6%] (maybe better)
startup:petclinic:sca:Agent 17.08 s 16.94 s [-0.0%; +1.6%] (no difference)
startup:petclinic:tracing:Agent 15.69 s 16.18 s [-7.1%; +1.1%] (no difference)

Commit: bef2ad61 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@daniel-mohedano
daniel-mohedano force-pushed the daniel.mohedano/agentless-logs-jboss-logmanager branch from 211ea5f to 41adf39 Compare October 2, 2026 09:43
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 94.57 95.12 $\color{green}{\blacktriangledown}$ -0.55 95.12 $\color{green}{\blacktriangledown}$ -0.55 30/169
agentless 88.31 91.39 $\color{green}{\blacktriangledown}$ -3.08 89.58 $\color{green}{\blacktriangledown}$ -1.27 29/140
agentlessCodeCoverage 98.47 101.00 $\color{green}{\blacktriangledown}$ -2.53 99.00 $\color{green}{\blacktriangledown}$ -0.53 29/141
agentlessLineCoverage 101.18 103.04 $\color{green}{\blacktriangledown}$ -1.86 101.00 $\color{red}{\blacktriangle}$ +0.18 30/144

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 17.95 19.59 $\color{green}{\blacktriangledown}$ -1.64 19.99 $\color{green}{\blacktriangledown}$ -2.04 29/169
agentless 17.55 18.82 $\color{green}{\blacktriangledown}$ -1.27 19.59 $\color{green}{\blacktriangledown}$ -2.04 30/142
agentlessCodeCoverage 21.19 22.09 $\color{green}{\blacktriangledown}$ -0.90 22.99 $\color{green}{\blacktriangledown}$ -1.80 31/145
agentlessLineCoverage 38.65 37.91 $\color{red}{\blacktriangle}$ +0.74 39.45 $\color{green}{\blacktriangledown}$ -0.80 26/137

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 16.28 16.36 $\color{green}{\blacktriangledown}$ -0.08 16.36 $\color{green}{\blacktriangledown}$ -0.08 29/165
agentless 9.83 9.92 $\color{green}{\blacktriangledown}$ -0.09 9.92 $\color{green}{\blacktriangledown}$ -0.09 29/137
agentlessCodeCoverage 13.91 13.67 $\color{red}{\blacktriangle}$ +0.24 13.67 $\color{red}{\blacktriangle}$ +0.24 29/139
agentlessLineCoverage 23.57 22.54 $\color{red}{\blacktriangle}$ +1.03 22.54 $\color{red}{\blacktriangle}$ +1.03 27/136

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent -17.74 17.38 $\color{green}{\blacktriangledown}$ -35.12 15.41 $\color{green}{\blacktriangledown}$ -33.15 29/167
agentless -13.34 16.69 $\color{green}{\blacktriangledown}$ -30.03 13.67 $\color{green}{\blacktriangledown}$ -27.01 29/139
agentlessCodeCoverage 68.20 95.12 $\color{green}{\blacktriangledown}$ -26.92 81.05 $\color{green}{\blacktriangledown}$ -12.85 29/139
agentlessLineCoverage 88.85 123.36 $\color{green}{\blacktriangledown}$ -34.51 123.36 $\color{green}{\blacktriangledown}$ -34.51 27/140

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

@daniel-mohedano
daniel-mohedano marked this pull request as ready for review October 5, 2026 12:37
@daniel-mohedano
daniel-mohedano requested review from a team as code owners October 5, 2026 12:37
@daniel-mohedano
daniel-mohedano requested review from vandonr and removed request for a team October 5, 2026 12:37
@daniel-mohedano
daniel-mohedano requested review from mcculls and sarahchen6 and removed request for a team October 5, 2026 12:37
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-05T12:57:46.872051Z 70d6cc0 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-05T12:56:56.861782Z 70d6cc0 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70d6cc022f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

JBoss submissions read the publisher thread’s live MDC instead of the record snapshot, while the class-wide call-depth guard suppresses distinct records emitted synchronously by handlers.

Open Bits AI session

🤖 Bits Code Review · Commit 70d6cc0 · @DataDog review to ask questions

@daniel-mohedano
daniel-mohedano force-pushed the daniel.mohedano/agentless-logs-jboss-logmanager branch from 70d6cc0 to 96cdb4c Compare October 6, 2026 11:21
@daniel-mohedano daniel-mohedano removed the comp: logs Logs Product label Oct 6, 2026
@daniel-mohedano
daniel-mohedano requested a review from a team as a code owner October 15, 2026 10:25
@daniel-mohedano
daniel-mohedano force-pushed the daniel.mohedano/agentless-logs-jboss-logmanager branch from 96cdb4c to c9d08f1 Compare October 15, 2026 10:35

@datadog-prod-us1-4 datadog-prod-us1-4 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

Agentless JBoss submission can discard MDC values stored on the log record and diverge from handler correlation when log-injection settings change during an active trace.

Open Bits AI session

🤖 Bits Code Review · Commit 96cdb4c

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant