Skip to content

feat(profiling): Add JFR to OTLP/P converter with CLI, benchmarks and validation - #12870

Draft
jbachorik wants to merge 1 commit into
jb/otlp-profile-recording-datafrom
jb/otlp-profile-converter
Draft

jbachorik wants to merge 1 commit into
jb/otlp-profile-recording-datafrom
jb/otlp-profile-converter

Conversation

@jbachorik

@jbachorik jbachorik commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Adds the dd-java-agent/agent-profiling/profiling-otel module. It converts JFR recordings to OTLP Profiles protobufs (OTLP/P).

  • Dictionary tables for strings, functions, locations, stacks, and links, with deduplication.
  • A custom protobuf encoder with pinned OTLP proto field numbers (OtlpProtoFields).
  • Converts jdk.ExecutionSample, jdk.NativeMethodSample, jdk.JavaMonitorEnter, jdk.JavaMonitorWait, jdk.ObjectAllocationSample, MethodSample, and jdk.CPUTimeSample to OTLP CPU samples.
  • Resource attribute sharing and deduplication.
  • A standalone CLI (JfrToOtlpConverterCLI, convert-jfr.sh).
  • A JMH benchmark suite: dictionary tables, protobuf encoder, stack trace conversion, and full file conversion.
  • A validation harness: OtlpProfileValidator with profcheck, pinned OTLP proto schemas, and an OTel collector test container. Includes docker/Dockerfile.profcheck.
  • Module docs: ARCHITECTURE, BENCHMARKS, CLI, INTEGRATION_PLAN, PROFCHECK_INTEGRATION.
  • Moves io.btrace:jafar to 0.26.2 and reworks the agent shadow jar bundling for jafar.

Motivation

The agent records JVM profiles as JFR recordings. An OTLP backend reads profiles only in the OTLP Profiles format. The agent must convert JFR to OTLP/P before it sends. This PR is that converter, as one module with no changes to existing code.

The jafar changes belong to this PR, because this module needs them. The converter reads JFR files through the jafar parser API (io.jafar.parser.api). Its tests write synthetic JFR recordings through org.openjdk.jmc:flightrecorder.writer. The version catalog gains the jafar-parser and jmc-flightrecorder-writer aliases for these two artifacts. jafar-tools is a fat jar that bundles an incomplete copy of the jafar-parser classes. The agent only needs the io.jafar.tools classes, and profiling-scrubber is their only consumer. The bundled parser classes carry no proper type metadata, and the GraalVM native-image builder fails on them. The new bundling drops the fat jar. The agent bundles jafar-parser through profiling-otel, and only the io.jafar.tools classes from the fat jar.

The module also carries a CLI, benchmarks, and a validation harness:

  • The CLI converts recordings outside the agent. Developers use it to inspect converter output and to reproduce converter defects without an agent run.
  • The benchmark suite measures the conversion cost. The conversion runs in the agent upload path, so its CPU and allocation cost must be known first.
  • The validation harness checks the output twice: against pinned OTLP proto schemas (profcheck), and against a real OTel collector in a test container. This checks the wire format against a real receiver, not only against our encoder.

Additional Notes

Part of a 5-PR stack. Each PR bases on the previous one.

  1. The payload and request body move (refactor: Move OtlpPayload and OTLP request bodies to communication/otlp-exporter #12880)
  2. The sender extraction and the canonical resource attributes (refactor: Extract OTLP sender infrastructure to communication/otlp-exporter #12868)
  3. RecordingData reference counting (feat(profiling): Reference-count RecordingData lifecycle #12869)
  4. this PR: the JFR to OTLP/P converter, profiling-otel, plus jafar 0.26.2
  5. OTLP profiles export wiring (feat(profiling): Integrate OTLP profiles export into the agent #12871)

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jbachorik jbachorik added type: feature request comp: profiling Profiling tag: ai generated Largely based on code generated by an AI or LLM labels Oct 14, 2026
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from d4cc2e2 to 3279bdf Compare October 14, 2026 15:34
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from c8393db to 09993b3 Compare October 14, 2026 15:34
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 3279bdf to 318c6c6 Compare October 14, 2026 16:04
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from 09993b3 to 5fb5676 Compare October 14, 2026 16:04
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 318c6c6 to 5f9a4f9 Compare October 14, 2026 21:47
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from 5fb5676 to 900b31a Compare October 14, 2026 21:47
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 5f9a4f9 to 7c38d31 Compare October 14, 2026 21:57
@jbachorik jbachorik closed this Oct 14, 2026
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from 900b31a to 7c38d31 Compare October 14, 2026 21:57
@jbachorik jbachorik reopened this Oct 14, 2026
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from c9eaa35 to af5329d Compare October 14, 2026 22:23
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 07c532b to ee2a515 Compare October 14, 2026 22:31
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from af5329d to a8eced5 Compare October 14, 2026 22:31
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from ee2a515 to eb9c523 Compare October 14, 2026 22:38
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from a8eced5 to c5b6483 Compare October 14, 2026 22:38
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from eb9c523 to 60154aa Compare October 14, 2026 22:44
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from c5b6483 to d4e3a66 Compare October 14, 2026 22:44
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 60154aa to b981f76 Compare October 14, 2026 22:45
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from d4e3a66 to 7efca57 Compare October 14, 2026 22:45
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from b981f76 to 69f3438 Compare October 14, 2026 22:50
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from 7efca57 to 6d01747 Compare October 14, 2026 22:50
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from 69f3438 to e366b70 Compare October 14, 2026 22:51
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from 6d01747 to e235a1c Compare October 14, 2026 22:51
… validation

New profiling-otel module: converts JFR recordings to OTLP Profiles
protobufs — string/function/location/stack/link dictionary tables,
hand-rolled protobuf encoder, jdk.* and jdk.CPUTimeSample event
handling, deduplicating resource attributes.

Ships a standalone CLI (convert-jfr.sh), JMH benchmark suite,
profcheck-based validation harness with pinned OTLP proto schemas and
an OTel collector testcontainer round-trip, plus module docs
(ARCHITECTURE, BENCHMARKS, CLI, INTEGRATION_PLAN,
PROFCHECK_INTEGRATION) and the Dockerfile.profcheck image.

Moves jafar to 0.26.2 (the version the converter is built against)
and reworks the agent shadow jar bundling: the jafar-tools fat jar
bundles an incomplete copy of the jafar-parser classes that breaks
the GraalVM native-image builder, so the agent bundles jafar-parser
(via profiling-otel) and only the io.jafar.tools classes from the fat
jar instead.

Also claims CODEOWNERS for the profcheck Dockerfile.
@jbachorik
jbachorik force-pushed the jb/otlp-profile-recording-data branch from e366b70 to 2f76dd4 Compare October 14, 2026 22:55
@jbachorik
jbachorik force-pushed the jb/otlp-profile-converter branch from e235a1c to 2684827 Compare October 14, 2026 22:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: profiling Profiling tag: ai generated Largely based on code generated by an AI or LLM type: feature request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant