feat!: combined tracer+profiler in ddtrace.so by morrisonlevi · Pull Request #4179 · DataDog/dd-trace-php

morrisonlevi · 2026-09-10T13:00:50Z

Description

This PR removes datadog-profiling.so and moves the profiler into ddtrace.so. This can affect customer code if they are checking extension_loaded("datadog-profiling") or similar. It can also affect build systems and management if it moves or alters INI entries, such as making assumptions that not loading datadog-profiling.so will mean the profiler isn't loaded. The datadog-setup.php script tries to understand some of these things and disable the profiler, but it's not guaranteed.

Motivation

Theoretically, there's a size benefit. This has not been realized yet, and actually things have gotten bigger. Personally, I am not sure if that should block merging this; this is a large PR that will have a lot of conflicts, so we may wish to merge it even if we regress, and work on size later. But honestly this was one of the drivers, if we fail to meet this, then we've failed, and maybe we shouldn't ship this!
Less cross-object dependencies, as the tracer and profiler are now in the same .so.
TODO

Testing

This adds a task to Loader test on <arch> libc jobs which ensures that we have a wall-time sample with the expected stack for SSI. It tries to avoid dependencies and workflows which can introduce flakiness, so it uses the version of Python built into the image, uses libzstd rather than a Python package for it, the profile is written locally to disk, and so on. It piggy-backs onto the loader test instead of a new job because it takes 2+ minutes to pull images and git repos and just mere seconds to run the test.

Reviewer checklist

Test coverage seems ok.
Appropriate labels assigned.

…on-2

# Conflicts: # .github/workflows/prof_correctness.yml

- loader: enable profiling in the JIT force-injection functional test cases so their assertions about profiler notification output match what the combined ddtrace.so actually does when DD_PROFILING_ENABLED is set. - profiling: gc_mem_caches_01.phpt assumed a fixed amount of incidental garbage is always reclaimable right after RINIT. That's not true once the tracer is active in the same process (the combined build), since the tracer's own request-lifetime allocations (e.g. the root span) can consume the small amount of cached/free memory the test relied on, making gc_mem_caches() legitimately return 0 for reasons unrelated to allocation profiling. The test now generates and frees its own garbage so the assertion is robust in both standalone and combined builds. - CI: the "profiling tests" job only ever built and exercised the standalone datadog-profiling.so, so gaps like the above were never caught. Added NTS and ZTS combined-mode (tracer + profiling in one ddtrace.so) build-and-test runs alongside the existing standalone runs. Verified locally against registry.ddbuild.io/ci/dd-trace-php/dd-trace-ci:php-8.5_bookworm-10 for both NTS and ZTS: built the real combined ddtrace.so and standalone datadog-profiling.so, reproduced the exact loader-test scenario (including the .ddtrace.profiling marker convention), and ran the full profiling/tests/phpt suite against both artifacts.

Add the generated Makefile as a prerequisite of the Rust archive rules so make rebuilds when php-config (NTS vs ZTS) changes, instead of relinking a stale ABI-incompatible archive. Also drop the standalone profiler build phases from CI's profiling tests job since we only ship combined.

…on-2

config::minit() now runs immediately after the module-conflict/config-count checks, since it's what installs the log crate's logger (gated by datadog.profiling.log_level). Previously it ran after the tracing-subscriber setup and PHP_VERSION detection, so any log/warn/error calls there were silent no-ops regardless of configured level, and debug builds separately hardcoded an early Trace-level logger that unconditionally leaked a MINIT trace line into every phpt test using the debug combined ddtrace.so (e.g. CI's min install tests).

Temporarily point at DataDog/system-tests@levi/common-extension-2 to pick up the install_ddtrace.sh profiling-marker detection fix until it's merged upstream.

The gradle SSI task and the CMake SSI build both invoke cargo/compile_rust.sh directly instead of going through the generated top-level Makefile, so they never got Make's INCLUDES. components-rs/config_codegen.rs now requires DDTRACE_PHP_INCLUDES to preprocess ext/configuration.h for the Rust config-ID codegen, so both call sites need to export it via php-config themselves.

…on-2 # Conflicts: # .claude/ci/shared-zai-tea-tests.md

This reverts commit 1fc782b.

bwoebi · 2026-09-11T16:35:00Z

+// Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
+// SPDX-License-Identifier: Apache-2.0
+


Btw. we have a default license for the repo and don't add these individually.

bwoebi · 2026-09-11T16:37:22Z

We definitely need to find a way to not do this.
Fine for the initial draft to test it, but this is not something I'm willing to accept. Also, like "preserve its historical ID ordering" wtf :-D

Okay, I'll await your experiments that you'd said you'd perform for this (from slack conversation).

bwoebi · 2026-09-11T17:05:22Z

Can we ensure that there's at least one loader test with a proper stack walk producing an usable profile with SSI?

…on-2

The libdatadog submodule bump crosses de67b226e (renamed datadog-live-debugger -> libdd-live-debugger for publishing), so the old crate name/path no longer resolves. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

… break Two prior macOS test-determinism fixes had been applied unconditionally in the shared Makefile, breaking Linux CI: - ALL_TEST_ENV_OVERRIDE forced DD_TRACE_IGNORE_AGENT_SAMPLING_RATES=1 on every platform, breaking tests/ext/background-sender/agent_sampling_sidecar.phpt, which specifically verifies behavior when agent sampling rates are NOT ignored. - RUN_TESTS_CMD unconditionally appended "-d datadog.agent_host=127.0.0.1 -d datadog.trace.agent_port=1", clobbering tests (e.g. tests/ext/live-debugger/debugger_log_probe_capture_size_limit.phpt) that explicitly point at the request-replayer mock agent via --ENV--. The env leakage these overrides guard against is specific to macOS's bare-metal CI runners, which GitLab auto-injects DD_* vars into for infra monitoring; Linux CI runs in Docker with no such stray agent and never needed either override. Scoped both to `uname -s` == Darwin, matching the existing pattern used for the macOS codesign step just below. Also bumps libdatadog to pick up a clippy::never_loop fix in datadog-sidecar/src/entry.rs that was breaking clippy/helper-rust CI jobs across every PHP version on Linux. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Standalone profiler actually goes through ext/datadog.c as well now

…on-2

morrisonlevi added 30 commits August 30, 2026 00:21

feat: combine tracing and profiling in ddtrace.so

f40c169

fix: validate standalone profiler CI artifacts

971b955

fix: use configured PHP metadata in Rust builds

ca6a1d1

fix: generate combined configuration metadata

6024083

fix: align profiling metadata with registry

98431bd

fix: pass PHP includes to ZAI Rust build

88621a8

fix: configure split Rust extension builds

9cf8445

fix: configure tracing sidecar Rust builds

de2d811

fix: rebuild profiler for selected PHP ABI

266c20a

fix: isolate PHP compiler flags on Windows

80941de

fix: initialize tracer defaults before request config

383a07f

build: silence Cargo with quiet Make

fd887c8

fix: share DD_TAGS map configuration

bcfbb62

Merge remote-tracking branch 'origin/master' into levi/common-extensi…

06df13e

…on-2

ci: build combined extension artifacts directly

e49d0b4

test: stabilize loader checks on Alpine

5e509a2

fix: weaken PHP symbols in Rust archives

05bd941

Merge remote-tracking branch origin/master into levi/common-extension-2

4fa7b2b

# Conflicts: # .github/workflows/prof_correctness.yml

ci: report package artifact sizes

44e6a29

test: expect combined loader configuration output

1a67c96

build: remove intermediate extension archives

24260eb

build: discard unused extension sections

04981fc

test: match combined JIT configuration order

1a2aa1b

build: optimize packaged PHP debug artifacts

71972a9

Merge remote-tracking branch 'origin/master' into levi/common-extensi…

1084a4f

…on-2

ci: allow overriding system-tests repo/ref for system_tests jobs

911a33c

Temporarily point at DataDog/system-tests@levi/common-extension-2 to pick up the install_ddtrace.sh profiling-marker detection fix until it's merged upstream.

morrisonlevi added 12 commits September 10, 2026 11:55

ci: use HTTPS for Alpine package repositories

1fc782b

Merge remote-tracking branch 'origin/master' into levi/common-extensi…

a7e1807

…on-2 # Conflicts: # .claude/ci/shared-zai-tea-tests.md

fix: use one ASan runtime for Rust builds

8418fde

ci: install libcurl for profiler correctness

cad341d

ci: preserve PHP runtime dependencies

2b2e827

fix(installer): support legacy profiler packages

a3052be

test(prof): accept combined module frames

ee98168

Merge branch 'master' into levi/common-extension-2

0d84a29

fix: block async signals on tracer writer

7f90345

fix: preserve PHP 7.1 ZTS property names

c1e8a8f

test: clear setup traces before filter checks

59915bc

Revert "ci: use HTTPS for Alpine package repositories"

198d94b

This reverts commit 1fc782b.

bwoebi reviewed Sep 11, 2026

View reviewed changes

morrisonlevi and others added 15 commits September 11, 2026 11:19

chore: remove redundant license header

e36d33e

test(loader): validate SSI profiling stacks

740ae23

test(loader): correlate time samples with stacks

93afe70

Merge branch 'master' into levi/common-extension-2

646d0f0

ci: reuse loader jobs for SSI profiles

1f08ff9

Merge remote-tracking branch 'origin/master' into levi/common-extensi…

c3191bc

…on-2

Bump libdatadog again

4103d00

Simplify changes to the profiler

35f8a4e

Standalone profiler actually goes through ext/datadog.c as well now

chore: update Cargo lock

781b001

chore: regenerate cbindgen headers

552f28d

Merge remote-tracking branch 'origin/master' into levi/common-extensi…

a64c4a4

…on-2

fix: handle ignored SIGCHLD when spawning sidecar

cff1a00

fix: remove stale configuration file references

ddf753c

Navigation Menu

Uh oh!

feat!: combined tracer+profiler in ddtrace.so - #4179

feat!: combined tracer+profiler in ddtrace.so#4179
morrisonlevi wants to merge 70 commits into
masterfrom
levi/common-extension-2

morrisonlevi commented Sep 10, 2026 •

edited

Loading

Uh oh!

bwoebi Sep 11, 2026

Uh oh!

bwoebi Sep 11, 2026

Uh oh!

morrisonlevi Sep 11, 2026 •

edited

Loading

Uh oh!

bwoebi commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		// Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
		// SPDX-License-Identifier: Apache-2.0

Uh oh!

Conversation

morrisonlevi commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation

Testing

Reviewer checklist

Uh oh!

bwoebi Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

bwoebi Sep 11, 2026

Choose a reason for hiding this comment

Uh oh!

morrisonlevi Sep 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bwoebi commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

morrisonlevi commented Sep 10, 2026 •

edited

Loading

morrisonlevi Sep 11, 2026 •

edited

Loading