Add BenchmarkUtils.warmUp to restore cross-arm branch profiles in benchmarks (perf toolkit - benchmarking tools)

[dougqh]

What Does This Do

Adds BenchmarkUtils.warmUp, a JMH setup helper that drives every path through a benchmark class (each @Benchmark method, plus each interesting outcome of a multi-outcome method) through many shuffled-order passes before measurement starts.

• Explicit form: warmUp(bench, [state...], bh, bench(...)...), with arities 0–2 for @State parameters. bench(...) is overloaded on a small set of java.util.function shapes (Function/Consumer, the Bi* versions, plus the existing datadog.trace.api.function.TriFunction/TriConsumer), so a lambda or method reference to an existing benchmark method fits directly. Primitive results get boxed on the way into the Blackhole. That cost lands only during warmup, never on the measured path.
• Reflective form: warmUp(Class<?>, bh) finds every @Benchmark method on the class, builds the benchmark class and its @State objects through their no-arg constructors, runs their no-arg @Setup methods, and drives one call per method. It needs no setup work, but drives only one outcome per method.
• The shuffle order changes on every pass. It's seeded, so runs are reproducible, and it reuses the same array, so passes don't allocate. That way, wherever C2 starts compiling during warmup, it sees the class's full mix of paths, not whichever one happens to come first.
• Adds WarmUpEscapeAnalysisDemoBenchmark, which shows the effect (see Additional Notes).

Motivation

JMH forks a fresh JVM for each @Benchmark method, so each fork's C2 compiles code that has only ever seen one benchmark's paths. Production is the opposite: one JVM runs everything. warmUpHashDispatch (#12294/#12298) already handles the receiver-type and class-hierarchy side of this. The remaining gap is one-sided branch profiles.

That gap can hide real allocations. In ThreadSafeMapD2Benchmark, @Setup pre-installs every key, so the miss branch of getOrCreate never runs. C2 drops that branch entirely (an unstable_if trap), the composite key no longer escapes, and escape analysis removes its allocation. The benchmark reports ≈0 B/op for a lookup that allocates a key on every call in a real cache, since a real cache's early misses leave that branch two-sided.

Additional Notes

Evidence. WarmUpEscapeAnalysisDemoBenchmark runs the same getOrCreate code in two arms, always hitting in the measured lookup. The only difference is whether warmUp first primes that call site once with a genuine miss, using scratch data:

pollute	ops/µs	B/op	gc.count
false	1610.2 ± 82.4	≈0	≈0
true	1154.7 ± 66.8	24.0	692

Zulu 17.0.7, Apple M1, 2 forks, 8 threads, -prof gc. Priming brings back the composite key's real 24 B/op and its ~28% throughput cost, without changing the measured method or the measured map. The demo clears its scratch map after warmUp, so the extra keys inserted on miss passes aren't kept live during measurement.

Known limits.

• The explicit form can only check that at least as many paths were registered as there are matching @Benchmark methods (including those inherited, since JMH runs @Setup on a generated subclass). It can't verify that every outcome of a multi-outcome method was driven, and registering only the hit path of getOrCreate is worse than not calling warmUp at all.
• The reflective form runs only no-arg @Setup methods (on the benchmark class and its state objects), leaves @Param fields at their Java defaults, and drives only the outcome that its reflectively built state happens to produce.

Not yet adopted. No production benchmark uses warmUp yet. Adopting it in ThreadSafeMapD2Benchmark means writing a miss path for absent keys, and every benchmark that adopts it has to be re-measured. That happens in a follow-up.

Stacked on #12298.

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels
• Avoid using close, fix, or any linking keywords when referencing an issue
  Use solves instead, and assign the PR milestone to the issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1870

🤖 Generated with Claude Code

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.97 s	13.86 s	[+0.1%; +1.5%] (maybe worse)
startup:insecure-bank:tracing:Agent	12.86 s	12.92 s	[-1.2%; +0.3%] (no difference)
startup:petclinic:appsec:Agent	17.13 s	17.05 s	[-0.6%; +1.5%] (no difference)
startup:petclinic:iast:Agent	16.82 s	16.91 s	[-1.3%; +0.2%] (no difference)
startup:petclinic:profiling:Agent	16.55 s	16.71 s	[-1.9%; -0.0%] (maybe better)
startup:petclinic:sca:Agent	17.13 s	16.87 s	[+0.7%; +2.3%] (maybe worse)
startup:petclinic:tracing:Agent	16.13 s	16.14 s	[-0.9%; +0.8%] (no difference)

Commit: c498d604 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.