feat(otlp-metrics): added support for histogram, exp. histogram, and summary#1853
feat(otlp-metrics): added support for histogram, exp. histogram, and summary#1853
Conversation
512959a to
5bc3dc5
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 37bf8c92cf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e0421e01c1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e5b1c795ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…summary (#1853) …m, and summary metrics Adds support for the remaining OTLP metrics (Exponential Histogram, Histogram, and Summary) in the correctness tests Will be used to test whether these metric types are being properly handled by ADP in the Saluki repo. A list of issues either fixed, containing architectural discussions, otherwise relevant for this Pull Request. Anything else we should know when reviewing? Original commits: * 460ec05 feat(OTLP metrics): added support for histogram, exponential histogram, and summary metrics * 7cfd22e chore(otlp metrics): changed some params and ran cargo fmt * d398476 test(otlp metrics): added further coverage for the correctness tests * 892cf93 fix(otlp metrics): added quick patch to fix issue of duplicate bins * b02dfdd style(OTLP metrics): applied brief clippy change * b2e3f1c style(otlp metrics): added line to bypass too many lines to abide by existing conventions * 5bc3dc5 chore(otlp metrics): added descriptors to CHANGELOG.md * ebd65a0 fix(otlp metric): removed accidental proptest regression seed file * 37bf8c9 fix(otlp metrics): restore entropy in histogram metric payloads * e0421e0 fix(otlp metrics): addressed codex comments about improper construction of summary metrics * e5b1c79 refactor(otlp): trim scaffolding for new metrics * 75669c1 refactor(otlp-metrics): removed synthetic labels and redundant tests * 782831c chore(otlp-metrics): updated vulnerable dependencies * b7a0e27 Merge branch 'main' into lt/add-otlp-metrics Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com>
b7a0e27 to
996be2d
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 996be2d185
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2a1274bcff
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f40bd8983b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be7e11b3d0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 002ae344dc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb9ff1c104
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc7b3305f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
When metric templates sample total_data_points, the loop invariant guarantees current <= max, so current..=current.min(max) is always a single-value range. This helper therefore only returns 1, 2, 4, 8, etc. instead of the documented exponentially weighted buckets like 2-3 and 4-7, which skews generated payload shapes and leaves most point counts in the configured 1-60 range unexercised.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f908b87b17
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 31d4db8537
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 228bef9f8b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1af42fdc6d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…togram (#1319) ## Summary <!-- Please provide a brief summary about what this PR does. This should help the reviewers give feedback faster and with higher quality. --> This PR adds supports for the remaining open telemetry metrics (summary, histogram, and exponential histogram). ## Change Type - [ ] Bug fix - [x] New feature - [ ] Non-functional (chore, refactoring, docs) - [ ] Performance ## How did you test this PR? I created a [draft PR in the lading repo](DataDog/lading#1853) that included the appropriate changes to emit the newly added metrics. I had my `Cargo.toml` point to the branch and ran `make build-datadog-intake-image build-millstone-image build-datadog-agent-image` and `make test-correctness` which yielded the following results: ```rust [*] Running 'dsd-plain' correctness test case... 2026-04-07T20:52:54.164747Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-plain/config.yaml'. 2026-04-07T20:52:54.164804Z INFO ground_truth: Test run starting... 2026-04-07T20:52:54.164890Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:52:54.164904Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/QC8smTa1 2026-04-07T20:52:54.165404Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/sTsNyjwJ 2026-04-07T20:53:17.025257Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:53:54.665292Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:53:56.963274Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:53:56.963322Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:53:56.974946Z INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target. 2026-04-07T20:53:56.979673Z INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:53:56.981558Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing... 2026-04-07T20:53:56.984571Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:53:56.984586Z INFO ground_truth: ground-truth stopped. [*] Running 'dsd-origin-detection' correctness test case... 2026-04-07T20:53:57.600962Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-origin-detection/config.yaml'. 2026-04-07T20:53:57.600989Z INFO ground_truth: Test run starting... 2026-04-07T20:53:57.601028Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:53:57.601036Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/Ulzgl0hU 2026-04-07T20:53:57.601180Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/hJrERcxw 2026-04-07T20:54:17.630782Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:54:54.133377Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:54:56.431092Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:54:56.431109Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:54:56.437142Z INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target. 2026-04-07T20:54:56.440374Z INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:54:56.442996Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing... 2026-04-07T20:54:56.447243Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:54:56.447257Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-metrics' correctness test case... 2026-04-07T20:54:57.125483Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-metrics/config.yaml'. 2026-04-07T20:54:57.125510Z INFO ground_truth: Test run starting... 2026-04-07T20:54:57.125550Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:54:57.125558Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/7S9Ofk63 2026-04-07T20:54:57.125678Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/3A4ZVXYw 2026-04-07T20:55:17.473838Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:55:58.954998Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:56:01.257443Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:56:01.257478Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:56:01.980027Z INFO ground_truth::analysis::metrics: Analyzing 3106 unfiltered metrics from baseline target, and 3104 unfiltered metrics from comparison target. 2026-04-07T20:56:02.014712Z INFO ground_truth::analysis::metrics: Filtered 69 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:56:02.019094Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3037 unique metrics. Continuing... 2026-04-07T20:56:02.241589Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:56:02.241605Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces' correctness test case... 2026-04-07T20:56:03.007379Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces/config.yaml'. 2026-04-07T20:56:03.007406Z INFO ground_truth: Test run starting... 2026-04-07T20:56:03.007444Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:56:03.007451Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/1cVwBPah 2026-04-07T20:56:03.007577Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/wUNiWwV6 2026-04-07T20:56:23.276549Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:57:03.869534Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:57:06.178544Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:57:06.178560Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:57:06.184466Z INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target. 2026-04-07T20:57:06.234992Z INFO ground_truth::analysis::traces: Analyzing 52 aggregated statistics groups from baseline and comparison target. 2026-04-07T20:57:06.239324Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:57:06.239335Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces-ottl-filtering' correctness test case... 2026-04-07T20:57:06.278783Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-filtering/config.yaml'. 2026-04-07T20:57:06.278797Z INFO ground_truth: Test run starting... 2026-04-07T20:57:06.278819Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:57:06.278822Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/ygy0OQ1f 2026-04-07T20:57:06.278939Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/AmhuzMjv 2026-04-07T20:57:39.420798Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:58:13.764492Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:58:16.096143Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:58:16.096210Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:58:16.105198Z INFO ground_truth::analysis::traces: Analyzing 1000 traces (2096 spans) from baseline and comparison target. 2026-04-07T20:58:16.141387Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:58:16.141407Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces-ottl-transform' correctness test case... 2026-04-07T20:58:16.812249Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-transform/config.yaml'. 2026-04-07T20:58:16.812272Z INFO ground_truth: Test run starting... 2026-04-07T20:58:16.812303Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:58:16.812308Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/DCexIxKF 2026-04-07T20:58:16.812427Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/6KiZgYPM 2026-04-07T20:58:51.020236Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:59:33.373262Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:59:35.683447Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:59:35.683463Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:59:35.688473Z INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target. 2026-04-07T20:59:35.745793Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:59:35.745809Z INFO ground_truth: ground-truth stopped. ``` ## References <!-- Please list any issues closed by this PR. --> <!-- - Closes: <issue link> --> <!-- Any other issues or PRs relevant to this PR? Feel free to list them here. --> --------- Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com>
…togram (#1319) ## Summary <!-- Please provide a brief summary about what this PR does. This should help the reviewers give feedback faster and with higher quality. --> This PR adds supports for the remaining open telemetry metrics (summary, histogram, and exponential histogram). ## Change Type - [ ] Bug fix - [x] New feature - [ ] Non-functional (chore, refactoring, docs) - [ ] Performance ## How did you test this PR? I created a [draft PR in the lading repo](DataDog/lading#1853) that included the appropriate changes to emit the newly added metrics. I had my `Cargo.toml` point to the branch and ran `make build-datadog-intake-image build-millstone-image build-datadog-agent-image` and `make test-correctness` which yielded the following results: ```rust [*] Running 'dsd-plain' correctness test case... 2026-04-07T20:52:54.164747Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-plain/config.yaml'. 2026-04-07T20:52:54.164804Z INFO ground_truth: Test run starting... 2026-04-07T20:52:54.164890Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:52:54.164904Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/QC8smTa1 2026-04-07T20:52:54.165404Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/sTsNyjwJ 2026-04-07T20:53:17.025257Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:53:54.665292Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:53:56.963274Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:53:56.963322Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:53:56.974946Z INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target. 2026-04-07T20:53:56.979673Z INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:53:56.981558Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing... 2026-04-07T20:53:56.984571Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:53:56.984586Z INFO ground_truth: ground-truth stopped. [*] Running 'dsd-origin-detection' correctness test case... 2026-04-07T20:53:57.600962Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-origin-detection/config.yaml'. 2026-04-07T20:53:57.600989Z INFO ground_truth: Test run starting... 2026-04-07T20:53:57.601028Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:53:57.601036Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/Ulzgl0hU 2026-04-07T20:53:57.601180Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/hJrERcxw 2026-04-07T20:54:17.630782Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:54:54.133377Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:54:56.431092Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:54:56.431109Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:54:56.437142Z INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target. 2026-04-07T20:54:56.440374Z INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:54:56.442996Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing... 2026-04-07T20:54:56.447243Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:54:56.447257Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-metrics' correctness test case... 2026-04-07T20:54:57.125483Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-metrics/config.yaml'. 2026-04-07T20:54:57.125510Z INFO ground_truth: Test run starting... 2026-04-07T20:54:57.125550Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:54:57.125558Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/7S9Ofk63 2026-04-07T20:54:57.125678Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/3A4ZVXYw 2026-04-07T20:55:17.473838Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:55:58.954998Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:56:01.257443Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:56:01.257478Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:56:01.980027Z INFO ground_truth::analysis::metrics: Analyzing 3106 unfiltered metrics from baseline target, and 3104 unfiltered metrics from comparison target. 2026-04-07T20:56:02.014712Z INFO ground_truth::analysis::metrics: Filtered 69 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison. 2026-04-07T20:56:02.019094Z INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3037 unique metrics. Continuing... 2026-04-07T20:56:02.241589Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:56:02.241605Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces' correctness test case... 2026-04-07T20:56:03.007379Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces/config.yaml'. 2026-04-07T20:56:03.007406Z INFO ground_truth: Test run starting... 2026-04-07T20:56:03.007444Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:56:03.007451Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/1cVwBPah 2026-04-07T20:56:03.007577Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/wUNiWwV6 2026-04-07T20:56:23.276549Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:57:03.869534Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:57:06.178544Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:57:06.178560Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:57:06.184466Z INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target. 2026-04-07T20:57:06.234992Z INFO ground_truth::analysis::traces: Analyzing 52 aggregated statistics groups from baseline and comparison target. 2026-04-07T20:57:06.239324Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:57:06.239335Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces-ottl-filtering' correctness test case... 2026-04-07T20:57:06.278783Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-filtering/config.yaml'. 2026-04-07T20:57:06.278797Z INFO ground_truth: Test run starting... 2026-04-07T20:57:06.278819Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:57:06.278822Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/ygy0OQ1f 2026-04-07T20:57:06.278939Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/AmhuzMjv 2026-04-07T20:57:39.420798Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:58:13.764492Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:58:16.096143Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:58:16.096210Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:58:16.105198Z INFO ground_truth::analysis::traces: Analyzing 1000 traces (2096 spans) from baseline and comparison target. 2026-04-07T20:58:16.141387Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:58:16.141407Z INFO ground_truth: ground-truth stopped. [*] Running 'otlp-traces-ottl-transform' correctness test case... 2026-04-07T20:58:16.812249Z INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-transform/config.yaml'. 2026-04-07T20:58:16.812272Z INFO ground_truth: Test run starting... 2026-04-07T20:58:16.812303Z INFO ground_truth::runner: Spawning containers for baseline and comparison targets... 2026-04-07T20:58:16.812308Z INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/DCexIxKF 2026-04-07T20:58:16.812427Z INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/6KiZgYPM 2026-04-07T20:58:51.020236Z INFO ground_truth::runner: Containers spawned successfully. Waiting for data... 2026-04-07T20:59:33.373262Z INFO ground_truth::runner: Cleaning up remaining containers and resources... 2026-04-07T20:59:35.683447Z INFO ground_truth::runner: Cleanup complete. 2026-04-07T20:59:35.683463Z INFO ground_truth: Test run complete. Analyzing results... 2026-04-07T20:59:35.688473Z INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target. 2026-04-07T20:59:35.745793Z INFO ground_truth: Analysis complete: no difference detected between baseline and comparison. 2026-04-07T20:59:35.745809Z INFO ground_truth: ground-truth stopped. ``` ## References <!-- Please list any issues closed by this PR. --> <!-- - Closes: <issue link> --> <!-- Any other issues or PRs relevant to this PR? Feel free to list them here. --> --------- Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com> 7ce97ac
preinlein
left a comment
There was a problem hiding this comment.
I realize there's a lot of feedback on this PR, happy to pair or talk offline about em
| } | ||
|
|
||
| pub(super) fn point_profile(seed: u64) -> PointProfile { | ||
| let sample = scaled_unit(seed, 0, 25.0, 1_000_000.0); |
There was a problem hiding this comment.
Are these mins/maxes sourced from any spec or are they arbitrary?
I don't want to over-engineer (we don't necessarily need this to be configurable - if possible, the generator should do the "right" thing where possible) things here so if they're arbitrary, let's comment that.
| } | ||
| } | ||
|
|
||
| fn histogram_bounds_from_profile(profile: PointProfile, n_bounds: usize) -> Vec<f64> { |
There was a problem hiding this comment.
imo this could be a function on PointProfile
I'd like to see some tests for this. There's enough magic numbers & special handling that I'd want this property tested to make sure we end up with the right values.
| bounds | ||
| } | ||
|
|
||
| pub(super) fn zero_threshold_from_profile(profile: PointProfile) -> f64 { |
There was a problem hiding this comment.
similar comment to above
(another reason I want tests is I could see us changing some of these magic numbers over time and I'd want a test in place already)
| } | ||
|
|
||
| fn scaled_unit(seed: u64, shift: u32, min: f64, max: f64) -> f64 { | ||
| let value = ((seed.rotate_right(shift) & 0xFFFF) as f64) / 65_535.0; |
There was a problem hiding this comment.
65535 is the biggest value for a u16, why are we using that here exactly? I'm not following the logic
| } | ||
|
|
||
| fn point_attributes(metadata: &[KeyValue], point_index: usize) -> Vec<KeyValue> { | ||
| let mut attributes = Vec::with_capacity(metadata.len().min(2) + 1); |
There was a problem hiding this comment.
Why are we doing metadata.len().min(2) (the min(2) part) here?
aka why are we exclusively grabbing 2 key-value pairs and nothing else?
| const POINT_INDEX_ATTRIBUTE_KEY: &str = "lading.point_index"; | ||
|
|
||
| #[derive(Clone, Copy, Debug)] | ||
| pub(super) struct PointProfile { |
There was a problem hiding this comment.
Could we put a small description as to what this is used for?
What I'm picking up is that it describes the pool of points we can randomly pick from?
| } | ||
|
|
||
| fn merge_bucket_counts(target: &mut [u64], update: &[u64]) { | ||
| debug_assert_eq!( |
There was a problem hiding this comment.
How do we imagine working with this?
imo I'd prefer to have property tests that exercise this in some way
| while tpl.encoded_len() > budget_before_fetch { | ||
| if !trim_one_data_point(&mut tpl) { | ||
| *budget = budget_before_fetch; | ||
| debug!( |
There was a problem hiding this comment.
Is this problematic or a "best effort" type of thing? Should these logs be error logs?
I wouldn't want silent failures in prod if it's an actual problem of some kind.
| drop(request.resource_metrics.pop()); | ||
| break; | ||
| } | ||
| debug_assert!(required_bytes <= max_bytes); |
There was a problem hiding this comment.
Do we need this to be a debug_assert? Could it be a real assertion?
I'm generally not a fan of debug assertions as I don't really know what to do with them when I encounter them: are they a real problem? Is it ok to silently fail/hit this code path?
Because if "it's fine for this to happen", why have the assertion at all.
If it's not fine, we should know about it.
There was a problem hiding this comment.
I'd like to see more tests for the added functionality in here.
The internals of the opentelemetry metrics are not readily available for individuals working in lading so we need the tests to really help describe the expected behavior of things.
…m, and summary metrics
What does this PR do?
Adds support for the remaining OTLP metrics (Exponential Histogram, Histogram, and Summary) in the correctness tests.
Motivation
Will be used to test whether these metric types are being properly handled by ADP in the Saluki repo when compared to the Agent.
Supports:
Related issues
A list of issues either fixed, containing architectural discussions, otherwise relevant
for this Pull Request.
Additional Notes
Anything else we should know when reviewing?
Listed below is some additional information for the metrics added (how they are constructed and how they are expected to behave):