Skip to content

feat(otlp-metrics): added support for histogram, exp. histogram, and summary#1853

Open
lucastemb wants to merge 12 commits intomainfrom
lt/add-otlp-metrics
Open

feat(otlp-metrics): added support for histogram, exp. histogram, and summary#1853
lucastemb wants to merge 12 commits intomainfrom
lt/add-otlp-metrics

Conversation

@lucastemb
Copy link
Copy Markdown

@lucastemb lucastemb commented Apr 7, 2026

…m, and summary metrics

What does this PR do?

Adds support for the remaining OTLP metrics (Exponential Histogram, Histogram, and Summary) in the correctness tests.

Motivation

Will be used to test whether these metric types are being properly handled by ADP in the Saluki repo when compared to the Agent.

Supports:

Related issues

A list of issues either fixed, containing architectural discussions, otherwise relevant
for this Pull Request.

Additional Notes

Anything else we should know when reviewing?

Listed below is some additional information for the metrics added (how they are constructed and how they are expected to behave):

@lucastemb lucastemb force-pushed the lt/add-otlp-metrics branch from 512959a to 5bc3dc5 Compare April 15, 2026 18:21
@lucastemb lucastemb changed the title feat(OTLP metrics): added support for histogram, exponential histogra… feat(otlp-metrics): added support for histogram, exp. histogram, and summary Apr 15, 2026
@lucastemb lucastemb marked this pull request as ready for review April 15, 2026 20:09
@lucastemb lucastemb requested a review from a team as a code owner April 15, 2026 20:09
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37bf8c92cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Comment thread lading_payload/src/opentelemetry/metric.rs
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e0421e01c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e5b1c795ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs
Comment thread lading_payload/src/opentelemetry/metric.rs
@lucastemb lucastemb marked this pull request as draft April 18, 2026 00:30
@lucastemb lucastemb marked this pull request as ready for review April 25, 2026 15:39
…summary (#1853)

…m, and summary metrics

Adds support for the remaining OTLP metrics (Exponential Histogram, Histogram, and Summary) in the correctness tests

Will be used to test whether these metric types are being properly handled by ADP in the Saluki repo.

A list of issues either fixed, containing architectural discussions, otherwise relevant
for this Pull Request.

Anything else we should know when reviewing?

Original commits:
* 460ec05 feat(OTLP metrics): added support for histogram, exponential histogram, and summary metrics
* 7cfd22e chore(otlp metrics): changed some params and ran cargo fmt
* d398476 test(otlp metrics): added further coverage for the correctness tests
* 892cf93 fix(otlp metrics): added quick patch to fix issue of duplicate bins
* b02dfdd style(OTLP metrics): applied brief clippy change
* b2e3f1c style(otlp metrics): added line to bypass too many lines to abide by existing conventions
* 5bc3dc5 chore(otlp metrics): added descriptors to CHANGELOG.md
* ebd65a0 fix(otlp metric): removed accidental proptest regression seed file
* 37bf8c9 fix(otlp metrics): restore entropy in histogram metric payloads
* e0421e0 fix(otlp metrics): addressed codex comments about improper construction of summary metrics
* e5b1c79 refactor(otlp): trim scaffolding for new metrics
* 75669c1 refactor(otlp-metrics): removed synthetic labels and redundant tests
* 782831c chore(otlp-metrics): updated vulnerable dependencies
* b7a0e27 Merge branch 'main' into lt/add-otlp-metrics

Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com>
@lucastemb lucastemb force-pushed the lt/add-otlp-metrics branch from b7a0e27 to 996be2d Compare April 25, 2026 15:45
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 996be2d185

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric/templates.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2a1274bcff

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f40bd8983b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: be7e11b3d0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 002ae344dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cb9ff1c104

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc7b3305f0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

return rng.random_range(current..=current.min(max));

P2 Badge Sample the full weighted range

When metric templates sample total_data_points, the loop invariant guarantees current <= max, so current..=current.min(max) is always a single-value range. This helper therefore only returns 1, 2, 4, 8, etc. instead of the documented exponentially weighted buckets like 2-3 and 4-7, which skews generated payload shapes and leaves most point counts in the configured 1-60 range unexercised.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f908b87b17

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31d4db8537

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 228bef9f8b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1af42fdc6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lading_payload/src/opentelemetry/metric.rs
lucastemb added a commit to DataDog/saluki that referenced this pull request Apr 27, 2026
…togram (#1319)

## Summary
<!-- Please provide a brief summary about what this PR does.
This should help the reviewers give feedback faster and with higher
quality. -->

This PR adds supports for the remaining open telemetry metrics (summary,
histogram, and exponential histogram).

## Change Type
- [ ] Bug fix
- [x] New feature
- [ ] Non-functional (chore, refactoring, docs)
- [ ] Performance


## How did you test this PR?
I created a [draft PR in the lading
repo](DataDog/lading#1853) that included the
appropriate changes to emit the newly added metrics.

I had my `Cargo.toml` point to the branch and ran `make
build-datadog-intake-image build-millstone-image
build-datadog-agent-image` and `make test-correctness` which yielded the
following results:

```rust [*] Running 'dsd-plain' correctness test case...
2026-04-07T20:52:54.164747Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-plain/config.yaml'.
2026-04-07T20:52:54.164804Z  INFO ground_truth: Test run starting...
2026-04-07T20:52:54.164890Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:52:54.164904Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/QC8smTa1
2026-04-07T20:52:54.165404Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/sTsNyjwJ
2026-04-07T20:53:17.025257Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:53:54.665292Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:53:56.963274Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:53:56.963322Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:53:56.974946Z  INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target.
2026-04-07T20:53:56.979673Z  INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:53:56.981558Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing...
2026-04-07T20:53:56.984571Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:53:56.984586Z  INFO ground_truth: ground-truth stopped.
[*] Running 'dsd-origin-detection' correctness test case...
2026-04-07T20:53:57.600962Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-origin-detection/config.yaml'.
2026-04-07T20:53:57.600989Z  INFO ground_truth: Test run starting...
2026-04-07T20:53:57.601028Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:53:57.601036Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/Ulzgl0hU
2026-04-07T20:53:57.601180Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/hJrERcxw
2026-04-07T20:54:17.630782Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:54:54.133377Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:54:56.431092Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:54:56.431109Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:54:56.437142Z  INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target.
2026-04-07T20:54:56.440374Z  INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:54:56.442996Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing...
2026-04-07T20:54:56.447243Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:54:56.447257Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-metrics' correctness test case...
2026-04-07T20:54:57.125483Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-metrics/config.yaml'.
2026-04-07T20:54:57.125510Z  INFO ground_truth: Test run starting...
2026-04-07T20:54:57.125550Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:54:57.125558Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/7S9Ofk63
2026-04-07T20:54:57.125678Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/3A4ZVXYw
2026-04-07T20:55:17.473838Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:55:58.954998Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:56:01.257443Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:56:01.257478Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:56:01.980027Z  INFO ground_truth::analysis::metrics: Analyzing 3106 unfiltered metrics from baseline target, and 3104 unfiltered metrics from comparison target.
2026-04-07T20:56:02.014712Z  INFO ground_truth::analysis::metrics: Filtered 69 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:56:02.019094Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3037 unique metrics. Continuing...
2026-04-07T20:56:02.241589Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:56:02.241605Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces' correctness test case...
2026-04-07T20:56:03.007379Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces/config.yaml'.
2026-04-07T20:56:03.007406Z  INFO ground_truth: Test run starting...
2026-04-07T20:56:03.007444Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:56:03.007451Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/1cVwBPah
2026-04-07T20:56:03.007577Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/wUNiWwV6
2026-04-07T20:56:23.276549Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:57:03.869534Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:57:06.178544Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:57:06.178560Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:57:06.184466Z  INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target.
2026-04-07T20:57:06.234992Z  INFO ground_truth::analysis::traces: Analyzing 52 aggregated statistics groups from baseline and comparison target.
2026-04-07T20:57:06.239324Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:57:06.239335Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces-ottl-filtering' correctness test case...
2026-04-07T20:57:06.278783Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-filtering/config.yaml'.
2026-04-07T20:57:06.278797Z  INFO ground_truth: Test run starting...
2026-04-07T20:57:06.278819Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:57:06.278822Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/ygy0OQ1f
2026-04-07T20:57:06.278939Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/AmhuzMjv
2026-04-07T20:57:39.420798Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:58:13.764492Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:58:16.096143Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:58:16.096210Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:58:16.105198Z  INFO ground_truth::analysis::traces: Analyzing 1000 traces (2096 spans) from baseline and comparison target.
2026-04-07T20:58:16.141387Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:58:16.141407Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces-ottl-transform' correctness test case...
2026-04-07T20:58:16.812249Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-transform/config.yaml'.
2026-04-07T20:58:16.812272Z  INFO ground_truth: Test run starting...
2026-04-07T20:58:16.812303Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:58:16.812308Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/DCexIxKF
2026-04-07T20:58:16.812427Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/6KiZgYPM
2026-04-07T20:58:51.020236Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:59:33.373262Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:59:35.683447Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:59:35.683463Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:59:35.688473Z  INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target.
2026-04-07T20:59:35.745793Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:59:35.745809Z  INFO ground_truth: ground-truth stopped.
```

## References

<!-- Please list any issues closed by this PR. -->

<!--
- Closes: <issue link>
-->

<!-- Any other issues or PRs relevant to this PR? Feel free to list them
here. -->

---------

Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com>
dd-octo-sts Bot pushed a commit to DataDog/saluki that referenced this pull request Apr 27, 2026
…togram (#1319)

## Summary
<!-- Please provide a brief summary about what this PR does.
This should help the reviewers give feedback faster and with higher
quality. -->

This PR adds supports for the remaining open telemetry metrics (summary,
histogram, and exponential histogram).

## Change Type
- [ ] Bug fix
- [x] New feature
- [ ] Non-functional (chore, refactoring, docs)
- [ ] Performance

## How did you test this PR?
I created a [draft PR in the lading
repo](DataDog/lading#1853) that included the
appropriate changes to emit the newly added metrics.

I had my `Cargo.toml` point to the branch and ran `make
build-datadog-intake-image build-millstone-image
build-datadog-agent-image` and `make test-correctness` which yielded the
following results:

```rust [*] Running 'dsd-plain' correctness test case...
2026-04-07T20:52:54.164747Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-plain/config.yaml'.
2026-04-07T20:52:54.164804Z  INFO ground_truth: Test run starting...
2026-04-07T20:52:54.164890Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:52:54.164904Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/QC8smTa1
2026-04-07T20:52:54.165404Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/sTsNyjwJ
2026-04-07T20:53:17.025257Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:53:54.665292Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:53:56.963274Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:53:56.963322Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:53:56.974946Z  INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target.
2026-04-07T20:53:56.979673Z  INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:53:56.981558Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing...
2026-04-07T20:53:56.984571Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:53:56.984586Z  INFO ground_truth: ground-truth stopped.
[*] Running 'dsd-origin-detection' correctness test case...
2026-04-07T20:53:57.600962Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/dsd-origin-detection/config.yaml'.
2026-04-07T20:53:57.600989Z  INFO ground_truth: Test run starting...
2026-04-07T20:53:57.601028Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:53:57.601036Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/Ulzgl0hU
2026-04-07T20:53:57.601180Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/hJrERcxw
2026-04-07T20:54:17.630782Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:54:54.133377Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:54:56.431092Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:54:56.431109Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:54:56.437142Z  INFO ground_truth::analysis::metrics: Analyzing 3095 unfiltered metrics from baseline target, and 3095 unfiltered metrics from comparison target.
2026-04-07T20:54:56.440374Z  INFO ground_truth::analysis::metrics: Filtered 67 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:54:56.442996Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3028 unique metrics. Continuing...
2026-04-07T20:54:56.447243Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:54:56.447257Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-metrics' correctness test case...
2026-04-07T20:54:57.125483Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-metrics/config.yaml'.
2026-04-07T20:54:57.125510Z  INFO ground_truth: Test run starting...
2026-04-07T20:54:57.125550Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:54:57.125558Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/7S9Ofk63
2026-04-07T20:54:57.125678Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/3A4ZVXYw
2026-04-07T20:55:17.473838Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:55:58.954998Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:56:01.257443Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:56:01.257478Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:56:01.980027Z  INFO ground_truth::analysis::metrics: Analyzing 3106 unfiltered metrics from baseline target, and 3104 unfiltered metrics from comparison target.
2026-04-07T20:56:02.014712Z  INFO ground_truth::analysis::metrics: Filtered 69 internal telemetry metric(s) from baseline, and 67 internal telemetry metric(s) from comparison.
2026-04-07T20:56:02.019094Z  INFO ground_truth::analysis::metrics: Baseline and comparison both emitted the same set of 3037 unique metrics. Continuing...
2026-04-07T20:56:02.241589Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:56:02.241605Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces' correctness test case...
2026-04-07T20:56:03.007379Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces/config.yaml'.
2026-04-07T20:56:03.007406Z  INFO ground_truth: Test run starting...
2026-04-07T20:56:03.007444Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:56:03.007451Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/1cVwBPah
2026-04-07T20:56:03.007577Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/wUNiWwV6
2026-04-07T20:56:23.276549Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:57:03.869534Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:57:06.178544Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:57:06.178560Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:57:06.184466Z  INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target.
2026-04-07T20:57:06.234992Z  INFO ground_truth::analysis::traces: Analyzing 52 aggregated statistics groups from baseline and comparison target.
2026-04-07T20:57:06.239324Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:57:06.239335Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces-ottl-filtering' correctness test case...
2026-04-07T20:57:06.278783Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-filtering/config.yaml'.
2026-04-07T20:57:06.278797Z  INFO ground_truth: Test run starting...
2026-04-07T20:57:06.278819Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:57:06.278822Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/ygy0OQ1f
2026-04-07T20:57:06.278939Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/AmhuzMjv
2026-04-07T20:57:39.420798Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:58:13.764492Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:58:16.096143Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:58:16.096210Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:58:16.105198Z  INFO ground_truth::analysis::traces: Analyzing 1000 traces (2096 spans) from baseline and comparison target.
2026-04-07T20:58:16.141387Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:58:16.141407Z  INFO ground_truth: ground-truth stopped.
[*] Running 'otlp-traces-ottl-transform' correctness test case...
2026-04-07T20:58:16.812249Z  INFO ground_truth: Loaded test case configuration from '/Users/lucas.tembras/saluki/test/correctness/otlp-traces-ottl-transform/config.yaml'.
2026-04-07T20:58:16.812272Z  INFO ground_truth: Test run starting...
2026-04-07T20:58:16.812303Z  INFO ground_truth::runner: Spawning containers for baseline and comparison targets...
2026-04-07T20:58:16.812308Z  INFO ground_truth::runner: Creating test group runner for target 'baseline'. Logs will be saved to /tmp/ground-truth/DCexIxKF
2026-04-07T20:58:16.812427Z  INFO ground_truth::runner: Creating test group runner for target 'comparison'. Logs will be saved to /tmp/ground-truth/6KiZgYPM
2026-04-07T20:58:51.020236Z  INFO ground_truth::runner: Containers spawned successfully. Waiting for data...
2026-04-07T20:59:33.373262Z  INFO ground_truth::runner: Cleaning up remaining containers and resources...
2026-04-07T20:59:35.683447Z  INFO ground_truth::runner: Cleanup complete.
2026-04-07T20:59:35.683463Z  INFO ground_truth: Test run complete. Analyzing results...
2026-04-07T20:59:35.688473Z  INFO ground_truth::analysis::traces: Analyzing 2000 traces (4096 spans) from baseline and comparison target.
2026-04-07T20:59:35.745793Z  INFO ground_truth: Analysis complete: no difference detected between baseline and comparison.
2026-04-07T20:59:35.745809Z  INFO ground_truth: ground-truth stopped.
```

## References

<!-- Please list any issues closed by this PR. -->

<!--
- Closes: <issue link>
-->

<!-- Any other issues or PRs relevant to this PR? Feel free to list them
here. -->

---------

Signed-off-by: Lucas Tembras <lucas.tembras@datadoghq.com> 7ce97ac
Copy link
Copy Markdown
Contributor

@preinlein preinlein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize there's a lot of feedback on this PR, happy to pair or talk offline about em

}

pub(super) fn point_profile(seed: u64) -> PointProfile {
let sample = scaled_unit(seed, 0, 25.0, 1_000_000.0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these mins/maxes sourced from any spec or are they arbitrary?

I don't want to over-engineer (we don't necessarily need this to be configurable - if possible, the generator should do the "right" thing where possible) things here so if they're arbitrary, let's comment that.

}
}

fn histogram_bounds_from_profile(profile: PointProfile, n_bounds: usize) -> Vec<f64> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo this could be a function on PointProfile

I'd like to see some tests for this. There's enough magic numbers & special handling that I'd want this property tested to make sure we end up with the right values.

bounds
}

pub(super) fn zero_threshold_from_profile(profile: PointProfile) -> f64 {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similar comment to above

(another reason I want tests is I could see us changing some of these magic numbers over time and I'd want a test in place already)

}

fn scaled_unit(seed: u64, shift: u32, min: f64, max: f64) -> f64 {
let value = ((seed.rotate_right(shift) & 0xFFFF) as f64) / 65_535.0;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

65535 is the biggest value for a u16, why are we using that here exactly? I'm not following the logic

}

fn point_attributes(metadata: &[KeyValue], point_index: usize) -> Vec<KeyValue> {
let mut attributes = Vec::with_capacity(metadata.len().min(2) + 1);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we doing metadata.len().min(2) (the min(2) part) here?

aka why are we exclusively grabbing 2 key-value pairs and nothing else?

const POINT_INDEX_ATTRIBUTE_KEY: &str = "lading.point_index";

#[derive(Clone, Copy, Debug)]
pub(super) struct PointProfile {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we put a small description as to what this is used for?

What I'm picking up is that it describes the pool of points we can randomly pick from?

}

fn merge_bucket_counts(target: &mut [u64], update: &[u64]) {
debug_assert_eq!(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we imagine working with this?

imo I'd prefer to have property tests that exercise this in some way

while tpl.encoded_len() > budget_before_fetch {
if !trim_one_data_point(&mut tpl) {
*budget = budget_before_fetch;
debug!(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this problematic or a "best effort" type of thing? Should these logs be error logs?

I wouldn't want silent failures in prod if it's an actual problem of some kind.

drop(request.resource_metrics.pop());
break;
}
debug_assert!(required_bytes <= max_bytes);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this to be a debug_assert? Could it be a real assertion?

I'm generally not a fan of debug assertions as I don't really know what to do with them when I encounter them: are they a real problem? Is it ok to silently fail/hit this code path?

Because if "it's fine for this to happen", why have the assertion at all.
If it's not fine, we should know about it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see more tests for the added functionality in here.

The internals of the opentelemetry metrics are not readily available for individuals working in lading so we need the tests to really help describe the expected behavior of things.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants