Skip to content

Add per-subject evaluation telemetry opt-out - #12884

Open
leoromanovsky wants to merge 3 commits into
masterfrom
leoromanovsky/evaluation-telemetry-opt-out
Open

leoromanovsky wants to merge 3 commits into
masterfrom
leoromanovsky/evaluation-telemetry-opt-out

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Applications need to evaluate flags while excluding selected subjects from evaluation telemetry. This adds the same explicit, per-subject control as the client SDKs. Applications keep ownership of their consent rules.

Changes and Decisions

  • Add Provider.Options.enableEvaluationTelemetryOptOutParameter(boolean), disabled by default.
  • Also support DD_FEATURE_FLAGS_EVALUATION_TELEMETRY_PER_SUBJECT_OPT_OUT_ENABLED=true. An explicit provider option overrides deployment configuration, including false.
  • Honor only boolean true at dd_evaluation_telemetry_opt_out. Missing, false, null, and other types preserve telemetry.
  • Suppress new evaluation EVP events/counts, OTel evaluation metrics, and automatic flag span enrichment.
  • Preserve flag results, defaults, exposures, experimentation, application hooks, unrelated tracing, and queued data.
  • Keep each decision within its evaluation. Concurrent and nested evaluations do not share subject state.
flowchart TD
    C[Per-evaluation subject context] --> P[Datadog provider]
    P --> E[Evaluate flag]
    E --> R[Return flag result]
    E --> X[Existing exposure handling: unchanged]
    E --> G{Evaluation telemetry opted out?}
    G -->|Yes| S[Skip new evaluation telemetry]
    G -->|No| T[Evaluation EVP, OTel metrics, flag span enrichment]
Loading
// Configure once.
OpenFeatureAPI api = OpenFeatureAPI.getInstance();
api.setProviderAndWait(new Provider(
    new Provider.Options().enableEvaluationTelemetryOptOutParameter(true)));
Client client = api.getClient();

// Supply each request's subject decision. Do not replace the global context.
boolean enabled = client.getBooleanValue("my-feature", false,
    new ImmutableContext(user.getId(), Collections.singletonMap(
        "dd_evaluation_telemetry_opt_out",
        new Value(user.hasDeclinedEvaluationTelemetry()))));

The parameter name is identical across languages. This control does not opt a subject out of experiments or exposure logging.

The module's existing coverage gate still fails on the unchanged base and this branch. This PR does not change its thresholds.

@leoromanovsky leoromanovsky added type: feature Enhancements and improvements tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Oct 15, 2026
@dd-octo-sts

dd-octo-sts Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.94 s 13.98 s [-1.1%; +0.5%] (no difference)
startup:insecure-bank:tracing:Agent 12.85 s 12.91 s [-1.1%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 17.20 s 17.08 s [-0.3%; +1.7%] (no difference)
startup:petclinic:iast:Agent 17.04 s 17.04 s [-1.0%; +0.9%] (no difference)
startup:petclinic:profiling:Agent 16.87 s 16.96 s [-1.8%; +0.8%] (no difference)
startup:petclinic:sca:Agent 17.07 s 16.76 s [+1.0%; +2.7%] (significantly worse)
startup:petclinic:tracing:Agent 16.18 s 16.09 s [-0.4%; +1.5%] (no difference)

Commit: f6d47500 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@leoromanovsky
leoromanovsky marked this pull request as ready for review October 15, 2026 12:11
@leoromanovsky
leoromanovsky requested a review from a team as a code owner October 15, 2026 12:11
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T12:15:13.953043Z 39ace88 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-15T12:16:35.864516Z 39ace88 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The opt-out gates evaluation telemetry independently of exposure logging and flag resolution. The completed static review identified no reportable defects.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 39ace88

@leoromanovsky
leoromanovsky requested a review from a team as a code owner October 15, 2026 21:37
@leoromanovsky
leoromanovsky requested review from AlexeyKuznetsov-DD and removed request for a team October 15, 2026 21:37
@datadog-datadog-prod-us1-2

This comment has been minimized.

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The evaluation telemetry gate leaves exposure handling and flag results intact. The completed static review identified no actionable defects.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 3787306

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant