Skip to content

Add agentless and standalone OpenFeature smoke tests - #12846

Draft
PerfectSlayer wants to merge 1 commit into
bbujon/ffe-agent-connectorfrom
bbujon/ffe-extras
Draft

PerfectSlayer wants to merge 1 commit into
bbujon/ffe-agent-connectorfrom
bbujon/ffe-extras

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

What Does This Do

Extracts the shared OpenFeature smoke test scenarios into AbstractOpenFeatureSmokeTest and runs them in three setups: SDK only (standalone), SDK + agent in agentless mode, and SDK + agent with Remote Configuration. Updates ARCHITECTURE.md.

Motivation

Covers each supported setup of the standalone SDK (APMLP-1512).

Stack

The full change is in #12709. It was split into this stack to ease the review.

  1. Resolve Feature Flags enablement from instrumenter config #12845
  2. Add dd-openfeature SDK flag configuration and evaluation core #12840
  3. Add dd-openfeature SDK settings and configuration delivery #12841
  4. Add dd-openfeature SDK exposure and flag evaluation pipelines #12842
  5. Add dd-openfeature SDK OpenFeature provider #12843
  6. Connect the dd-openfeature SDK to the agent #12844
  7. Add agentless and standalone OpenFeature smoke tests #12846 👈 this PR

Jira ticket: APMLP-1512

🤖 Generated with Claude Code

@datadog-official

This comment has been minimized.

@dd-octo-sts

dd-octo-sts Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.97 s 14.05 s [-1.2%; -0.0%] (maybe better)
startup:insecure-bank:tracing:Agent 12.98 s 13.07 s [-1.3%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent 17.13 s 16.99 s [-0.3%; +1.9%] (no difference)
startup:petclinic:iast:Agent 16.98 s 17.06 s [-1.1%; +0.3%] (no difference)
startup:petclinic:profiling:Agent 16.57 s 16.95 s [-3.2%; -1.2%] (significantly better)
startup:petclinic:sca:Agent 17.04 s 16.82 s [+0.4%; +2.2%] (maybe worse)
startup:petclinic:tracing:Agent 16.26 s 16.05 s [+0.1%; +2.6%] (maybe worse)

Commit: 4db3136f · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

Extract the shared OpenFeature smoke test scenarios and run them against
the SDK alone, the SDK with the agent in agentless mode, and the SDK
with the agent through Remote Configuration. Document the new SDK
layout in ARCHITECTURE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes tag: override groovy enforcement Override the "Enforce Groovy Migration" check type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant