Skip to content

Enable EVP flagevaluation system tests for Node.js#7155

Draft
leoromanovsky wants to merge 15 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-nodejs
Draft

Enable EVP flagevaluation system tests for Node.js#7155
leoromanovsky wants to merge 15 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-nodejs

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Motivation

Node.js now has local passing evidence for the server-side EVP flagevaluation system tests added in #7146. This stacked PR enables the manifest entry for Node.js only, keeping other SDK languages disabled until their own local validation passes.

Changes

  • Enables tests/ffe/test_flag_eval_evp.py for Node.js from v6.0.0.
  • Leaves all other Node.js FFE manifest entries unchanged.
  • Keeps this PR based on leo.romanovsky/ffe-evp-flagevaluation-system-tests, not stacked on another language enablement branch.

Decisions

  • This PR stays draft until the Node.js SDK PR and base system-tests PR are reviewed together.
  • The enablement is language-scoped; other SDK manifests remain disabled until their own evidence is collected.
  • Local validation used the existing Node.js system-tests local tracer workflow with binaries/nodejs-load-from-local pointing at the SDK PR checkout.

Evidence

Local run on 2026-06-17 UTC:

  • SDK commit: 7ed3ffc08913cf25cc6aab8fffb417536851cebe
  • system-tests commit: 739deccd4e8c84e416706eb20315c9f706fcb07f
  • Weblog: nodejs / express4
  • Runtime context: Agent 7.80.1; library nodejs@6.0.0-pre; Linux aarch64

Commands:

./utils/build/build_nodejs_base_images.sh express4
./build.sh nodejs -w express4
./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION -k "test_flag_eval_evp"

Result: 8 passed, 2577 deselected in 57.08s.

Agent-visible EVP checks from /api/v2/flagevaluations payloads:

  • evp-count-flag: 5 evaluations -> 1 event, evaluation_count=5
  • evp-burst-aggregation-flag: 512 evaluations -> 1 event, evaluation_count=512
  • evp-high-cardinality-aggregation-flag: 128 distinct buckets -> 128 events, total evaluation_count=128
  • evp-degradation-flag: 10,050 evaluations -> 10,001 events, including 1 degraded bucket with evaluation_count=50

Local logs:

  • .tmp/evidence/nodejs-build-express4-base-local-20260617T030300Z.log
  • .tmp/evidence/nodejs-build-express4-local-20260617T030359Z.log
  • .tmp/evidence/nodejs-test_flag_eval_evp-local-20260617T030436Z.log

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/nodejs.yml                                                    @DataDog/dd-trace-js

@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Jun 17, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 7 Pipeline jobs failed

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / express4 1   View in Datadog   GitHub Actions

🧪 4 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[express4] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f5d1ee5e700>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f5d4d10fd10>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[express4] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-burst-aggregation-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f5d1f6f6520>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7f5d4d10eb40>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
...
View all 4 test failures

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / express4-typescript 1   View in Datadog   GitHub Actions

🧪 4 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[express4-typescript] from system_tests_suite   View in Datadog
AssertionError: Flag evaluation failed: <!DOCTYPE html>
  <html lang="en">
  <head>
  <meta charset="utf-8">
  <title>Error</title>
  </head>
  <body>
  <pre>Cannot POST /ffe</pre>
  </body>
  </html>
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[express4-typescript] from system_tests_suite   View in Datadog
AssertionError: Request 1 failed: <!DOCTYPE html>
  <html lang="en">
  <head>
  <meta charset="utf-8">
  <title>Error</title>
  </head>
  <body>
  <pre>Cannot POST /ffe</pre>
  </body>
  </html>
...
View all 4 test failures

Testing the test | System Tests (nodejs, dev) / End-to-end #1 / express5 1   View in Datadog   GitHub Actions

🧪 3 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[express5] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7fa959df7c40>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fa9885a13d0>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...
tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[express5] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-burst-aggregation-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7fa959f54fe0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7fa9885a05c0>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
...
View all 3 test failures

View all 7 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 739decc | Docs | Datadog PR Page | Give us feedback!

Base automatically changed from leo.romanovsky/ffe-evp-flagevaluation-system-tests to main June 17, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant