Enable EVP flagevaluation system tests for Ruby

[leoromanovsky]

Motivation

tests/ffe/test_flag_eval_evp.py covers the EVP flagevaluation contract for server-side SDKs. Ruby has a passing focused local run against the implementation PR, including natural degradation over the production per-flag full-tier cap, so its manifest can move from missing_feature (FFL-2446) to the validated version.

Changes

• Enables tests/ffe/test_flag_eval_evp.py for Ruby in manifests/ruby.yml at v2.36.0-dev.
• Leaves the test implementation unchanged from Add EVP flagevaluation system tests #7146.
• Keeps this as a sibling PR based on leo.romanovsky/ffe-evp-flagevaluation-system-tests, not stacked on other language enablement branches.

Decisions

• This PR enables only Ruby; other languages are enabled by separate sibling PRs after their own local pass evidence.
• Existing OTel metric coverage stays in tests/ffe/test_flag_eval_metrics.py.

Validation

• Date: 2026-06-17
• SDK PR: feat(open_feature): emit server-side EVP flagevaluation dd-trace-rb#5896
• SDK commit: c9a1399d7b066de72c78fc4d6cbb3a0ce5a986d2
• system-tests commit: dcaf396e91980abca8e7b8a58aeb1b576dca063a
• Command: ./build.sh ruby -w rails72
• Result: PASS
• Command: ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION -k "test_flag_eval_evp"
• Result: PASS, 8 passed, 2577 deselected in 50.61s
• Runtime: Agent 7.80.1; library ruby@2.36.0-dev; weblog rails72; Linux aarch64
• Payload evidence from agent flagevaluation intake:
  • evp-count-flag: 1 event, total evaluation_count=5
  • evp-burst-aggregation-flag: 1 event, total evaluation_count=512
  • evp-high-cardinality-aggregation-flag: 128 events, total evaluation_count=128
  • evp-degradation-flag: 10,001 events, total evaluation_count=10050, with 10,000 full rows and one degraded bucket carrying evaluation_count=50

[github-actions]

CODEOWNERS have been resolved as:

manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby

[datadog-prod-us1-3]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 13 Pipeline jobs failed

Testing the test | System Tests (ruby, dev) / End-to-end #1 / rack 1     

🧪 2 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[rack] from system_tests_suite  

AssertionError: Flag evaluation failed: not found
assert 404 == 200
 +  where 404 = HttpResponse(status_code:404, headers:{'Content-Type': 'text/plain', 'Content-Length': '9'}, text:not found).status_code
 +    where HttpResponse(status_code:404, headers:{'Content-Type': 'text/plain', 'Content-Length': '9'}, text:not found) = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fc119a65cd0>.r

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fc119a65cd0>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
>       assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
E       AssertionError: Flag evaluation failed: not found
...

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[rack] from system_tests_suite  

AssertionError: Request 1 failed: not found
assert 404 == 200
 +  where 404 = HttpResponse(status_code:404, headers:{'Content-Type': 'text/plain', 'Content-Length': '9'}, text:not found).status_code

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation object at 0x7fc119a66120>

    def test_ffe_evp_flagevaluation_burst_aggregation(self) -> None:
        for index, response in enumerate(self.responses):
>           assert response.status_code == 200, f"Request {index + 1} failed: {response.text}"
E           AssertionError: Request 1 failed: not found
...

Testing the test | System Tests (ruby, dev) / End-to-end #1 / rails72 1     

🧪 3 Tests failed

tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key.test_ffe_exp_5_missing_targeting_key[rails72] from system_tests_suite  

AssertionError: EXP.5 FAILED: Expected exposure event for flag 'exp-5-missing-targeting-key-flag' with subject.id = '', but no matching event was found. Events received for this flag: []. The tracer must NOT skip exposures when targeting key is empty.
assert None is not None

self = <tests.ffe.test_exposures.Test_FFE_EXP_5_Missing_Targeting_Key object at 0x7f0aaefb8bc0>

    def test_ffe_exp_5_missing_targeting_key(self):
        """EXP.5: Test that empty targeting key generates exposure with subject.id = ''."""
        assert self.response.status_code == 200, f"Flag evaluation failed: {self.response.text}"
    
        result = json.loads(self.response.text)
...

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[rails72] from system_tests_suite  

AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f0a77676980>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f0aae6cefc0>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...

View all 3 test failures

Testing the test | System Tests (ruby, dev) / End-to-end #1 / sinatra14 1     

🧪 2 Tests failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[sinatra14] from system_tests_suite  

AssertionError: Flag evaluation failed: <!DOCTYPE html>
  <html>
  <head>
    <style type="text/css">
    body { text-align:center;font-family:helvetica,arial;font-size:22px;
      color:#888;margin:20px}
    #c {margin:0 auto;width:500px;text-align:left}
    </style>
  </head>
  <body>
...

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Burst_Aggregation.test_ffe_evp_flagevaluation_burst_aggregation[sinatra14] from system_tests_suite  

AssertionError: Request 1 failed: <!DOCTYPE html>
  <html>
  <head>
    <style type="text/css">
    body { text-align:center;font-family:helvetica,arial;font-size:22px;
      color:#888;margin:20px}
    #c {margin:0 auto;width:500px;text-align:left}
    </style>
  </head>
  <body>
...

View all 13 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: dcaf396 | Docs | Datadog PR Page | Give us feedback!}