Skip to content

Enable EVP flagevaluation system tests for PHP#7160

Draft
leoromanovsky wants to merge 15 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-php
Draft

Enable EVP flagevaluation system tests for PHP#7160
leoromanovsky wants to merge 15 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-php

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Motivation

tests/ffe/test_flag_eval_evp.py now covers the server-side EVP flagevaluation contract. PHP has a passing local focused run against DataDog/dd-trace-php#3984, so its manifest can move from missing_feature (FFL-2446) to the validated version.

Changes

  • Enables tests/ffe/test_flag_eval_evp.py for PHP in manifests/php.yml at v1.21.0-dev.
  • Keeps the test implementation in the base system-tests PR Add EVP flagevaluation system tests #7146.
  • Keeps this as a sibling PR based on leo.romanovsky/ffe-evp-flagevaluation-system-tests, not stacked on other language enablement branches.

Decisions

  • This PR enables only PHP; other languages are enabled by separate sibling PRs after their own local pass evidence.
  • The degradation case is active for PHP and uses the production per-flag full-tier cap through the base test.
  • Existing OTel metric coverage stays in tests/ffe/test_flag_eval_metrics.py.

Validation

Validated on 2026-06-17 with system-tests PHP enable commit a29bfcfa1b2fa172adfee5f61a2a3424456763ae, base #7146 commit 2e7ad616bc525ee76698f34a686da549701b1914, PHP SDK commit 696c397f3ad7fd9b8b2135e16c9424c49a55da1f, and libdatadog commit 53f81e56510f9981dd7db95a9dc7dec592cb9678.

  • ./build.sh php -w apache-mod-8.2 - PASS.
  • ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION --library php --weblog apache-mod-8.2 -k "Flagevaluation_Degradation" - PASS, 1 passed, 2584 deselected in 34.38s.
  • ./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION --library php --weblog apache-mod-8.2 -k "test_flag_eval_evp" - PASS, 8 passed, 2577 deselected in 63.53s.
  • Agent 7.80.1; library php@1.21.0; weblog apache-mod-8.2; Linux aarch64.
  • Degradation payload evidence from the passing full run: events=10001, total=10050, full_total=10000, degraded_count=1, degraded_total=50.
  • PHP local validation used a PHP 8.2 aarch64 ddtrace.so built from DataDog/dd-trace-php#3984 with libdatadog 53f81e565, installed through the system-tests /binaries/ddtrace.so override.

@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php

@datadog-official

datadog-official Bot commented Jun 17, 2026

Copy link
Copy Markdown

Pipelines  Tests

Fix all issues with BitsAI

⚠️ Warnings

🚦 51 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.0] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7ff2fb765760>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7ff3292c4740>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.0-zts] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f666fbb51c0>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f669ca5c950>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.1] from system_tests_suite   View in Datadog
AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 +  where False = <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')>(<function wait_for_evp_flagevaluation_event.<locals>.<lambda> at 0x7f20f86fe480>, timeout=30)
 +    where <bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator('agent')> = AgentInterfaceValidator('agent').wait_for
 +      where AgentInterfaceValidator('agent') = interfaces.agent

self = <tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7f2124f680e0>

    def test_ffe_evp_flagevaluation_basic(self) -> None:
        assert self.r.status_code == 200, f"Flag evaluation failed: {self.r.text}"
...

View all 51 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a29bfcf | Docs | Datadog PR Page | Give us feedback!

Base automatically changed from leo.romanovsky/ffe-evp-flagevaluation-system-tests to main June 17, 2026 15:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant