Skip to content

feat(ffe): add future-experiments-only opt-out mode - #2710

Merged
leoromanovsky merged 1 commit into
leoromanovsky/experimentation-opt-out-immediatefrom
leoromanovsky/experimentation-opt-out-future
Oct 22, 2026
Merged

feat(ffe): add future-experiments-only opt-out mode#2710
leoromanovsky merged 1 commit into
leoromanovsky/experimentation-opt-out-immediatefrom
leoromanovsky/experimentation-opt-out-future

Conversation

@leoromanovsky

@leoromanovsky leoromanovsky commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Adds FutureExperimentsOnly evaluation and an enum-based C entry point using the existing dd_experimentation_opt_out_at context attribute. The existing immediate-mode C interface remains unchanged.

Motivation

Customers can stop participation in future experiments while allowing older allocations to continue evaluation. Allocations activated at or after the subject's opt-out time are skipped; missing or null activation does not qualify an allocation as older.

Additional Notes

Stack: immediate opt-out #2698 → this PR → Ruby native integration on dd-trace-rb #6521.
This adds randomizationActivatedAt decoding with per-flag error isolation and pins shared fixtures #52.
The cutoff matches Go #5700 and Java #12950; it does not establish previous subject enrollment or change evaluation telemetry.

How to test the change?

Tests cover all three modes, nanosecond boundaries, timezone offsets, malformed input, unknown activation, ordinary fallback, exposure selection, and both C entry points.
The shared suite exercises 361 baseline cases, 26 immediate cases, and 42 future cases.
Packaged C-example execution and Ruby process-level system tests remain pending because local disk space is exhausted.

@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 944 documentation warning(s) found

📦 libdd-ffe-ffi - 553 warning(s)

📦 libdd-ffe-test-suite - ✅ No warnings

📦 libdd-ffe - 391 warning(s)

Updated: 2026-10-21 23:10:18 UTC | Commit: 1dac1e4 | missing-docs job results

@github-actions

github-actions Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 19 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-ffe-ffi - 6 error(s)

Show output 
info: syncing channel updates for 1.87.0-x86_64-unknown-linux-gnu
info: latest update on 2025-05-15 for version 1.87.0 (17067e9ac 2025-05-09)
info: downloading 6 components
warn: the missing active toolchain `1.87.0-x86_64-unknown-linux-gnu` has been auto-installed
warn: this might cause rustup commands to take longer time to finish than expected
info: you may opt out with `RUSTUP_AUTO_INSTALL=0` or `rustup set auto-install disable`
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:25:1
   │
25 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v2.0.0
         └── libdd-ffe-ffi v1.0.2

error[vulnerability]: TLS 1.3 handshake messages incorrectly accepted across encryption level boundaries
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:195:1
    │
195 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0285
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0285
    ├ Rustls accepted TLS 1.3 handshake messages sent at the wrong encryption level
      when they followed a key-changing message in the same record. For example,
      a plaintext `EncryptedExtensions` message packed into the same record as the
      `ServerHello` was accepted.
      
      RFC 8446 section 5.1 requires that handshake messages do not span key changes,
      and that implementations terminate the connection with an "unexpected_message"
      alert if they do.
      
      The handshake transcript is still authenticated, so a network-position attacker
      cannot use this to alter or complete a handshake; the practical effect is that
      a peer could send handshake messages that should be encrypted in plaintext
      without rustls rejecting the connection.
      
      This is functionally the same bug as Go's
      [GO-2026-4340](https://pkg.go.dev/vuln/GO-2026-4340) (CVE-2025-61730).
    ├ Announcement: https://github.com/rustls/rustls/pull/3265
    ├ Solution: Upgrade to >=0.23.45 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── libdd-ffe-ffi v1.0.2
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-common-ffi v46.0.0
      │       │   └── libdd-ffe-ffi v1.0.2 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: `Debug` impls for client session values expose TLS secrets
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:195:1
    │
195 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0346
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0346
    ├ The `Debug` implementations of `Tls12ClientSessionValue` and
      `Tls13ClientSessionValue` were derived, and so included the TLS 1.2 master
      secret or TLS 1.3 resumption PSK in their output.
      
      Custom `ClientSessionStore` implementations that format these values with
      `Debug` (for example, when logging) could disclose these secrets to readers of
      the logs. This conflicts with rustls's
      [key-material redaction policy](https://github.com/rustls/rustls/blob/99f2358cae2954837dbb866faf6727de75489ab9/SECURITY.md#boundary-public-api).
      
      Ordinary connection and built-in session cache `Debug` output is redacted and
      is not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3299
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── libdd-ffe-ffi v1.0.2
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-common-ffi v46.0.0
      │       │   └── libdd-ffe-ffi v1.0.2 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Brotli certificate decompression can allocate up to 1 GiB before authentication
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:195:1
    │
195 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0348
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0348
    ├ The built-in Brotli certificate decompressor (enabled by the non-default
      `brotli` crate feature) bounded the size of the decompressed output, but
      accepted a peer-selected history window. Brotli's large-window extension was
      enabled, so a five-byte header could cause an allocation of 1 GiB. A peer can
      abort a process whose allocator cannot satisfy that request.
      
      No trusted certificate is required by the attacker. Affected configurations are:
      
      - clients that enable the `brotli` feature and connect to a malicious server, and
      - servers that enable the `brotli` feature and request client certificates.
      
      Applications that do not enable the `brotli` feature are not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3297
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── libdd-ffe-ffi v1.0.2
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-common-ffi v46.0.0
      │       │   └── libdd-ffe-ffi v1.0.2 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Unauthenticated ServerHello can downgrade later connections to classical key exchange
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:195:1
    │
195 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0349
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0349
    ├ The client recorded the server-selected key exchange group for each server
      name when processing the ServerHello, before the server was authenticated.
      This record is used to predict which key share to send in future ClientHellos
      to that server name.
      
      A network attacker can answer one connection attempt with a ServerHello
      selecting X25519, and then close the connection. Later ClientHellos for that
      name carry only an X25519 key share, while `supported_groups` still lists
      X25519MLKEM768 first. If the genuine server accepts that share rather than
      sending a HelloRetryRequest for its preferred hybrid group, the handshake
      completes with classical-only key exchange and no error. This persists for as
      long as the entry stays in the client's session store.
      
      The impact is loss of post-quantum confidentiality for affected connections
      ("harvest now, decrypt later"). There is no key compromise and no
      authentication bypass.
      
      The following are not affected:
      
      - connections to servers which send a HelloRetryRequest for their preferred group,
      - clients with resumption disabled (as nothing is stored), and
      - other server names.
      
      This behaviour is discussed in
      [draft-ietf-tls-key-share-prediction](https://www.ietf.org/archive/id/draft-ietf-tls-key-share-prediction-01.html#name-security-considerations),
      so the practical impact depends largely on server policy.
    ├ Announcement: https://github.com/rustls/rustls/pull/3355
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── libdd-ffe-ffi v1.0.2
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-common-ffi v46.0.0
      │       │   └── libdd-ffe-ffi v1.0.2 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Certificate verification may accept distrusted intermediates
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:200:1
    │
200 │ rustls-webpki 0.103.13 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0382
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0382
    ├ Affected versions may accept certificate paths containing distrusted emSign
      intermediate public keys when the configured trust anchors allow such paths.
      The private keys corresponding to these intermediates are now well known.
      
      Versions 0.103.16 and 0.104.0-alpha.8 reject paths containing these public keys.
    ├ Announcement: https://github.com/rustls/webpki/pull/546
    ├ Solution: Upgrade to >=0.103.16, <0.104.0-alpha.1 OR >=0.104.0-alpha.8 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.13
      ├── rustls v0.23.37
      │   ├── hyper-rustls v0.27.7
      │   │   └── libdd-common v8.0.0
      │   │       ├── libdd-capabilities-impl v7.0.0
      │   │       │   ├── (dev) libdd-ffe v2.0.0
      │   │       │   │   └── libdd-ffe-ffi v1.0.2
      │   │       │   └── libdd-remote-config v7.0.0
      │   │       │       ├── libdd-ffe v2.0.0 (*)
      │   │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │   │       ├── libdd-common-ffi v46.0.0
      │   │       │   └── libdd-ffe-ffi v1.0.2 (*)
      │   │       ├── libdd-ffe v2.0.0 (*)
      │   │       └── libdd-remote-config v7.0.0 (*)
      │   ├── libdd-common v8.0.0 (*)
      │   ├── rustls-platform-verifier v0.6.2
      │   │   └── libdd-common v8.0.0 (*)
      │   └── tokio-rustls v0.26.0
      │       └── hyper-rustls v0.27.7 (*)
      └── rustls-platform-verifier v0.6.2 (*)

advisories FAILED, bans ok, sources ok

📦 libdd-ffe-test-suite - 6 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:24:1
   │
24 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v2.0.0
         └── (dev) libdd-ffe-test-suite v46.0.0

error[vulnerability]: TLS 1.3 handshake messages incorrectly accepted across encryption level boundaries
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:194:1
    │
194 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0285
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0285
    ├ Rustls accepted TLS 1.3 handshake messages sent at the wrong encryption level
      when they followed a key-changing message in the same record. For example,
      a plaintext `EncryptedExtensions` message packed into the same record as the
      `ServerHello` was accepted.
      
      RFC 8446 section 5.1 requires that handshake messages do not span key changes,
      and that implementations terminate the connection with an "unexpected_message"
      alert if they do.
      
      The handshake transcript is still authenticated, so a network-position attacker
      cannot use this to alter or complete a handshake; the practical effect is that
      a peer could send handshake messages that should be encrypted in plaintext
      without rustls rejecting the connection.
      
      This is functionally the same bug as Go's
      [GO-2026-4340](https://pkg.go.dev/vuln/GO-2026-4340) (CVE-2025-61730).
    ├ Announcement: https://github.com/rustls/rustls/pull/3265
    ├ Solution: Upgrade to >=0.23.45 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── (dev) libdd-ffe-test-suite v46.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: `Debug` impls for client session values expose TLS secrets
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:194:1
    │
194 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0346
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0346
    ├ The `Debug` implementations of `Tls12ClientSessionValue` and
      `Tls13ClientSessionValue` were derived, and so included the TLS 1.2 master
      secret or TLS 1.3 resumption PSK in their output.
      
      Custom `ClientSessionStore` implementations that format these values with
      `Debug` (for example, when logging) could disclose these secrets to readers of
      the logs. This conflicts with rustls's
      [key-material redaction policy](https://github.com/rustls/rustls/blob/99f2358cae2954837dbb866faf6727de75489ab9/SECURITY.md#boundary-public-api).
      
      Ordinary connection and built-in session cache `Debug` output is redacted and
      is not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3299
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── (dev) libdd-ffe-test-suite v46.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Brotli certificate decompression can allocate up to 1 GiB before authentication
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:194:1
    │
194 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0348
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0348
    ├ The built-in Brotli certificate decompressor (enabled by the non-default
      `brotli` crate feature) bounded the size of the decompressed output, but
      accepted a peer-selected history window. Brotli's large-window extension was
      enabled, so a five-byte header could cause an allocation of 1 GiB. A peer can
      abort a process whose allocator cannot satisfy that request.
      
      No trusted certificate is required by the attacker. Affected configurations are:
      
      - clients that enable the `brotli` feature and connect to a malicious server, and
      - servers that enable the `brotli` feature and request client certificates.
      
      Applications that do not enable the `brotli` feature are not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3297
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── (dev) libdd-ffe-test-suite v46.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Unauthenticated ServerHello can downgrade later connections to classical key exchange
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:194:1
    │
194 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0349
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0349
    ├ The client recorded the server-selected key exchange group for each server
      name when processing the ServerHello, before the server was authenticated.
      This record is used to predict which key share to send in future ClientHellos
      to that server name.
      
      A network attacker can answer one connection attempt with a ServerHello
      selecting X25519, and then close the connection. Later ClientHellos for that
      name carry only an X25519 key share, while `supported_groups` still lists
      X25519MLKEM768 first. If the genuine server accepts that share rather than
      sending a HelloRetryRequest for its preferred hybrid group, the handshake
      completes with classical-only key exchange and no error. This persists for as
      long as the entry stays in the client's session store.
      
      The impact is loss of post-quantum confidentiality for affected connections
      ("harvest now, decrypt later"). There is no key compromise and no
      authentication bypass.
      
      The following are not affected:
      
      - connections to servers which send a HelloRetryRequest for their preferred group,
      - clients with resumption disabled (as nothing is stored), and
      - other server names.
      
      This behaviour is discussed in
      [draft-ietf-tls-key-share-prediction](https://www.ietf.org/archive/id/draft-ietf-tls-key-share-prediction-01.html#name-security-considerations),
      so the practical impact depends largely on server policy.
    ├ Announcement: https://github.com/rustls/rustls/pull/3355
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   │   └── (dev) libdd-ffe-test-suite v46.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Certificate verification may accept distrusted intermediates
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:199:1
    │
199 │ rustls-webpki 0.103.13 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0382
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0382
    ├ Affected versions may accept certificate paths containing distrusted emSign
      intermediate public keys when the configured trust anchors allow such paths.
      The private keys corresponding to these intermediates are now well known.
      
      Versions 0.103.16 and 0.104.0-alpha.8 reject paths containing these public keys.
    ├ Announcement: https://github.com/rustls/webpki/pull/546
    ├ Solution: Upgrade to >=0.103.16, <0.104.0-alpha.1 OR >=0.104.0-alpha.8 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.13
      ├── rustls v0.23.37
      │   ├── hyper-rustls v0.27.7
      │   │   └── libdd-common v8.0.0
      │   │       ├── libdd-capabilities-impl v7.0.0
      │   │       │   ├── (dev) libdd-ffe v2.0.0
      │   │       │   │   └── (dev) libdd-ffe-test-suite v46.0.0
      │   │       │   └── libdd-remote-config v7.0.0
      │   │       │       ├── libdd-ffe v2.0.0 (*)
      │   │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │   │       ├── libdd-ffe v2.0.0 (*)
      │   │       └── libdd-remote-config v7.0.0 (*)
      │   ├── libdd-common v8.0.0 (*)
      │   ├── rustls-platform-verifier v0.6.2
      │   │   └── libdd-common v8.0.0 (*)
      │   └── tokio-rustls v0.26.0
      │       └── hyper-rustls v0.27.7 (*)
      └── rustls-platform-verifier v0.6.2 (*)

advisories FAILED, bans ok, sources ok

📦 libdd-ffe - 7 error(s)

Show output 
error[unmaintained]: Bincode is unmaintained
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:22:1
   │
22 │ bincode 1.3.3 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unmaintained advisory detected
   │
   ├ ID: RUSTSEC-2025-0141
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2025-0141
   ├ Due to a doxxing and harassment incident, the bincode team has taken the decision to cease development permanently.
     
     The team considers version 1.3.3 a complete version of bincode that is not in need of any updates.
     
     ## Alternatives to consider
     
     * [wincode](https://crates.io/crates/wincode)
     * [postcard](https://crates.io/crates/postcard)
     * [bitcode](https://crates.io/crates/bitcode)
     * [rkyv](https://crates.io/crates/rkyv)
   ├ Announcement: https://git.sr.ht/~stygianentity/bincode/tree/v3.0/item/README.md
   ├ Solution: No safe upgrade is available!
   ├ bincode v1.3.3
     └── (dev) libdd-ffe v2.0.0

error[unsound]: Potential use-after-free due to lack of panic safety in `LruCache::pop()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:128:1
    │
128 │ lru 0.16.4 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0253
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0253
    ├ `LruCache::pop()` in `lru` was not panic-safe. If the `Drop` implementation of a stored key panics during `pop()`, `self.detach()` is never called, leaving dangling pointers in the internal doubly-linked list.
      
      A subsequent cache operation that triggers eviction can then dereference these dangling pointers:
      - The node is freed from the map, but remains linked in the LRU list due to the skipped `detach()` call
      - When a new insertion causes eviction, the LRU traversal encounters the dangling pointer
      - This results in a write to already-freed memory during the eviction process
      
      ## Impact
      
      - **CWE-416 (Use-After-Free):** memory corruption when subsequent cache operations access freed node pointers in the linked list
      - **CWE-415 (Double Free):** potential heap corruption when the same memory is freed multiple times
      
      Both types of undefined behavior can be invoked in safe Rust, but only if unwinding panics are enabled and `std::panic::catch_unwind` is used with key types that have potentially-panicking `Drop` implementations.
      
      ## Fix
      
      Fixed in `lru` 0.18.2 by detaching the node from the linked list before freeing it and dropping the key ([lru-rs#238](https://github.com/jeromefroe/lru-rs/pull/238)).
    ├ Announcement: https://github.com/jeromefroe/lru-rs/pull/238
    ├ Solution: Upgrade to >=0.18.2 (try `cargo update -p lru`)
    ├ lru v0.16.4
      └── libdd-ffe v2.0.0

error[vulnerability]: TLS 1.3 handshake messages incorrectly accepted across encryption level boundaries
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:170:1
    │
170 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0285
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0285
    ├ Rustls accepted TLS 1.3 handshake messages sent at the wrong encryption level
      when they followed a key-changing message in the same record. For example,
      a plaintext `EncryptedExtensions` message packed into the same record as the
      `ServerHello` was accepted.
      
      RFC 8446 section 5.1 requires that handshake messages do not span key changes,
      and that implementations terminate the connection with an "unexpected_message"
      alert if they do.
      
      The handshake transcript is still authenticated, so a network-position attacker
      cannot use this to alter or complete a handshake; the practical effect is that
      a peer could send handshake messages that should be encrypted in plaintext
      without rustls rejecting the connection.
      
      This is functionally the same bug as Go's
      [GO-2026-4340](https://pkg.go.dev/vuln/GO-2026-4340) (CVE-2025-61730).
    ├ Announcement: https://github.com/rustls/rustls/pull/3265
    ├ Solution: Upgrade to >=0.23.45 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: `Debug` impls for client session values expose TLS secrets
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:170:1
    │
170 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0346
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0346
    ├ The `Debug` implementations of `Tls12ClientSessionValue` and
      `Tls13ClientSessionValue` were derived, and so included the TLS 1.2 master
      secret or TLS 1.3 resumption PSK in their output.
      
      Custom `ClientSessionStore` implementations that format these values with
      `Debug` (for example, when logging) could disclose these secrets to readers of
      the logs. This conflicts with rustls's
      [key-material redaction policy](https://github.com/rustls/rustls/blob/99f2358cae2954837dbb866faf6727de75489ab9/SECURITY.md#boundary-public-api).
      
      Ordinary connection and built-in session cache `Debug` output is redacted and
      is not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3299
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Brotli certificate decompression can allocate up to 1 GiB before authentication
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:170:1
    │
170 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0348
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0348
    ├ The built-in Brotli certificate decompressor (enabled by the non-default
      `brotli` crate feature) bounded the size of the decompressed output, but
      accepted a peer-selected history window. Brotli's large-window extension was
      enabled, so a five-byte header could cause an allocation of 1 GiB. A peer can
      abort a process whose allocator cannot satisfy that request.
      
      No trusted certificate is required by the attacker. Affected configurations are:
      
      - clients that enable the `brotli` feature and connect to a malicious server, and
      - servers that enable the `brotli` feature and request client certificates.
      
      Applications that do not enable the `brotli` feature are not affected.
    ├ Announcement: https://github.com/rustls/rustls/pull/3297
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Unauthenticated ServerHello can downgrade later connections to classical key exchange
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:170:1
    │
170 │ rustls 0.23.37 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0349
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0349
    ├ The client recorded the server-selected key exchange group for each server
      name when processing the ServerHello, before the server was authenticated.
      This record is used to predict which key share to send in future ClientHellos
      to that server name.
      
      A network attacker can answer one connection attempt with a ServerHello
      selecting X25519, and then close the connection. Later ClientHellos for that
      name carry only an X25519 key share, while `supported_groups` still lists
      X25519MLKEM768 first. If the genuine server accepts that share rather than
      sending a HelloRetryRequest for its preferred hybrid group, the handshake
      completes with classical-only key exchange and no error. This persists for as
      long as the entry stays in the client's session store.
      
      The impact is loss of post-quantum confidentiality for affected connections
      ("harvest now, decrypt later"). There is no key compromise and no
      authentication bypass.
      
      The following are not affected:
      
      - connections to servers which send a HelloRetryRequest for their preferred group,
      - clients with resumption disabled (as nothing is stored), and
      - other server names.
      
      This behaviour is discussed in
      [draft-ietf-tls-key-share-prediction](https://www.ietf.org/archive/id/draft-ietf-tls-key-share-prediction-01.html#name-security-considerations),
      so the practical impact depends largely on server policy.
    ├ Announcement: https://github.com/rustls/rustls/pull/3355
    ├ Solution: Upgrade to >=0.23.46 (try `cargo update -p rustls`)
    ├ rustls v0.23.37
      ├── hyper-rustls v0.27.7
      │   └── libdd-common v8.0.0
      │       ├── libdd-capabilities-impl v7.0.0
      │       │   ├── (dev) libdd-ffe v2.0.0
      │       │   └── libdd-remote-config v7.0.0
      │       │       ├── libdd-ffe v2.0.0 (*)
      │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │       ├── libdd-ffe v2.0.0 (*)
      │       └── libdd-remote-config v7.0.0 (*)
      ├── libdd-common v8.0.0 (*)
      ├── rustls-platform-verifier v0.6.2
      │   └── libdd-common v8.0.0 (*)
      └── tokio-rustls v0.26.0
          └── hyper-rustls v0.27.7 (*)

error[vulnerability]: Certificate verification may accept distrusted intermediates
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:175:1
    │
175 │ rustls-webpki 0.103.13 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0382
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0382
    ├ Affected versions may accept certificate paths containing distrusted emSign
      intermediate public keys when the configured trust anchors allow such paths.
      The private keys corresponding to these intermediates are now well known.
      
      Versions 0.103.16 and 0.104.0-alpha.8 reject paths containing these public keys.
    ├ Announcement: https://github.com/rustls/webpki/pull/546
    ├ Solution: Upgrade to >=0.103.16, <0.104.0-alpha.1 OR >=0.104.0-alpha.8 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.13
      ├── rustls v0.23.37
      │   ├── hyper-rustls v0.27.7
      │   │   └── libdd-common v8.0.0
      │   │       ├── libdd-capabilities-impl v7.0.0
      │   │       │   ├── (dev) libdd-ffe v2.0.0
      │   │       │   └── libdd-remote-config v7.0.0
      │   │       │       ├── libdd-ffe v2.0.0 (*)
      │   │       │       └── (dev) libdd-remote-config v7.0.0 (*)
      │   │       ├── libdd-ffe v2.0.0 (*)
      │   │       └── libdd-remote-config v7.0.0 (*)
      │   ├── libdd-common v8.0.0 (*)
      │   ├── rustls-platform-verifier v0.6.2
      │   │   └── libdd-common v8.0.0 (*)
      │   └── tokio-rustls v0.26.0
      │       └── hyper-rustls v0.27.7 (*)
      └── rustls-platform-verifier v0.6.2 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-10-21 23:12:02 UTC | Commit: 1dac1e4 | dependency-check job results

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Oct 21, 2026 •
edited by datadog-prod-us1-5 Bot
Loading

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 95.00%
• Overall Coverage: 78.02% (+0.43%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ec56508 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-10-21 23:20:52

Comparing candidate commit ec56508 in PR branch leoromanovsky/experimentation-opt-out-future with baseline commit 0200047 in branch leoromanovsky/experimentation-opt-out-immediate.

📊 Benchmarking dashboard

Found 0 performance improvements and 4 performance regressions! Performance is the same for 10 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:flagevaluation_evp/coalescer/stress/10flags_1000users_250fields

  • 🟥 execution_time [+14.534ms; +15.167ms] or [+7.651%; +7.984%]
  • 🟥 throughput [-389.770op/s; -373.737op/s] or [-7.404%; -7.099%]

scenario:flagevaluation_evp/payloads/stress/10flags_1000users_250fields

  • 🟥 execution_time [+15.081ms; +15.228ms] or [+8.353%; +8.435%]
  • 🟥 throughput [-430.998op/s; -426.869op/s] or [-7.781%; -7.707%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz ec56508 1792624034 leoromanovsky/experimentation-opt-out-future
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 157.326ns 160.002ns ± 2.112ns 159.533ns ± 1.307ns 161.103ns 163.771ns 166.304ns 169.020ns 5.95% 1.218 1.604 1.32% 0.149ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [159.710ns; 160.295ns] or [-0.183%; +0.183%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz ec56508 1792624034 leoromanovsky/experimentation-opt-out-future
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time 34.655ms 35.197ms ± 0.194ms 35.196ms ± 0.117ms 35.322ms 35.526ms 35.619ms 35.676ms 1.36% -0.166 -0.022 0.55% 0.014ms 1 200
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput 70074.576op/s 71031.571op/s ± 391.582op/s 71030.605op/s ± 236.042op/s 71244.187op/s 71717.747op/s 72040.231op/s 72140.179op/s 1.56% 0.198 -0.004 0.55% 27.689op/s 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time 194.594ms 195.693ms ± 0.402ms 195.673ms ± 0.285ms 195.968ms 196.336ms 196.687ms 197.092ms 0.72% 0.172 0.277 0.20% 0.028ms 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput 5073.785op/s 5110.056op/s ± 10.482op/s 5110.555op/s ± 7.450op/s 5117.159op/s 5127.387op/s 5132.139op/s 5138.898op/s 0.55% -0.158 0.264 0.20% 0.741op/s 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time 846.527µs 848.839µs ± 1.193µs 848.641µs ± 0.690µs 849.481µs 851.156µs 851.882µs 852.203µs 0.42% 0.652 -0.009 0.14% 0.084µs 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput 117343.000op/s 117808.217op/s ± 165.476op/s 117835.471op/s ± 95.756op/s 117922.430op/s 118042.259op/s 118086.997op/s 118129.771op/s 0.25% -0.646 -0.019 0.14% 11.701op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time [35.170ms; 35.224ms] or [-0.076%; +0.076%] None None None
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput [70977.302op/s; 71085.841op/s] or [-0.076%; +0.076%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time [195.638ms; 195.749ms] or [-0.028%; +0.028%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput [5108.603op/s; 5111.508op/s] or [-0.028%; +0.028%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time [848.673µs; 849.004µs] or [-0.019%; +0.019%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput [117785.284op/s; 117831.151op/s] or [-0.019%; +0.019%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz ec56508 1792624034 leoromanovsky/experimentation-opt-out-future
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 265.089µs 271.484µs ± 12.794µs 268.379µs ± 1.378µs 270.538µs 284.356µs 322.862µs 402.747µs 50.07% 6.899 59.354 4.70% 0.905µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [269.711µs; 273.257µs] or [-0.653%; +0.653%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz ec56508 1792624034 leoromanovsky/experimentation-opt-out-future
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time 31.818ms 31.977ms ± 0.070ms 31.980ms ± 0.045ms 32.024ms 32.064ms 32.168ms 32.219ms 0.75% 0.052 0.493 0.22% 0.005ms 1 200
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput 77592.844op/s 78182.333op/s ± 170.138op/s 78173.631op/s ± 110.205op/s 78288.436op/s 78464.615op/s 78572.217op/s 78572.418op/s 0.51% -0.036 0.467 0.22% 12.031op/s 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time 203.781ms 204.818ms ± 1.608ms 204.307ms ± 0.191ms 204.531ms 209.738ms 209.869ms 209.921ms 2.75% 2.643 5.257 0.78% 0.114ms 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput 4763.699op/s 4882.670op/s ± 37.552op/s 4894.594op/s ± 4.575op/s 4898.562op/s 4903.612op/s 4905.785op/s 4907.232op/s 0.26% -2.637 5.234 0.77% 2.655op/s 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time 703.565µs 705.292µs ± 1.263µs 705.252µs ± 0.447µs 705.698µs 706.325µs 706.575µs 720.550µs 2.17% 8.812 104.830 0.18% 0.089µs 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput 138782.781op/s 141785.759op/s ± 249.983op/s 141793.368op/s ± 89.860op/s 141881.121op/s 142036.628op/s 142116.547op/s 142133.221op/s 0.24% -8.656 102.384 0.18% 17.676op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time [31.967ms; 31.986ms] or [-0.030%; +0.030%] None None None
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput [78158.753op/s; 78205.913op/s] or [-0.030%; +0.030%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time [204.595ms; 205.041ms] or [-0.109%; +0.109%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput [4877.465op/s; 4887.874op/s] or [-0.107%; +0.107%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time [705.117µs; 705.467µs] or [-0.025%; +0.025%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput [141751.114op/s; 141820.404op/s] or [-0.024%; +0.024%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 0200047 1792534561 leoromanovsky/experimentation-opt-out-immediate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 157.780ns 160.676ns ± 2.374ns 160.523ns ± 1.748ns 161.769ns 164.681ns 168.435ns 170.222ns 6.04% 1.479 2.922 1.47% 0.168ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [160.347ns; 161.005ns] or [-0.205%; +0.205%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 0200047 1792534561 leoromanovsky/experimentation-opt-out-immediate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time 34.216ms 34.497ms ± 0.118ms 34.495ms ± 0.087ms 34.584ms 34.669ms 34.733ms 34.811ms 0.92% -0.088 -0.315 0.34% 0.008ms 1 200
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput 71816.353op/s 72470.401op/s ± 247.867op/s 72475.249op/s ± 183.087op/s 72656.875op/s 72913.243op/s 73046.364op/s 73064.996op/s 0.81% 0.106 -0.313 0.34% 17.527op/s 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time 179.731ms 180.539ms ± 0.345ms 180.564ms ± 0.255ms 180.789ms 181.062ms 181.360ms 181.638ms 0.60% 0.019 -0.148 0.19% 0.024ms 1 200
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput 5505.449op/s 5538.989op/s ± 10.588op/s 5538.217op/s ± 7.800op/s 5546.779op/s 5557.435op/s 5560.710op/s 5563.864op/s 0.46% -0.008 -0.155 0.19% 0.749op/s 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time 839.845µs 842.156µs ± 1.003µs 842.193µs ± 0.761µs 842.918µs 843.880µs 844.179µs 844.758µs 0.30% 0.132 -0.600 0.12% 0.071µs 1 200
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput 118377.160op/s 118743.083op/s ± 141.449op/s 118737.593op/s ± 107.334op/s 118855.069op/s 118958.215op/s 119017.479op/s 119069.522op/s 0.28% -0.127 -0.603 0.12% 10.002op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/payloads/scale/2500flags_500users_20fields execution_time [34.481ms; 34.514ms] or [-0.047%; +0.047%] None None None
flagevaluation_evp/payloads/scale/2500flags_500users_20fields throughput [72436.049op/s; 72504.753op/s] or [-0.047%; +0.047%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields execution_time [180.491ms; 180.587ms] or [-0.026%; +0.026%] None None None
flagevaluation_evp/payloads/stress/10flags_1000users_250fields throughput [5537.522op/s; 5540.457op/s] or [-0.026%; +0.026%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields execution_time [842.016µs; 842.295µs] or [-0.017%; +0.017%] None None None
flagevaluation_evp/payloads/typical/100flags_50users_10fields throughput [118723.480op/s; 118762.687op/s] or [-0.017%; +0.017%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 0200047 1792534561 leoromanovsky/experimentation-opt-out-immediate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 257.222µs 262.976µs ± 12.642µs 259.665µs ± 1.292µs 261.753µs 276.438µs 312.822µs 388.313µs 49.54% 6.588 53.767 4.80% 0.894µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [261.224µs; 264.728µs] or [-0.666%; +0.666%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz 0200047 1792534561 leoromanovsky/experimentation-opt-out-immediate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time 31.164ms 31.417ms ± 0.093ms 31.423ms ± 0.061ms 31.481ms 31.551ms 31.601ms 31.774ms 1.12% -0.126 0.718 0.30% 0.007ms 1 200
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput 78680.024op/s 79575.986op/s ± 236.314op/s 79560.404op/s ± 155.192op/s 79738.173op/s 80015.467op/s 80166.984op/s 80220.695op/s 0.83% 0.150 0.697 0.30% 16.710op/s 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time 188.755ms 189.968ms ± 1.621ms 189.452ms ± 0.249ms 189.714ms 194.688ms 195.093ms 195.241ms 3.06% 2.495 4.625 0.85% 0.115ms 1 200
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput 5121.877op/s 5264.423op/s ± 43.996op/s 5278.388op/s ± 6.936op/s 5284.540op/s 5291.277op/s 5295.277op/s 5297.864op/s 0.37% -2.485 4.588 0.83% 3.111op/s 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time 694.462µs 695.882µs ± 0.740µs 695.850µs ± 0.506µs 696.363µs 697.141µs 697.844µs 698.112µs 0.33% 0.357 -0.182 0.11% 0.052µs 1 200
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput 143243.509op/s 143702.627op/s ± 152.852op/s 143709.124op/s ± 104.483op/s 143810.904op/s 143939.697op/s 143975.662op/s 143996.259op/s 0.20% -0.352 -0.189 0.11% 10.808op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields execution_time [31.404ms; 31.430ms] or [-0.041%; +0.041%] None None None
flagevaluation_evp/coalescer/scale/2500flags_500users_20fields throughput [79543.235op/s; 79608.736op/s] or [-0.041%; +0.041%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields execution_time [189.743ms; 190.192ms] or [-0.118%; +0.118%] None None None
flagevaluation_evp/coalescer/stress/10flags_1000users_250fields throughput [5258.326op/s; 5270.521op/s] or [-0.116%; +0.116%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields execution_time [695.780µs; 695.985µs] or [-0.015%; +0.015%] None None None
flagevaluation_evp/coalescer/typical/100flags_50users_10fields throughput [143681.443op/s; 143723.811op/s] or [-0.015%; +0.015%] None None None

@leoromanovsky leoromanovsky mentioned this pull request Oct 22, 2026
Merged
@leoromanovsky
leoromanovsky added this pull request to stack #2711 October 22, 2026 01:22
@leoromanovsky
leoromanovsky merged commit ec56508 into main Oct 22, 2026
63 of 68 checks passed
@leoromanovsky
leoromanovsky deleted the leoromanovsky/experimentation-opt-out-future branch October 22, 2026 01:30
@leoromanovsky leoromanovsky mentioned this pull request Oct 22, 2026
Open
@leoromanovsky

leoromanovsky commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Consolidated into #2698, which now includes immediate and future-experiments-only modes and pins DataDog/ffe-system-test-data#52. GitHub marked this stacked PR as merged when its commit was fast-forwarded into the #2698 branch. These changes have not merged into main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@leoromanovsky