From 252e68a9ecacdcf09ececc972d32c27357532508 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 18:38:48 -0400
Subject: [PATCH 01/44] Offer legacy ciphers by default

The custom OpenSSL build exists so blasthttp can reach servers that only
speak deprecated ciphers, but nothing was reaching them. `SslConnector::
builder` installs its own cipher list, `DEFAULT:!aNULL:!eNULL:!MD5:!3DES:
!DES:!RC4:!IDEA:!SEED:...`, and we never replaced it, so RC4, DES, 3DES
and SEED never made it into the ClientHello. A server speaking only one
of them was unreachable unless the caller passed `cipher_string` by hand.

`set_security_level(0)` looked like it covered this and does not. The
security level decides how weak a negotiated cipher may be; the cipher
list decides which ones are offered at all.

Set `ALL` when the caller names nothing, on both the pooled path and the
`connect_stream` path used by `raw_connect`, `resolve_ip` and
`request_target`, which build their SSL contexts separately.

Null-encryption suites stay out of the default. They remain available
through an explicit `cipher_string`, but negotiating one by accident
would return a connection that looks like TLS and encrypts nothing.

tests/legacy_default.rs covers this end to end: each test stands up a
real TLS server pinned to one legacy cipher or protocol version and
connects with a client that sets no TLS options at all. All five cipher
cases failed before this change.

Side effect worth knowing: the ClientHello grows from 31 cipher suites
to 105, so the client's JA3/JA4 fingerprint changes.

README dropped its claims about export ciphers and SSLv3. OpenSSL
removed export ciphers in 1.1.0, and SSLv3 is disabled in our build
despite `enable-ssl3` being passed, so neither has ever worked.
---
 CHANGELOG.md            |   8 ++
 README.md               |  10 +-
 src/client/hyper.rs     |  44 +++++++--
 tests/legacy_default.rs | 204 ++++++++++++++++++++++++++++++++++++++++
 4 files changed, 252 insertions(+), 14 deletions(-)
 create mode 100644 tests/legacy_default.rs

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 599bd49..4cbbd37 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.10.1
+
+- Legacy ciphers are offered by default, which is what the custom OpenSSL build has always been for. A server speaking only RC4, RC4-MD5, 3DES, SEED, Camellia or anonymous DH is now reachable without passing `cipher_string` yourself. Previously `SslConnector::builder` installed its own list (`DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:...`) and nothing replaced it, so those suites never reached the wire and such a server was unreachable at any setting. `set_security_level(0)` did not help: the security level governs how weak a negotiated cipher may be, not which ones are offered
+- Null-encryption suites are the deliberate exception. They stay reachable through an explicit `cipher_string` but are never offered by default, since negotiating one by accident returns a connection that looks like TLS and encrypts nothing
+- The default applies on both TLS paths, the pooled client and the `resolve_ip` / `request_target` / `raw_connect` path, which build their SSL contexts separately
+- This widens the ClientHello from 31 cipher suites to 105, which changes the client's TLS fingerprint. Anything matching on JA3/JA4 will see a different value than it did on 0.10.0
+- README no longer claims export ciphers and SSLv3 work. OpenSSL removed export ciphers in 1.1.0, and SSLv3 ends up disabled in our build despite `enable-ssl3` being passed, so neither has ever been reachable
+
 ## 0.10.0
 
 - Cookies set on a redirect hop are applied to the hops that follow it, the way a browser does. What a chain collects lives for that one request, so nothing carries between requests
diff --git a/README.md b/README.md
index d6e47ca..66fbcb1 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,7 @@ cargo add blasthttp
 - **Batch connection reuse** — connections are pooled and reused within a batch, dramatically reducing overhead when scanning many URLs on the same hosts
 - **Rust performance** — async I/O, zero-copy where possible, and native concurrency give significant speed improvements over pure Python HTTP clients
 - **SSL cert info on every request** — extracts CN, SANs, emails, issuer, validity dates, and fingerprint during the TLS handshake that's already happening, eliminating the need for a separate connection to gather this information
-- **All TLS ciphers available by default** — custom-compiled OpenSSL 3.3.2 with legacy provider baked in (RC4, 3DES, export ciphers, SSLv3) so you can connect to anything
+- **Legacy TLS ciphers offered by default**: custom-compiled OpenSSL 3.3.2 with the legacy provider baked in, so a server that speaks only RC4, 3DES, SEED, Camellia or anonymous DH is reachable without naming the cipher yourself. TLS 1.0 and 1.1 are offered too. Null-encryption suites are the one exception, available through `cipher_string` but never offered by default
 - **No cert validation by default** — offensive-first: connects to self-signed, expired, and misconfigured TLS without extra config
 - **HTTP/2 support** — automatic via ALPN negotiation, falls back to HTTP/1.1
 - **Low-level primitives** — `RawConnection` for byte-level TCP/TLS I/O (bypasses HTTP framing entirely) and `blasthttp.h2` for manual H2 frame construction (includes a permissive HPACK encoder that emits bytes a strict encoder refuses, the building block for H2 smuggling / CRLF-injection tooling)
@@ -115,7 +115,7 @@ Output is JSON (one object per response), including status, headers, redirect ch
 | `--verify` | Enable TLS cert validation | off |
 | `-x, --proxy` | HTTP/SOCKS proxy URL | |
 | `--no-proxy` | Host that bypasses the proxy: `host`, `*.suffix`, IP, CIDR, or `*` (repeatable) | |
-| `--ciphers` | OpenSSL cipher string | all |
+| `--ciphers` | OpenSSL cipher string | `ALL` (every suite except null-encryption) |
 | `--min-tls` | Minimum TLS version (1.0–1.3) | |
 | `--max-tls` | Maximum TLS version (1.0–1.3) | |
 | `-v, --verbose` | Verbose output (-vv includes body) | |
@@ -239,7 +239,7 @@ All parameters except `url` are optional:
 | `verify_certs` | `bool` | Enable TLS cert validation (default `False`) |
 | `proxy` | `str` | HTTP/SOCKS proxy URL |
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
-| `cipher_string` | `str` | OpenSSL cipher string |
+| `cipher_string` | `str` | OpenSSL cipher string (default `ALL`: every suite the build provides except null-encryption) |
 | `min_tls_version` | `str` | Minimum TLS version (`"1.0"`–`"1.3"`) |
 | `max_tls_version` | `str` | Maximum TLS version (`"1.0"`–`"1.3"`) |
 | `retries` | `int` | Number of retries on failure |
@@ -572,7 +572,9 @@ def http_mock():
 
 ### 1. Build custom OpenSSL
 
-blasthttp ships a script that downloads OpenSSL 3.3.2, compiles it with weak cipher support (`RC4`, `DES`, `3DES`, export ciphers, `SSLv3`), and installs it to `vendor/openssl/install/`. This only needs to be run once — the result is cached and reused.
+blasthttp ships a script that downloads OpenSSL 3.3.2, compiles it with weak cipher support (`RC4`, `DES`, `3DES`, `SEED`), and installs it to `vendor/openssl/install/`. This only needs to be run once — the result is cached and reused.
+
+Two things the script asks for that the build does not currently produce: export ciphers, which OpenSSL removed outright in 1.1.0, and `SSLv3`, which ends up disabled despite `enable-ssl3` being passed (`configuration.h` defines `OPENSSL_NO_SSL3`, and `libssl.a` has no `SSLv3_method`). Neither is reachable today.
 
 ```bash
 ./scripts/build-openssl.sh
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index a8a31c9..9721899 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -58,6 +58,26 @@ fn sanitize_uri(url: &str) -> String {
     out
 }
 
+/// Cipher list used when the caller doesn't name one.
+///
+/// This has to be set explicitly. `SslConnector::builder` installs its own
+/// list first, `DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:...`,
+/// and `set_security_level(0)` does NOT undo it. The security level governs
+/// how weak a cipher may be; the cipher list governs which ones are offered at
+/// all. Leaving the list alone means RC4, DES, 3DES and SEED never reach the
+/// wire whatever the security level says, so a server speaking only one of
+/// them is unreachable, which is the whole reason this project builds its own
+/// OpenSSL.
+///
+/// `ALL` is every suite the build provides except the eNULL (no encryption)
+/// ones. It does include the aNULL suites, which skip authentication: those
+/// are worth reaching for a scanner, and this client already defaults to not
+/// verifying certificates, so they give up nothing that was being enforced.
+/// Null *encryption* stays opt-in through `cipher_string`, because negotiating
+/// it by default would hand back a connection that looks like TLS and encrypts
+/// nothing.
+const DEFAULT_CIPHER_LIST: &str = "ALL";
+
 // Load the OpenSSL legacy provider once (for RC4, DES, etc.).
 // The provider is statically compiled into libcrypto via `no-module` build flag.
 // `Once` ensures this runs exactly once even across threads.
@@ -346,11 +366,13 @@ impl OpenSslConnector {
             load_system_ca_certs(&mut builder)?;
         }
 
-        if let Some(ref ciphers) = config.cipher_string {
-            builder.set_cipher_list(ciphers).map_err(|e| {
-                ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e))
-            })?;
-        }
+        let ciphers = config
+            .cipher_string
+            .as_deref()
+            .unwrap_or(DEFAULT_CIPHER_LIST);
+        builder
+            .set_cipher_list(ciphers)
+            .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
 
         if let Some(ref min_ver) = config.min_tls_version {
             let version = parse_tls_version(min_ver)?;
@@ -975,11 +997,13 @@ pub(crate) async fn connect_stream(
     } else {
         load_system_ca_certs(&mut ssl_builder)?;
     }
-    if let Some(ref ciphers) = config.cipher_string {
-        ssl_builder
-            .set_cipher_list(ciphers)
-            .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
-    }
+    let ciphers = config
+        .cipher_string
+        .as_deref()
+        .unwrap_or(DEFAULT_CIPHER_LIST);
+    ssl_builder
+        .set_cipher_list(ciphers)
+        .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
     if let Some(ref min_ver) = config.min_tls_version {
         let version = parse_tls_version(min_ver)?;
         ssl_builder
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
new file mode 100644
index 0000000..e37cbad
--- /dev/null
+++ b/tests/legacy_default.rs
@@ -0,0 +1,204 @@
+// Full-stack tests: legacy ciphers and protocols must work WITHOUT the caller
+// naming them.
+//
+// The custom OpenSSL build (scripts/build-openssl.sh, enable-weak-ssl-ciphers)
+// exists so blasthttp can reach ancient servers. README.md advertises "All TLS
+// ciphers available by default". These tests hold that promise to its word:
+// every client config below sets `verify_certs` (the cert is self-signed) and
+// a timeout, and nothing else. No `cipher_string`, no `min_tls_version`, no
+// `max_tls_version`.
+//
+// Each test stands up a real TLS server pinned to one legacy cipher or
+// protocol version, then makes a real request through the ordinary client
+// path.
+
+mod tls_server;
+
+use blasthttp::client::HttpClient;
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::config::RequestConfig;
+use openssl::ssl::SslVersion;
+use tls_server::{TlsServerConfig, TlsTestServer};
+
+/// A client config with NO TLS tuning of any kind.
+///
+/// `verify_certs` is off because the test server's cert is self-signed, and
+/// that is orthogonal to which ciphers we offer. Everything else is left at
+/// the default, which is the entire point of these tests.
+fn default_config(url: &str) -> RequestConfig {
+    let mut config = RequestConfig::new(url.to_string());
+    config.verify_certs = Some(false);
+    config.timeout_seconds = Some(5);
+    config
+}
+
+/// Stand up a server that speaks exactly one legacy cipher, then connect with
+/// a stock client and assert we got the page.
+async fn assert_reachable_with_cipher(cipher: &str) {
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some(cipher.to_string()),
+        // None of these ciphers exist in TLS 1.3, so the server has to sit at
+        // 1.2 or below for the pin to mean anything.
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let config = default_config(&server.url());
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "a stock client should reach a {cipher}-only server, but got: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
+/// Same, for a server pinned to one legacy protocol version.
+async fn assert_reachable_with_version(label: &str, version: SslVersion) {
+    let server = TlsTestServer::start(TlsServerConfig {
+        min_tls_version: Some(version),
+        max_tls_version: Some(version),
+        ..Default::default()
+    })
+    .await;
+
+    let config = default_config(&server.url());
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "a stock client should reach a {label}-only server, but got: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
+// ── Legacy ciphers, no client configuration ───────────────────────
+
+#[tokio::test]
+async fn test_rc4_reachable_by_default() {
+    assert_reachable_with_cipher("RC4-SHA").await;
+}
+
+#[tokio::test]
+async fn test_rc4_md5_reachable_by_default() {
+    assert_reachable_with_cipher("RC4-MD5").await;
+}
+
+#[tokio::test]
+async fn test_3des_reachable_by_default() {
+    assert_reachable_with_cipher("DES-CBC3-SHA").await;
+}
+
+#[tokio::test]
+async fn test_seed_reachable_by_default() {
+    assert_reachable_with_cipher("SEED-SHA").await;
+}
+
+#[tokio::test]
+async fn test_camellia_reachable_by_default() {
+    assert_reachable_with_cipher("CAMELLIA128-SHA").await;
+}
+
+#[tokio::test]
+async fn test_anon_dh_reachable_by_default() {
+    // Anonymous DH: no certificate at all. Worth reaching, since a server
+    // that allows it is itself the finding.
+    assert_reachable_with_cipher("ADH-AES128-SHA").await;
+}
+
+// ── Null encryption is deliberately NOT in the default ────────────
+
+#[tokio::test]
+async fn test_null_cipher_not_offered_by_default() {
+    // A server offering only eNULL should be out of reach for a stock client.
+    // Connecting would look like TLS while encrypting nothing, so it has to be
+    // an explicit decision rather than something we fall into.
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("NULL-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let config = default_config(&server.url());
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_err(),
+        "null encryption must not be reachable without asking for it"
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_null_cipher_reachable_when_asked_for() {
+    // ...but naming it explicitly still works, so the capability is available
+    // to anyone who actually wants it.
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("NULL-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let mut config = default_config(&server.url());
+    config.cipher_string = Some("NULL-SHA".to_string());
+
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "explicit NULL-SHA should still connect: {:?}",
+        result.as_ref().err()
+    );
+
+    server.shutdown().await;
+}
+
+// ── Legacy protocol versions, no client configuration ─────────────
+
+#[tokio::test]
+async fn test_tls10_reachable_by_default() {
+    assert_reachable_with_version("TLS 1.0", SslVersion::TLS1).await;
+}
+
+#[tokio::test]
+async fn test_tls11_reachable_by_default() {
+    assert_reachable_with_version("TLS 1.1", SslVersion::TLS1_1).await;
+}
+
+// ── Control: this one already works, so a failure here means the
+//    harness is broken rather than the client ──────────────────────
+
+#[tokio::test]
+async fn test_modern_still_reachable_by_default() {
+    let server = TlsTestServer::start(TlsServerConfig {
+        min_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let config = default_config(&server.url());
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "control case failed, so the harness is suspect: {:?}",
+        result.as_ref().err()
+    );
+
+    server.shutdown().await;
+}

From 019c80842aa85d062f8c375ff08f1d0dfa39aa81 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 19:53:47 -0400
Subject: [PATCH 02/44] Make SSLv3 actually work

SSLv3 has never been reachable in any build, despite the README
advertising it and the build script asking for it. Three separate things
were in the way.

The build. `scripts/build-openssl.sh` passed `enable-ssl3`, but
OpenSSL's Configure carries a disable cascade that reads "if ssl3-method
is off, turn ssl3 off too", and ssl3-method is off by default. So the
flag was undone during configure. `configdata.pm` recorded both
`enable-ssl3` and `no-ssl3`, and every shipped build came out with
OPENSSL_NO_SSL3 defined and no SSLv3_method symbols in libssl.a. Passing
`enable-ssl3-method` alongside it fixes this.

The SSL options. `SslConnector::builder` sets NO_SSLV3 and nothing
cleared it, so even a build with SSLv3 compiled in would refuse to
negotiate it. Cleared on both the pooled path and `connect_stream`.

The spelling. `parse_tls_version` understood 1.0 through 1.3 and nothing
else, so there was no way to name SSLv3 at all. It now takes `3.0`,
`ssl3` and `sslv3`, and the error message mentions it.

An SSLv3-only server is now reachable without asking, matching how the
legacy ciphers behave after the previous commit, and can also be pinned
explicitly. The test server clears NO_SSLV3 too so a test can pin it.

Separately, the build cache was keyed only on the target, so changing
what the build asks for was silently ignored and a stale install stayed
in place. It now keys on the version and feature flags as well, which is
what makes this fix reach anyone with an existing checkout. Upgrading
triggers one rebuild.

Measured: JA4 is unchanged, `supported_versions` gains 0300, and google,
github, cloudflare, amazon, reddit, wikipedia, microsoft and apple all
still return 200.
---
 CHANGELOG.md             |  5 ++++-
 README.md                | 14 ++++++------
 scripts/build-openssl.sh | 48 ++++++++++++++++++++++++++++++----------
 src/client/hyper.rs      | 13 ++++++++++-
 tests/legacy_default.rs  | 34 ++++++++++++++++++++++++++++
 tests/tls_server.rs      |  6 +++--
 6 files changed, 97 insertions(+), 23 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4cbbd37..d408874 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,7 +6,10 @@
 - Null-encryption suites are the deliberate exception. They stay reachable through an explicit `cipher_string` but are never offered by default, since negotiating one by accident returns a connection that looks like TLS and encrypts nothing
 - The default applies on both TLS paths, the pooled client and the `resolve_ip` / `request_target` / `raw_connect` path, which build their SSL contexts separately
 - This widens the ClientHello from 31 cipher suites to 105, which changes the client's TLS fingerprint. Anything matching on JA3/JA4 will see a different value than it did on 0.10.0
-- README no longer claims export ciphers and SSLv3 work. OpenSSL removed export ciphers in 1.1.0, and SSLv3 ends up disabled in our build despite `enable-ssl3` being passed, so neither has ever been reachable
+- SSLv3 works, for the first time. It had never been reachable in any build: `scripts/build-openssl.sh` passed `enable-ssl3`, but OpenSSL's Configure has a disable cascade reading "if `ssl3-method` is off, turn `ssl3` off too", and `ssl3-method` is off by default. The flag was silently undone, so every shipped build had `OPENSSL_NO_SSL3` defined and no `SSLv3_method` symbols. The script now passes `enable-ssl3-method` as well
+- `SslConnector::builder` also sets `NO_SSLV3`, which nothing cleared, and `parse_tls_version` only understood 1.0 through 1.3, so SSLv3 had no spelling even if the build had provided it. Both are fixed: the option is cleared on both TLS paths, and `min_tls_version` / `max_tls_version` / `--min-tls` / `--max-tls` now accept `3.0` (also `ssl3`, `sslv3`). An SSLv3-only server is reachable without asking for it
+- The OpenSSL build cache is keyed on the feature flags and version as well as the target, so changing what the build asks for now triggers a rebuild instead of silently reusing a build made a different way. The first run after upgrading rebuilds once
+- README no longer claims export ciphers work. OpenSSL removed them outright in 1.1.0, so no build flag brings them back
 
 ## 0.10.0
 
diff --git a/README.md b/README.md
index 66fbcb1..b55ca1e 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,7 @@ cargo add blasthttp
 - **Batch connection reuse** — connections are pooled and reused within a batch, dramatically reducing overhead when scanning many URLs on the same hosts
 - **Rust performance** — async I/O, zero-copy where possible, and native concurrency give significant speed improvements over pure Python HTTP clients
 - **SSL cert info on every request** — extracts CN, SANs, emails, issuer, validity dates, and fingerprint during the TLS handshake that's already happening, eliminating the need for a separate connection to gather this information
-- **Legacy TLS ciphers offered by default**: custom-compiled OpenSSL 3.3.2 with the legacy provider baked in, so a server that speaks only RC4, 3DES, SEED, Camellia or anonymous DH is reachable without naming the cipher yourself. TLS 1.0 and 1.1 are offered too. Null-encryption suites are the one exception, available through `cipher_string` but never offered by default
+- **Legacy TLS offered by default**: custom-compiled OpenSSL 3.3.2 with the legacy provider baked in, so a server that speaks only RC4, 3DES, SEED, Camellia or anonymous DH is reachable without naming the cipher yourself. SSLv3, TLS 1.0 and TLS 1.1 are offered too. Null-encryption suites are the one exception, available through `cipher_string` but never offered by default
 - **No cert validation by default** — offensive-first: connects to self-signed, expired, and misconfigured TLS without extra config
 - **HTTP/2 support** — automatic via ALPN negotiation, falls back to HTTP/1.1
 - **Low-level primitives** — `RawConnection` for byte-level TCP/TLS I/O (bypasses HTTP framing entirely) and `blasthttp.h2` for manual H2 frame construction (includes a permissive HPACK encoder that emits bytes a strict encoder refuses, the building block for H2 smuggling / CRLF-injection tooling)
@@ -116,8 +116,8 @@ Output is JSON (one object per response), including status, headers, redirect ch
 | `-x, --proxy` | HTTP/SOCKS proxy URL | |
 | `--no-proxy` | Host that bypasses the proxy: `host`, `*.suffix`, IP, CIDR, or `*` (repeatable) | |
 | `--ciphers` | OpenSSL cipher string | `ALL` (every suite except null-encryption) |
-| `--min-tls` | Minimum TLS version (1.0–1.3) | |
-| `--max-tls` | Maximum TLS version (1.0–1.3) | |
+| `--min-tls` | Minimum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
+| `--max-tls` | Maximum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
 | `-v, --verbose` | Verbose output (-vv includes body) | |
 
 ## Python API
@@ -240,8 +240,8 @@ All parameters except `url` are optional:
 | `proxy` | `str` | HTTP/SOCKS proxy URL |
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
 | `cipher_string` | `str` | OpenSSL cipher string (default `ALL`: every suite the build provides except null-encryption) |
-| `min_tls_version` | `str` | Minimum TLS version (`"1.0"`–`"1.3"`) |
-| `max_tls_version` | `str` | Maximum TLS version (`"1.0"`–`"1.3"`) |
+| `min_tls_version` | `str` | Minimum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
+| `max_tls_version` | `str` | Maximum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `retries` | `int` | Number of retries on failure |
 | `retry_wait_min_ms` | `int` | Minimum backoff between retries (ms) |
 | `retry_wait_max_ms` | `int` | Maximum backoff between retries (ms) |
@@ -572,9 +572,9 @@ def http_mock():
 
 ### 1. Build custom OpenSSL
 
-blasthttp ships a script that downloads OpenSSL 3.3.2, compiles it with weak cipher support (`RC4`, `DES`, `3DES`, `SEED`), and installs it to `vendor/openssl/install/`. This only needs to be run once — the result is cached and reused.
+blasthttp ships a script that downloads OpenSSL 3.3.2, compiles it with weak cipher support (`RC4`, `DES`, `3DES`, `SEED`) and `SSLv3`, and installs it to `vendor/openssl/install/`. The result is cached, keyed on the OpenSSL version, the target, and the feature flags, so changing any of those triggers a rebuild on the next run.
 
-Two things the script asks for that the build does not currently produce: export ciphers, which OpenSSL removed outright in 1.1.0, and `SSLv3`, which ends up disabled despite `enable-ssl3` being passed (`configuration.h` defines `OPENSSL_NO_SSL3`, and `libssl.a` has no `SSLv3_method`). Neither is reachable today.
+Export ciphers are the one thing the build cannot provide: OpenSSL removed them outright in 1.1.0, so no build flag brings them back.
 
 ```bash
 ./scripts/build-openssl.sh
diff --git a/scripts/build-openssl.sh b/scripts/build-openssl.sh
index 74a0805..39101ca 100755
--- a/scripts/build-openssl.sh
+++ b/scripts/build-openssl.sh
@@ -25,6 +25,26 @@ INSTALL_DIR="${VENDOR_DIR}/install"
 TARBALL="${VENDOR_DIR}/openssl-${OPENSSL_VERSION}.tar.gz"
 MARKER="${INSTALL_DIR}/.blasthttp-built"
 
+# The features that make this build different from a stock OpenSSL.
+#
+# `enable-ssl3` and `enable-ssl3-method` both have to be here. OpenSSL's
+# Configure has a disable cascade that reads "if ssl3-method is off, turn ssl3
+# off too", and ssl3-method is off by default. So enable-ssl3 on its own gets
+# cascaded straight back to no-ssl3, which is exactly what happened here for
+# a long time: the flag was passed, configdata.pm recorded both `enable-ssl3`
+# and `no-ssl3`, and the build came out with OPENSSL_NO_SSL3 defined and no
+# SSLv3_method symbols.
+FEATURE_FLAGS=(
+    enable-weak-ssl-ciphers
+    enable-ssl3
+    enable-ssl3-method
+    no-shared
+    no-module
+    no-tests
+    -fPIC
+)
+
+
 # --- Cross-compilation support ---
 TARGET="${CARGO_BUILD_TARGET:-}"
 
@@ -48,15 +68,24 @@ case "$TARGET" in
         ;;
 esac
 
-# Skip if already built for the same target
+# Identifies what the cached build actually is. Both the target and the
+# feature flags belong in here: an existing checkout whose flags changed needs
+# a rebuild just as much as one whose target changed, and keying on the target
+# alone means a flag change is silently ignored until someone deletes the
+# install directory by hand.
+BUILD_RECIPE="${TARGET:-native}|${OPENSSL_VERSION}|${FEATURE_FLAGS[*]}"
+
+# Skip only if the cached build was made the same way
 if [ -f "$MARKER" ]; then
-    BUILT_TARGET=$(cat "$MARKER" 2>/dev/null || true)
-    if [ "$BUILT_TARGET" = "$TARGET" ]; then
+    BUILT_RECIPE=$(cat "$MARKER" 2>/dev/null || true)
+    if [ "$BUILT_RECIPE" = "$BUILD_RECIPE" ]; then
         echo "=== OpenSSL ${OPENSSL_VERSION} already built for '${TARGET:-native}' at ${INSTALL_DIR} ==="
         echo "=== Delete ${INSTALL_DIR} to force rebuild ==="
         exit 0
     fi
-    echo "=== Rebuilding OpenSSL: target changed from '${BUILT_TARGET:-native}' to '${TARGET:-native}' ==="
+    echo "=== Rebuilding OpenSSL: build recipe changed ==="
+    echo "===   was: ${BUILT_RECIPE:-(unknown)}"
+    echo "===   now: ${BUILD_RECIPE}"
     rm -rf "$INSTALL_DIR"
 fi
 
@@ -147,12 +176,7 @@ cd "$SOURCE_DIR"
 
 COMMON_ARGS=(
     --prefix="$INSTALL_DIR"
-    enable-weak-ssl-ciphers
-    enable-ssl3
-    no-shared
-    no-module
-    no-tests
-    -fPIC
+    "${FEATURE_FLAGS[@]}"
 )
 
 if [ -n "$openssl_target" ]; then
@@ -186,9 +210,9 @@ make -j"$NUM_JOBS"
 echo "Installing..."
 make install_sw
 
-# Mark as complete (store target for cache invalidation)
+# Mark as complete (store the full recipe so a flag or target change invalidates)
 mkdir -p "$INSTALL_DIR"
-echo "$TARGET" > "$MARKER"
+echo "$BUILD_RECIPE" > "$MARKER"
 
 echo ""
 echo "=== OpenSSL ${OPENSSL_VERSION} built successfully ==="
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 9721899..d85931a 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -257,12 +257,13 @@ fn encode_alpn_protocols(protos: &[String]) -> Result<Vec<u8>, ClientError> {
 
 fn parse_tls_version(s: &str) -> Result<openssl::ssl::SslVersion, ClientError> {
     match s.to_lowercase().as_str() {
+        "3.0" | "ssl3" | "sslv3" | "ssl3.0" => Ok(openssl::ssl::SslVersion::SSL3),
         "1.0" | "tls1.0" | "tlsv1.0" => Ok(openssl::ssl::SslVersion::TLS1),
         "1.1" | "tls1.1" | "tlsv1.1" => Ok(openssl::ssl::SslVersion::TLS1_1),
         "1.2" | "tls1.2" | "tlsv1.2" => Ok(openssl::ssl::SslVersion::TLS1_2),
         "1.3" | "tls1.3" | "tlsv1.3" => Ok(openssl::ssl::SslVersion::TLS1_3),
         _ => Err(ClientError::other(format!(
-            "unknown TLS version '{}' (use 1.0, 1.1, 1.2, 1.3)",
+            "unknown TLS version '{}' (use 3.0 for SSLv3, or 1.0, 1.1, 1.2, 1.3)",
             s
         ))),
     }
@@ -360,6 +361,13 @@ impl OpenSslConnector {
         // This is an offensive-first tool — we need to connect to anything.
         builder.set_security_level(0);
 
+        // `SslConnector::builder` sets NO_SSLV3 for us. Clear it, or a server
+        // that speaks nothing newer stays unreachable even when the caller
+        // asks for SSLv3 by name. Clearing the option only permits the
+        // protocol; which versions actually get offered is still decided by
+        // the min/max proto version below.
+        builder.clear_options(openssl::ssl::SslOptions::NO_SSLV3);
+
         if !config.should_verify_certs() {
             builder.set_verify(openssl::ssl::SslVerifyMode::NONE);
         } else {
@@ -991,6 +999,9 @@ pub(crate) async fn connect_stream(
             .map_err(|e| ClientError::tls(format!("SSL setup failed: {}", e)))?;
 
     ssl_builder.set_security_level(0);
+    // Same reason as the pooled path: openssl-rs sets NO_SSLV3 and an SSLv3
+    // server is unreachable until it is cleared.
+    ssl_builder.clear_options(openssl::ssl::SslOptions::NO_SSLV3);
 
     if !config.should_verify_certs() {
         ssl_builder.set_verify(openssl::ssl::SslVerifyMode::NONE);
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index e37cbad..d12664d 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -169,6 +169,40 @@ async fn test_null_cipher_reachable_when_asked_for() {
 
 // ── Legacy protocol versions, no client configuration ─────────────
 
+#[tokio::test]
+async fn test_sslv3_reachable_by_default() {
+    assert_reachable_with_version("SSLv3", SslVersion::SSL3).await;
+}
+
+#[tokio::test]
+async fn test_sslv3_reachable_when_pinned_explicitly() {
+    // The version can also be named, which had no spelling at all before:
+    // `parse_tls_version` only knew 1.0 through 1.3, so SSLv3 was
+    // unrequestable regardless of what the build supported.
+    let server = TlsTestServer::start(TlsServerConfig {
+        min_tls_version: Some(SslVersion::SSL3),
+        max_tls_version: Some(SslVersion::SSL3),
+        ..Default::default()
+    })
+    .await;
+
+    let mut config = default_config(&server.url());
+    config.min_tls_version = Some("3.0".to_string());
+    config.max_tls_version = Some("3.0".to_string());
+
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "explicitly pinned SSLv3 should connect: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
 #[tokio::test]
 async fn test_tls10_reachable_by_default() {
     assert_reachable_with_version("TLS 1.0", SslVersion::TLS1).await;
diff --git a/tests/tls_server.rs b/tests/tls_server.rs
index 5988139..65a0a13 100644
--- a/tests/tls_server.rs
+++ b/tests/tls_server.rs
@@ -99,12 +99,14 @@ fn build_acceptor(config: &TlsServerConfig) -> SslAcceptor {
     // Clear ALL version restrictions and options
     builder.set_min_proto_version(None).unwrap();
     builder.set_max_proto_version(None).unwrap();
-    // Remove any SSL options that might disable specific TLS versions
+    // Remove any SSL options that might disable specific protocol versions,
+    // SSLv3 included, so a test can pin the server to it.
     builder.clear_options(
         SslOptions::NO_TLSV1_3
             | SslOptions::NO_TLSV1_2
             | SslOptions::NO_TLSV1_1
-            | SslOptions::NO_TLSV1,
+            | SslOptions::NO_TLSV1
+            | SslOptions::NO_SSLV3,
     );
     // Accept all ciphers by default
     builder

From ce2594b47a4d3ed011bd1cd2088cc19158906cb5 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 21:15:37 -0400
Subject: [PATCH 03/44] Add offline ClientHello capture and JA4

First piece of the 1.x benchmark work. Nothing in the impersonation plan
is verifiable without a way to see what we actually put on the wire, and
doing that against public fingerprinting services does not scale: they
rate-limit, several that the ecosystem still cites are dead, and one has
been hijacked and now serves a gambling site.

So measure it locally. The test server installs a client_hello callback,
which fires after the hello is parsed and before a cipher is chosen, and
records the offer. That is the only point where the full ClientHello is
still visible. The safe openssl wrapper covers ciphers and versions but
has no accessor for the extension list, which is most of a fingerprint,
so that part goes through openssl-sys. Both new dev-dependencies were
already in the tree as transitive dependencies of openssl.

JA4 rather than JA3, for a reason that decides the shape of the stealth
work: JA4 sorts its lists and strips GREASE, and extension order plus
GREASE are exactly the two things OpenSSL cannot control. They cost us
nothing on this metric. JA3 is also a poor target in its own right, since
Chrome permutes extensions per connection and one Chrome build yields a
different JA3 on every handshake.

The implementation is checked against ground truth rather than against
itself: ja4_conformance.rs computes the JA4 of a captured Chrome 150
hello from lexiforest/curl-impersonate's signature corpus (MIT) and
asserts the published value, which four independent projects agree on
across three fingerprinting services. All three segments match.

fingerprint_baseline.rs pins where we stand: t13i9911h2 locally, against
Chrome's t13d1516h2. It also records the gaps as assertions that will
flip as each is closed, so no GREASE anywhere, none of the six
browser-only extensions, and every protocol version from SSLv3 up on
offer.

Shared test helpers move to tests/support/. Cargo compiles every
top-level file in tests/ as its own test binary, so tls_server.rs was
already building as a target with no tests in it, and a second helper
next to it failed outright once it referenced a sibling.

One thing worth knowing that fell out of this: JA4's count fields are
two digits, so our 105 cipher suites render as 99. Past that point JA4
cannot tell us apart from any other client with an unreasonable cipher
list.
---
 Cargo.lock                        |   2 +
 Cargo.toml                        |   9 ++
 tests/fingerprint_baseline.rs     | 182 ++++++++++++++++++++++++++++++
 tests/ja4_conformance.rs          | 141 +++++++++++++++++++++++
 tests/legacy_default.rs           |   4 +-
 tests/support/ja4.rs              | 182 ++++++++++++++++++++++++++++++
 tests/support/mod.rs              |  14 +++
 tests/{ => support}/tls_server.rs | 159 +++++++++++++++++++++++++-
 tests/tls_integration.rs          |   4 +-
 9 files changed, 691 insertions(+), 6 deletions(-)
 create mode 100644 tests/fingerprint_baseline.rs
 create mode 100644 tests/ja4_conformance.rs
 create mode 100644 tests/support/ja4.rs
 create mode 100644 tests/support/mod.rs
 rename tests/{ => support}/tls_server.rs (55%)

diff --git a/Cargo.lock b/Cargo.lock
index fc635ff..0ac2e7f 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -105,6 +105,7 @@ dependencies = [
  "bytes",
  "clap",
  "flate2",
+ "foreign-types-shared",
  "futures",
  "http",
  "http-body-util",
@@ -112,6 +113,7 @@ dependencies = [
  "hyper-util",
  "murmur3",
  "openssl",
+ "openssl-sys",
  "psl",
  "pyo3",
  "pyo3-async-runtimes",
diff --git a/Cargo.toml b/Cargo.toml
index 072a839..ca98715 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -70,3 +70,12 @@ pyo3-async-runtimes = { version = "0.27", features = ["attributes", "tokio-runti
 [dev-dependencies]
 # TLS test server — uses our custom OpenSSL to serve with weak ciphers
 tokio-test = "0.4"
+
+# Raw SSL_client_hello_get* bindings, for reading back the ClientHello we sent.
+# The safe `openssl` wrapper exposes ciphers and versions but not the extension
+# list, and the extension list is most of a fingerprint. Both of these are
+# already in the tree as transitive dependencies of `openssl`, so they pull
+# nothing new in; `foreign-types-shared` is only here for the `as_ptr()` that
+# hands those bindings an `SSL *`.
+openssl-sys = "0.9"
+foreign-types-shared = "0.1"
diff --git a/tests/fingerprint_baseline.rs b/tests/fingerprint_baseline.rs
new file mode 100644
index 0000000..fc11de4
--- /dev/null
+++ b/tests/fingerprint_baseline.rs
@@ -0,0 +1,182 @@
+// What our ClientHello actually looks like, measured offline.
+//
+// These tests stand up a local TLS server that records the hello the client
+// sends, so the fingerprint can be checked in CI with no network and no
+// third-party service. That matters for two reasons: the public fingerprint
+// endpoints rate-limit, and several of the ones the ecosystem cites are dead
+// or, in one case, now serving a gambling site.
+//
+// The pinned values below are a regression gate, not an aspiration. Changing
+// the TLS configuration is supposed to change them; the point is that it
+// cannot happen silently. When one of these fails, read the `ja4_raw` output
+// in the failure message, confirm the move was intended, and update the pin.
+
+mod support;
+
+use blasthttp::client::HttpClient;
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::config::RequestConfig;
+use support::tls_server::{CapturedHello, TlsServerConfig, TlsTestServer};
+
+/// Make one ordinary request at a capturing server and return what it saw.
+async fn capture_default_hello() -> CapturedHello {
+    let server = TlsTestServer::start(TlsServerConfig::default()).await;
+
+    let mut config = RequestConfig::new(server.url());
+    config.verify_certs = Some(false);
+    config.timeout_seconds = Some(5);
+
+    let client = HyperClient::new();
+    let _ = client.send(&config).await;
+
+    let hello = server
+        .hello
+        .lock()
+        .expect("hello slot poisoned")
+        .clone()
+        .expect("server recorded no ClientHello");
+
+    server.shutdown().await;
+    hello
+}
+
+#[tokio::test]
+async fn test_capture_sees_a_plausible_hello() {
+    // Guards the harness itself. If capture breaks, every other test in this
+    // file would go green against an empty hello and prove nothing.
+    let hello = capture_default_hello().await;
+
+    assert!(
+        !hello.ciphers.is_empty(),
+        "captured no cipher suites, so capture is broken"
+    );
+    assert!(
+        !hello.extensions.is_empty(),
+        "captured no extensions, so capture is broken"
+    );
+    assert!(
+        hello.extensions.contains(&0x000d),
+        "expected signature_algorithms in the hello"
+    );
+    assert!(
+        !hello.sig_algs.is_empty(),
+        "signature_algorithms extension present but parsed empty"
+    );
+}
+
+#[tokio::test]
+async fn test_no_sni_for_an_ip_literal() {
+    // The server is addressed by IP, and SNI is not sent for IP literals. This
+    // pins the `i` in the JA4 header and is the reason these tests read `t13i`
+    // rather than the `t13d` a hostname would give.
+    let hello = capture_default_hello().await;
+    assert!(
+        !hello.has_sni,
+        "SNI should be absent when connecting to an IP literal"
+    );
+}
+
+#[tokio::test]
+async fn test_default_ja4_is_pinned() {
+    let hello = capture_default_hello().await;
+    let actual = support::ja4::ja4(&hello);
+
+    // Current default: the broad legacy cipher list, which is 105 suites.
+    // JA4 renders that as `99`, because the count field is two digits wide and
+    // the spec caps it. Worth knowing on its own: past 99 suites, JA4 cannot
+    // tell 105 from 200, so the header alone stops distinguishing us from any
+    // other client with an unreasonable cipher list.
+    //
+    // The extension count is 11 rather than 12 because this server is
+    // addressed by IP, so no SNI is sent. Against a hostname it reads
+    // t13d9912h2.
+    //
+    // Chrome 150 for contrast is t13d1516h2_8daaf6152771_806a8c22fdea: 15
+    // ciphers, 16 extensions. Closing that gap is the point of the stealth
+    // work, and this pin is how we will see it move.
+    const EXPECTED: &str = "t13i9911h2";
+
+    let header = actual.split('_').next().unwrap_or_default();
+    assert_eq!(
+        header,
+        EXPECTED,
+        "JA4 header moved.\n  was: {}\n  now: {}\n  raw: {}",
+        EXPECTED,
+        header,
+        support::ja4::ja4_raw(&hello),
+    );
+}
+
+#[tokio::test]
+async fn test_grease_is_absent() {
+    // We send no GREASE anywhere, which JA4 ignores but a detector checking
+    // the raw hello does not: every modern browser emits GREASE in several
+    // places and almost nothing scripted does, so its absence is a cheap and
+    // reliable tell. OpenSSL has no API for it at any version.
+    //
+    // When stealth mode arrives this expectation inverts. Until then it
+    // records where we stand.
+    let hello = capture_default_hello().await;
+
+    let greased: Vec<u16> = hello
+        .ciphers
+        .iter()
+        .chain(hello.extensions.iter())
+        .chain(hello.groups.iter())
+        .copied()
+        .filter(|v| support::ja4::is_grease(*v))
+        .collect();
+
+    assert!(
+        greased.is_empty(),
+        "expected no GREASE from the OpenSSL stack, found {:04x?}",
+        greased
+    );
+}
+
+#[tokio::test]
+async fn test_offers_legacy_protocol_versions() {
+    // The companion to the cipher work: SSLv3 through TLS 1.3 are all offered,
+    // which is what makes an ancient server reachable without configuration.
+    // It is also conspicuous, since a browser offers 1.2 and 1.3 only.
+    let hello = capture_default_hello().await;
+
+    for (version, label) in [
+        (0x0300u16, "SSLv3"),
+        (0x0301, "TLS 1.0"),
+        (0x0302, "TLS 1.1"),
+        (0x0303, "TLS 1.2"),
+        (0x0304, "TLS 1.3"),
+    ] {
+        assert!(
+            hello.supported_versions.contains(&version),
+            "{} missing from supported_versions: {:04x?}",
+            label,
+            hello.supported_versions
+        );
+    }
+}
+
+#[tokio::test]
+async fn test_no_browser_only_extensions_yet() {
+    // The six extensions a JA4 match against Chrome needs, none of which we
+    // send today. Named here so the gap is legible and so each one flips this
+    // test as it lands.
+    let hello = capture_default_hello().await;
+
+    for (ext, name) in [
+        (0x0005u16, "status_request"),
+        (0x0012, "signed_certificate_timestamp"),
+        (0x001b, "compress_certificate"),
+        (0x44cd, "application_settings (ALPS)"),
+        (0xfe0d, "encrypted_client_hello"),
+        (0xff01, "renegotiation_info"),
+    ] {
+        assert!(
+            !hello.extensions.contains(&ext),
+            "{} ({:#06x}) is now sent; stealth work has started, update this test",
+            name,
+            ext
+        );
+    }
+}
diff --git a/tests/ja4_conformance.rs b/tests/ja4_conformance.rs
new file mode 100644
index 0000000..fafad93
--- /dev/null
+++ b/tests/ja4_conformance.rs
@@ -0,0 +1,141 @@
+// Does our JA4 agree with everyone else's?
+//
+// The baseline tests pin what we send, but a JA4 implementation that is wrong
+// in a self-consistent way would pass all of them and still be useless for
+// comparing against a browser. This checks the algorithm itself against a
+// captured real-browser ClientHello whose JA4 is published.
+//
+// Reference data is the Chrome 150 signature from lexiforest/curl-impersonate,
+// `tests/signatures/chrome_150.0.7871.127_macOS.yaml` (MIT licensed, copyright
+// the curl_cffi developers). Its JA4 is corroborated by four independent
+// projects across three different fingerprinting services, which is about as
+// much agreement as this area offers.
+//
+// Worth knowing when adding vectors: fingerprint values are not portable
+// between calculators. One handshake observed by seven services produced three
+// different JA3 hashes and two different JA4s. Only add a vector whose expected
+// value has independent corroboration, or this file starts asserting one
+// service's quirks.
+
+mod support;
+
+use support::tls_server::CapturedHello;
+
+/// GREASE placeholder. Which of the sixteen values appears does not matter,
+/// since every one of them is stripped before hashing.
+const GREASE: u16 = 0x0a0a;
+
+/// Chrome 150 on macOS, as captured by curl-impersonate.
+fn chrome_150() -> CapturedHello {
+    CapturedHello {
+        // Modern hellos pin this at TLS 1.2 and put the real list in
+        // supported_versions.
+        legacy_version: 0x0303,
+        ciphers: vec![
+            GREASE, 4865, 4866, 4867, 49195, 49199, 49196, 49200, 52393, 52392, 49171, 49172, 156,
+            157, 47, 53,
+        ],
+        // In the order Chrome sent them. Chrome permutes this per connection,
+        // which is exactly why JA4 sorts before hashing and why the order here
+        // does not affect the result.
+        extensions: vec![
+            GREASE, 0x44cd, 0x002d, 0x001b, 0x0033, 0x002b, 0xff01, 0xfe0d, 0x0017, 0x000b, 0x0005,
+            0x0023, 0x000a, 0x0012, 0x0010, 0x000d, 0x0000, 0x1a1a,
+        ],
+        groups: vec![GREASE, 4588, 29, 23, 24],
+        // The first three are ML-DSA, new in recent Chrome. Order is
+        // significant: JA4 does not sort this list.
+        sig_algs: vec![
+            2308, 2309, 2310, 1027, 2052, 1025, 1283, 2053, 1281, 2054, 1537,
+        ],
+        alpn: vec!["h2".to_string(), "http/1.1".to_string()],
+        supported_versions: vec![GREASE, 0x0304, 0x0303],
+        has_sni: true,
+    }
+}
+
+#[test]
+fn test_ja4_matches_published_chrome_150() {
+    const PUBLISHED: &str = "t13d1516h2_8daaf6152771_806a8c22fdea";
+
+    let computed = support::ja4::ja4(&chrome_150());
+
+    assert_eq!(
+        computed,
+        PUBLISHED,
+        "JA4 disagrees with the published value for Chrome 150.\n  raw: {}",
+        support::ja4::ja4_raw(&chrome_150()),
+    );
+}
+
+#[test]
+fn test_ja4_header_decodes_as_expected() {
+    // Spelled out so a failure says which field moved rather than just that a
+    // string differs. 16 ciphers and 18 extensions were sent; one and two of
+    // those are GREASE, giving the 15 and 16 in the header.
+    let hello = chrome_150();
+    let computed = support::ja4::ja4(&hello);
+    let header = computed.split('_').next().unwrap();
+
+    assert_eq!(&header[0..1], "t", "protocol should be TCP");
+    assert_eq!(
+        &header[1..3],
+        "13",
+        "supported_versions tops out at TLS 1.3"
+    );
+    assert_eq!(&header[3..4], "d", "SNI was present");
+    assert_eq!(&header[4..6], "15", "16 ciphers less 1 GREASE");
+    assert_eq!(&header[6..8], "16", "18 extensions less 2 GREASE");
+    assert_eq!(&header[8..10], "h2", "first and last char of first ALPN");
+}
+
+#[test]
+fn test_extension_order_does_not_change_ja4() {
+    // The property the whole stealth plan rests on. Chrome shuffles its
+    // extension order every connection, so if ordering moved the hash, no
+    // stable Chrome JA4 could exist. It also means our inability to control
+    // extension order under OpenSSL costs us nothing on this metric.
+    let mut shuffled = chrome_150();
+    shuffled.extensions.reverse();
+
+    assert_eq!(
+        support::ja4::ja4(&chrome_150()),
+        support::ja4::ja4(&shuffled),
+        "reversing the extension order changed the JA4"
+    );
+}
+
+#[test]
+fn test_grease_does_not_change_ja4() {
+    // The other half of the same argument: GREASE is stripped, so not emitting
+    // any costs nothing here either. It is still visible to anything reading
+    // the raw hello, which is tracked separately in fingerprint_baseline.rs.
+    let mut without = chrome_150();
+    without.ciphers.retain(|c| !support::ja4::is_grease(*c));
+    without.extensions.retain(|e| !support::ja4::is_grease(*e));
+    without.groups.retain(|g| !support::ja4::is_grease(*g));
+    without
+        .supported_versions
+        .retain(|v| !support::ja4::is_grease(*v));
+
+    assert_eq!(
+        support::ja4::ja4(&chrome_150()),
+        support::ja4::ja4(&without),
+        "removing GREASE changed the JA4"
+    );
+}
+
+#[test]
+fn test_sig_alg_order_does_change_ja4() {
+    // Signature algorithms are the one list JA4 leaves in the order sent, so
+    // a profile that gets the values right and the order wrong still fails to
+    // match. Guards against anyone "tidying up" by sorting them.
+    let mut reordered = chrome_150();
+    reordered.sig_algs.reverse();
+
+    assert_ne!(
+        support::ja4::ja4(&chrome_150()),
+        support::ja4::ja4(&reordered),
+        "signature algorithm order should be significant"
+    );
+}
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index d12664d..ddc7869 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -12,13 +12,13 @@
 // protocol version, then makes a real request through the ordinary client
 // path.
 
-mod tls_server;
+mod support;
 
 use blasthttp::client::HttpClient;
 use blasthttp::client::hyper::HyperClient;
 use blasthttp::config::RequestConfig;
 use openssl::ssl::SslVersion;
-use tls_server::{TlsServerConfig, TlsTestServer};
+use support::tls_server::{TlsServerConfig, TlsTestServer};
 
 /// A client config with NO TLS tuning of any kind.
 ///
diff --git a/tests/support/ja4.rs b/tests/support/ja4.rs
new file mode 100644
index 0000000..6e06060
--- /dev/null
+++ b/tests/support/ja4.rs
@@ -0,0 +1,182 @@
+// JA4 over a captured ClientHello.
+//
+// JA4 is FoxIO's TLS client fingerprint (BSD-3; the JA4+ family that includes
+// JA4H is under a separate non-commercial licence and is deliberately not
+// implemented here). It replaced JA3 because JA3 preserves ordering and counts
+// GREASE, and Chrome shuffles its extension order and emits fresh GREASE on
+// every connection, so one Chrome build produces a different JA3 per
+// handshake. JA4 sorts and strips, so it holds still.
+//
+// That property is why this is the metric worth gating on: extension order and
+// GREASE are the two things OpenSSL cannot control, and JA4 ignores both.
+//
+//     JA4 = <a>_<b>_<c>
+//       a  protocol, version, SNI, cipher count, extension count, ALPN
+//       b  first 12 hex of sha256 over the sorted cipher list
+//       c  first 12 hex of sha256 over the sorted extension list, then the
+//          signature algorithms in the order sent
+//
+// Operates on a `CapturedHello` from the sibling `tls_server` module.
+
+use super::tls_server::CapturedHello;
+
+/// RFC 8701 GREASE values: both bytes equal, low nibble `a`.
+/// 0x0a0a, 0x1a1a, ... 0xfafa.
+pub fn is_grease(v: u16) -> bool {
+    (v >> 8) == (v & 0x00ff) && (v & 0x000f) == 0x000a
+}
+
+fn strip_grease(values: &[u16]) -> Vec<u16> {
+    values.iter().copied().filter(|v| !is_grease(*v)).collect()
+}
+
+fn sha256_prefix12(input: &str) -> String {
+    let digest = openssl::hash::hash(openssl::hash::MessageDigest::sha256(), input.as_bytes())
+        .expect("sha256 of an in-memory string cannot fail");
+    digest
+        .iter()
+        .map(|b| format!("{:02x}", b))
+        .collect::<String>()[..12]
+        .to_string()
+}
+
+/// A list that contributes nothing hashes to zeros rather than to the hash of
+/// an empty string, which is what the spec calls for.
+fn hash_list(values: &[u16]) -> String {
+    if values.is_empty() {
+        return "000000000000".to_string();
+    }
+    let joined = values
+        .iter()
+        .map(|v| format!("{:04x}", v))
+        .collect::<Vec<_>>()
+        .join(",");
+    sha256_prefix12(&joined)
+}
+
+/// Two characters naming the negotiated TLS version.
+fn version_code(hello: &CapturedHello) -> &'static str {
+    // `supported_versions` is authoritative when present. The legacy field is
+    // pinned at TLS 1.2 on any modern hello regardless of what is really
+    // offered, so reading it alone would call every TLS 1.3 client 1.2.
+    let versions = strip_grease(&hello.supported_versions);
+    let highest = versions
+        .iter()
+        .copied()
+        .max()
+        .unwrap_or(hello.legacy_version);
+    match highest {
+        0x0304 => "13",
+        0x0303 => "12",
+        0x0302 => "11",
+        0x0301 => "10",
+        0x0300 => "s3",
+        _ => "00",
+    }
+}
+
+/// First and last character of the first ALPN value: `h2` stays `h2`,
+/// `http/1.1` becomes `h1`. `00` when no ALPN was offered.
+fn alpn_code(hello: &CapturedHello) -> String {
+    match hello.alpn.first() {
+        None => "00".to_string(),
+        Some(first) => {
+            let chars: Vec<char> = first.chars().collect();
+            match (chars.first(), chars.last()) {
+                (Some(a), Some(z)) if a.is_ascii() && z.is_ascii() => format!("{}{}", a, z),
+                _ => "99".to_string(),
+            }
+        }
+    }
+}
+
+/// Compute the JA4 string for a captured hello.
+pub fn ja4(hello: &CapturedHello) -> String {
+    let ciphers = strip_grease(&hello.ciphers);
+    let extensions = strip_grease(&hello.extensions);
+
+    // Counts are capped at 99 because the field is two digits wide.
+    let cipher_count = ciphers.len().min(99);
+    let ext_count = extensions.len().min(99);
+
+    let a = format!(
+        "t{}{}{:02}{:02}{}",
+        version_code(hello),
+        if hello.has_sni { "d" } else { "i" },
+        cipher_count,
+        ext_count,
+        alpn_code(hello),
+    );
+
+    let mut sorted_ciphers = ciphers.clone();
+    sorted_ciphers.sort_unstable();
+    let b = hash_list(&sorted_ciphers);
+
+    // server_name and ALPN are counted in the header but left out of the hash,
+    // since both vary with the request rather than with the client.
+    let mut sorted_exts: Vec<u16> = extensions
+        .iter()
+        .copied()
+        .filter(|e| *e != 0x0000 && *e != 0x0010)
+        .collect();
+    sorted_exts.sort_unstable();
+
+    // Signature algorithms keep the order they were sent in, and GREASE has to
+    // come out of them too. Chrome 152 puts a fresh greased value at the head
+    // of this list every handshake, so an implementation that keeps it reports
+    // a different third segment on every connection.
+    let sig_algs = strip_grease(&hello.sig_algs);
+
+    let c = if sorted_exts.is_empty() {
+        "000000000000".to_string()
+    } else {
+        let ext_part = sorted_exts
+            .iter()
+            .map(|v| format!("{:04x}", v))
+            .collect::<Vec<_>>()
+            .join(",");
+        let sig_part = sig_algs
+            .iter()
+            .map(|v| format!("{:04x}", v))
+            .collect::<Vec<_>>()
+            .join(",");
+        sha256_prefix12(&format!("{}_{}", ext_part, sig_part))
+    };
+
+    format!("{}_{}_{}", a, b, c)
+}
+
+/// The readable form: same fields, nothing hashed. What you actually want when
+/// a fingerprint moved and you need to know which byte did it.
+pub fn ja4_raw(hello: &CapturedHello) -> String {
+    let ciphers = strip_grease(&hello.ciphers);
+    let extensions = strip_grease(&hello.extensions);
+
+    let mut sorted_ciphers = ciphers.clone();
+    sorted_ciphers.sort_unstable();
+    let mut sorted_exts: Vec<u16> = extensions
+        .iter()
+        .copied()
+        .filter(|e| *e != 0x0000 && *e != 0x0010)
+        .collect();
+    sorted_exts.sort_unstable();
+
+    let hex = |vs: &[u16]| {
+        vs.iter()
+            .map(|v| format!("{:04x}", v))
+            .collect::<Vec<_>>()
+            .join(",")
+    };
+
+    format!(
+        "t{}{}{:02}{:02}{}_{}_{}_{}",
+        version_code(hello),
+        if hello.has_sni { "d" } else { "i" },
+        ciphers.len().min(99),
+        extensions.len().min(99),
+        alpn_code(hello),
+        hex(&sorted_ciphers),
+        hex(&sorted_exts),
+        hex(&strip_grease(&hello.sig_algs)),
+    )
+}
diff --git a/tests/support/mod.rs b/tests/support/mod.rs
new file mode 100644
index 0000000..1163676
--- /dev/null
+++ b/tests/support/mod.rs
@@ -0,0 +1,14 @@
+// Shared helpers for the integration tests.
+//
+// These live under `tests/support/` rather than directly in `tests/` because
+// Cargo compiles every top-level file there as its own test binary. A helper
+// module sitting at that level becomes a test target with no tests in it, and
+// one that fails to build on its own the moment it references a sibling.
+//
+// Not every test uses every helper, so the modules allow dead code rather than
+// making each consumer carry an allow attribute.
+
+#![allow(dead_code)]
+
+pub mod ja4;
+pub mod tls_server;
diff --git a/tests/tls_server.rs b/tests/support/tls_server.rs
similarity index 55%
rename from tests/tls_server.rs
rename to tests/support/tls_server.rs
index 65a0a13..1d06f67 100644
--- a/tests/tls_server.rs
+++ b/tests/support/tls_server.rs
@@ -14,8 +14,42 @@ use std::net::SocketAddr;
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
 use tokio::net::TcpListener;
 
+/// The parts of a ClientHello that make up a TLS fingerprint.
+///
+/// Read back server-side from the handshake the client under test actually
+/// sent, so it reflects what went on the wire rather than what the config
+/// asked for. Those differ more often than you would like: `cipher_string`
+/// only governs TLS 1.2 and below, for instance, so the 1.3 suite list comes
+/// from somewhere else entirely.
+#[derive(Debug, Clone, Default)]
+pub struct CapturedHello {
+    /// The legacy `client_version` field, which is pinned at TLS 1.2 on any
+    /// modern hello. The real version lives in `supported_versions`.
+    pub legacy_version: u16,
+    /// Cipher suites in the order offered, GREASE included.
+    pub ciphers: Vec<u16>,
+    /// Extension types in the order sent, GREASE included.
+    pub extensions: Vec<u16>,
+    /// `supported_groups` (0x000a) contents, GREASE included.
+    pub groups: Vec<u16>,
+    /// `signature_algorithms` (0x000d) contents, in the order sent.
+    pub sig_algs: Vec<u16>,
+    /// ALPN protocol names offered, in order.
+    pub alpn: Vec<String>,
+    /// `supported_versions` (0x002b) contents, GREASE included.
+    pub supported_versions: Vec<u16>,
+    /// Whether a `server_name` (0x0000) extension was present.
+    pub has_sni: bool,
+}
+
+/// Where a capturing server leaves what it saw. `None` until a client
+/// completes enough of a handshake to send its hello.
+pub type HelloSlot = std::sync::Arc<std::sync::Mutex<Option<CapturedHello>>>;
+
 pub struct TlsTestServer {
     pub addr: SocketAddr,
+    /// What the client sent, once it has connected.
+    pub hello: HelloSlot,
     shutdown_tx: tokio::sync::oneshot::Sender<()>,
     handle: tokio::task::JoinHandle<()>,
 }
@@ -88,7 +122,117 @@ fn generate_self_signed(config: &TlsServerConfig) -> (PKey<openssl::pkey::Privat
     (pkey, cert)
 }
 
-fn build_acceptor(config: &TlsServerConfig) -> SslAcceptor {
+// ── ClientHello capture ───────────────────────────────────────────
+//
+// `SSL_CTX_set_client_hello_cb` fires after the hello is parsed and before a
+// cipher is chosen, which is the only point where the full offer is still
+// visible. The safe `openssl` wrapper covers ciphers and versions but has no
+// accessor for the extension list, so those go through openssl-sys.
+
+/// Split a big-endian `u16` array, as TLS encodes cipher and extension lists.
+fn be_u16s(bytes: &[u8]) -> Vec<u16> {
+    bytes
+        .chunks_exact(2)
+        .map(|c| u16::from_be_bytes([c[0], c[1]]))
+        .collect()
+}
+
+/// Read one extension's body, or `None` when the client didn't send it.
+fn extension_body(ssl: &mut openssl::ssl::SslRef, ext_type: u16) -> Option<Vec<u8>> {
+    use foreign_types_shared::ForeignTypeRef;
+    let mut out: *const std::ffi::c_uchar = std::ptr::null();
+    let mut outlen: usize = 0;
+    let found = unsafe {
+        openssl_sys::SSL_client_hello_get0_ext(
+            ssl.as_ptr(),
+            ext_type as std::ffi::c_uint,
+            &mut out,
+            &mut outlen,
+        )
+    };
+    if found != 1 || out.is_null() {
+        return None;
+    }
+    Some(unsafe { std::slice::from_raw_parts(out, outlen) }.to_vec())
+}
+
+/// A TLS vector whose length prefix is two bytes, which is how
+/// `supported_groups` and `signature_algorithms` wrap their contents.
+fn u16_vector_body(body: &[u8]) -> Vec<u16> {
+    if body.len() < 2 {
+        return Vec::new();
+    }
+    let len = u16::from_be_bytes([body[0], body[1]]) as usize;
+    be_u16s(&body[2..(2 + len).min(body.len())])
+}
+
+fn parse_alpn(body: &[u8]) -> Vec<String> {
+    // Two-byte list length, then repeated one-byte-length-prefixed names.
+    if body.len() < 2 {
+        return Vec::new();
+    }
+    let mut out = Vec::new();
+    let mut i = 2;
+    while i < body.len() {
+        let n = body[i] as usize;
+        i += 1;
+        if i + n > body.len() {
+            break;
+        }
+        out.push(String::from_utf8_lossy(&body[i..i + n]).into_owned());
+        i += n;
+    }
+    out
+}
+
+fn capture_hello(ssl: &mut openssl::ssl::SslRef) -> CapturedHello {
+    use foreign_types_shared::ForeignTypeRef;
+
+    let mut captured = CapturedHello {
+        legacy_version: unsafe {
+            openssl_sys::SSL_client_hello_get0_legacy_version(ssl.as_ptr()) as u16
+        },
+        ciphers: ssl.client_hello_ciphers().map(be_u16s).unwrap_or_default(),
+        ..Default::default()
+    };
+
+    // Extension types, in the order the client sent them. The array is
+    // allocated by OpenSSL and is ours to free.
+    let mut types: *mut std::ffi::c_int = std::ptr::null_mut();
+    let mut count: usize = 0;
+    let ok = unsafe {
+        openssl_sys::SSL_client_hello_get1_extensions_present(ssl.as_ptr(), &mut types, &mut count)
+    };
+    if ok == 1 && !types.is_null() {
+        captured.extensions = unsafe { std::slice::from_raw_parts(types, count) }
+            .iter()
+            .map(|t| *t as u16)
+            .collect();
+        unsafe { openssl_sys::OPENSSL_free(types as *mut std::ffi::c_void) };
+    }
+
+    captured.has_sni = captured.extensions.contains(&0x0000);
+    if let Some(b) = extension_body(ssl, 0x000a) {
+        captured.groups = u16_vector_body(&b);
+    }
+    if let Some(b) = extension_body(ssl, 0x000d) {
+        captured.sig_algs = u16_vector_body(&b);
+    }
+    if let Some(b) = extension_body(ssl, 0x0010) {
+        captured.alpn = parse_alpn(&b);
+    }
+    if let Some(b) = extension_body(ssl, 0x002b) {
+        // supported_versions uses a ONE-byte length prefix, unlike the others.
+        if !b.is_empty() {
+            let len = b[0] as usize;
+            captured.supported_versions = be_u16s(&b[1..(1 + len).min(b.len())]);
+        }
+    }
+
+    captured
+}
+
+fn build_acceptor(config: &TlsServerConfig, hello: HelloSlot) -> SslAcceptor {
     // Load the legacy provider so the server can use weak ciphers too
     load_legacy_provider();
 
@@ -119,6 +263,15 @@ fn build_acceptor(config: &TlsServerConfig) -> SslAcceptor {
         )
         .unwrap();
 
+    // Record the client's offer before a cipher is picked.
+    builder.set_client_hello_callback(move |ssl, _alert| {
+        let captured = capture_hello(ssl);
+        if let Ok(mut slot) = hello.lock() {
+            *slot = Some(captured);
+        }
+        Ok(openssl::ssl::ClientHelloResponse::SUCCESS)
+    });
+
     let (pkey, cert) = generate_self_signed(config);
     builder.set_private_key(&pkey).unwrap();
     builder.set_certificate(&cert).unwrap();
@@ -154,7 +307,8 @@ fn load_legacy_provider() {
 
 impl TlsTestServer {
     pub async fn start(config: TlsServerConfig) -> Self {
-        let acceptor = build_acceptor(&config);
+        let hello: HelloSlot = std::sync::Arc::new(std::sync::Mutex::new(None));
+        let acceptor = build_acceptor(&config, hello.clone());
 
         // Bind to random port on localhost
         let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -203,6 +357,7 @@ impl TlsTestServer {
 
         TlsTestServer {
             addr,
+            hello,
             shutdown_tx,
             handle,
         }
diff --git a/tests/tls_integration.rs b/tests/tls_integration.rs
index e4b39a7..a4ffa9c 100644
--- a/tests/tls_integration.rs
+++ b/tests/tls_integration.rs
@@ -2,13 +2,13 @@
 // These spin up a real TLS server using our custom OpenSSL build,
 // then connect with blasthttp's HyperClient to verify end-to-end behavior.
 
-mod tls_server;
+mod support;
 
 use blasthttp::client::HttpClient;
 use blasthttp::client::hyper::HyperClient;
 use blasthttp::config::RequestConfig;
 use openssl::ssl::SslVersion;
-use tls_server::{TlsServerConfig, TlsTestServer};
+use support::tls_server::{TlsServerConfig, TlsTestServer};
 
 fn make_config(url: &str) -> RequestConfig {
     let mut config = RequestConfig::new(url.to_string());

From 0446221770a155a88044e9dc3cec2f41fbb85b83 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 21:25:14 -0400
Subject: [PATCH 04/44] Add outcome classifier and live benchmark

Second piece of the 1.x benchmark work. The capture side answers what we
send; this answers what happens to us.

A scan has to tell three things apart that all arrive looking like a
response: the origin answered, a protection product answered instead, or
the product answered and handed us a session anyway. Collapsing those is
what turns a protected host into an apparently dead one, which is the
false negative the whole effort is about.

Classification keys on headers, cookies and status, never on a vendor's
name in the body. A first pass did the latter and scored datadome.co,
imperva.com and kasada.io as challenges from their own marketing copy;
that case is now a test.

Two things only showed up by running it against the real thing, and both
came from a fixture I had assumed rather than captured:

  - www.akamai.com's 403 carries no `server` header at all. It
    identifies itself with akamai-grn, x-akam-sw-version and an ak_p
    server-timing entry, so requiring `server: AkamaiGHost` matched
    nothing.
  - that page entity-encodes its own punctuation, so the body holds
    `errors&#46;edgesuite&#46;net` and a search for the dotted hostname
    never fires. Matching the bare label survives it.

The live benchmark is opt-in (`--ignored`) and prints a scorecard rather
than asserting, since "we are not blocked" would fail for reasons
outside this repository. It also cross-checks our local JA4 against a
remote observer, which surfaced a known divergence worth recording: the
JA4 header is fixed width and the spec caps counts at 99, our legacy
list is 105 suites, and tls.peet.ws prints 105 where we print 99. Both
hash segments agree exactly, so only the rendering differs. A stealth
profile at 15 suites removes it.

Current scorecard: 1/4 through. cloudflare.com present, akamai.com
blocked, both scrapingcourse challenge pages unreachable as expected
since that tier wants JavaScript.
---
 tests/antibot_classify.rs | 261 ++++++++++++++++++++++++++++++++++++++
 tests/ja4_conformance.rs  |  26 ++++
 tests/live_benchmark.rs   | 199 +++++++++++++++++++++++++++++
 tests/support/antibot.rs  | 261 ++++++++++++++++++++++++++++++++++++++
 tests/support/mod.rs      |   1 +
 5 files changed, 748 insertions(+)
 create mode 100644 tests/antibot_classify.rs
 create mode 100644 tests/live_benchmark.rs
 create mode 100644 tests/support/antibot.rs

diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
new file mode 100644
index 0000000..fc82e3e
--- /dev/null
+++ b/tests/antibot_classify.rs
@@ -0,0 +1,261 @@
+// The outcome classifier, against real captured responses.
+//
+// Fixtures are trimmed from responses actually observed on 2026-09-28, not
+// invented. The Akamai one in particular is the exact page `www.akamai.com`
+// returned to blasthttp while returning 200 with real content to a
+// Chrome-impersonating client from the same address in the same minute. That
+// pair is the clearest single measurement of the problem this effort exists to
+// fix, so it is worth having as a regression fixture.
+
+mod support;
+
+use support::antibot::{Outcome, ResponseFacts, Vendor, classify};
+
+fn hdrs(pairs: &[(&str, &str)]) -> Vec<(String, String)> {
+    pairs
+        .iter()
+        .map(|(k, v)| (k.to_string(), v.to_string()))
+        .collect()
+}
+
+fn facts<'a>(status: u16, headers: &'a [(String, String)], body: &'a str) -> ResponseFacts<'a> {
+    ResponseFacts {
+        status,
+        headers,
+        body,
+    }
+}
+
+// ── Real captures ─────────────────────────────────────────────────
+
+/// Exactly what www.akamai.com served blasthttp, 366 bytes, reproduced 3/3.
+const AKAMAI_DENIED: &str = concat!(
+    "<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD><BODY>\n<H1>Access Denied</H1>\n \n",
+    "You don't have permission to access \"http&#58;&#47;&#47;www&#46;akamai&#46;com&#47;\" ",
+    "on this server.<P>\nReference&#32;&#35;18&#46;d3952a17&#46;1790609813&#46;f486746\n",
+    "<P>https&#58;&#47;&#47;errors&#46;edgesuite&#46;net&#47;18&#46;d3952a17&#46;1790609813&#46;f486746\n",
+    "</BODY>\n</HTML>\n"
+);
+
+#[test]
+fn test_akamai_access_denied_is_a_block() {
+    // These are the headers www.akamai.com actually returned, not a plausible
+    // set. Two things about them broke the first version of the classifier and
+    // are the reason this fixture is verbatim:
+    //
+    //   - there is no `server` header at all, so a rule requiring
+    //     `server: AkamaiGHost` never fired
+    //   - the body entity-encodes its own punctuation, so a search for
+    //     `errors.edgesuite.net` never matched `errors&#46;edgesuite&#46;net`
+    //
+    // Both were caught by running the live benchmark, which classified this
+    // exact response as Blocked(Unknown).
+    let h = hdrs(&[
+        ("mime-version", "1.0"),
+        ("content-type", "text/html"),
+        ("content-length", "368"),
+        ("server-timing", "cdn-cache; desc=HIT"),
+        (
+            "server-timing",
+            "ak_p; desc=\"1790644996525_35083629_828334756_15_11554_9_12_-\";dur=1",
+        ),
+        ("x-akam-sw-version", "0.5.0"),
+        ("akamai-grn", "0.6d551702.1790644996.315f62a4"),
+        (
+            "set-cookie",
+            "akaas_production=2147483647~rv=32~id=e124a124de5a16eb7688b22c80f8e95d;",
+        ),
+    ]);
+    assert_eq!(
+        classify(&facts(403, &h, AKAMAI_DENIED)),
+        Outcome::Blocked(Vendor::Akamai)
+    );
+}
+
+#[test]
+fn test_akamai_block_detected_without_a_server_header() {
+    // Narrower version of the above: the edge markers alone have to carry it.
+    let h = hdrs(&[("akamai-grn", "0.6d551702.1790644996.315f62a4")]);
+    assert_eq!(
+        classify(&facts(
+            403,
+            &h,
+            "<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD></HTML>"
+        )),
+        Outcome::Blocked(Vendor::Akamai)
+    );
+}
+
+#[test]
+fn test_entity_encoded_edgesuite_still_matches() {
+    // No Akamai headers whatsoever, just the encoded error page body.
+    let h = hdrs(&[("content-type", "text/html")]);
+    assert_eq!(
+        classify(&facts(403, &h, AKAMAI_DENIED)),
+        Outcome::Blocked(Vendor::Akamai)
+    );
+}
+
+#[test]
+fn test_akamai_session_cookie_on_success_is_present() {
+    // The other half of the same comparison: through, and handed a session.
+    let h = hdrs(&[
+        ("server", "AkamaiNetStorage"),
+        (
+            "set-cookie",
+            "_abck=A7F1...~0~YAAQ...; Path=/; Domain=.akamai.com",
+        ),
+        ("set-cookie", "bm_sz=8C2...; Path=/"),
+    ]);
+    let outcome = classify(&facts(200, &h, "<html><body>real content</body></html>"));
+    assert_eq!(outcome, Outcome::Present(Vendor::Akamai));
+    assert!(outcome.got_through());
+}
+
+#[test]
+fn test_cloudflare_managed_challenge() {
+    // nowsecure.nl and the scrapingcourse challenge pages both answer this
+    // way. curl_cffi does not pass it either: this tier wants JavaScript, and
+    // no amount of fingerprint work reaches it.
+    let h = hdrs(&[
+        ("server", "cloudflare"),
+        ("cf-mitigated", "challenge"),
+        ("cf-ray", "a423d7b45e21b0e8-ATL"),
+    ]);
+    assert_eq!(
+        classify(&facts(403, &h, "<html>Just a moment...</html>")),
+        Outcome::Challenge(Vendor::Cloudflare)
+    );
+}
+
+#[test]
+fn test_cloudflare_waf_block_is_not_a_challenge() {
+    // A plain WAF refusal carries no cf-mitigated header. Calling it a
+    // challenge would suggest a JS tier that is not actually there.
+    let h = hdrs(&[("server", "cloudflare"), ("cf-ray", "a423d7b45e21b0e8-ATL")]);
+    let outcome = classify(&facts(403, &h, "<html>error 1020</html>"));
+    assert!(
+        matches!(outcome, Outcome::Blocked(_)),
+        "expected a block, got {:?}",
+        outcome
+    );
+    assert_ne!(outcome, Outcome::Challenge(Vendor::Cloudflare));
+}
+
+#[test]
+fn test_cloudflare_bm_cookie_on_success_is_present() {
+    // www.cloudflare.com sets __cf_bm on an ordinary 200.
+    let h = hdrs(&[
+        ("server", "cloudflare"),
+        ("set-cookie", "__cf_bm=hNs8cwmP7rgTn3Us...; Path=/; Secure"),
+    ]);
+    assert_eq!(
+        classify(&facts(200, &h, "<html>content</html>")),
+        Outcome::Present(Vendor::Cloudflare)
+    );
+}
+
+#[test]
+fn test_akamai_sec_cpt_428_is_its_own_outcome() {
+    // Fires after _abck has already passed, and answers 428 rather than 403.
+    // Folding it into "blocked" would hide that we got further than usual.
+    let h = hdrs(&[
+        ("server", "AkamaiGHost"),
+        ("set-cookie", "sec_cpt=abc123; Path=/"),
+    ]);
+    assert_eq!(
+        classify(&facts(428, &h, r#"{"chlg_duration":30,"difficulty":5}"#)),
+        Outcome::Challenge(Vendor::Akamai)
+    );
+}
+
+// ── The false positives that caught the first pass ────────────────
+
+#[test]
+fn test_vendor_homepages_are_not_challenges() {
+    // Every one of these scored CHALLENGE on the first attempt, purely
+    // because a body grep found the vendor's own name in its marketing copy.
+    // A vendor homepage is a terrible probe and the classifier must not be
+    // fooled by one.
+    for (host, body) in [
+        (
+            "datadome.co",
+            "<html><h1>DataDome</h1><p>DataDome protects against bots. \
+             Learn why DataDome is trusted.</p></html>",
+        ),
+        (
+            "imperva.com",
+            "<html><h1>Imperva</h1><p>Imperva Incapsula delivers \
+             application security.</p></html>",
+        ),
+        (
+            "kasada.io",
+            "<html><h1>Kasada</h1><p>Kasada stops bots cold.</p></html>",
+        ),
+        (
+            "humansecurity.com",
+            "<html><p>HUMAN, formerly PerimeterX, defends against bots.</p></html>",
+        ),
+    ] {
+        let h = hdrs(&[("content-type", "text/html")]);
+        assert_eq!(
+            classify(&facts(200, &h, body)),
+            Outcome::Ok,
+            "{} marketing copy was misread as a protection response",
+            host
+        );
+    }
+}
+
+#[test]
+fn test_reference_number_alone_is_not_akamai() {
+    // Plenty of error pages carry a reference number. Without the edgesuite
+    // host or the AkamaiGHost server header it means nothing.
+    let h = hdrs(&[("server", "nginx")]);
+    let outcome = classify(&facts(
+        403,
+        &h,
+        "<html><h1>Access Denied</h1><p>Reference #12345</p></html>",
+    ));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+}
+
+// ── Ordinary responses ────────────────────────────────────────────
+
+#[test]
+fn test_plain_success_is_ok() {
+    let h = hdrs(&[("server", "nginx"), ("content-type", "text/html")]);
+    let outcome = classify(&facts(200, &h, "<html><body>hello</body></html>"));
+    assert_eq!(outcome, Outcome::Ok);
+    assert!(outcome.got_through());
+}
+
+#[test]
+fn test_ordinary_404_is_not_a_block() {
+    // A missing page is a real answer from the origin, and counting it as a
+    // block would make an ordinary scan look like it was being refused.
+    let h = hdrs(&[("server", "nginx")]);
+    assert_eq!(classify(&facts(404, &h, "not found")), Outcome::Error);
+}
+
+#[test]
+fn test_unidentified_refusal_reports_unknown_vendor() {
+    // Better to say a block happened and admit we cannot name who did it than
+    // to guess a vendor.
+    let h = hdrs(&[("server", "nginx")]);
+    let outcome = classify(&facts(403, &h, "forbidden"));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+    assert!(!outcome.got_through());
+}
+
+#[test]
+fn test_product_cookie_with_refusal_still_counts_as_blocked() {
+    // Setting a session cookie while refusing is still a refusal.
+    let h = hdrs(&[
+        ("server", "cloudflare"),
+        ("set-cookie", "__cf_bm=xyz; Path=/"),
+    ]);
+    let outcome = classify(&facts(403, &h, "<html>denied</html>"));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::Cloudflare));
+    assert!(!outcome.got_through());
+}
diff --git a/tests/ja4_conformance.rs b/tests/ja4_conformance.rs
index fafad93..4959953 100644
--- a/tests/ja4_conformance.rs
+++ b/tests/ja4_conformance.rs
@@ -125,6 +125,32 @@ fn test_grease_does_not_change_ja4() {
     );
 }
 
+#[test]
+fn test_counts_cap_at_99() {
+    // The JA4 header is fixed width: one character of protocol, two of
+    // version, one of SNI, two of cipher count, two of extension count, two of
+    // ALPN. A three-digit count would make it unparseable, so the spec caps
+    // both counts at 99.
+    //
+    // This is not academic for us. Our legacy cipher list is 105 suites, so we
+    // sit past the cap, and `tls.peet.ws` renders that as `105` where we
+    // render `99`. Their hashes and ours agree exactly, so the underlying
+    // reading is identical and only the count field diverges. We follow the
+    // spec, because the whole purpose here is comparing against browser JA4s
+    // and every browser is far below the cap. Once a stealth profile ships at
+    // 15 suites the divergence disappears on its own.
+    let mut absurd = chrome_150();
+    absurd.ciphers = (1u16..=150).collect();
+    absurd.extensions = (1u16..=150).collect();
+
+    let header = support::ja4::ja4(&absurd);
+    let header = header.split('_').next().unwrap();
+
+    assert_eq!(&header[4..6], "99", "cipher count should cap at 99");
+    assert_eq!(&header[6..8], "99", "extension count should cap at 99");
+    assert_eq!(header.len(), 10, "header must stay 10 characters wide");
+}
+
 #[test]
 fn test_sig_alg_order_does_change_ja4() {
     // Signature algorithms are the one list JA4 leaves in the order sent, so
diff --git a/tests/live_benchmark.rs b/tests/live_benchmark.rs
new file mode 100644
index 0000000..a93ea22
--- /dev/null
+++ b/tests/live_benchmark.rs
@@ -0,0 +1,199 @@
+// Live measurement against real services. Opt-in, never part of CI.
+//
+//     cargo test --test live_benchmark -- --ignored --nocapture
+//
+// Everything here needs the network and some of it needs third-party services
+// that rate-limit, go away, or change behaviour between runs. None of that
+// belongs in a gate. These print a scorecard and assert only that the
+// measurement itself worked, because an assertion like "we are not blocked"
+// would fail for reasons entirely outside this repository.
+//
+// The deterministic gates live in fingerprint_baseline.rs (what we send),
+// ja4_conformance.rs (whether our JA4 agrees with everyone else's) and
+// antibot_classify.rs (how outcomes are read). Those run offline in CI.
+
+mod support;
+
+use blasthttp::client::HttpClient;
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::config::RequestConfig;
+use support::antibot::{Outcome, ResponseFacts, classify};
+use support::tls_server::{TlsServerConfig, TlsTestServer};
+
+fn config(url: &str) -> RequestConfig {
+    let mut c = RequestConfig::new(url.to_string());
+    c.verify_certs = Some(false);
+    c.timeout_seconds = Some(30);
+    c.follow_redirects = Some(true);
+    c
+}
+
+async fn fetch(url: &str) -> Option<(u16, Vec<(String, String)>, String)> {
+    let client = HyperClient::new();
+    match client.send(&config(url)).await {
+        Ok(r) => {
+            let body = r.body().to_string();
+            Some((r.status, r.headers.clone(), body))
+        }
+        Err(e) => {
+            println!("    request failed: {}", e.message);
+            None
+        }
+    }
+}
+
+/// Our own ClientHello, captured locally, addressed by hostname so SNI is sent
+/// and the result is directly comparable with what a remote service computes.
+async fn local_ja4() -> String {
+    let server = TlsTestServer::start(TlsServerConfig::default()).await;
+    let url = format!("https://localhost:{}/", server.addr.port());
+
+    let client = HyperClient::new();
+    let _ = client.send(&config(&url)).await;
+
+    let hello = server
+        .hello
+        .lock()
+        .expect("hello slot poisoned")
+        .clone()
+        .expect("no ClientHello captured");
+    server.shutdown().await;
+
+    support::ja4::ja4(&hello)
+}
+
+#[tokio::test]
+#[ignore = "needs network; run with --ignored"]
+async fn live_ja4_agrees_with_a_remote_observer() {
+    // Cross-checks the whole capture path. Our local reading and a third
+    // party's should describe the same client, since it is the same client.
+    // A mismatch means either the capture is wrong or something between us and
+    // them is rewriting the handshake, and both are worth knowing.
+    let local = local_ja4().await;
+    println!("\n  local JA4:  {}", local);
+
+    let Some((status, _, body)) = fetch("https://tls.peet.ws/api/all").await else {
+        println!("  remote observer unreachable, skipping comparison");
+        return;
+    };
+    assert_eq!(status, 200, "fingerprint service did not answer");
+
+    let parsed: serde_json::Value = serde_json::from_str(&body).expect("expected JSON");
+    let remote = parsed["tls"]["ja4"].as_str().unwrap_or("(absent)");
+    println!("  remote JA4: {}", remote);
+
+    // Not an assertion either way. JA4 implementations genuinely disagree:
+    // one handshake shown to seven services produced three different JA3s and
+    // two different JA4s.
+    //
+    // One divergence here is known and expected. The JA4 header is fixed
+    // width and the spec caps both counts at 99; our legacy list is 105
+    // suites, so we print `99` where peet.ws prints `105`. The two hash
+    // segments match exactly, which says the underlying reading is the same
+    // and only the rendering differs. A stealth profile at 15 suites makes it
+    // go away.
+    if local == remote {
+        println!("  agree");
+    } else {
+        let (l, r) = (local.split('_').next(), remote.split('_').next());
+        if local.split('_').skip(1).eq(remote.split('_').skip(1)) {
+            println!("  headers differ ({:?} vs {:?}), hashes agree", l, r);
+            println!("  (expected while our cipher count is over the 99 cap)");
+        } else {
+            println!("  hashes differ, worth investigating which of us is wrong");
+        }
+    }
+}
+
+#[tokio::test]
+#[ignore = "needs network; run with --ignored"]
+async fn live_fingerprint_prevalence() {
+    // The check nothing else provides: does our ClientHello occur in real
+    // traffic at all? A profile can match a published JA4 exactly and still be
+    // a shape no real client emits, and no signature comparison catches that.
+    //
+    // Measured 2026-09-28: before the legacy cipher work our hello had been
+    // seen 903 times across roughly 17B observations; after, it does not
+    // appear at all. A real Chrome sits above 450M.
+    let Some((_, _, body)) = fetch("https://tls.tlsfingerprint.io/api/client-fingerprint").await
+    else {
+        println!("  fingerprint database unreachable, skipping");
+        return;
+    };
+
+    let parsed: serde_json::Value = match serde_json::from_str(&body) {
+        Ok(v) => v,
+        Err(_) => {
+            println!("  unexpected response shape, skipping");
+            return;
+        }
+    };
+    let Some(id) = parsed["norm_hex_id"].as_str() else {
+        println!("  no norm_hex_id in response, skipping");
+        return;
+    };
+
+    println!("\n  our ClientHello id: {}", id);
+
+    let url = format!(
+        "https://tlsfingerprint.io/api/tls/fingerprints/{}/exists",
+        id
+    );
+    let Some((_, _, seen)) = fetch(&url).await else {
+        return;
+    };
+
+    let parsed: serde_json::Value = serde_json::from_str(&seen).unwrap_or_default();
+    match parsed["total"].as_u64() {
+        Some(n) => println!("  observed {} times in real traffic", n),
+        None => println!("  never observed in real traffic (the loudest signal we emit)"),
+    }
+}
+
+#[tokio::test]
+#[ignore = "needs network; run with --ignored"]
+async fn live_block_scorecard() {
+    // One GET per target, classified structurally.
+    //
+    // Target choice is deliberate. The scrapingcourse pages exist to be
+    // scraped against. akamai.com is the vendor's own property and the
+    // clearest discriminator we have: it refused blasthttp and served a
+    // Chrome-impersonating client 219KB from the same address in the same
+    // minute. cloudflare.com is a control that should always pass.
+    //
+    // Do not grow this into a list of ordinary commercial sites, and do not
+    // put it on a schedule.
+    let targets = [
+        ("cloudflare.com (control)", "https://www.cloudflare.com/"),
+        ("akamai.com", "https://www.akamai.com/"),
+        (
+            "scrapingcourse cf-challenge",
+            "https://www.scrapingcourse.com/cloudflare-challenge",
+        ),
+        (
+            "scrapingcourse antibot",
+            "https://www.scrapingcourse.com/antibot-challenge",
+        ),
+    ];
+
+    println!();
+    let mut through = 0;
+    for (label, url) in targets {
+        let outcome = match fetch(url).await {
+            Some((status, headers, body)) => classify(&ResponseFacts {
+                status,
+                headers: &headers,
+                body: &body,
+            }),
+            None => Outcome::Error,
+        };
+        if outcome.got_through() {
+            through += 1;
+        }
+        println!("  {:32} {:?}", label, outcome);
+    }
+    println!("\n  got through: {}/{}", through, targets.len());
+    println!("  (the JS challenge tier is out of reach for any HTTP client,");
+    println!("   curl_cffi included, so the scrapingcourse pages are expected");
+    println!("   to stay unreachable until something executes JavaScript)");
+}
diff --git a/tests/support/antibot.rs b/tests/support/antibot.rs
new file mode 100644
index 0000000..340e307
--- /dev/null
+++ b/tests/support/antibot.rs
@@ -0,0 +1,261 @@
+// Classifying what a bot-management product did to a request.
+//
+// A scan needs to tell three things apart that all look like "a response":
+// the origin answered, a protection product answered instead, or the product
+// answered *and* handed us a session. Without that, an operator sees a wall of
+// 403s and cannot tell a blocked host from a dead one, which is the false
+// negative this whole effort exists to fix.
+//
+// Two rules learned the hard way, both from a first pass that got them wrong:
+//
+// 1. Never match a vendor's name in the body. `datadome.co`, `imperva.com` and
+//    `kasada.io` all scored as challenges purely from their own marketing
+//    copy. Everything here keys on headers, cookies, status, or a body string
+//    that cannot occur in ordinary prose.
+//
+// 2. Always classify a known-good reference client in the same pass. A block
+//    and a site that simply changed look identical from one sample.
+
+#![allow(dead_code)]
+
+use std::fmt;
+
+/// Which product answered. Not exhaustive, and deliberately conservative:
+/// guessing a vendor wrong is worse than reporting `Unknown`.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum Vendor {
+    Cloudflare,
+    Akamai,
+    DataDome,
+    PerimeterX,
+    Imperva,
+    F5,
+    Kasada,
+    Unknown,
+}
+
+impl fmt::Display for Vendor {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let s = match self {
+            Vendor::Cloudflare => "cloudflare",
+            Vendor::Akamai => "akamai",
+            Vendor::DataDome => "datadome",
+            Vendor::PerimeterX => "perimeterx",
+            Vendor::Imperva => "imperva",
+            Vendor::F5 => "f5",
+            Vendor::Kasada => "kasada",
+            Vendor::Unknown => "unknown",
+        };
+        f.write_str(s)
+    }
+}
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum Outcome {
+    /// Ordinary response, no protection product detected.
+    Ok,
+    /// A product is in front of this host and let us through, usually having
+    /// set a session cookie. The most useful signal there is: we passed
+    /// something that was actively looking.
+    Present(Vendor),
+    /// An interactive challenge. Out of reach for any HTTP client, since
+    /// answering means executing the page's JavaScript.
+    Challenge(Vendor),
+    /// A hard refusal, no challenge offered.
+    Blocked(Vendor),
+    /// Never got a usable response.
+    Error,
+}
+
+impl Outcome {
+    /// Did we get real content? The bar the benchmark scores against.
+    pub fn got_through(&self) -> bool {
+        matches!(self, Outcome::Ok | Outcome::Present(_))
+    }
+
+    pub fn vendor(&self) -> Option<Vendor> {
+        match self {
+            Outcome::Present(v) | Outcome::Challenge(v) | Outcome::Blocked(v) => Some(*v),
+            _ => None,
+        }
+    }
+}
+
+/// Everything a classification looks at. Headers and cookies are expected
+/// lowercase-insensitive; `headers` holds `(name, value)` pairs as received,
+/// duplicates and all.
+pub struct ResponseFacts<'a> {
+    pub status: u16,
+    pub headers: &'a [(String, String)],
+    pub body: &'a str,
+}
+
+impl ResponseFacts<'_> {
+    fn header_matches(&self, name: &str, needle: &str) -> bool {
+        self.headers
+            .iter()
+            .any(|(k, v)| k.eq_ignore_ascii_case(name) && v.to_ascii_lowercase().contains(needle))
+    }
+
+    fn has_header(&self, name: &str) -> bool {
+        self.headers
+            .iter()
+            .any(|(k, _)| k.eq_ignore_ascii_case(name))
+    }
+
+    /// Cookie names set by this response. Only the name is examined; values
+    /// are session data and carry nothing we want to match on.
+    fn set_cookie_names(&self) -> Vec<String> {
+        self.headers
+            .iter()
+            .filter(|(k, _)| k.eq_ignore_ascii_case("set-cookie"))
+            .filter_map(|(_, v)| v.split('=').next())
+            .map(|n| n.trim().to_ascii_lowercase())
+            .collect()
+    }
+
+    fn sets_cookie(&self, name: &str) -> bool {
+        let want = name.to_ascii_lowercase();
+        self.set_cookie_names().contains(&want)
+    }
+
+    fn sets_cookie_prefixed(&self, prefix: &str) -> bool {
+        let want = prefix.to_ascii_lowercase();
+        self.set_cookie_names().iter().any(|n| n.starts_with(&want))
+    }
+
+    fn body_contains(&self, needle: &str) -> bool {
+        // Bodies can be large; a challenge marker is always near the top.
+        const WINDOW: usize = 64 * 1024;
+        let end = self.body.len().min(WINDOW);
+        self.body[..end].contains(needle)
+    }
+}
+
+/// Classify one response.
+///
+/// Order matters: a challenge outranks a mere sighting of the vendor, and a
+/// block outranks both, because a host that challenges us has still told us
+/// more than one that refuses outright.
+pub fn classify(facts: &ResponseFacts) -> Outcome {
+    // ── Cloudflare ────────────────────────────────────────────────
+    // `cf-mitigated: challenge` is the unambiguous one and the only header
+    // Cloudflare sets specifically to say it intervened.
+    if facts.header_matches("cf-mitigated", "challenge") {
+        return Outcome::Challenge(Vendor::Cloudflare);
+    }
+    if facts.body_contains("/cdn-cgi/challenge-platform/")
+        || facts.body_contains("window._cf_chl_opt")
+    {
+        return Outcome::Challenge(Vendor::Cloudflare);
+    }
+
+    // ── Akamai ────────────────────────────────────────────────────
+    // The edge error page, which is fussier to spot than it looks.
+    //
+    // It HTML-entity-encodes its own punctuation, so the body carries
+    // `errors&#46;edgesuite&#46;net` and a search for `errors.edgesuite.net`
+    // never fires. Matching the bare label `edgesuite` survives that, since
+    // only the dots and slashes get encoded.
+    //
+    // `server: AkamaiGHost` is also not reliably present: www.akamai.com's own
+    // 403 sends no `server` header at all and identifies itself through
+    // `akamai-grn`, `x-akam-sw-version` and an `ak_p` server-timing entry
+    // instead. So the server header is one signal among several rather than a
+    // requirement. `Reference #` on its own stays insufficient, because plenty
+    // of unrelated error pages carry one.
+    if facts.body_contains("edgesuite")
+        || (is_akamai_edge(facts) && facts.body_contains("Access Denied"))
+    {
+        return Outcome::Blocked(Vendor::Akamai);
+    }
+    // sec-cpt is an adaptive challenge that fires after _abck already passed,
+    // and it answers 428 rather than 403. Its own outcome, not a block.
+    if facts.status == 428 && (facts.sets_cookie("sec_cpt") || facts.body_contains("sec-cpt")) {
+        return Outcome::Challenge(Vendor::Akamai);
+    }
+
+    // ── DataDome ──────────────────────────────────────────────────
+    if facts.body_contains("geo.captcha-delivery.com") {
+        return Outcome::Challenge(Vendor::DataDome);
+    }
+
+    // ── PerimeterX / HUMAN ────────────────────────────────────────
+    if facts.body_contains("/px/captcha") || facts.body_contains("px-captcha") {
+        return Outcome::Challenge(Vendor::PerimeterX);
+    }
+
+    // ── Imperva / Incapsula ───────────────────────────────────────
+    if facts.body_contains("_Incapsula_Resource") {
+        return Outcome::Challenge(Vendor::Imperva);
+    }
+
+    // ── Kasada ────────────────────────────────────────────────────
+    // Kasada answers 429 rather than 403 for a refusal.
+    if facts.has_header("x-kpsdk-ct") || facts.body_contains("KPSDK") {
+        return if facts.status == 429 {
+            Outcome::Blocked(Vendor::Kasada)
+        } else {
+            Outcome::Present(Vendor::Kasada)
+        };
+    }
+
+    // ── Product present, and we got through ───────────────────────
+    // Reached only when nothing above fired, so these are sightings on a
+    // response that actually carried content.
+    let present = if facts.sets_cookie("__cf_bm") || facts.sets_cookie("cf_clearance") {
+        Some(Vendor::Cloudflare)
+    } else if facts.sets_cookie("_abck")
+        || facts.sets_cookie("ak_bmsc")
+        || facts.sets_cookie("bm_sz")
+    {
+        Some(Vendor::Akamai)
+    } else if facts.sets_cookie("datadome") {
+        Some(Vendor::DataDome)
+    } else if facts.sets_cookie_prefixed("_px") {
+        Some(Vendor::PerimeterX)
+    } else if facts.sets_cookie_prefixed("visid_incap_") || facts.sets_cookie_prefixed("incap_ses_")
+    {
+        Some(Vendor::Imperva)
+    } else if facts.header_matches("server", "volt-adc") {
+        Some(Vendor::F5)
+    } else {
+        None
+    };
+
+    if let Some(vendor) = present {
+        // A product that set its cookie while refusing us is still a block.
+        return if is_refusal(facts.status) {
+            Outcome::Blocked(vendor)
+        } else {
+            Outcome::Present(vendor)
+        };
+    }
+
+    // ── Nothing identifiable ──────────────────────────────────────
+    if (200..400).contains(&facts.status) {
+        Outcome::Ok
+    } else if is_refusal(facts.status) {
+        Outcome::Blocked(Vendor::Unknown)
+    } else {
+        Outcome::Error
+    }
+}
+
+/// Statuses a protection product uses to refuse. 429 is in here because a rate
+/// limit and a bot block are the same outcome for a scan, whoever sent it.
+fn is_refusal(status: u16) -> bool {
+    matches!(status, 401 | 403 | 405 | 406 | 429 | 503)
+}
+
+/// Is this response coming off Akamai's edge at all?
+///
+/// Any one of these is enough. They are the headers the edge attaches to its
+/// own responses regardless of whether it is serving content or refusing.
+fn is_akamai_edge(facts: &ResponseFacts) -> bool {
+    facts.header_matches("server", "akamaighost")
+        || facts.has_header("akamai-grn")
+        || facts.has_header("x-akam-sw-version")
+        || facts.has_header("x-akamai-transformed")
+        || facts.header_matches("server-timing", "ak_p")
+}
diff --git a/tests/support/mod.rs b/tests/support/mod.rs
index 1163676..67b68db 100644
--- a/tests/support/mod.rs
+++ b/tests/support/mod.rs
@@ -10,5 +10,6 @@
 
 #![allow(dead_code)]
 
+pub mod antibot;
 pub mod ja4;
 pub mod tls_server;

From e3af326f54e7c785e3273f1071210cfc29a63ac5 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 21:28:32 -0400
Subject: [PATCH 05/44] Score the benchmark per tier, not as one number

The scorecard reported "1/4 got through" when the real bypass count was
zero. Two ways that number lied.

The one success was cloudflare.com, which is a control. Its homepage
challenges nobody, so every client reaches it and passing says nothing
about us. Counting it made a 0% bypass rate read as 25%.

It also put the JavaScript tier in the denominator. Cloudflare's managed
challenge and the scrapingcourse pages want the browser to execute their
code and report back, which no HTTP client does, curl_cffi included. So
the score could never approach 4/4 however good the fingerprint work
got, and a permanently unreachable denominator makes the metric useless
for tracking progress.

Now targets carry a tier. Controls are verified and not scored, and a
control that fails aborts the run, since a blocked control means the
network path or the site changed and nothing else in the run can be
trusted. The JavaScript tier is reported so a change would be visible
but never counted against us. Only the passive-fingerprint tier is
scored, because it is the only one this work can move.

Reads 0/1 on the passive tier today. That is the number stealth mode has
to change, and curl_cffi already scores 1/1 on it.
---
 tests/live_benchmark.rs | 93 +++++++++++++++++++++++++++++++++--------
 1 file changed, 75 insertions(+), 18 deletions(-)

diff --git a/tests/live_benchmark.rs b/tests/live_benchmark.rs
index a93ea22..2c51900 100644
--- a/tests/live_benchmark.rs
+++ b/tests/live_benchmark.rs
@@ -153,32 +153,65 @@ async fn live_fingerprint_prevalence() {
 #[tokio::test]
 #[ignore = "needs network; run with --ignored"]
 async fn live_block_scorecard() {
-    // One GET per target, classified structurally.
+    // One GET per target, classified structurally, scored per tier.
     //
-    // Target choice is deliberate. The scrapingcourse pages exist to be
-    // scraped against. akamai.com is the vendor's own property and the
-    // clearest discriminator we have: it refused blasthttp and served a
-    // Chrome-impersonating client 219KB from the same address in the same
-    // minute. cloudflare.com is a control that should always pass.
+    // Scoring across tiers as one number is worse than useless, which an
+    // earlier version of this proved by reporting "1/4" when the true bypass
+    // count was zero. Two ways that number lied:
+    //
+    //   - the one success was the control, which protects nothing, so passing
+    //     it demonstrates nothing
+    //   - it counted the JS tier in the denominator, which no HTTP client
+    //     reaches, so the score could never approach 4/4 however good the
+    //     work got
+    //
+    // So: controls are verified and not scored, and only the passive tier is
+    // scored, because that is the only tier this work can move.
     //
     // Do not grow this into a list of ordinary commercial sites, and do not
     // put it on a schedule.
+
+    /// What a target is for, which decides whether and how it is scored.
+    enum Tier {
+        /// Protects nothing. Confirms the harness reaches the internet and
+        /// reads an ordinary response correctly.
+        Control,
+        /// Passive fingerprinting. The tier stealth mode exists to beat, and
+        /// the only one worth a score.
+        Passive,
+        /// Interactive JavaScript challenge. Out of reach for any HTTP client,
+        /// curl_cffi included. Reported so a change would be visible, never
+        /// counted against us.
+        Javascript,
+    }
+
     let targets = [
-        ("cloudflare.com (control)", "https://www.cloudflare.com/"),
-        ("akamai.com", "https://www.akamai.com/"),
         (
+            Tier::Control,
+            "cloudflare.com",
+            "https://www.cloudflare.com/",
+        ),
+        // The vendor's own property, and the clearest discriminator we have:
+        // it refused blasthttp and served a Chrome-impersonating client 219KB
+        // from the same address in the same minute.
+        (Tier::Passive, "akamai.com", "https://www.akamai.com/"),
+        (
+            Tier::Javascript,
             "scrapingcourse cf-challenge",
             "https://www.scrapingcourse.com/cloudflare-challenge",
         ),
         (
+            Tier::Javascript,
             "scrapingcourse antibot",
             "https://www.scrapingcourse.com/antibot-challenge",
         ),
     ];
 
     println!();
-    let mut through = 0;
-    for (label, url) in targets {
+    let (mut passive_through, mut passive_total) = (0, 0);
+    let mut control_failed = false;
+
+    for (tier, label, url) in targets {
         let outcome = match fetch(url).await {
             Some((status, headers, body)) => classify(&ResponseFacts {
                 status,
@@ -187,13 +220,37 @@ async fn live_block_scorecard() {
             }),
             None => Outcome::Error,
         };
-        if outcome.got_through() {
-            through += 1;
-        }
-        println!("  {:32} {:?}", label, outcome);
+
+        let tag = match tier {
+            Tier::Control => {
+                if !outcome.got_through() {
+                    control_failed = true;
+                }
+                "control"
+            }
+            Tier::Passive => {
+                passive_total += 1;
+                if outcome.got_through() {
+                    passive_through += 1;
+                }
+                "passive"
+            }
+            Tier::Javascript => "js",
+        };
+        println!("  [{:7}] {:30} {:?}", tag, label, outcome);
     }
-    println!("\n  got through: {}/{}", through, targets.len());
-    println!("  (the JS challenge tier is out of reach for any HTTP client,");
-    println!("   curl_cffi included, so the scrapingcourse pages are expected");
-    println!("   to stay unreachable until something executes JavaScript)");
+
+    println!();
+    println!(
+        "  passive-fingerprint tier: {}/{} reached",
+        passive_through, passive_total
+    );
+    println!("  javascript tier: not scored, unreachable without a browser");
+    println!("  controls: not scored");
+
+    assert!(
+        !control_failed,
+        "a control target was blocked, so this run says nothing about us: \
+         either the network path is interfering or the site changed"
+    );
 }

From e1b4905e0f2466af3147469bdbeb28445cb2c0b2 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 21:42:31 -0400
Subject: [PATCH 06/44] Add browser profiles and a Chrome TLS profile

Starts closing the gap the benchmark measures. A profile is data rather
than code, because browsers ship every few weeks and anything needing a
rewrite per release will not be kept current.

Chrome 131 rather than something newer, deliberately. Chrome 133 and
later sign with ML-DSA, codepoints OpenSSL 3.3.2 does not know, so their
signature_algorithms list cannot be reproduced and their JA4 is out of
reach. The cipher and extension sets are identical from Chrome 124
through 150, so only the signature algorithms are dated.

Both TLS builders now go through one `apply_tls_settings`, which also
closes a standing hazard: they were copy-pasted siblings, so anything
applied to only one silently gave raw_connect, resolve_ip and
request_target requests a different fingerprint from ordinary requests.

Where this leaves the JA4, against Chrome 131's
t13d1516h2_8daaf6152771_02713d6af862:

  was     t13d9911h2   105 ciphers, 11 extensions
  now     t13d1614h2    16 ciphers, 14 extensions

Signature algorithms match exactly. Every cipher matches except one, and
four extensions are now sent that were not: 0005, 0012, 4469 and fe0d.

Three gaps remain, and all three are things OpenSSL keeps for itself.
Probed rather than assumed: it refuses a custom extension for 0x0005,
0x001b and 0xff01, and accepts 0x0012, 0x4469 and 0xfe0d. status_request
was solved without a patch by setting it per-connection, which is the
only place OpenSSL's own API for it works. The other two need build
changes, along with suppressing the TLS_EMPTY_RENEGOTIATION_INFO_SCSV
that inflates the cipher count by one; suppressing that and sending
renegotiation_info are the same change, since OpenSSL sends the SCSV
precisely when it is not sending the extension.

Two traps found by measurement, both worth knowing:

The `openssl` crate stores a custom extension's payload in ex_data keyed
on the payload TYPE, so every extension returning Vec<u8> shares one
slot and overwrites the others. Three registered that way send one.
Distinct newtypes give each its own slot.

The harness was under-reporting. SSL_client_hello_get1_extensions_present
only returns extensions the server's OpenSSL has a definition for, so
ALPS and ECH were invisible to it, which is to say it was blind to
exactly the extensions being added. Capture now parses the ClientHello
off the wire, which also preserves extension order and GREASE for later.
Local and remote readings now agree exactly.
---
 src/client/hyper.rs           | 304 +++++++++++++++++++++++++++++-----
 src/config.rs                 |  21 +++
 src/lib.rs                    |   1 +
 src/main.rs                   |   5 +
 src/profile.rs                | 162 ++++++++++++++++++
 src/python.rs                 |  14 ++
 tests/fingerprint_baseline.rs |  51 ++++++
 tests/support/tls_server.rs   | 227 +++++++++++++++----------
 8 files changed, 651 insertions(+), 134 deletions(-)
 create mode 100644 src/profile.rs

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index d85931a..3dd4a1a 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -78,6 +78,235 @@ fn sanitize_uri(url: &str) -> String {
 /// nothing.
 const DEFAULT_CIPHER_LIST: &str = "ALL";
 
+/// Add the extensions a browser sends that OpenSSL does not send on its own.
+///
+/// JA4 hashes the list of extension *types*, not their contents, so presence
+/// is what the fingerprint turns on. Payload fidelity still matters to
+/// anything reading the raw hello, and where a payload is cheap to get right
+/// it is written out properly rather than left empty.
+///
+/// `SSL_CTX_add_custom_ext` refuses any extension OpenSSL handles itself, so
+/// this only covers the ones it has never heard of. `status_request` is set
+/// through its own API for that reason, and `renegotiation_info` cannot be
+/// done from out here at all.
+fn add_browser_extensions(
+    builder: &mut openssl::ssl::SslConnectorBuilder,
+) -> Result<(), ClientError> {
+    use openssl::ssl::ExtensionContext;
+
+    // Each extension's payload needs its OWN type, and this is not a style
+    // choice. The `openssl` crate stashes the bytes an add-callback returns in
+    // per-connection ex_data keyed on the payload type:
+    //
+    //     let idx = Ssl::cached_ex_index::<CustomExtAddState<T>>();
+    //
+    // so every custom extension returning `Vec<u8>` shares one slot and
+    // overwrites the others. Registering three that way sends exactly one,
+    // whichever wrote last, and the rest hand OpenSSL a pointer into a buffer
+    // that has already moved. Distinct newtypes give each its own slot.
+    macro_rules! payload_type {
+        ($name:ident) => {
+            struct $name(Vec<u8>);
+            impl AsRef<[u8]> for $name {
+                fn as_ref(&self) -> &[u8] {
+                    &self.0
+                }
+            }
+        };
+    }
+    payload_type!(StatusRequest);
+    payload_type!(SignedCertTimestamp);
+    payload_type!(CertCompression);
+    payload_type!(Alps);
+    payload_type!(EchGrease);
+
+    // Only the ClientHello. Sending these anywhere else would be wrong and,
+    // for ALPS, would confuse a server that actually implements it.
+    let ctx = ExtensionContext::CLIENT_HELLO;
+
+    // Which codepoints OpenSSL will cede, probed rather than assumed:
+    //
+    //   0x0005 status_request        refused, OpenSSL owns it
+    //   0x0012 signed_cert_timestamp accepted
+    //   0x001b compress_certificate  refused, OpenSSL owns it
+    //   0x4469 application_settings  accepted
+    //   0xfe0d encrypted_client_hello accepted
+    //   0xff01 renegotiation_info    refused, OpenSSL owns it
+    //
+    // The refusals are attempted anyway and ignored: they are cheap, they
+    // would start working if the build gained the relevant support, and none
+    // is worth failing a request over. Closing those three is what needs a
+    // patch to our OpenSSL build.
+
+    // status_request (0x0005): OCSP, then two empty lists. Five bytes, the
+    // same as Chrome.
+    let _ = builder.add_custom_ext(
+        0x0005,
+        ctx,
+        |_ssl, _ctx, _cert| Ok(Some(StatusRequest(vec![0x01, 0x00, 0x00, 0x00, 0x00]))),
+        |_ssl, _ctx, _data, _cert| Ok(()),
+    );
+
+    // signed_certificate_timestamp (0x0012): empty, as a client sends it.
+    let _ = builder.add_custom_ext(
+        0x0012,
+        ctx,
+        |_ssl, _ctx, _cert| Ok(Some(SignedCertTimestamp(Vec::new()))),
+        |_ssl, _ctx, _data, _cert| Ok(()),
+    );
+
+    // compress_certificate (0x001b): one algorithm, brotli.
+    let _ = builder.add_custom_ext(
+        0x001b,
+        ctx,
+        |_ssl, _ctx, _cert| Ok(Some(CertCompression(vec![0x02, 0x00, 0x02]))),
+        |_ssl, _ctx, _data, _cert| Ok(()),
+    );
+
+    // application_settings, ALPS (0x4469). Chrome 124 through 131 use this
+    // codepoint; 133 and later moved to 0x44cd. The body is an ALPN-shaped
+    // list holding just h2.
+    builder
+        .add_custom_ext(
+            0x4469,
+            ctx,
+            |_ssl, _ctx, _cert| Ok(Some(Alps(vec![0x00, 0x03, 0x02, b'h', b'2']))),
+            |_ssl, _ctx, _data, _cert| Ok(()),
+        )
+        .map_err(|e| ClientError::tls(format!("failed to add ALPS extension: {}", e)))?;
+
+    // encrypted_client_hello (0xfe0d). Chrome sends a GREASE ECH: a decoy
+    // shaped like the real thing that decrypts to nothing. OpenSSL 3.3.2 has
+    // no ECH at all, so this is hand-built. Nothing reads it, and its only job
+    // is to occupy the codepoint, but it is shaped correctly so that a server
+    // which does parse ECH sees something well-formed.
+    builder
+        .add_custom_ext(
+            0xfe0d,
+            ctx,
+            |_ssl, _ctx, _cert| {
+                let mut body = Vec::with_capacity(140);
+                body.push(0x00); // ClientHelloOuter
+                body.push(0x00); // HKDF-SHA256
+                body.extend_from_slice(&[0x00, 0x01]); // AES-128-GCM
+                body.extend_from_slice(&[0x00, 0x00]); // config id
+                body.extend_from_slice(&[0x00, 0x20]); // 32-byte enc
+                body.extend_from_slice(&[0x42; 32]);
+                body.extend_from_slice(&[0x00, 0x60]); // 96-byte payload
+                body.extend_from_slice(&[0x17; 96]);
+                Ok(Some(EchGrease(body)))
+            },
+            |_ssl, _ctx, _data, _cert| Ok(()),
+        )
+        .map_err(|e| ClientError::tls(format!("failed to add ECH extension: {}", e)))?;
+
+    Ok(())
+}
+
+/// Apply the cipher, signature-algorithm, group and version settings to a
+/// connector builder.
+///
+/// Shared because there are two places that build an SSL context, the pooled
+/// connector and `connect_stream`, and they have historically been
+/// copy-pasted siblings. Anything applied in only one of them silently gives
+/// `raw_connect`, `resolve_ip` and `request_target` requests a different
+/// fingerprint from ordinary ones.
+///
+/// Precedence is: an explicit setting beats the profile, and the profile beats
+/// the default. A caller who names a cipher string means it, and several
+/// existing tests depend on that still being true.
+fn apply_tls_settings(
+    builder: &mut openssl::ssl::SslConnectorBuilder,
+    config: &RequestConfig,
+) -> Result<(), ClientError> {
+    use openssl::ssl::SslOptions;
+
+    let profile = config.resolved_profile();
+
+    // Security level 0: allow all ciphers including RC4 and DES. This is an
+    // offensive-first tool and needs to connect to anything.
+    builder.set_security_level(0);
+
+    // `SslConnector::builder` sets NO_SSLV3. Clear it, or a server that speaks
+    // nothing newer stays unreachable even when asked for by name. Clearing
+    // the option only permits the protocol; what is actually offered is
+    // decided by the min/max version below, and a browser profile pins that
+    // floor at 1.2 so this never widens a profile's offer.
+    builder.clear_options(SslOptions::NO_SSLV3);
+
+    if let Some(p) = profile {
+        // Removing two extensions a browser does not send.
+        //
+        // encrypt_then_mac (0x0016) goes away by disabling the feature.
+        // padding (0x0015) is added by SSL_OP_TLSEXT_PADDING, which arrives
+        // switched on inside SSL_OP_ALL, so it has to be cleared rather than
+        // set. JA4 counts extensions, so both of these move the fingerprint.
+        //
+        // The `openssl` crate exposes neither flag, so the bits are taken from
+        // ssl.h directly: SSL_OP_TLSEXT_PADDING is SSL_OP_BIT(4) and
+        // SSL_OP_NO_ENCRYPT_THEN_MAC is SSL_OP_BIT(19). Both are stable parts
+        // of the public ABI.
+        const SSL_OP_TLSEXT_PADDING: u64 = 1 << 4;
+        const SSL_OP_NO_ENCRYPT_THEN_MAC: u64 = 1 << 19;
+        builder.set_options(SslOptions::from_bits_retain(SSL_OP_NO_ENCRYPT_THEN_MAC));
+        builder.clear_options(SslOptions::from_bits_retain(SSL_OP_TLSEXT_PADDING));
+
+        builder.set_ciphersuites(p.tls.ciphersuites).map_err(|e| {
+            ClientError::tls(format!(
+                "profile '{}' has an invalid TLS 1.3 suite list: {}",
+                p.name, e
+            ))
+        })?;
+        builder.set_sigalgs_list(p.tls.sigalgs).map_err(|e| {
+            ClientError::tls(format!(
+                "profile '{}' has an invalid signature algorithm list: {}",
+                p.name, e
+            ))
+        })?;
+        builder.set_groups_list(p.tls.groups).map_err(|e| {
+            ClientError::tls(format!(
+                "profile '{}' has an invalid group list: {}",
+                p.name, e
+            ))
+        })?;
+
+        add_browser_extensions(builder)?;
+    }
+
+    let ciphers = config
+        .cipher_string
+        .as_deref()
+        .or(profile.map(|p| p.tls.cipher_list))
+        .unwrap_or(DEFAULT_CIPHER_LIST);
+    builder
+        .set_cipher_list(ciphers)
+        .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
+
+    let min_version = config
+        .min_tls_version
+        .as_deref()
+        .or(profile.map(|p| p.tls.min_version));
+    if let Some(v) = min_version {
+        let version = parse_tls_version(v)?;
+        builder
+            .set_min_proto_version(Some(version))
+            .map_err(|e| ClientError::tls(format!("failed to set min TLS version: {}", e)))?;
+    }
+
+    let max_version = config
+        .max_tls_version
+        .as_deref()
+        .or(profile.map(|p| p.tls.max_version));
+    if let Some(v) = max_version {
+        let version = parse_tls_version(v)?;
+        builder
+            .set_max_proto_version(Some(version))
+            .map_err(|e| ClientError::tls(format!("failed to set max TLS version: {}", e)))?;
+    }
+
+    Ok(())
+}
+
 // Load the OpenSSL legacy provider once (for RC4, DES, etc.).
 // The provider is statically compiled into libcrypto via `no-module` build flag.
 // `Once` ensures this runs exactly once even across threads.
@@ -217,6 +446,9 @@ fn extract_cert_info(ssl: &openssl::ssl::SslRef) -> Option<CertInfo> {
 // Wraps HttpConnector with TLS handshake via openssl + tokio-openssl.
 #[derive(Clone)]
 struct OpenSslConnector {
+    /// Whether a browser profile is active. Needed in `call()` because
+    /// status_request can only be set once an `Ssl` exists.
+    profile_active: bool,
     http: HttpConnector,
     ssl: openssl::ssl::SslConnector,
     // Shared slot for cert info — written during handshake, read after response
@@ -357,44 +589,13 @@ impl OpenSslConnector {
             openssl::ssl::SslConnector::builder(openssl::ssl::SslMethod::tls_client())
                 .map_err(|e| ClientError::tls(format!("SSL setup failed: {}", e)))?;
 
-        // Security level 0: allow all ciphers including RC4, DES, export.
-        // This is an offensive-first tool — we need to connect to anything.
-        builder.set_security_level(0);
-
-        // `SslConnector::builder` sets NO_SSLV3 for us. Clear it, or a server
-        // that speaks nothing newer stays unreachable even when the caller
-        // asks for SSLv3 by name. Clearing the option only permits the
-        // protocol; which versions actually get offered is still decided by
-        // the min/max proto version below.
-        builder.clear_options(openssl::ssl::SslOptions::NO_SSLV3);
-
         if !config.should_verify_certs() {
             builder.set_verify(openssl::ssl::SslVerifyMode::NONE);
         } else {
             load_system_ca_certs(&mut builder)?;
         }
 
-        let ciphers = config
-            .cipher_string
-            .as_deref()
-            .unwrap_or(DEFAULT_CIPHER_LIST);
-        builder
-            .set_cipher_list(ciphers)
-            .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
-
-        if let Some(ref min_ver) = config.min_tls_version {
-            let version = parse_tls_version(min_ver)?;
-            builder
-                .set_min_proto_version(Some(version))
-                .map_err(|e| ClientError::tls(format!("failed to set min TLS version: {}", e)))?;
-        }
-
-        if let Some(ref max_ver) = config.max_tls_version {
-            let version = parse_tls_version(max_ver)?;
-            builder
-                .set_max_proto_version(Some(version))
-                .map_err(|e| ClientError::tls(format!("failed to set max TLS version: {}", e)))?;
-        }
+        apply_tls_settings(&mut builder, config)?;
 
         // ALPN: advertise HTTP/2 and HTTP/1.1 support during TLS handshake.
         // The wire format is length-prefixed: [2, b'h', b'2', 8, b'h', b't', ...].
@@ -421,6 +622,7 @@ impl OpenSslConnector {
         http.set_connect_timeout(Some(connect_timeout));
 
         Ok(OpenSslConnector {
+            profile_active: config.resolved_profile().is_some(),
             http,
             ssl,
             cert_slot,
@@ -535,6 +737,7 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
         let slot_key = peer_slot_key(&uri);
         let http_fut = self.http.call(uri);
         let ssl_connector = self.ssl.clone();
+        let profile_active = self.profile_active;
         let cert_slot = self.cert_slot.clone();
         let peer_slot = self.peer_slot.clone();
         let connect_timeout = self.connect_timeout;
@@ -558,6 +761,15 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
 
             let mut ssl_conf = openssl::ssl::Ssl::new(ssl_connector.context())
                 .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
+
+            // status_request (0x0005). OpenSSL keeps this codepoint for itself
+            // and refuses a custom extension for it, and its own API,
+            // SSL_set_tlsext_status_type, is per-connection rather than
+            // per-context. So it has to be set here, where an Ssl finally
+            // exists, rather than alongside the other profile extensions.
+            if profile_active {
+                let _ = ssl_conf.set_status_type(openssl::ssl::StatusType::OCSP);
+            }
             // Only set SNI for hostnames, not IP addresses (SNI with IPs is invalid per RFC)
             if host.parse::<std::net::IpAddr>().is_err() {
                 ssl_conf
@@ -646,6 +858,12 @@ enum AnyClient {
 #[derive(Clone, Hash, Eq, PartialEq)]
 struct TlsKey {
     verify_certs: bool,
+    /// Part of the key because a profile changes the cipher list, the
+    /// signature algorithms, the groups and the version floor. Leaving it out
+    /// means the first request to a host fixes the TLS configuration for every
+    /// later one, so an impersonated request and a plain one would quietly
+    /// share a connection.
+    profile: Option<String>,
     cipher_string: Option<String>,
     min_tls_version: Option<String>,
     max_tls_version: Option<String>,
@@ -656,6 +874,7 @@ impl TlsKey {
     fn from_config(config: &RequestConfig) -> Self {
         TlsKey {
             verify_certs: config.should_verify_certs(),
+            profile: config.profile.clone(),
             cipher_string: config.cipher_string.clone(),
             min_tls_version: config.min_tls_version.clone(),
             max_tls_version: config.max_tls_version.clone(),
@@ -998,23 +1217,13 @@ pub(crate) async fn connect_stream(
         openssl::ssl::SslConnector::builder(openssl::ssl::SslMethod::tls_client())
             .map_err(|e| ClientError::tls(format!("SSL setup failed: {}", e)))?;
 
-    ssl_builder.set_security_level(0);
-    // Same reason as the pooled path: openssl-rs sets NO_SSLV3 and an SSLv3
-    // server is unreachable until it is cleared.
-    ssl_builder.clear_options(openssl::ssl::SslOptions::NO_SSLV3);
-
     if !config.should_verify_certs() {
         ssl_builder.set_verify(openssl::ssl::SslVerifyMode::NONE);
     } else {
         load_system_ca_certs(&mut ssl_builder)?;
     }
-    let ciphers = config
-        .cipher_string
-        .as_deref()
-        .unwrap_or(DEFAULT_CIPHER_LIST);
-    ssl_builder
-        .set_cipher_list(ciphers)
-        .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
+
+    apply_tls_settings(&mut ssl_builder, config)?;
     if let Some(ref min_ver) = config.min_tls_version {
         let version = parse_tls_version(min_ver)?;
         ssl_builder
@@ -1041,9 +1250,16 @@ pub(crate) async fn connect_stream(
         .map_err(|e| ClientError::tls(format!("failed to set ALPN: {}", e)))?;
 
     let ssl_connector = ssl_builder.build();
+    let profile_active = config.resolved_profile().is_some();
     let mut ssl_conf = openssl::ssl::Ssl::new(ssl_connector.context())
         .map_err(|e| ClientError::tls(format!("SSL conf failed: {}", e)))?;
 
+    // See the pooled path: OpenSSL keeps status_request for itself and its API
+    // for it is per-connection, so it can only be set once an Ssl exists.
+    if profile_active {
+        let _ = ssl_conf.set_status_type(openssl::ssl::StatusType::OCSP);
+    }
+
     // SNI = original hostname, NOT the resolved IP
     if host.parse::<std::net::IpAddr>().is_err() {
         ssl_conf
diff --git a/src/config.rs b/src/config.rs
index be31eaa..d3f243b 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -28,6 +28,17 @@ pub struct RequestConfig {
     /// `corp`), single IPs, CIDR ranges (`10.0.0.0/8`), or `*` for all.
     #[serde(default)]
     pub no_proxy: Vec<String>,
+    /// Browser profile to impersonate, e.g. `"chrome"`. Applies a matching
+    /// TLS configuration, HTTP/2 settings and default headers together.
+    ///
+    /// Applied as a unit on purpose. A profile whose layers disagree is worse
+    /// than none: a client claiming to be Chrome whose handshake says
+    /// otherwise has stated something checkably false.
+    ///
+    /// An explicit `cipher_string`, `min_tls_version` or `max_tls_version`
+    /// overrides the profile's value for that field, on the grounds that a
+    /// caller naming something specific means it.
+    pub profile: Option<String>,
     pub cipher_string: Option<String>,
     pub min_tls_version: Option<String>,
     pub max_tls_version: Option<String>,
@@ -70,6 +81,7 @@ impl RequestConfig {
             verify_certs: None,
             proxy: None,
             no_proxy: Vec::new(),
+            profile: None,
             cipher_string: None,
             min_tls_version: None,
             max_tls_version: None,
@@ -84,6 +96,15 @@ impl RequestConfig {
         }
     }
 
+    /// The browser profile this request should imitate, if it named one.
+    ///
+    /// An unknown name resolves to `None` rather than erroring, so a profile
+    /// that has not been written yet degrades to ordinary behaviour instead of
+    /// failing the request.
+    pub fn resolved_profile(&self) -> Option<&'static crate::profile::BrowserProfile> {
+        self.profile.as_deref().and_then(crate::profile::by_name)
+    }
+
     pub fn method(&self) -> &str {
         self.method.as_deref().unwrap_or("GET")
     }
diff --git a/src/lib.rs b/src/lib.rs
index a2f112b..34590e0 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -4,6 +4,7 @@ pub mod config;
 pub mod cookies;
 pub mod debug;
 pub mod h2;
+pub mod profile;
 pub mod response;
 
 #[cfg(feature = "python")]
diff --git a/src/main.rs b/src/main.rs
index 691c41d..0b82dfa 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -69,6 +69,10 @@ struct Cli {
     #[arg(long = "no-proxy", num_args = 1)]
     no_proxy: Vec<String>,
 
+    /// Browser profile to imitate (e.g. "chrome")
+    #[arg(long)]
+    profile: Option<String>,
+
     /// OpenSSL cipher string (e.g. "ALL", "HIGH", "RC4-SHA")
     #[arg(long)]
     ciphers: Option<String>,
@@ -120,6 +124,7 @@ fn build_config(cli: &Cli, url: String) -> RequestConfig {
     config.max_body_size = cli.max_body_size;
     config.proxy = cli.proxy.clone();
     config.no_proxy = cli.no_proxy.clone();
+    config.profile = cli.profile.clone();
     config.cipher_string = cli.ciphers.clone();
     config.min_tls_version = cli.min_tls.clone();
     config.max_tls_version = cli.max_tls.clone();
diff --git a/src/profile.rs b/src/profile.rs
new file mode 100644
index 0000000..540918b
--- /dev/null
+++ b/src/profile.rs
@@ -0,0 +1,162 @@
+//! Browser profiles: what a real browser puts on the wire.
+//!
+//! A profile is data, not code, because browsers ship every few weeks and a
+//! profile that has to be rewritten each time will not be kept current.
+//!
+//! The rule that shapes all of this: a profile has to be applied whole or not
+//! at all. Sending browser headers over a non-browser handshake is worse than
+//! sending honest ones, because a client claiming to be Chrome whose first
+//! packet says otherwise has stated something checkably false, and that is a
+//! stronger signal than merely being unfamiliar. Measured: `udemy.com` returns
+//! 200 to an honest curl and 403 with `cf-mitigated: challenge` to the same
+//! client wearing only a Chrome User-Agent.
+
+/// The TLS half of a browser profile.
+///
+/// Fields map onto the OpenSSL calls that set them. TLS 1.3 suites and TLS 1.2
+/// suites are separate because OpenSSL configures them through different
+/// functions: `SSL_CTX_set_ciphersuites` and `SSL_CTX_set_cipher_list`. Setting
+/// only the latter, which is all blasthttp did before this, leaves the 1.3
+/// suites at OpenSSL's defaults in OpenSSL's order.
+#[derive(Debug, Clone, Copy)]
+pub struct TlsProfile {
+    /// TLS 1.3 suites, in order, colon separated (`SSL_CTX_set_ciphersuites`).
+    pub ciphersuites: &'static str,
+    /// TLS 1.2 and below, in order, colon separated (`SSL_CTX_set_cipher_list`).
+    pub cipher_list: &'static str,
+    /// Signature algorithms, in order. JA4 hashes this list as sent rather
+    /// than sorted, so the order is part of the fingerprint.
+    pub sigalgs: &'static str,
+    /// Supported groups, in order. Note these do not affect JA4: only the
+    /// presence of the extension is counted, not its contents. They are still
+    /// visible to anything reading the raw hello.
+    pub groups: &'static str,
+    pub min_version: &'static str,
+    pub max_version: &'static str,
+    pub alpn: &'static [&'static str],
+}
+
+/// A complete profile: TLS, plus the HTTP layers that have to agree with it.
+#[derive(Debug, Clone, Copy)]
+pub struct BrowserProfile {
+    pub name: &'static str,
+    pub tls: TlsProfile,
+    /// Default request headers in the order the browser sends them. HTTP/1.1
+    /// header order is preserved by this client, so this is the order that
+    /// goes on the wire.
+    pub headers: &'static [(&'static str, &'static str)],
+}
+
+/// Chrome 131 on Windows.
+///
+/// Chosen over a newer Chrome deliberately. Chrome 133 and later sign with
+/// ML-DSA (signature algorithms 0x0904 through 0x0906), which OpenSSL 3.3.2
+/// does not know, so its signature_algorithms list cannot be reproduced and
+/// its JA4 is out of reach. Chrome 131's list is entirely standard.
+///
+/// The cipher and extension sets are identical between Chrome 124 and 150, so
+/// this is not a stale profile in the way the version number suggests; only
+/// the signature algorithms moved. Both Chrome 131 and 124 produce
+/// `t13d1516h2_8daaf6152771_02713d6af862`.
+pub const CHROME_131: BrowserProfile = BrowserProfile {
+    name: "chrome131",
+    tls: TlsProfile {
+        // 1301, 1302, 1303.
+        ciphersuites: "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256",
+        // c02b, c02f, c02c, c030, cca9, cca8, c013, c014, 009c, 009d, 002f, 0035.
+        cipher_list: "ECDHE-ECDSA-AES128-GCM-SHA256:\
+                      ECDHE-RSA-AES128-GCM-SHA256:\
+                      ECDHE-ECDSA-AES256-GCM-SHA384:\
+                      ECDHE-RSA-AES256-GCM-SHA384:\
+                      ECDHE-ECDSA-CHACHA20-POLY1305:\
+                      ECDHE-RSA-CHACHA20-POLY1305:\
+                      ECDHE-RSA-AES128-SHA:\
+                      ECDHE-RSA-AES256-SHA:\
+                      AES128-GCM-SHA256:\
+                      AES256-GCM-SHA384:\
+                      AES128-SHA:\
+                      AES256-SHA",
+        // 0403, 0804, 0401, 0503, 0805, 0501, 0806, 0601, in that order.
+        sigalgs: "ECDSA+SHA256:RSA-PSS+SHA256:RSA+SHA256:\
+                  ECDSA+SHA384:RSA-PSS+SHA384:RSA+SHA384:\
+                  RSA-PSS+SHA512:RSA+SHA512",
+        // Chrome also offers X25519MLKEM768 (4588) ahead of these, which
+        // OpenSSL gained in 3.5 and ours cannot send. It does not affect JA4,
+        // but it is visible in the raw hello, and a client claiming to be a
+        // current Chrome while offering no post-quantum share is itself worth
+        // noticing.
+        groups: "X25519:P-256:P-384",
+        min_version: "1.2",
+        max_version: "1.3",
+        alpn: &["h2", "http/1.1"],
+    },
+    headers: &[
+        (
+            "sec-ch-ua",
+            "\"Google Chrome\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\"",
+        ),
+        ("sec-ch-ua-mobile", "?0"),
+        ("sec-ch-ua-platform", "\"Windows\""),
+        ("upgrade-insecure-requests", "1"),
+        (
+            "user-agent",
+            "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 \
+             (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
+        ),
+        (
+            "accept",
+            "text/html,application/xhtml+xml,application/xml;q=0.9,\
+             image/avif,image/webp,image/apng,*/*;q=0.8,\
+             application/signed-exchange;v=b3;q=0.7",
+        ),
+        ("sec-fetch-site", "none"),
+        ("sec-fetch-mode", "navigate"),
+        ("sec-fetch-user", "?1"),
+        ("sec-fetch-dest", "document"),
+        ("accept-encoding", "gzip, deflate, br, zstd"),
+        ("accept-language", "en-US,en;q=0.9"),
+        ("priority", "u=0, i"),
+    ],
+};
+
+/// Look a profile up by name.
+pub fn by_name(name: &str) -> Option<&'static BrowserProfile> {
+    match name.to_ascii_lowercase().as_str() {
+        // "chrome" tracks the newest profile we can actually reproduce, which
+        // is what a caller asking for "chrome" wants.
+        "chrome" | "chrome131" => Some(&CHROME_131),
+        _ => None,
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_chrome_alias_resolves() {
+        assert_eq!(by_name("chrome").unwrap().name, "chrome131");
+        assert_eq!(by_name("CHROME131").unwrap().name, "chrome131");
+        assert!(by_name("netscape").is_none());
+    }
+
+    #[test]
+    fn test_cipher_counts_match_the_target() {
+        // 12 TLS 1.2 suites plus 3 TLS 1.3 suites is the 15 that JA4 counts
+        // for Chrome. Getting this wrong moves the JA4 header, so it is worth
+        // asserting next to the data rather than only end to end.
+        let p = CHROME_131.tls;
+        assert_eq!(p.cipher_list.split(':').count(), 12);
+        assert_eq!(p.ciphersuites.split(':').count(), 3);
+        assert_eq!(p.sigalgs.split(':').count(), 8);
+    }
+
+    #[test]
+    fn test_headers_are_in_browser_order() {
+        // Order is part of the fingerprint, so guard the two ends of it.
+        let names: Vec<&str> = CHROME_131.headers.iter().map(|(n, _)| *n).collect();
+        assert_eq!(names.first(), Some(&"sec-ch-ua"));
+        assert_eq!(names.last(), Some(&"priority"));
+        assert!(names.contains(&"sec-fetch-dest"));
+    }
+}
diff --git a/src/python.rs b/src/python.rs
index bfd05e8..267767f 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -919,6 +919,7 @@ impl BlastHTTP {
         verify_certs=None,
         proxy=None,
         no_proxy=None,
+        profile=None,
         cipher_string=None,
         min_tls_version=None,
         max_tls_version=None,
@@ -947,6 +948,7 @@ impl BlastHTTP {
         verify_certs: Option<bool>,
         proxy: Option<String>,
         no_proxy: Option<Vec<String>>,
+        profile: Option<String>,
         cipher_string: Option<String>,
         min_tls_version: Option<String>,
         max_tls_version: Option<String>,
@@ -973,6 +975,7 @@ impl BlastHTTP {
             verify_certs,
             proxy,
             no_proxy: no_proxy.unwrap_or_default(),
+            profile,
             cipher_string,
             min_tls_version,
             max_tls_version,
@@ -1166,6 +1169,7 @@ impl BlastHTTP {
             verify_certs,
             proxy,
             no_proxy: no_proxy.unwrap_or_default(),
+            profile: None,
             cipher_string: None,
             min_tls_version: None,
             max_tls_version: None,
@@ -1218,6 +1222,7 @@ impl BlastHTTP {
     #[pyo3(signature = (
         url,
         verify_certs=None,
+        profile=None,
         cipher_string=None,
         min_tls_version=None,
         max_tls_version=None,
@@ -1233,6 +1238,7 @@ impl BlastHTTP {
         py: Python<'py>,
         url: String,
         verify_certs: Option<bool>,
+        profile: Option<String>,
         cipher_string: Option<String>,
         min_tls_version: Option<String>,
         max_tls_version: Option<String>,
@@ -1244,6 +1250,7 @@ impl BlastHTTP {
     ) -> PyResult<Bound<'py, PyAny>> {
         let mut config = RequestConfig::new(url.clone());
         config.verify_certs = verify_certs;
+        config.profile = profile;
         config.cipher_string = cipher_string;
         config.min_tls_version = min_tls_version;
         config.max_tls_version = max_tls_version;
@@ -1402,6 +1409,8 @@ struct PyBatchConfig {
     #[pyo3(get, set)]
     no_proxy: Option<Vec<String>>,
     #[pyo3(get, set)]
+    profile: Option<String>,
+    #[pyo3(get, set)]
     cipher_string: Option<String>,
     #[pyo3(get, set)]
     min_tls_version: Option<String>,
@@ -1437,6 +1446,7 @@ impl PyBatchConfig {
         verify_certs=None,
         proxy=None,
         no_proxy=None,
+        profile=None,
         cipher_string=None,
         min_tls_version=None,
         max_tls_version=None,
@@ -1462,6 +1472,7 @@ impl PyBatchConfig {
         verify_certs: Option<bool>,
         proxy: Option<String>,
         no_proxy: Option<Vec<String>>,
+        profile: Option<String>,
         cipher_string: Option<String>,
         min_tls_version: Option<String>,
         max_tls_version: Option<String>,
@@ -1486,6 +1497,7 @@ impl PyBatchConfig {
             verify_certs,
             proxy,
             no_proxy,
+            profile,
             cipher_string,
             min_tls_version,
             max_tls_version,
@@ -1515,6 +1527,7 @@ impl Clone for PyBatchConfig {
             verify_certs: self.verify_certs,
             proxy: self.proxy.clone(),
             no_proxy: self.no_proxy.clone(),
+            profile: self.profile.clone(),
             cipher_string: self.cipher_string.clone(),
             min_tls_version: self.min_tls_version.clone(),
             max_tls_version: self.max_tls_version.clone(),
@@ -1547,6 +1560,7 @@ impl PyBatchConfig {
             verify_certs: self.verify_certs,
             proxy: self.proxy,
             no_proxy: self.no_proxy.unwrap_or_default(),
+            profile: self.profile,
             cipher_string: self.cipher_string,
             min_tls_version: self.min_tls_version,
             max_tls_version: self.max_tls_version,
diff --git a/tests/fingerprint_baseline.rs b/tests/fingerprint_baseline.rs
index fc11de4..9946c92 100644
--- a/tests/fingerprint_baseline.rs
+++ b/tests/fingerprint_baseline.rs
@@ -180,3 +180,54 @@ async fn test_no_browser_only_extensions_yet() {
         );
     }
 }
+
+// ── Chrome profile ────────────────────────────────────────────────
+
+/// Capture a hello made with a named profile, addressed by hostname so SNI is
+/// sent and the result is comparable with a browser's.
+async fn capture_profile_hello(profile: &str) -> CapturedHello {
+    let server = TlsTestServer::start(TlsServerConfig::default()).await;
+    let url = format!("https://localhost:{}/", server.addr.port());
+
+    let mut config = RequestConfig::new(url);
+    config.verify_certs = Some(false);
+    config.timeout_seconds = Some(5);
+    config.profile = Some(profile.to_string());
+
+    let client = HyperClient::new();
+    let _ = client.send(&config).await;
+
+    let hello = server
+        .hello
+        .lock()
+        .expect("hello slot poisoned")
+        .clone()
+        .expect("server recorded no ClientHello");
+    server.shutdown().await;
+    hello
+}
+
+#[tokio::test]
+async fn test_chrome_profile_progress() {
+    // Not a pass/fail gate yet. Prints the distance to Chrome 131 so the
+    // remaining gap is visible while it is being closed.
+    let hello = capture_profile_hello("chrome").await;
+    let got = support::ja4::ja4(&hello);
+
+    // Chrome 131 and 124, confirmed against tls.peet.ws.
+    const TARGET: &str = "t13d1516h2_8daaf6152771_02713d6af862";
+
+    println!("\n  target: {}", TARGET);
+    println!("  actual: {}", got);
+    println!("  raw   : {}", support::ja4::ja4_raw(&hello));
+
+    let want: Vec<&str> = TARGET.split('_').collect();
+    let have: Vec<&str> = got.split('_').collect();
+    for (label, w, h) in [
+        ("header", want[0], have[0]),
+        ("ciphers", want[1], have[1]),
+        ("exts+sigalgs", want[2], have[2]),
+    ] {
+        println!("  {:13} {}", label, if w == h { "match" } else { "differ" });
+    }
+}
diff --git a/tests/support/tls_server.rs b/tests/support/tls_server.rs
index 1d06f67..85b3c53 100644
--- a/tests/support/tls_server.rs
+++ b/tests/support/tls_server.rs
@@ -124,10 +124,16 @@ fn generate_self_signed(config: &TlsServerConfig) -> (PKey<openssl::pkey::Privat
 
 // ── ClientHello capture ───────────────────────────────────────────
 //
-// `SSL_CTX_set_client_hello_cb` fires after the hello is parsed and before a
-// cipher is chosen, which is the only point where the full offer is still
-// visible. The safe `openssl` wrapper covers ciphers and versions but has no
-// accessor for the extension list, so those go through openssl-sys.
+// Parsed straight off the wire rather than through OpenSSL's
+// `SSL_CTX_set_client_hello_cb`. That callback looks like the obvious tool and
+// quietly under-reports: `SSL_client_hello_get1_extensions_present` only
+// returns extensions the *server's* OpenSSL has a definition for, so anything
+// it has not heard of is invisible. Since the extensions worth testing are
+// precisely the ones OpenSSL does not implement, ALPS and ECH among them, that
+// made the harness blind to the thing it exists to measure.
+//
+// Reading the bytes has two further benefits we need later: extension order
+// survives, and so does GREASE. Both are invisible through the callback.
 
 /// Split a big-endian `u16` array, as TLS encodes cipher and extension lists.
 fn be_u16s(bytes: &[u8]) -> Vec<u16> {
@@ -137,102 +143,138 @@ fn be_u16s(bytes: &[u8]) -> Vec<u16> {
         .collect()
 }
 
-/// Read one extension's body, or `None` when the client didn't send it.
-fn extension_body(ssl: &mut openssl::ssl::SslRef, ext_type: u16) -> Option<Vec<u8>> {
-    use foreign_types_shared::ForeignTypeRef;
-    let mut out: *const std::ffi::c_uchar = std::ptr::null();
-    let mut outlen: usize = 0;
-    let found = unsafe {
-        openssl_sys::SSL_client_hello_get0_ext(
-            ssl.as_ptr(),
-            ext_type as std::ffi::c_uint,
-            &mut out,
-            &mut outlen,
-        )
-    };
-    if found != 1 || out.is_null() {
-        return None;
-    }
-    Some(unsafe { std::slice::from_raw_parts(out, outlen) }.to_vec())
+/// A cursor that refuses to run off the end, so a malformed or truncated hello
+/// returns what was understood instead of panicking inside a test server.
+struct Reader<'a> {
+    buf: &'a [u8],
+    pos: usize,
 }
 
-/// A TLS vector whose length prefix is two bytes, which is how
-/// `supported_groups` and `signature_algorithms` wrap their contents.
-fn u16_vector_body(body: &[u8]) -> Vec<u16> {
-    if body.len() < 2 {
-        return Vec::new();
+impl<'a> Reader<'a> {
+    fn new(buf: &'a [u8]) -> Self {
+        Reader { buf, pos: 0 }
+    }
+    fn take(&mut self, n: usize) -> Option<&'a [u8]> {
+        let end = self.pos.checked_add(n)?;
+        if end > self.buf.len() {
+            return None;
+        }
+        let out = &self.buf[self.pos..end];
+        self.pos = end;
+        Some(out)
+    }
+    fn u8(&mut self) -> Option<u8> {
+        self.take(1).map(|b| b[0])
+    }
+    fn u16(&mut self) -> Option<u16> {
+        self.take(2).map(|b| u16::from_be_bytes([b[0], b[1]]))
+    }
+    /// A TLS vector with a one- or two-byte length prefix.
+    fn vec8(&mut self) -> Option<&'a [u8]> {
+        let n = self.u8()? as usize;
+        self.take(n)
+    }
+    fn vec16(&mut self) -> Option<&'a [u8]> {
+        let n = self.u16()? as usize;
+        self.take(n)
     }
-    let len = u16::from_be_bytes([body[0], body[1]]) as usize;
-    be_u16s(&body[2..(2 + len).min(body.len())])
 }
 
-fn parse_alpn(body: &[u8]) -> Vec<String> {
-    // Two-byte list length, then repeated one-byte-length-prefixed names.
-    if body.len() < 2 {
-        return Vec::new();
-    }
-    let mut out = Vec::new();
-    let mut i = 2;
-    while i < body.len() {
-        let n = body[i] as usize;
-        i += 1;
-        if i + n > body.len() {
-            break;
+/// Parse a ClientHello out of one or more TLS records.
+///
+/// Returns `None` for anything that is not a well-formed handshake, which in a
+/// test means the client failed before saying anything useful.
+pub fn parse_client_hello(raw: &[u8]) -> Option<CapturedHello> {
+    // Reassemble the handshake across records. A hello carrying a real
+    // key_share and an ECH extension runs past 1400 bytes and is routinely
+    // split, so reading only the first record would truncate it.
+    let mut handshake = Vec::new();
+    let mut r = Reader::new(raw);
+    while r.pos < raw.len() {
+        let content_type = r.u8()?;
+        let _record_version = r.u16()?;
+        let fragment = r.vec16()?;
+        if content_type != 0x16 {
+            return None; // not a handshake record
         }
-        out.push(String::from_utf8_lossy(&body[i..i + n]).into_owned());
-        i += n;
+        handshake.extend_from_slice(fragment);
     }
-    out
-}
 
-fn capture_hello(ssl: &mut openssl::ssl::SslRef) -> CapturedHello {
-    use foreign_types_shared::ForeignTypeRef;
+    let mut h = Reader::new(&handshake);
+    if h.u8()? != 0x01 {
+        return None; // not a ClientHello
+    }
+    let len = {
+        let b = h.take(3)?;
+        ((b[0] as usize) << 16) | ((b[1] as usize) << 8) | b[2] as usize
+    };
+    let body = h.take(len)?;
+
+    let mut b = Reader::new(body);
+    let legacy_version = b.u16()?;
+    b.take(32)?; // random
+    b.vec8()?; // session id
+    let ciphers = be_u16s(b.vec16()?);
+    b.vec8()?; // compression methods
 
     let mut captured = CapturedHello {
-        legacy_version: unsafe {
-            openssl_sys::SSL_client_hello_get0_legacy_version(ssl.as_ptr()) as u16
-        },
-        ciphers: ssl.client_hello_ciphers().map(be_u16s).unwrap_or_default(),
+        legacy_version,
+        ciphers,
         ..Default::default()
     };
 
-    // Extension types, in the order the client sent them. The array is
-    // allocated by OpenSSL and is ours to free.
-    let mut types: *mut std::ffi::c_int = std::ptr::null_mut();
-    let mut count: usize = 0;
-    let ok = unsafe {
-        openssl_sys::SSL_client_hello_get1_extensions_present(ssl.as_ptr(), &mut types, &mut count)
+    // Extensions are optional in the grammar, though nothing modern omits them.
+    let Some(ext_block) = b.vec16() else {
+        return Some(captured);
     };
-    if ok == 1 && !types.is_null() {
-        captured.extensions = unsafe { std::slice::from_raw_parts(types, count) }
-            .iter()
-            .map(|t| *t as u16)
-            .collect();
-        unsafe { openssl_sys::OPENSSL_free(types as *mut std::ffi::c_void) };
-    }
 
-    captured.has_sni = captured.extensions.contains(&0x0000);
-    if let Some(b) = extension_body(ssl, 0x000a) {
-        captured.groups = u16_vector_body(&b);
-    }
-    if let Some(b) = extension_body(ssl, 0x000d) {
-        captured.sig_algs = u16_vector_body(&b);
-    }
-    if let Some(b) = extension_body(ssl, 0x0010) {
-        captured.alpn = parse_alpn(&b);
-    }
-    if let Some(b) = extension_body(ssl, 0x002b) {
-        // supported_versions uses a ONE-byte length prefix, unlike the others.
-        if !b.is_empty() {
-            let len = b[0] as usize;
-            captured.supported_versions = be_u16s(&b[1..(1 + len).min(b.len())]);
+    let mut e = Reader::new(ext_block);
+    while e.pos < ext_block.len() {
+        let Some(ext_type) = e.u16() else { break };
+        let Some(data) = e.vec16() else { break };
+        captured.extensions.push(ext_type);
+
+        let mut d = Reader::new(data);
+        match ext_type {
+            0x0000 => captured.has_sni = true,
+            0x000a => {
+                if let Some(v) = d.vec16() {
+                    captured.groups = be_u16s(v);
+                }
+            }
+            0x000d => {
+                if let Some(v) = d.vec16() {
+                    captured.sig_algs = be_u16s(v);
+                }
+            }
+            0x0010 => {
+                if let Some(list) = d.vec16() {
+                    let mut l = Reader::new(list);
+                    while l.pos < list.len() {
+                        match l.vec8() {
+                            Some(name) => captured
+                                .alpn
+                                .push(String::from_utf8_lossy(name).into_owned()),
+                            None => break,
+                        }
+                    }
+                }
+            }
+            // supported_versions is the odd one out: a one-byte length prefix
+            // where the others use two.
+            0x002b => {
+                if let Some(v) = d.vec8() {
+                    captured.supported_versions = be_u16s(v);
+                }
+            }
+            _ => {}
         }
     }
 
-    captured
+    Some(captured)
 }
 
-fn build_acceptor(config: &TlsServerConfig, hello: HelloSlot) -> SslAcceptor {
+fn build_acceptor(config: &TlsServerConfig) -> SslAcceptor {
     // Load the legacy provider so the server can use weak ciphers too
     load_legacy_provider();
 
@@ -263,15 +305,6 @@ fn build_acceptor(config: &TlsServerConfig, hello: HelloSlot) -> SslAcceptor {
         )
         .unwrap();
 
-    // Record the client's offer before a cipher is picked.
-    builder.set_client_hello_callback(move |ssl, _alert| {
-        let captured = capture_hello(ssl);
-        if let Ok(mut slot) = hello.lock() {
-            *slot = Some(captured);
-        }
-        Ok(openssl::ssl::ClientHelloResponse::SUCCESS)
-    });
-
     let (pkey, cert) = generate_self_signed(config);
     builder.set_private_key(&pkey).unwrap();
     builder.set_certificate(&cert).unwrap();
@@ -308,7 +341,7 @@ fn load_legacy_provider() {
 impl TlsTestServer {
     pub async fn start(config: TlsServerConfig) -> Self {
         let hello: HelloSlot = std::sync::Arc::new(std::sync::Mutex::new(None));
-        let acceptor = build_acceptor(&config, hello.clone());
+        let acceptor = build_acceptor(&config);
 
         // Bind to random port on localhost
         let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
@@ -316,6 +349,7 @@ impl TlsTestServer {
 
         let (shutdown_tx, shutdown_rx) = tokio::sync::oneshot::channel::<()>();
 
+        let hello_slot = hello.clone();
         let handle = tokio::spawn(async move {
             let response = format!(
                 "HTTP/1.1 {} OK\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{}",
@@ -327,6 +361,19 @@ impl TlsTestServer {
             tokio::select! {
                 result = listener.accept() => {
                     if let Ok((tcp_stream, _)) = result {
+                        // Read the ClientHello off the wire and parse it before
+                        // TLS touches it, then hand the same bytes to the
+                        // handshake. Peeking rather than consuming keeps the
+                        // stream intact, so no replay buffer is needed and the
+                        // handshake proceeds exactly as it would have.
+                        let mut peek = vec![0u8; 8192];
+                        if let Ok(n) = tcp_stream.peek(&mut peek).await
+                            && let Some(parsed) = parse_client_hello(&peek[..n])
+                            && let Ok(mut slot) = hello_slot.lock()
+                        {
+                            *slot = Some(parsed);
+                        }
+
                         // Async TLS handshake via tokio-openssl
                         let ssl = openssl::ssl::Ssl::new(acceptor.context()).unwrap();
                         let mut tls_stream = match tokio_openssl::SslStream::new(ssl, tcp_stream) {

From f8d82c89b27baccac1ee4dcf9818019a1ca37642 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 28 Sep 2026 22:37:25 -0400
Subject: [PATCH 07/44] Patch OpenSSL for renegotiation_info, and make the
 profile connect

akamai.com now answers 200 with a real session where it returned a hard
403 before. That is the measurement this work exists to move: the
passive-fingerprint tier goes 0/1 to 1/1.

The OpenSSL patch does most of the TLS work. Stock OpenSSL signals
renegotiation-info support with the SCSV pseudo-cipher where every
browser uses the extension, which costs a match twice over: a cipher
nothing else sends, and a missing extension everything else sends.
Switching it fixes both, and the cipher segment of our JA4 now equals
Chrome 131's exactly.

The patch is conditioned on the TLS floor rather than applied
unconditionally, because the SCSV exists to protect pre-RFC5746 servers
that mishandle unfamiliar extensions. A client whose floor is TLS 1.2 is
modern and sends the extension; one willing to speak older protocols
keeps stock behaviour. The compatibility path is unchanged, and a test
asserts it.

Build patches are now part of the cached build recipe, so editing one
invalidates the build. Without that a patch change would silently do
nothing, which is how the SSLv3 flag went unnoticed for months.

Three of Chrome's extensions are deliberately NOT sent, each tried and
each withdrawn for breaking real sites:

  ALPS (0x4469): google.com negotiates it, then expects the settings
  exchange inside HTTP/2 that we do not implement. TLS completes and the
  request then fails.

  ECH (0xfe0d): a hand-built GREASE blob is not shaped precisely enough.
  reddit.com rejects it at the handshake. Registering the server-reply
  contexts fixed cloudflare.com but not reddit, so the payload is the
  problem rather than the registration.

  compress_certificate (0x001b): our build is OPENSSL_NO_COMP_ALG and
  cannot decompress a certificate. Enabling one means a compression
  library across six cross-compiled wheel targets.

That last point generalises, and is the finding worth carrying forward:
advertising a protocol feature is a promise to implement it, and a
server that takes you up on it gets a client that cannot follow through.

So the JA4 does not match Chrome exactly, and did not need to. Thirteen
extensions against Chrome's sixteen was enough to turn Akamai around.
Exact parity was never the bar; plausibility was, and a client that
connects beats one that matches a hash and cannot.

Verified on akamai, cloudflare, google, github, wikipedia, reddit and
microsoft: all reachable with the profile.
---
 scripts/build-openssl.sh                      |  28 +++-
 ...1-renegotiation-info-instead-of-scsv.patch |  67 ++++++++
 src/client/hyper.rs                           | 150 +++++++++---------
 tests/fingerprint_baseline.rs                 |  97 +++++++++--
 tests/live_benchmark.rs                       |  61 +++++--
 5 files changed, 297 insertions(+), 106 deletions(-)
 create mode 100644 scripts/openssl-patches/0001-renegotiation-info-instead-of-scsv.patch

diff --git a/scripts/build-openssl.sh b/scripts/build-openssl.sh
index 39101ca..eb53aee 100755
--- a/scripts/build-openssl.sh
+++ b/scripts/build-openssl.sh
@@ -24,6 +24,7 @@ SOURCE_DIR="${VENDOR_DIR}/openssl-${OPENSSL_VERSION}"
 INSTALL_DIR="${VENDOR_DIR}/install"
 TARBALL="${VENDOR_DIR}/openssl-${OPENSSL_VERSION}.tar.gz"
 MARKER="${INSTALL_DIR}/.blasthttp-built"
+PATCH_DIR="${SCRIPT_DIR}/openssl-patches"
 
 # The features that make this build different from a stock OpenSSL.
 #
@@ -73,7 +74,13 @@ esac
 # a rebuild just as much as one whose target changed, and keying on the target
 # alone means a flag change is silently ignored until someone deletes the
 # install directory by hand.
-BUILD_RECIPE="${TARGET:-native}|${OPENSSL_VERSION}|${FEATURE_FLAGS[*]}"
+# Patches are part of the recipe too: editing one has to invalidate the cached
+# build exactly as changing a flag does, or the change silently does nothing.
+PATCH_HASH="none"
+if [ -d "$PATCH_DIR" ] && compgen -G "$PATCH_DIR/*.patch" > /dev/null; then
+    PATCH_HASH=$(cat "$PATCH_DIR"/*.patch | sha256sum | cut -c1-16)
+fi
+BUILD_RECIPE="${TARGET:-native}|${OPENSSL_VERSION}|${FEATURE_FLAGS[*]}|patches=${PATCH_HASH}"
 
 # Skip only if the cached build was made the same way
 if [ -f "$MARKER" ]; then
@@ -120,6 +127,25 @@ fi
 echo "Extracting..."
 tar xzf "$TARBALL" -C "$VENDOR_DIR"
 
+# --- Apply our patches ---
+#
+# Applied to a freshly extracted tree every time, so the patches never stack
+# and a failure is always a real conflict rather than a second application.
+# Every patch is required: a silent skip would produce a build that looks fine
+# and behaves differently, which is the failure mode that hid the SSLv3
+# problem for months.
+if [ -d "$PATCH_DIR" ]; then
+    for patch in "$PATCH_DIR"/*.patch; do
+        [ -e "$patch" ] || continue
+        echo "Applying $(basename "$patch")"
+        if ! patch -p1 -d "$SOURCE_DIR" -s < "$patch"; then
+            echo "ERROR: failed to apply $(basename "$patch")" >&2
+            echo "The patch probably needs rebasing onto OpenSSL ${OPENSSL_VERSION}." >&2
+            exit 1
+        fi
+    done
+fi
+
 # --- Find cross-compiler if needed ---
 find_cross_cc() {
     local target="$1"
diff --git a/scripts/openssl-patches/0001-renegotiation-info-instead-of-scsv.patch b/scripts/openssl-patches/0001-renegotiation-info-instead-of-scsv.patch
new file mode 100644
index 0000000..8754e8c
--- /dev/null
+++ b/scripts/openssl-patches/0001-renegotiation-info-instead-of-scsv.patch
@@ -0,0 +1,67 @@
+Signal renegotiation-info support with the extension, not the SCSV
+
+RFC 5746 section 3.4 lets a client advertise renegotiation-info support on an
+initial ClientHello in either of two ways: the
+TLS_EMPTY_RENEGOTIATION_INFO_SCSV pseudo-cipher (0x00ff), or an empty
+renegotiation_info extension (0xff01).
+
+OpenSSL picks the SCSV. Every browser picks the extension. That one choice
+costs a fingerprint match twice over: it adds a cipher suite nothing else
+sends and omits an extension everything else sends, so both the cipher count
+and the extension set differ from any browser.
+
+The switch is conditioned on the minimum protocol version rather than made
+unconditional, because the SCSV exists for a reason. Pre-RFC5746 servers, the
+very ones that motivated it, can mishandle an unfamiliar extension. So a
+client whose floor is TLS 1.2 is treated as modern and sends the extension,
+while one willing to speak TLS 1.1 or older keeps the SCSV and stock
+behaviour. blasthttp's compatibility path stays byte-identical to unpatched
+OpenSSL; only the impersonation path changes, and it always pins its floor at
+TLS 1.2.
+
+On an initial handshake previous_client_finished_len is zero, so the existing
+WPACKET_sub_memcpy_u8 call already emits the empty renegotiated_connection
+field RFC 5746 requires. No new code is needed to build the body.
+
+--- a/ssl/statem/extensions_clnt.c
++++ b/ssl/statem/extensions_clnt.c
+@@ -16,8 +16,19 @@
+                                           unsigned int context, X509 *x,
+                                           size_t chainidx)
+ {
+-    /* Add RI if renegotiating */
+-    if (!s->renegotiate)
++    /*
++     * Add RI if renegotiating.
++     *
++     * On an initial handshake RFC 5746 3.4 allows either this extension or
++     * the SCSV. Browsers always send the extension, so a client whose floor
++     * is TLS 1.2 does too. One willing to speak older protocols keeps the
++     * SCSV, which is safer against pre-RFC5746 servers that mishandle
++     * unfamiliar extensions. ssl_cipher_list_to_bytes() is gated on the same
++     * condition so exactly one of the two is ever sent.
++     */
++    if (!s->renegotiate
++            && (SSL_CONNECTION_IS_DTLS(s)
++                || s->min_proto_version < TLS1_2_VERSION))
+         return EXT_RETURN_NOT_SENT;
+ 
+     if (!WPACKET_put_bytes_u16(pkt, TLSEXT_TYPE_renegotiate)
+--- a/ssl/statem/statem_clnt.c
++++ b/ssl/statem/statem_clnt.c
+@@ -4063,9 +4063,14 @@
+ {
+     int i;
+     size_t totlen = 0, len, maxlen, maxverok = 0;
++    /*
++     * Send the SCSV only when the renegotiation_info extension is not being
++     * sent. tls_construct_ctos_renegotiate() is gated on the same version
++     * test, so support is signalled exactly once, one way or the other.
++     */
+     int empty_reneg_info_scsv = !s->renegotiate
+                                 && (SSL_CONNECTION_IS_DTLS(s)
+-                                    || s->min_proto_version < TLS1_3_VERSION);
++                                    || s->min_proto_version < TLS1_2_VERSION);
+     SSL *ssl = SSL_CONNECTION_GET_SSL(s);
+ 
+     /* Set disabled masks for this session */
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 3dd4a1a..0f68534 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -101,9 +101,8 @@ fn add_browser_extensions(
     //     let idx = Ssl::cached_ex_index::<CustomExtAddState<T>>();
     //
     // so every custom extension returning `Vec<u8>` shares one slot and
-    // overwrites the others. Registering three that way sends exactly one,
-    // whichever wrote last, and the rest hand OpenSSL a pointer into a buffer
-    // that has already moved. Distinct newtypes give each its own slot.
+    // overwrites the others. Registering three that way sends exactly one.
+    // Distinct newtypes give each its own slot.
     macro_rules! payload_type {
         ($name:ident) => {
             struct $name(Vec<u8>);
@@ -114,38 +113,31 @@ fn add_browser_extensions(
             }
         };
     }
-    payload_type!(StatusRequest);
     payload_type!(SignedCertTimestamp);
-    payload_type!(CertCompression);
-    payload_type!(Alps);
-    payload_type!(EchGrease);
 
-    // Only the ClientHello. Sending these anywhere else would be wrong and,
-    // for ALPS, would confuse a server that actually implements it.
-    let ctx = ExtensionContext::CLIENT_HELLO;
+    // The ClientHello for sending, plus the places a server may answer.
+    // Registering CLIENT_HELLO alone makes OpenSSL reject a reply that arrives
+    // in a context with no handler, which breaks the handshake outright rather
+    // than merely looking wrong.
+    let ctx = ExtensionContext::CLIENT_HELLO
+        | ExtensionContext::TLS1_3_SERVER_HELLO
+        | ExtensionContext::TLS1_3_ENCRYPTED_EXTENSIONS
+        | ExtensionContext::TLS1_2_SERVER_HELLO;
 
     // Which codepoints OpenSSL will cede, probed rather than assumed:
     //
-    //   0x0005 status_request        refused, OpenSSL owns it
-    //   0x0012 signed_cert_timestamp accepted
-    //   0x001b compress_certificate  refused, OpenSSL owns it
-    //   0x4469 application_settings  accepted
+    //   0x0005 status_request         refused for custom use, OpenSSL owns it
+    //   0x0012 signed_cert_timestamp  accepted
+    //   0x001b compress_certificate   refused, OpenSSL owns it
+    //   0x4469 application_settings   accepted
     //   0xfe0d encrypted_client_hello accepted
-    //   0xff01 renegotiation_info    refused, OpenSSL owns it
+    //   0xff01 renegotiation_info     refused, OpenSSL owns it
     //
-    // The refusals are attempted anyway and ignored: they are cheap, they
-    // would start working if the build gained the relevant support, and none
-    // is worth failing a request over. Closing those three is what needs a
+    // status_request is set per-connection instead, in the one place OpenSSL's
+    // own API for it works. renegotiation_info and the SCSV are handled by a
     // patch to our OpenSSL build.
 
-    // status_request (0x0005): OCSP, then two empty lists. Five bytes, the
-    // same as Chrome.
-    let _ = builder.add_custom_ext(
-        0x0005,
-        ctx,
-        |_ssl, _ctx, _cert| Ok(Some(StatusRequest(vec![0x01, 0x00, 0x00, 0x00, 0x00]))),
-        |_ssl, _ctx, _data, _cert| Ok(()),
-    );
+    // status_request (0x0005) is handled per-connection; see the connector.
 
     // signed_certificate_timestamp (0x0012): empty, as a client sends it.
     let _ = builder.add_custom_ext(
@@ -155,50 +147,35 @@ fn add_browser_extensions(
         |_ssl, _ctx, _data, _cert| Ok(()),
     );
 
-    // compress_certificate (0x001b): one algorithm, brotli.
-    let _ = builder.add_custom_ext(
-        0x001b,
-        ctx,
-        |_ssl, _ctx, _cert| Ok(Some(CertCompression(vec![0x02, 0x00, 0x02]))),
-        |_ssl, _ctx, _data, _cert| Ok(()),
-    );
-
-    // application_settings, ALPS (0x4469). Chrome 124 through 131 use this
-    // codepoint; 133 and later moved to 0x44cd. The body is an ALPN-shaped
-    // list holding just h2.
-    builder
-        .add_custom_ext(
-            0x4469,
-            ctx,
-            |_ssl, _ctx, _cert| Ok(Some(Alps(vec![0x00, 0x03, 0x02, b'h', b'2']))),
-            |_ssl, _ctx, _data, _cert| Ok(()),
-        )
-        .map_err(|e| ClientError::tls(format!("failed to add ALPS extension: {}", e)))?;
-
-    // encrypted_client_hello (0xfe0d). Chrome sends a GREASE ECH: a decoy
-    // shaped like the real thing that decrypts to nothing. OpenSSL 3.3.2 has
-    // no ECH at all, so this is hand-built. Nothing reads it, and its only job
-    // is to occupy the codepoint, but it is shaped correctly so that a server
-    // which does parse ECH sees something well-formed.
-    builder
-        .add_custom_ext(
-            0xfe0d,
-            ctx,
-            |_ssl, _ctx, _cert| {
-                let mut body = Vec::with_capacity(140);
-                body.push(0x00); // ClientHelloOuter
-                body.push(0x00); // HKDF-SHA256
-                body.extend_from_slice(&[0x00, 0x01]); // AES-128-GCM
-                body.extend_from_slice(&[0x00, 0x00]); // config id
-                body.extend_from_slice(&[0x00, 0x20]); // 32-byte enc
-                body.extend_from_slice(&[0x42; 32]);
-                body.extend_from_slice(&[0x00, 0x60]); // 96-byte payload
-                body.extend_from_slice(&[0x17; 96]);
-                Ok(Some(EchGrease(body)))
-            },
-            |_ssl, _ctx, _data, _cert| Ok(()),
-        )
-        .map_err(|e| ClientError::tls(format!("failed to add ECH extension: {}", e)))?;
+    // ── Three Chrome extensions deliberately NOT sent ─────────────
+    //
+    // Chrome sends application_settings (0x4469), encrypted_client_hello
+    // (0xfe0d) and compress_certificate (0x001b). We send none of them, which
+    // costs three extensions against Chrome's JA4. Each was tried and each
+    // broke real sites, because advertising a protocol feature is a promise to
+    // implement it and a server that takes you up on it gets a client that
+    // cannot follow through.
+    //
+    //   ALPS (0x4469): google.com negotiates it and then expects the ALPS
+    //     settings exchange inside HTTP/2. We do not implement that, so the
+    //     handshake completes and the request then fails. Needs the HTTP/2
+    //     work before it can be sent honestly.
+    //
+    //   ECH (0xfe0d): Chrome sends a GREASE ECH, a decoy shaped precisely
+    //     enough to pass a real parser. A hand-built imitation is not, and
+    //     reddit.com rejects it at the handshake. Registering the server-reply
+    //     contexts fixed cloudflare.com but not reddit.com, so the payload
+    //     itself is the problem. Needs real ECH, which arrived in OpenSSL 3.5.
+    //
+    //   compress_certificate (0x001b): our OpenSSL is built with
+    //     OPENSSL_NO_COMP_ALG, so it cannot decompress a certificate. Enabling
+    //     it means a compression library in the build, and the wheels
+    //     cross-compile to six targets.
+    //
+    // The measurement that matters: without all three, akamai.com went from a
+    // hard 403 to 200 with a real session. Exact JA4 parity turned out not to
+    // be the bar. Plausibility was enough there, and a client that connects is
+    // worth more than one that matches a hash and cannot.
 
     Ok(())
 }
@@ -1563,11 +1540,34 @@ fn build_request(
         builder = builder.header("Host", authority.as_str());
     }
 
-    if !has_custom_ua {
-        builder = builder.header("User-Agent", "blasthttp/0.1.0");
-    }
-    if !has_custom_ae {
-        builder = builder.header("Accept-Encoding", "gzip, deflate, br");
+    // With a profile active, send its full header set in the browser's order
+    // rather than this client's two defaults.
+    //
+    // This has to travel with the TLS half or it makes matters worse. Browser
+    // headers over a non-browser handshake state something checkably false,
+    // and a contradiction is easier to act on than unfamiliarity: udemy.com
+    // answers an honest curl with 200 and the same client wearing only a
+    // Chrome User-Agent with 403.
+    //
+    // A caller's own header still wins over the profile's, since naming one
+    // explicitly means it.
+    if let Some(p) = config.resolved_profile() {
+        for (name, value) in p.headers {
+            let already_set = custom.iter().any(|(k, _)| k.eq_ignore_ascii_case(name));
+            if !already_set {
+                builder = builder.header(*name, *value);
+            }
+        }
+    } else {
+        if !has_custom_ua {
+            builder = builder.header(
+                "User-Agent",
+                concat!("blasthttp/", env!("CARGO_PKG_VERSION")),
+            );
+        }
+        if !has_custom_ae {
+            builder = builder.header("Accept-Encoding", "gzip, deflate, br");
+        }
     }
 
     // Emit the caller's headers, folding any redirect-chain cookies into the
diff --git a/tests/fingerprint_baseline.rs b/tests/fingerprint_baseline.rs
index 9946c92..672c45c 100644
--- a/tests/fingerprint_baseline.rs
+++ b/tests/fingerprint_baseline.rs
@@ -208,26 +208,91 @@ async fn capture_profile_hello(profile: &str) -> CapturedHello {
 }
 
 #[tokio::test]
-async fn test_chrome_profile_progress() {
-    // Not a pass/fail gate yet. Prints the distance to Chrome 131 so the
-    // remaining gap is visible while it is being closed.
+async fn test_chrome_profile_ja4_is_pinned() {
     let hello = capture_profile_hello("chrome").await;
     let got = support::ja4::ja4(&hello);
 
-    // Chrome 131 and 124, confirmed against tls.peet.ws.
-    const TARGET: &str = "t13d1516h2_8daaf6152771_02713d6af862";
+    // Chrome 131 and 124, confirmed against tls.peet.ws, is
+    // t13d1516h2_8daaf6152771_02713d6af862.
+    //
+    // The cipher segment matches it exactly. The extension count does not, and
+    // is not meant to: three of Chrome's extensions are deliberately not sent
+    // because advertising them broke real sites. See add_browser_extensions()
+    // for which and why.
+    //
+    // Worth recording next to the pin, because it is the finding that shaped
+    // the whole approach: this non-matching JA4 is enough to turn akamai.com
+    // from a hard 403 into a 200 with a real session. Exact parity was not the
+    // bar. A client that connects beats one that matches a hash and cannot.
+    const EXPECTED: &str = "t13d1513h2_8daaf6152771_1eb89897b454";
 
-    println!("\n  target: {}", TARGET);
-    println!("  actual: {}", got);
-    println!("  raw   : {}", support::ja4::ja4_raw(&hello));
+    assert_eq!(
+        got,
+        EXPECTED,
+        "Chrome profile JA4 moved.\n  raw: {}",
+        support::ja4::ja4_raw(&hello),
+    );
+}
 
-    let want: Vec<&str> = TARGET.split('_').collect();
-    let have: Vec<&str> = got.split('_').collect();
-    for (label, w, h) in [
-        ("header", want[0], have[0]),
-        ("ciphers", want[1], have[1]),
-        ("exts+sigalgs", want[2], have[2]),
-    ] {
-        println!("  {:13} {}", label, if w == h { "match" } else { "differ" });
+#[tokio::test]
+async fn test_chrome_profile_cipher_list_matches_chrome_exactly() {
+    // The cipher segment is the one part that does match Chrome, and it only
+    // does because of the OpenSSL patch: stock OpenSSL appends
+    // TLS_EMPTY_RENEGOTIATION_INFO_SCSV to the list, which no browser sends,
+    // and that one extra suite changed the hash. Guarded separately so a
+    // regression there is unmistakable.
+    let hello = capture_profile_hello("chrome").await;
+    let got = support::ja4::ja4(&hello);
+    let cipher_hash = got.split('_').nth(1).unwrap_or_default();
+
+    assert_eq!(
+        cipher_hash, "8daaf6152771",
+        "cipher list no longer matches Chrome 131"
+    );
+    assert!(
+        !hello.ciphers.contains(&0x00ff),
+        "the renegotiation SCSV is back in the cipher list: {:04x?}",
+        hello.ciphers
+    );
+}
+
+#[tokio::test]
+async fn test_chrome_profile_sends_renegotiation_info() {
+    // The other half of the same patch. OpenSSL sends the SCSV precisely when
+    // it is not sending this extension, so these two tests fail together.
+    let hello = capture_profile_hello("chrome").await;
+    assert!(
+        hello.extensions.contains(&0xff01),
+        "renegotiation_info missing, so the OpenSSL patch is not in this build"
+    );
+}
+
+#[tokio::test]
+async fn test_chrome_profile_drops_non_browser_extensions() {
+    // padding and encrypt_then_mac are OpenSSL habits no browser has.
+    let hello = capture_profile_hello("chrome").await;
+    for (ext, name) in [(0x0015u16, "padding"), (0x0016, "encrypt_then_mac")] {
+        assert!(
+            !hello.extensions.contains(&ext),
+            "{} ({:#06x}) should not be sent under a browser profile",
+            name,
+            ext
+        );
     }
 }
+
+#[tokio::test]
+async fn test_compat_path_keeps_the_scsv() {
+    // The patch is conditioned on the TLS floor so the compatibility path is
+    // byte-identical to stock OpenSSL. Without a profile we should still send
+    // the SCSV and still not send renegotiation_info.
+    let hello = capture_default_hello().await;
+    assert!(
+        hello.ciphers.contains(&0x00ff),
+        "compatibility path lost the SCSV, so the patch is too broad"
+    );
+    assert!(
+        !hello.extensions.contains(&0xff01),
+        "compatibility path gained renegotiation_info, so the patch is too broad"
+    );
+}
diff --git a/tests/live_benchmark.rs b/tests/live_benchmark.rs
index 2c51900..51c517b 100644
--- a/tests/live_benchmark.rs
+++ b/tests/live_benchmark.rs
@@ -21,16 +21,28 @@ use support::antibot::{Outcome, ResponseFacts, classify};
 use support::tls_server::{TlsServerConfig, TlsTestServer};
 
 fn config(url: &str) -> RequestConfig {
+    config_with(url, None)
+}
+
+fn config_with(url: &str, profile: Option<&str>) -> RequestConfig {
     let mut c = RequestConfig::new(url.to_string());
     c.verify_certs = Some(false);
     c.timeout_seconds = Some(30);
     c.follow_redirects = Some(true);
+    c.profile = profile.map(|p| p.to_string());
     c
 }
 
 async fn fetch(url: &str) -> Option<(u16, Vec<(String, String)>, String)> {
+    fetch_with(url, None).await
+}
+
+async fn fetch_with(
+    url: &str,
+    profile: Option<&str>,
+) -> Option<(u16, Vec<(String, String)>, String)> {
     let client = HyperClient::new();
-    match client.send(&config(url)).await {
+    match client.send(&config_with(url, profile)).await {
         Ok(r) => {
             let body = r.body().to_string();
             Some((r.status, r.headers.clone(), body))
@@ -207,43 +219,64 @@ async fn live_block_scorecard() {
         ),
     ];
 
-    println!();
-    let (mut passive_through, mut passive_total) = (0, 0);
-    let mut control_failed = false;
-
-    for (tier, label, url) in targets {
-        let outcome = match fetch(url).await {
+    // Each target twice: once plain, once impersonating. The delta between
+    // the two columns is the only number that says whether this work does
+    // anything, and running them in one pass keeps the comparison honest
+    // about time of day, network path and whatever the site is doing today.
+    async fn outcome_of(url: &str, profile: Option<&str>) -> Outcome {
+        match fetch_with(url, profile).await {
             Some((status, headers, body)) => classify(&ResponseFacts {
                 status,
                 headers: &headers,
                 body: &body,
             }),
             None => Outcome::Error,
-        };
+        }
+    }
+
+    println!(
+        "  {:9} {:30} {:24} profile=chrome",
+        "tier", "target", "plain"
+    );
+    let (mut passive_plain, mut passive_profiled, mut passive_total) = (0, 0, 0);
+    let mut control_failed = false;
+
+    for (tier, label, url) in targets {
+        let plain = outcome_of(url, None).await;
+        let profiled = outcome_of(url, Some("chrome")).await;
 
         let tag = match tier {
             Tier::Control => {
-                if !outcome.got_through() {
+                if !plain.got_through() {
                     control_failed = true;
                 }
                 "control"
             }
             Tier::Passive => {
                 passive_total += 1;
-                if outcome.got_through() {
-                    passive_through += 1;
+                if plain.got_through() {
+                    passive_plain += 1;
+                }
+                if profiled.got_through() {
+                    passive_profiled += 1;
                 }
                 "passive"
             }
             Tier::Javascript => "js",
         };
-        println!("  [{:7}] {:30} {:?}", tag, label, outcome);
+        println!(
+            "  [{:7}] {:30} {:24} {:?}",
+            tag,
+            label,
+            format!("{:?}", plain),
+            profiled
+        );
     }
 
     println!();
     println!(
-        "  passive-fingerprint tier: {}/{} reached",
-        passive_through, passive_total
+        "  passive-fingerprint tier: {}/{} plain, {}/{} with profile",
+        passive_plain, passive_total, passive_profiled, passive_total
     );
     println!("  javascript tier: not scored, unreachable without a browser");
     println!("  controls: not scored");

From 03f0b057a9aa58fe77d4b7e381e8e878cc9819e0 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 09:23:13 -0400
Subject: [PATCH 08/44] Shape the HTTP/2 settings to the profile

src/client/hyper.rs made no http2_* calls at all, so every SETTINGS value
and the connection window were hyper-util defaults, which match no
browser. Four of them are reachable through hyper-util and now follow the
profile:

  before  2:0;4:2097152;5:16384;6:16384|5177345|0|m,s,a,p
  after   2:0;4:6291456;6:262144|15663105|0|m,s,a,p
  chrome  1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p

INITIAL_WINDOW_SIZE, MAX_HEADER_LIST_SIZE and the WINDOW_UPDATE now match
Chrome exactly. Omitting MAX_FRAME_SIZE is part of the match rather than
an oversight: our default announced 16384 and Chrome announces nothing.
Adaptive window is off because it resizes the connection window as
traffic flows, emitting WINDOW_UPDATE frames no browser sends.

Two fields are left alone, both fork-gated rather than forgotten.
HEADER_TABLE_SIZE exists in hyper's own config but hyper-util does not
expose it. The pseudo-header order is hardcoded in h2's Iter::next(), and
reaching it means forking h2, hyper and hyper-util, since each layer
translates a fixed option set rather than passing options through. Worth
noting that curl_cffi gets Chrome's order for free: it is nghttp2's
natural order, and Rust's h2 simply chose differently.

Verified reachable with the profile: akamai, cloudflare, google, github,
reddit.
---
 src/client/hyper.rs | 24 +++++++++++++++++++++++-
 src/profile.rs      | 43 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 0f68534..3b4e2e2 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -948,7 +948,29 @@ impl HyperClient {
         let cert_slot: CertSlot = Arc::new(Mutex::new(None));
         let peer_slot: PeerSlot = Arc::new(Mutex::new(HashMap::new()));
         let connector = OpenSslConnector::new(config, cert_slot.clone(), peer_slot.clone())?;
-        let builder = Client::builder(TokioExecutor::new());
+        let mut builder = Client::builder(TokioExecutor::new());
+
+        // Shape the HTTP/2 SETTINGS and connection window to the profile.
+        //
+        // Nothing here was being set at all before, so the values on the wire
+        // were hyper-util's defaults, which match no browser. These four are
+        // everything hyper-util exposes of the Akamai HTTP/2 fingerprint; the
+        // remaining two fields, HEADER_TABLE_SIZE and the pseudo-header order,
+        // need forks of hyper-util and h2 respectively and are left alone.
+        if let Some(p) = config.resolved_profile() {
+            builder
+                .http2_initial_stream_window_size(p.http2.initial_stream_window)
+                .http2_initial_connection_window_size(p.http2.initial_connection_window)
+                .http2_max_header_list_size(p.http2.max_header_list_size)
+                // `None` omits MAX_FRAME_SIZE entirely, which is what Chrome
+                // does. Our default announces 16384 and Chrome announces
+                // nothing, so leaving this unset is itself part of the match.
+                .http2_max_frame_size(p.http2.max_frame_size)
+                // Adaptive window resizes the connection window as traffic
+                // flows, which would emit WINDOW_UPDATE frames no browser
+                // sends and undo the fixed value set above.
+                .http2_adaptive_window(false);
+        }
 
         let inner = match mode {
             ConnMode::Direct(_) => AnyClient::Direct(builder.build(connector)),
diff --git a/src/profile.rs b/src/profile.rs
index 540918b..e428a42 100644
--- a/src/profile.rs
+++ b/src/profile.rs
@@ -36,11 +36,43 @@ pub struct TlsProfile {
     pub alpn: &'static [&'static str],
 }
 
+/// The HTTP/2 half of a profile.
+///
+/// Values are what the browser puts in its SETTINGS frame and its
+/// connection-level WINDOW_UPDATE, which together make up most of what is
+/// commonly called the Akamai HTTP/2 fingerprint.
+///
+/// Two fields of that fingerprint are deliberately absent because our stack
+/// cannot reach them without forking dependencies:
+///
+///   HEADER_TABLE_SIZE (setting 1): hyper carries it internally but
+///   hyper-util does not expose it, so there is no way to set it from here.
+///
+///   Pseudo-header order: `h2` hardcodes method, scheme, authority, path in
+///   `Iter::next()`, and Chrome sends method, authority, scheme, path.
+///   Changing it means forking h2, hyper and hyper-util, since each layer
+///   translates a fixed set of options rather than passing them through.
+#[derive(Debug, Clone, Copy)]
+pub struct Http2Profile {
+    /// SETTINGS 4, INITIAL_WINDOW_SIZE.
+    pub initial_stream_window: u32,
+    /// The connection window. The WINDOW_UPDATE a browser sends is this minus
+    /// the protocol default of 65535, so Chrome's 15663105 increment comes
+    /// from asking for 15 MiB.
+    pub initial_connection_window: u32,
+    /// SETTINGS 6, MAX_HEADER_LIST_SIZE.
+    pub max_header_list_size: u32,
+    /// SETTINGS 5, MAX_FRAME_SIZE. `None` leaves it out, which is what Chrome
+    /// does; our default sends 16384 and Chrome sends nothing.
+    pub max_frame_size: Option<u32>,
+}
+
 /// A complete profile: TLS, plus the HTTP layers that have to agree with it.
 #[derive(Debug, Clone, Copy)]
 pub struct BrowserProfile {
     pub name: &'static str,
     pub tls: TlsProfile,
+    pub http2: Http2Profile,
     /// Default request headers in the order the browser sends them. HTTP/1.1
     /// header order is preserved by this client, so this is the order that
     /// goes on the wire.
@@ -90,6 +122,17 @@ pub const CHROME_131: BrowserProfile = BrowserProfile {
         max_version: "1.3",
         alpn: &["h2", "http/1.1"],
     },
+    // Chrome's Akamai fingerprint is
+    // 1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p. ENABLE_PUSH is
+    // already 0 because hyper always disables push.
+    http2: Http2Profile {
+        initial_stream_window: 6_291_456,
+        // 15 MiB. The WINDOW_UPDATE that results is 15728640 - 65535 =
+        // 15663105, which is what Chrome sends.
+        initial_connection_window: 15_728_640,
+        max_header_list_size: 262_144,
+        max_frame_size: None,
+    },
     headers: &[
         (
             "sec-ch-ua",

From 21e1caa2374e9bb960dad68871af2a2b49584dc6 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 10:19:54 -0400
Subject: [PATCH 09/44] Add BLASTHTTP_BISECT for isolating which profile layer
 a detector sees

"We are blocked" does not say which part of the imitation failed, and
rebuilding between experiments makes finding out slow enough that you
guess instead. BLASTHTTP_BISECT=tls,headers,http2 turns off any
combination of the three layers at runtime.

It paid for itself immediately. PerimeterX blocks blasthttp+chrome while
letting both plain blasthttp and curl_cffi through, which I had assumed
meant our TLS fingerprint was not good enough. It is not the TLS.

On priceline.com, three passes each:

  plain (all off)            3/3
  tls only                   3/3
  headers only               0/3
  http2 only                 3/3
  tls + http2, no headers    3/3
  full profile               0/3

Narrowing into the headers, it is the User-Agent on its own. Adding only
sec-ch-ua, only sec-fetch-*, or only accept/accept-language/priority to
an otherwise plain client changes nothing; adding only a browser
User-Agent drops it to zero.

And it is any browser, not Chrome specifically:

  blasthttp/0.10.1 (default)   3/3
  Chrome 131 Windows           0/3
  Chrome 150 macOS             0/3   (byte-identical to curl_cffi's)
  Firefox 147                  0/3
  NotARealBrowser/1.0          3/3

curl_cffi sends that same Chrome 150 macOS string and passes 3/3. So the
User-Agent is not what is scored; it selects which scoring applies. Claim
to be a browser and the client gets checked against that claim. Claim
anything else, including nonsense, and it does not.

That model explains results that looked contradictory. Akamai and
DataDome reward the profile because they fingerprint passively and ours
is close enough. PerimeterX verifies the claim, curl_cffi survives the
verification and we do not, so against it our partial imitation is worse
than being honest. It also generalises the udemy.com case from the
research: not "browser headers over script TLS" but "any browser claim
invites verification".

Two consequences. A browser profile should not become a global default,
since it makes us worse against anything that verifies. And the case for
closing the remaining gaps, GREASE, extension order, ALPS, ECH,
post-quantum key share, is now specific rather than aesthetic: they are
what the verification is looking at.
---
 src/client/hyper.rs | 33 ++++++++++++++++++++++++++++++---
 1 file changed, 30 insertions(+), 3 deletions(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 3b4e2e2..f3bc9a9 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -180,6 +180,25 @@ fn add_browser_extensions(
     Ok(())
 }
 
+/// Layers of a profile that can be disabled independently, for working out
+/// which one a particular detector is reacting to.
+///
+/// Driven by `BLASTHTTP_BISECT`, a comma-separated list of layers to leave
+/// off: `tls`, `headers`, `http2`. This exists because "we are blocked" does
+/// not say which part of the imitation is failing, and the alternative is
+/// rebuilding between every experiment.
+///
+/// ```text
+/// BLASTHTTP_BISECT=headers,http2   # profile TLS only
+/// BLASTHTTP_BISECT=tls             # everything but TLS
+/// ```
+pub(crate) fn bisect_disabled(layer: &str) -> bool {
+    match std::env::var("BLASTHTTP_BISECT") {
+        Ok(v) => v.split(',').any(|p| p.trim().eq_ignore_ascii_case(layer)),
+        Err(_) => false,
+    }
+}
+
 /// Apply the cipher, signature-algorithm, group and version settings to a
 /// connector builder.
 ///
@@ -198,7 +217,9 @@ fn apply_tls_settings(
 ) -> Result<(), ClientError> {
     use openssl::ssl::SslOptions;
 
-    let profile = config.resolved_profile();
+    let profile = config
+        .resolved_profile()
+        .filter(|_| !bisect_disabled("tls"));
 
     // Security level 0: allow all ciphers including RC4 and DES. This is an
     // offensive-first tool and needs to connect to anything.
@@ -957,7 +978,10 @@ impl HyperClient {
         // everything hyper-util exposes of the Akamai HTTP/2 fingerprint; the
         // remaining two fields, HEADER_TABLE_SIZE and the pseudo-header order,
         // need forks of hyper-util and h2 respectively and are left alone.
-        if let Some(p) = config.resolved_profile() {
+        if let Some(p) = config
+            .resolved_profile()
+            .filter(|_| !bisect_disabled("http2"))
+        {
             builder
                 .http2_initial_stream_window_size(p.http2.initial_stream_window)
                 .http2_initial_connection_window_size(p.http2.initial_connection_window)
@@ -1573,7 +1597,10 @@ fn build_request(
     //
     // A caller's own header still wins over the profile's, since naming one
     // explicitly means it.
-    if let Some(p) = config.resolved_profile() {
+    if let Some(p) = config
+        .resolved_profile()
+        .filter(|_| !bisect_disabled("headers"))
+    {
         for (name, value) in p.headers {
             let already_set = custom.iter().any(|(k, _)| k.eq_ignore_ascii_case(name));
             if !already_set {

From 5de87fa2f8561b2c2d13e5b297b266868f73d8e3 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 11:48:06 -0400
Subject: [PATCH 10/44] Recover why a TLS handshake failed

Phase 1 of the profile plan, and a live bug on its own. Nothing that
chooses a connection profile can work until a failure says what went
wrong, and until now none of them did.

The reason was being destroyed on the way out. The connector hands
hyper-util a Box<dyn Error>; hyper-util wraps it in an error whose
Display is the literal string "client error (Connect)" for every
connect-time failure; nothing ever calls .source() to get back down to
the openssl error underneath. So the substring test that was supposed to
tell TLS failures apart could never match, and a cipher mismatch, a
rejected certificate, a DNS failure and a refused connection all arrived
identically as ErrorKind::Connection.

That also meant TLS failures were being retried, since Connection is
retryable and Tls is not, contradicting test_tls_error_is_not_retryable.

Now the connector records the reason before boxing it away, into a map
keyed by host and port. Keyed per host because a cached client serves
many hosts at once, and CertSlot already shows what a single shared slot
does under concurrency. The map lives on HyperClient rather than on
CachedClient: a retry with different TLS settings builds a *different*
cached client and still needs to read why the previous attempt failed,
so hanging it off the cached client would partition the map by exactly
the thing it exists to inform.

TlsFailure is derived from OpenSSL reason codes rather than message
text, and distinguishes the cases that lead somewhere different:
NoSharedCipher and UnsupportedProtocol mean a wider offer might work,
CertificateVerify means it never will, Reset and Timeout mean we learned
nothing about our ciphers.

Alert counts as "try wider" too, and that is the case that matters.
Running it against an RC4-only server offered AES returns
`ssl/tls alert handshake failure`, not `no shared cipher`: the latter is
what OpenSSL raises when *we* work it out locally, while a real server
just sends a fatal alert and declines to explain. Excluding alerts would
have meant never widening the offer in the single most common case. The
first version of this did exclude them; three test servers found it.

The stack is scanned for a code we recognise rather than read at index
zero, because OpenSSL pushes several errors per failure and a generic
"handshake failure" often sits on top of the one that explains it.

Python gets a TransportError (subclassing RuntimeError, so existing
`except RuntimeError` still works) carrying `kind`, `tls_failure` and
`retryable` as attributes. Six call sites were doing
PyRuntimeError::new_err(e.message) and dropping everything else.

Measured end to end against local servers:

  cipher mismatch   TLS handshake failed: peer sent an alert: ssl/tls
                    alert handshake failure
  version mismatch  TLS handshake failed: no protocol version in common
  nothing listening request failed: client error (Connect)

The third is the one that matters as much as the other two: a refused
connection must not claim to be a TLS problem, or the ladder would widen
its cipher offer at a host that was never listening.
---
 src/client/hyper.rs | 122 ++++++++++++++++++++++---
 src/client/mock.rs  |   2 +
 src/client/mod.rs   | 212 ++++++++++++++++++++++++++++++++++++++++++++
 src/python.rs       |  81 +++++++++++++----
 4 files changed, 385 insertions(+), 32 deletions(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index f3bc9a9..6641fbd 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1,4 +1,4 @@
-use super::{ClientError, HttpClient};
+use super::{ClientError, HttpClient, TlsFailure};
 use crate::config::RequestConfig;
 use crate::debug::{DebugLog, debug_record, new_debug_log};
 use crate::response::{CertInfo, RedirectHop, Response};
@@ -337,6 +337,21 @@ type CertSlot = Arc<Mutex<Option<CertInfo>>>;
 /// will lag, but the value will always be an IP that *was* used).
 type PeerSlot = Arc<Mutex<HashMap<String, IpAddr>>>;
 
+/// Where the connector leaves the reason a TLS handshake failed, keyed the
+/// same way as `PeerSlot`.
+///
+/// This exists because the reason is destroyed on the way out. The connector
+/// hands hyper-util a `Box<dyn Error>`, hyper-util wraps it in an error whose
+/// `Display` is the literal string "client error (Connect)", and nothing ever
+/// calls `.source()` to get back down to it. So by the time a failure reaches
+/// the caller, a cipher mismatch, a rejected certificate, a DNS failure and a
+/// refused connection are indistinguishable.
+///
+/// Keyed per host rather than held in a single slot because a cached client
+/// serves many hosts concurrently, and `CertSlot` already demonstrates what
+/// last-writer-wins does to a shared slot under load.
+type TlsFailureSlot = Arc<Mutex<HashMap<String, TlsFailure>>>;
+
 /// Build the lookup key for `PeerSlot` from a URI's host and port.
 /// HTTPS defaults to 443, everything else to 80 — matches what
 /// `HttpConnector` uses when it dials the OS resolver.
@@ -455,6 +470,8 @@ struct OpenSslConnector {
     // opened, read after each redirect hop so the right IP gets stamped
     // on the `RedirectHop` (or final `Response`).
     peer_slot: PeerSlot,
+    // Why the last handshake to a given host failed. See `TlsFailureSlot`.
+    tls_failure_slot: TlsFailureSlot,
     connect_timeout: Duration,
 }
 
@@ -579,6 +596,7 @@ impl OpenSslConnector {
         config: &RequestConfig,
         cert_slot: CertSlot,
         peer_slot: PeerSlot,
+        tls_failure_slot: TlsFailureSlot,
     ) -> Result<Self, ClientError> {
         // Ensure legacy ciphers (RC4, DES, etc.) are available
         ensure_legacy_provider();
@@ -625,6 +643,7 @@ impl OpenSslConnector {
             ssl,
             cert_slot,
             peer_slot,
+            tls_failure_slot,
             connect_timeout,
         })
     }
@@ -737,6 +756,7 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
         let ssl_connector = self.ssl.clone();
         let profile_active = self.profile_active;
         let cert_slot = self.cert_slot.clone();
+        let tls_failure_slot = self.tls_failure_slot.clone();
         let peer_slot = self.peer_slot.clone();
         let connect_timeout = self.connect_timeout;
 
@@ -746,7 +766,7 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
 
             // Record peer IP for this fresh connection. Best-effort —
             // failure to read peer_addr (vanishingly rare) is not fatal.
-            if let (Some(key), Ok(peer)) = (slot_key, tcp_stream.peer_addr())
+            if let (Some(key), Ok(peer)) = (slot_key.clone(), tcp_stream.peer_addr())
                 && let Ok(mut map) = peer_slot.lock()
             {
                 map.insert(key, peer.ip());
@@ -791,9 +811,20 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
             let mut stream = tokio_openssl::SslStream::new(ssl_conf, tcp_stream)
                 .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
 
+            // Record why, then box the error away as hyper-util requires.
+            // This is the only point where the reason still exists.
+            let record = |failure: TlsFailure| {
+                if let Some(ref key) = slot_key
+                    && let Ok(mut slot) = tls_failure_slot.lock()
+                {
+                    slot.insert(key.clone(), failure);
+                }
+            };
+
             tokio::time::timeout(connect_timeout, Pin::new(&mut stream).connect())
                 .await
                 .map_err(|_| -> Box<dyn std::error::Error + Send + Sync> {
+                    record(TlsFailure::Timeout);
                     Box::new(std::io::Error::new(
                         std::io::ErrorKind::TimedOut,
                         format!(
@@ -803,7 +834,10 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
                         ),
                     ))
                 })?
-                .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
+                .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> {
+                    record(TlsFailure::from_openssl(&e));
+                    Box::new(e)
+                })?;
 
             // Extract cert info after successful handshake
             let cert_info = extract_cert_info(stream.ssl());
@@ -899,6 +933,13 @@ pub struct HyperClient {
     // both a ForwardProxy client (for HTTP targets) and a Tunnel client (for
     // HTTPS targets), so we cache per-mode rather than a single client.
     cached: Mutex<std::collections::HashMap<ConnMode, CachedClient>>,
+    // Why the last handshake to each host failed.
+    //
+    // Deliberately on the client rather than on `CachedClient`: a retry with
+    // different TLS settings builds a *different* cached client, and it still
+    // needs to read why the previous attempt failed. Per-cached-client would
+    // partition this map by exactly the thing it exists to inform.
+    tls_failures: TlsFailureSlot,
 }
 
 impl Default for HyperClient {
@@ -911,6 +952,7 @@ impl HyperClient {
     pub fn new() -> Self {
         HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
+            tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
 
@@ -968,7 +1010,12 @@ impl HyperClient {
 
         let cert_slot: CertSlot = Arc::new(Mutex::new(None));
         let peer_slot: PeerSlot = Arc::new(Mutex::new(HashMap::new()));
-        let connector = OpenSslConnector::new(config, cert_slot.clone(), peer_slot.clone())?;
+        let connector = OpenSslConnector::new(
+            config,
+            cert_slot.clone(),
+            peer_slot.clone(),
+            self.tls_failures.clone(),
+        )?;
         let mut builder = Client::builder(TokioExecutor::new());
 
         // Shape the HTTP/2 SETTINGS and connection window to the profile.
@@ -1042,6 +1089,7 @@ async fn dispatch_request(
     config: &RequestConfig,
     log: &DebugLog,
     redirect_cookies: Option<&str>,
+    tls_failures: &TlsFailureSlot,
 ) -> Result<SingleResponse, ClientError> {
     // The pooled high-level client populates Host / :authority from the URI
     // itself, so we don't add a Host header here. Adding it would cause
@@ -1065,19 +1113,58 @@ async fn dispatch_request(
         AnyClient::Tunnel(c) => c.request(request).await,
         AnyClient::Socks5(c) => c.request(request).await,
     }
-    .map_err(|e| {
-        let msg = format!("request failed: {}", e);
-        let err_str = e.to_string().to_lowercase();
-        if err_str.contains("ssl") || err_str.contains("tls") || err_str.contains("certificate") {
-            ClientError::tls(msg)
-        } else {
-            ClientError::connection(msg)
-        }
-    })?;
+    .map_err(|e| classify_dispatch_error(&e, uri, tls_failures))?;
 
     parse_response(hyper_response, config, log).await
 }
 
+/// Turn a hyper-util client error into something that says what went wrong.
+///
+/// The error itself carries almost nothing: its `Display` is the literal
+/// `"client error (Connect)"` for every connect-time failure, so the substring
+/// test this replaces could never match and classified every TLS failure as a
+/// connection error. That also meant TLS failures were being retried, which
+/// `test_tls_error_is_not_retryable` says they should not be.
+///
+/// The real reason was recorded by the connector on its way out, so look it up
+/// by host. Absent an entry, the failure happened before TLS and connection is
+/// the honest answer.
+fn classify_dispatch_error(
+    e: &hyper_util::client::legacy::Error,
+    uri: &http::Uri,
+    tls_failures: &TlsFailureSlot,
+) -> ClientError {
+    let recorded = peer_slot_key(uri).and_then(|key| {
+        tls_failures
+            .lock()
+            .ok()
+            .and_then(|slot| slot.get(&key).cloned())
+    });
+
+    match recorded {
+        Some(failure) => ClientError::tls_detailed(
+            format!("TLS handshake failed: {}", describe_tls_failure(&failure)),
+            failure,
+        ),
+        None => ClientError::connection(format!("request failed: {}", e)),
+    }
+}
+
+/// A human-readable one-liner for a `TlsFailure`.
+fn describe_tls_failure(failure: &TlsFailure) -> String {
+    match failure {
+        TlsFailure::NoSharedCipher => "no cipher suite in common".to_string(),
+        TlsFailure::UnsupportedProtocol => "no protocol version in common".to_string(),
+        TlsFailure::Alert { description } => format!("peer sent an alert: {}", description),
+        TlsFailure::CertificateVerify { detail } => {
+            format!("certificate verification failed: {}", detail)
+        }
+        TlsFailure::Reset => "connection closed during the handshake".to_string(),
+        TlsFailure::Timeout => "handshake timed out".to_string(),
+        TlsFailure::Other { detail } => detail.clone(),
+    }
+}
+
 /// Opens a fresh TCP connection (optionally to a resolved IP instead of DNS)
 /// and performs TLS if HTTPS (with SNI set to the original hostname).
 /// Returns the connected stream and any certificate info collected during
@@ -1315,7 +1402,13 @@ pub(crate) async fn connect_stream(
                 config.timeout()
             ))
         })?
-        .map_err(|e| ClientError::tls(format!("TLS handshake failed: {}", e)))?;
+        .map_err(|e| {
+            let failure = TlsFailure::from_openssl(&e);
+            ClientError::tls_detailed(
+                format!("TLS handshake failed: {}", describe_tls_failure(&failure)),
+                failure,
+            )
+        })?;
 
     let cert_info = extract_cert_info(tls_stream.ssl());
     let negotiated_alpn = tls_stream
@@ -2060,6 +2153,7 @@ impl HyperClient {
                     config,
                     log,
                     hop_cookies.as_deref(),
+                    &self.tls_failures,
                 )
                 .await?
             };
diff --git a/src/client/mock.rs b/src/client/mock.rs
index 0dbd625..4916224 100644
--- a/src/client/mock.rs
+++ b/src/client/mock.rs
@@ -75,6 +75,7 @@ impl HttpClient for ErrorKindMockClient {
         Err(ClientError {
             message: self.message.clone(),
             kind: self.kind.clone(),
+            tls_failure: None,
         })
     }
 }
@@ -117,6 +118,7 @@ impl HttpClient for SequenceMockClient {
             return Err(ClientError {
                 message: format!("mock error on call {}", call),
                 kind: self.fail_kind.clone(),
+                tls_failure: None,
             });
         }
 
diff --git a/src/client/mod.rs b/src/client/mod.rs
index c16ced2..7695324 100644
--- a/src/client/mod.rs
+++ b/src/client/mod.rs
@@ -42,10 +42,150 @@ impl ErrorKind {
     }
 }
 
+/// Why a TLS handshake failed, in enough detail to decide what to do about it.
+///
+/// The distinction that matters is whether a *broader* offer would plausibly
+/// help. A peer that shares no cipher with us might accept an older one; a peer
+/// whose certificate we rejected will reject it again just as hard, and
+/// retrying with weaker crypto there would be both futile and alarming.
+///
+/// Derived from OpenSSL's reason codes rather than its message text. The text
+/// is stable enough to log and far too loose to branch on.
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum TlsFailure {
+    /// No cipher suite in common. The clearest signal that a wider offer is
+    /// worth trying.
+    NoSharedCipher,
+    /// No protocol version in common, or the peer rejected the one we picked.
+    UnsupportedProtocol,
+    /// The peer sent a fatal alert. `description` is OpenSSL's reason text.
+    /// handshake_failure most often means the same thing as NoSharedCipher,
+    /// but the peer declined to say which, so it is kept separate.
+    Alert {
+        description: String,
+    },
+    /// Certificate verification failed. Never a reason to widen the offer.
+    CertificateVerify {
+        detail: String,
+    },
+    /// The connection died mid-handshake with no alert. Typical of old servers
+    /// and middleboxes choking on a large or unfamiliar ClientHello, and also
+    /// of something actively refusing us, which is why it is not on its own
+    /// treated as a cipher problem.
+    Reset,
+    /// The handshake did not complete in time.
+    Timeout,
+    Other {
+        detail: String,
+    },
+}
+
+impl TlsFailure {
+    /// Would offering more ciphers and older protocol versions plausibly help?
+    ///
+    /// `Alert` is in here, and that is the case that matters in practice. A
+    /// server with no cipher in common with us does not send a neatly labelled
+    /// "no shared cipher"; that reason code is what OpenSSL raises when *we*
+    /// work it out locally. What actually comes back from the wire is a fatal
+    /// handshake_failure alert with no explanation. Measured against an
+    /// RC4-only server offered AES: the error is
+    /// `ssl/tls alert handshake failure`, not `no shared cipher`. Excluding
+    /// alerts here would mean never widening the offer in the single most
+    /// common case.
+    ///
+    /// Still conservative about the rest. A certificate rejection, a timeout
+    /// and a reset all say nothing about our cipher list, and widening in
+    /// response would send deprecated crypto to a peer that never asked for
+    /// it.
+    pub fn suggests_wider_offer(&self) -> bool {
+        matches!(
+            self,
+            TlsFailure::NoSharedCipher | TlsFailure::UnsupportedProtocol | TlsFailure::Alert { .. }
+        )
+    }
+
+    /// Classify an OpenSSL handshake error.
+    pub fn from_openssl(err: &openssl::ssl::Error) -> Self {
+        // Reason codes from the vendored sslerr.h.
+        const NO_SHARED_CIPHER: i32 = 193;
+        const NO_PROTOCOLS_AVAILABLE: i32 = 191;
+        const UNSUPPORTED_PROTOCOL: i32 = 258;
+        const UNSUPPORTED_SSL_VERSION: i32 = 259;
+        const WRONG_VERSION_NUMBER: i32 = 267;
+        const VERSION_TOO_HIGH: i32 = 166;
+        const VERSION_TOO_LOW: i32 = 396;
+        const CERTIFICATE_VERIFY_FAILED: i32 = 134;
+        const TLSV1_ALERT_PROTOCOL_VERSION: i32 = 1070;
+        const TLSV1_ALERT_INSUFFICIENT_SECURITY: i32 = 1071;
+        const SSLV3_ALERT_HANDSHAKE_FAILURE: i32 = 1040;
+        const SSLV3_ALERT_ILLEGAL_PARAMETER: i32 = 1047;
+
+        // Scan the whole stack for a code we recognise rather than reading
+        // only the first. OpenSSL pushes several errors for one failure and
+        // the most specific is not reliably at the front: a generic
+        // "ssl handshake failure" often sits on top of the reason that
+        // actually explains it.
+        if let Some(stack) = err.ssl_error() {
+            let mut fallback: Option<TlsFailure> = None;
+
+            for e in stack.errors() {
+                let reason = e.reason().unwrap_or("").to_string();
+                let classified = match e.reason_code() {
+                    NO_SHARED_CIPHER => Some(TlsFailure::NoSharedCipher),
+                    // The peer telling us our ciphers are too weak is the same
+                    // actionable fact as having none in common, from the other
+                    // direction.
+                    TLSV1_ALERT_INSUFFICIENT_SECURITY => Some(TlsFailure::NoSharedCipher),
+                    NO_PROTOCOLS_AVAILABLE
+                    | UNSUPPORTED_PROTOCOL
+                    | UNSUPPORTED_SSL_VERSION
+                    | WRONG_VERSION_NUMBER
+                    | VERSION_TOO_HIGH
+                    | VERSION_TOO_LOW
+                    | TLSV1_ALERT_PROTOCOL_VERSION => Some(TlsFailure::UnsupportedProtocol),
+                    CERTIFICATE_VERIFY_FAILED => {
+                        Some(TlsFailure::CertificateVerify { detail: reason })
+                    }
+                    SSLV3_ALERT_HANDSHAKE_FAILURE | SSLV3_ALERT_ILLEGAL_PARAMETER => {
+                        Some(TlsFailure::Alert {
+                            description: reason,
+                        })
+                    }
+                    _ => {
+                        // Unrecognised: keep the first one as a description of
+                        // last resort, but carry on looking for a code we know.
+                        fallback.get_or_insert(TlsFailure::Other { detail: reason });
+                        None
+                    }
+                };
+                if let Some(failure) = classified {
+                    return failure;
+                }
+            }
+
+            if let Some(failure) = fallback {
+                return failure;
+            }
+        }
+
+        // No reason stack: the failure was below TLS. SYSCALL with an io cause
+        // is a reset; without one it is an unexpected EOF, which OpenSSL
+        // reports the same way.
+        match err.io_error() {
+            Some(io) if io.kind() == std::io::ErrorKind::TimedOut => TlsFailure::Timeout,
+            Some(_) => TlsFailure::Reset,
+            None => TlsFailure::Reset,
+        }
+    }
+}
+
 #[derive(Debug, Clone)]
 pub struct ClientError {
     pub message: String,
     pub kind: ErrorKind,
+    /// Structured detail when the failure was a TLS handshake, so callers can
+    /// branch on why rather than substring-matching `message`.
+    pub tls_failure: Option<TlsFailure>,
 }
 
 impl ClientError {
@@ -53,6 +193,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::Connection,
+            tls_failure: None,
         }
     }
 
@@ -60,6 +201,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::Timeout,
+            tls_failure: None,
         }
     }
 
@@ -67,6 +209,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::Tls,
+            tls_failure: None,
         }
     }
 
@@ -74,6 +217,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::InvalidUrl,
+            tls_failure: None,
         }
     }
 
@@ -81,6 +225,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::TooManyRedirects,
+            tls_failure: None,
         }
     }
 
@@ -88,6 +233,18 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::Status(status),
+            tls_failure: None,
+        }
+    }
+
+    /// A TLS failure that knows why. Prefer this over `tls` wherever an
+    /// `openssl::ssl::Error` is in hand, since the reason is what the profile
+    /// ladder branches on and it cannot be recovered from the message text.
+    pub fn tls_detailed(message: String, failure: TlsFailure) -> Self {
+        ClientError {
+            message,
+            kind: ErrorKind::Tls,
+            tls_failure: Some(failure),
         }
     }
 
@@ -95,6 +252,7 @@ impl ClientError {
         ClientError {
             message,
             kind: ErrorKind::Other,
+            tls_failure: None,
         }
     }
 }
@@ -329,3 +487,57 @@ mod tests {
         assert_eq!(config.max_retries(), 0);
     }
 }
+
+#[cfg(test)]
+mod tls_failure_tests {
+    use super::*;
+
+    #[test]
+    fn test_negotiation_failures_suggest_a_wider_offer() {
+        // The three shapes a "we could not agree" answer arrives in.
+        assert!(TlsFailure::NoSharedCipher.suggests_wider_offer());
+        assert!(TlsFailure::UnsupportedProtocol.suggests_wider_offer());
+        assert!(
+            TlsFailure::Alert {
+                description: "ssl/tls alert handshake failure".to_string(),
+            }
+            .suggests_wider_offer(),
+            "a handshake_failure alert is what a real cipher mismatch looks \
+             like on the wire; excluding it would defeat the ladder in its \
+             most common case"
+        );
+    }
+
+    #[test]
+    fn test_other_failures_do_not_suggest_a_wider_offer() {
+        // Widening the offer in response to any of these would send weaker
+        // crypto to a peer whose objection had nothing to do with our ciphers.
+        assert!(
+            !TlsFailure::CertificateVerify {
+                detail: "self signed certificate".to_string(),
+            }
+            .suggests_wider_offer()
+        );
+        assert!(!TlsFailure::Reset.suggests_wider_offer());
+        assert!(!TlsFailure::Timeout.suggests_wider_offer());
+        assert!(
+            !TlsFailure::Other {
+                detail: "something else".to_string(),
+            }
+            .suggests_wider_offer()
+        );
+    }
+
+    #[test]
+    fn test_constructors_leave_tls_failure_unset() {
+        // Only the TLS-aware constructor carries detail; everything else must
+        // not claim to know why a handshake failed.
+        assert!(ClientError::connection("x".into()).tls_failure.is_none());
+        assert!(ClientError::timeout("x".into()).tls_failure.is_none());
+        assert!(ClientError::tls("x".into()).tls_failure.is_none());
+
+        let detailed = ClientError::tls_detailed("x".into(), TlsFailure::NoSharedCipher);
+        assert_eq!(detailed.kind, ErrorKind::Tls);
+        assert_eq!(detailed.tls_failure, Some(TlsFailure::NoSharedCipher));
+    }
+}
diff --git a/src/python.rs b/src/python.rs
index 267767f..4c394c8 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -19,6 +19,7 @@ use crate::batch::{self, BatchResult, RateLimiter};
 use crate::client::HttpClient;
 use crate::client::hyper::HyperClient;
 use crate::client::raw;
+use crate::client::{ClientError, ErrorKind, TlsFailure};
 use crate::config::RequestConfig;
 use crate::multipart;
 use crate::response::{CertInfo, RedirectHop, Response, ResponseHash};
@@ -380,6 +381,61 @@ pyo3::create_exception!(
     "Raised by Response.raise_for_status() when the response status is 4xx or 5xx."
 );
 
+// ── Turning a ClientError into a Python exception ─────────────────
+
+pyo3::create_exception!(
+    blasthttp,
+    TransportError,
+    pyo3::exceptions::PyRuntimeError,
+    "Raised when a request never produced a response: DNS, TCP, TLS or timeout."
+);
+
+/// Convert a `ClientError` into a Python exception that keeps what we know.
+///
+/// Every call site used to do `PyRuntimeError::new_err(e.message)`, which threw
+/// away `kind` and left Python to substring-match a sentence to find out
+/// whether a host was unreachable, timed out, or refused our ciphers. The
+/// message stays the same so existing `except RuntimeError` keeps working
+/// (TransportError subclasses it), but the detail is now attached as
+/// attributes:
+///
+///   `kind`         one of connection, timeout, tls, invalid_url,
+///                  too_many_redirects, status, other
+///   `tls_failure`  when the handshake failed, why: no_shared_cipher,
+///                  unsupported_protocol, alert, certificate_verify, reset,
+///                  timeout, other. `None` otherwise.
+///   `retryable`    whether trying the same request again might help.
+fn client_error_to_py(e: ClientError) -> PyErr {
+    let kind = match &e.kind {
+        ErrorKind::Connection => "connection",
+        ErrorKind::Timeout => "timeout",
+        ErrorKind::Tls => "tls",
+        ErrorKind::InvalidUrl => "invalid_url",
+        ErrorKind::TooManyRedirects => "too_many_redirects",
+        ErrorKind::Status(_) => "status",
+        ErrorKind::Other => "other",
+    };
+    let tls_failure = e.tls_failure.as_ref().map(|f| match f {
+        TlsFailure::NoSharedCipher => "no_shared_cipher",
+        TlsFailure::UnsupportedProtocol => "unsupported_protocol",
+        TlsFailure::Alert { .. } => "alert",
+        TlsFailure::CertificateVerify { .. } => "certificate_verify",
+        TlsFailure::Reset => "reset",
+        TlsFailure::Timeout => "timeout",
+        TlsFailure::Other { .. } => "other",
+    });
+    let retryable = e.kind.is_retryable();
+
+    let err = TransportError::new_err(e.message);
+    Python::attach(|py| {
+        let v = err.value(py);
+        let _ = v.setattr("kind", kind);
+        let _ = v.setattr("tls_failure", tls_failure);
+        let _ = v.setattr("retryable", retryable);
+    });
+    err
+}
+
 // ── Response wrapper ──────────────────────────────────────────────
 
 #[pyclass(name = "Response")]
@@ -996,10 +1052,7 @@ impl BlastHTTP {
             if let Some(ref limiter) = limiter {
                 limiter.acquire().await;
             }
-            let response = client
-                .send(&config)
-                .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+            let response = client.send(&config).await.map_err(client_error_to_py)?;
             Ok(PyResponse {
                 inner: response,
                 headers_cache: OnceLock::new(),
@@ -1190,10 +1243,7 @@ impl BlastHTTP {
             if let Some(ref limiter) = limiter {
                 limiter.acquire().await;
             }
-            let response = client
-                .send(&config)
-                .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+            let response = client.send(&config).await.map_err(client_error_to_py)?;
 
             // Write body bytes to file
             let mut file = std::fs::File::create(&path).map_err(|e| {
@@ -1271,7 +1321,7 @@ impl BlastHTTP {
             }
             let conn = raw::RawConnection::connect(&url, &config)
                 .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+                .map_err(client_error_to_py)?;
             Ok(PyRawConnection {
                 inner: Arc::new(conn),
                 rate_limiter: limiter_for_conn,
@@ -1306,10 +1356,7 @@ impl PyRawConnection {
             if let Some(ref limiter) = limiter {
                 limiter.acquire().await;
             }
-            inner
-                .send_bytes(&data)
-                .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+            inner.send_bytes(&data).await.map_err(client_error_to_py)?;
             Ok(())
         })
     }
@@ -1337,7 +1384,7 @@ impl PyRawConnection {
             let data = inner
                 .read_raw(max_bytes, timeout_ms)
                 .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+                .map_err(client_error_to_py)?;
             Ok(data)
         })
     }
@@ -1346,10 +1393,7 @@ impl PyRawConnection {
     fn close<'py>(&self, py: Python<'py>) -> PyResult<Bound<'py, PyAny>> {
         let inner = self.inner.clone();
         future_into_py(py, async move {
-            inner
-                .close()
-                .await
-                .map_err(|e| PyRuntimeError::new_err(e.message))?;
+            inner.close().await.map_err(client_error_to_py)?;
             Ok(())
         })
     }
@@ -2052,6 +2096,7 @@ fn blasthttp(m: &Bound<'_, PyModule>) -> PyResult<()> {
     m.add_class::<PyRequest>()?;
     m.add_class::<PyRawConnection>()?;
     m.add("HTTPStatusError", m.py().get_type::<HTTPStatusError>())?;
+    m.add("TransportError", m.py().get_type::<TransportError>())?;
     register_h2_submodule(m)?;
     crate::mock::register_mock_submodule(m)?;
     register_headers_as_mapping(m)?;

From 8bbccc5a059b4c2869c909fe60c135eec35363d8 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 12:46:23 -0400
Subject: [PATCH 11/44] Promote the antibot classifier into the library

Phase 2. The classifier already existed under tests/, regression-tested
against real captured responses, and took exactly the shape of a
response. It was just unreachable from anything that ships.

Now on Response::protection(), computed rather than stored so it stays
right for a response built by hand, and exposed to Python as a
(outcome, vendor) pair.

The distinction worth having is challenge versus blocked. A blocked
request might succeed with different connection settings; a challenge
never will, because answering it means running the page's JavaScript.
That is the signal to hand a URL to a real browser instead of retrying,
and BBOT should not have to find it by grepping our response body for
cf-mitigated.

Verified from Python against the wheel:

  akamai.com                  403  ('blocked', 'akamai')
  scrapingcourse cf-challenge 403  ('challenge', 'cloudflare')
  google.com                  200  ('ok', '-')

And the error path from the same build, which Phase 1 made possible:

  refused connection  TransportError kind=connection tls_failure=None
                      retryable=True
  cipher mismatch     kind=tls tls_failure=alert retryable=False
  version mismatch    kind=tls tls_failure=unsupported_protocol
                      retryable=False

Together those are the whole contract a consumer needs: whether we got a
response, whether something intercepted it, whether retrying could help,
and whether a browser is the only way through.
---
 {tests/support => src}/antibot.rs | 37 +++++++-------
 src/lib.rs                        |  1 +
 src/python.rs                     | 34 +++++++++++++
 src/response.rs                   | 85 +++++++++++++++++++++++++++++++
 tests/antibot_classify.rs         |  4 +-
 tests/live_benchmark.rs           |  2 +-
 tests/support/mod.rs              |  1 -
 7 files changed, 140 insertions(+), 24 deletions(-)
 rename {tests/support => src}/antibot.rs (90%)

diff --git a/tests/support/antibot.rs b/src/antibot.rs
similarity index 90%
rename from tests/support/antibot.rs
rename to src/antibot.rs
index 340e307..35bd27b 100644
--- a/tests/support/antibot.rs
+++ b/src/antibot.rs
@@ -1,22 +1,21 @@
-// Classifying what a bot-management product did to a request.
-//
-// A scan needs to tell three things apart that all look like "a response":
-// the origin answered, a protection product answered instead, or the product
-// answered *and* handed us a session. Without that, an operator sees a wall of
-// 403s and cannot tell a blocked host from a dead one, which is the false
-// negative this whole effort exists to fix.
-//
-// Two rules learned the hard way, both from a first pass that got them wrong:
-//
-// 1. Never match a vendor's name in the body. `datadome.co`, `imperva.com` and
-//    `kasada.io` all scored as challenges purely from their own marketing
-//    copy. Everything here keys on headers, cookies, status, or a body string
-//    that cannot occur in ordinary prose.
-//
-// 2. Always classify a known-good reference client in the same pass. A block
-//    and a site that simply changed look identical from one sample.
-
-#![allow(dead_code)]
+//! Working out what a bot-management product did to a request.
+//!
+//! A scan has to tell three things apart that all arrive looking like a
+//! response: the origin answered, a protection product answered instead, or
+//! the product answered and handed us a session anyway. Collapsing those is
+//! what turns a protected host into an apparently dead one, which is the
+//! false negative this exists to prevent.
+//!
+//! Two rules, both learned by getting them wrong:
+//!
+//! 1. Never match a vendor's name in the body. `datadome.co`, `imperva.com`
+//!    and `kasada.io` all scored as challenges purely from their own marketing
+//!    copy. Everything here keys on headers, cookies, status, or a body string
+//!    that cannot occur in ordinary prose.
+//!
+//! 2. A response that merely *references* a vendor's script is not a challenge
+//!    page. Every page on a protected site carries the loader; only a page
+//!    shaped like an interception is a block.
 
 use std::fmt;
 
diff --git a/src/lib.rs b/src/lib.rs
index 34590e0..d2549dc 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -1,3 +1,4 @@
+pub mod antibot;
 pub mod batch;
 pub mod client;
 pub mod config;
diff --git a/src/python.rs b/src/python.rs
index 4c394c8..ae1e54f 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -15,6 +15,7 @@ use std::time::Duration;
 use tokio::sync::Mutex as TokioMutex;
 use tokio::time::Instant;
 
+use crate::antibot as blasthttp_antibot;
 use crate::batch::{self, BatchResult, RateLimiter};
 use crate::client::HttpClient;
 use crate::client::hyper::HyperClient;
@@ -705,6 +706,39 @@ impl PyResponse {
 
     /// Debug messages collected during the request.
     /// Always populated — Python side can log/display as needed.
+    /// What a bot-management product did to this request, if anything.
+    ///
+    /// Returns a tuple of `(outcome, vendor)`, both lowercase strings:
+    ///
+    ///   outcome  ok | present | challenge | blocked | error
+    ///   vendor   cloudflare | akamai | datadome | perimeterx | imperva |
+    ///            f5 | kasada | unknown | "-" when nothing was detected
+    ///
+    /// `present` means a product is in front of the host and let us through,
+    /// usually having issued a session cookie. That is a more useful answer
+    /// than a bare 200: it says we passed something that was actively looking.
+    ///
+    /// `challenge` is the one to branch on. A blocked request might succeed
+    /// with different settings; a challenge will not, because answering it
+    /// means running the page's JavaScript. It is the signal to hand the URL
+    /// to a real browser rather than retry.
+    #[getter]
+    fn protection(&self) -> (String, String) {
+        let outcome = self.inner.protection();
+        let name = match &outcome {
+            blasthttp_antibot::Outcome::Ok => "ok",
+            blasthttp_antibot::Outcome::Present(_) => "present",
+            blasthttp_antibot::Outcome::Challenge(_) => "challenge",
+            blasthttp_antibot::Outcome::Blocked(_) => "blocked",
+            blasthttp_antibot::Outcome::Error => "error",
+        };
+        let vendor = outcome
+            .vendor()
+            .map(|v| v.to_string())
+            .unwrap_or_else(|| "-".to_string());
+        (name.to_string(), vendor)
+    }
+
     #[getter]
     fn debug_log(&self) -> Vec<String> {
         self.inner.debug_log.clone()
diff --git a/src/response.rs b/src/response.rs
index b9651f4..c2a1ed1 100644
--- a/src/response.rs
+++ b/src/response.rs
@@ -147,6 +147,24 @@ impl Response {
 
     /// Body decoded as UTF-8 (lossy — invalid sequences become U+FFFD).
     /// Computed on first access and cached.
+    /// What, if anything, a bot-management product did to this request.
+    ///
+    /// Computed from the response rather than stored, so it stays correct if a
+    /// caller builds a `Response` by hand. Cheap: the classifier reads headers,
+    /// cookies, status and at most the first 64 KB of the body.
+    ///
+    /// The distinction worth acting on is `Challenge`. A blocked request might
+    /// succeed with different connection settings; a challenge will not,
+    /// because answering it means executing the page's JavaScript. That is the
+    /// signal to hand the URL to a real browser instead of retrying.
+    pub fn protection(&self) -> crate::antibot::Outcome {
+        crate::antibot::classify(&crate::antibot::ResponseFacts {
+            status: self.status,
+            headers: &self.headers,
+            body: self.body(),
+        })
+    }
+
     pub fn body(&self) -> &str {
         self.body_cache
             .get_or_init(|| match std::str::from_utf8(&self.body_bytes) {
@@ -452,3 +470,70 @@ mod tests {
         assert!(cookies.is_empty());
     }
 }
+
+#[cfg(test)]
+mod protection_tests {
+    use super::*;
+    use crate::antibot::{Outcome, Vendor};
+
+    fn response_with(status: u16, headers: &[(&str, &str)], body: &str) -> Response {
+        Response {
+            url: "https://example.com/".to_string(),
+            status,
+            headers: headers
+                .iter()
+                .map(|(k, v)| (k.to_string(), v.to_string()))
+                .collect(),
+            body_bytes: body.as_bytes().to_vec(),
+            elapsed_ms: 1,
+            redirect_chain: Vec::new(),
+            cert_info: None,
+            peer_ip: None,
+            request_url: "https://example.com/".to_string(),
+            request_method: "GET".to_string(),
+            debug_log: Vec::new(),
+            decode_error: None,
+            body_cache: OnceLock::new(),
+            raw_headers_cache: OnceLock::new(),
+            cookies_cache: OnceLock::new(),
+            hash_cache: OnceLock::new(),
+        }
+    }
+
+    #[test]
+    fn test_response_classifies_itself() {
+        // The classifier is only useful if a caller can reach it from the
+        // response it already has. This is the path BBOT takes.
+        //
+        // Headers are the ones www.akamai.com actually returns, which notably
+        // include no `server` header at all.
+        let blocked = response_with(
+            403,
+            &[
+                ("akamai-grn", "0.6d551702.1790644996.315f62a4"),
+                ("content-type", "text/html"),
+            ],
+            "<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD></HTML>",
+        );
+        assert_eq!(blocked.protection(), Outcome::Blocked(Vendor::Akamai));
+    }
+
+    #[test]
+    fn test_challenge_is_distinguishable_from_a_block() {
+        // The distinction the whole escalation path rests on: a block might
+        // succeed with different settings, a challenge never will.
+        let challenge = response_with(
+            403,
+            &[("cf-mitigated", "challenge"), ("server", "cloudflare")],
+            "<html>Just a moment...</html>",
+        );
+        assert_eq!(
+            challenge.protection(),
+            Outcome::Challenge(Vendor::Cloudflare)
+        );
+
+        let ordinary = response_with(200, &[("server", "nginx")], "<html>hello</html>");
+        assert_eq!(ordinary.protection(), Outcome::Ok);
+        assert!(ordinary.protection().got_through());
+    }
+}
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index fc82e3e..6ac0a9b 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -7,9 +7,7 @@
 // pair is the clearest single measurement of the problem this effort exists to
 // fix, so it is worth having as a regression fixture.
 
-mod support;
-
-use support::antibot::{Outcome, ResponseFacts, Vendor, classify};
+use blasthttp::antibot::{Outcome, ResponseFacts, Vendor, classify};
 
 fn hdrs(pairs: &[(&str, &str)]) -> Vec<(String, String)> {
     pairs
diff --git a/tests/live_benchmark.rs b/tests/live_benchmark.rs
index 51c517b..cc9c8ec 100644
--- a/tests/live_benchmark.rs
+++ b/tests/live_benchmark.rs
@@ -14,10 +14,10 @@
 
 mod support;
 
+use blasthttp::antibot::{Outcome, ResponseFacts, classify};
 use blasthttp::client::HttpClient;
 use blasthttp::client::hyper::HyperClient;
 use blasthttp::config::RequestConfig;
-use support::antibot::{Outcome, ResponseFacts, classify};
 use support::tls_server::{TlsServerConfig, TlsTestServer};
 
 fn config(url: &str) -> RequestConfig {
diff --git a/tests/support/mod.rs b/tests/support/mod.rs
index 67b68db..1163676 100644
--- a/tests/support/mod.rs
+++ b/tests/support/mod.rs
@@ -10,6 +10,5 @@
 
 #![allow(dead_code)]
 
-pub mod antibot;
 pub mod ja4;
 pub mod tls_server;

From 8bbf86675702bfc974a1db3b676674303922846d Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 14:44:12 -0400
Subject: [PATCH 12/44] Generalise to three connection profiles

Phase 3. BrowserProfile becomes ConnectionProfile, and there are now
three: compatibility, modern, chrome131.

The trap this had to avoid: several behaviours were gated on *whether a
profile existed* rather than on anything in it. The padding and
encrypt_then_mac flips, add_browser_extensions, the OCSP status_request
and http2_adaptive_window were all "if a profile is set". Naming today's
default behaviour as a profile would have silently switched every one of
them on for the configuration that is supposed to be unchanged. They are
now explicit fields: `browser_extensions` on the TLS half, `http2` as an
Option, headers as a possibly-empty list.

So compatibility is a transcription, not a cleanup, and every None in it
is load-bearing. tests/legacy_default.rs and test_compat_path_keeps_the_scsv
are its specification, and all 314 tests pass unchanged. Confirmed on the
wire too: no profile and --profile compatibility both produce
t13d10512h2_c86627d460ee_c50a3655fff1 with 105 suites, bit for bit.

resolved_profile() is now infallible. Naming no profile means the default
rather than "no profile", which collapses five Option branches, and an
unknown name is an error instead of silently resolving to whatever the
default happened to be.

modern is new: 11 suites instead of 105, TLS 1.2 and 1.3 only, no browser
claim.

One correction, because the measurement contradicted the argument I wrote
it on. modern was justified as "unremarkable in real traffic". It is not.
Checked against tlsfingerprint.io, which indexes around 17 billion
observed connections, all three profiles come back never seen, modern
included. Narrowing the cipher list fixes the JA4 and does nothing for the
rest of the hello, which stays distinctive for reasons OpenSSL gives no
way to change: no GREASE anywhere, and extensions in OpenSSL's fixed order
rather than a browser's shuffled one.

The case for modern therefore rests on the JA4 going from t13d10512h2 to
t13d1113h2, and on stopping the SSLv3 and TLS 1.0 offer, not on blending
in. Blending in is not available on this TLS stack, and the profile docs
now say so rather than claiming otherwise.
---
 src/client/hyper.rs | 110 ++++++++-------
 src/config.rs       |  28 +++-
 src/profile.rs      | 328 +++++++++++++++++++++++++++++++++++++-------
 3 files changed, 358 insertions(+), 108 deletions(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 6641fbd..69d84a4 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -58,26 +58,6 @@ fn sanitize_uri(url: &str) -> String {
     out
 }
 
-/// Cipher list used when the caller doesn't name one.
-///
-/// This has to be set explicitly. `SslConnector::builder` installs its own
-/// list first, `DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:...`,
-/// and `set_security_level(0)` does NOT undo it. The security level governs
-/// how weak a cipher may be; the cipher list governs which ones are offered at
-/// all. Leaving the list alone means RC4, DES, 3DES and SEED never reach the
-/// wire whatever the security level says, so a server speaking only one of
-/// them is unreachable, which is the whole reason this project builds its own
-/// OpenSSL.
-///
-/// `ALL` is every suite the build provides except the eNULL (no encryption)
-/// ones. It does include the aNULL suites, which skip authentication: those
-/// are worth reaching for a scanner, and this client already defaults to not
-/// verifying certificates, so they give up nothing that was being enforced.
-/// Null *encryption* stays opt-in through `cipher_string`, because negotiating
-/// it by default would hand back a connection that looks like TLS and encrypts
-/// nothing.
-const DEFAULT_CIPHER_LIST: &str = "ALL";
-
 /// Add the extensions a browser sends that OpenSSL does not send on its own.
 ///
 /// JA4 hashes the list of extension *types*, not their contents, so presence
@@ -217,9 +197,13 @@ fn apply_tls_settings(
 ) -> Result<(), ClientError> {
     use openssl::ssl::SslOptions;
 
-    let profile = config
-        .resolved_profile()
-        .filter(|_| !bisect_disabled("tls"));
+    // A bisect of the TLS layer means "what the default would have sent",
+    // which is now a named profile rather than an absence.
+    let profile = if bisect_disabled("tls") {
+        crate::profile::default_profile()
+    } else {
+        config.resolved_profile()
+    };
 
     // Security level 0: allow all ciphers including RC4 and DES. This is an
     // offensive-first tool and needs to connect to anything.
@@ -232,7 +216,7 @@ fn apply_tls_settings(
     // floor at 1.2 so this never widens a profile's offer.
     builder.clear_options(SslOptions::NO_SSLV3);
 
-    if let Some(p) = profile {
+    if profile.tls.browser_extensions {
         // Removing two extensions a browser does not send.
         //
         // encrypt_then_mac (0x0016) goes away by disabling the feature.
@@ -249,33 +233,51 @@ fn apply_tls_settings(
         builder.set_options(SslOptions::from_bits_retain(SSL_OP_NO_ENCRYPT_THEN_MAC));
         builder.clear_options(SslOptions::from_bits_retain(SSL_OP_TLSEXT_PADDING));
 
-        builder.set_ciphersuites(p.tls.ciphersuites).map_err(|e| {
+        add_browser_extensions(builder)?;
+    }
+
+    // Each of these is optional: leaving one unset keeps OpenSSL's own list
+    // and order, which is what a profile imitating an ordinary OpenSSL client
+    // wants.
+    if let Some(suites) = profile.tls.ciphersuites {
+        builder.set_ciphersuites(suites).map_err(|e| {
             ClientError::tls(format!(
                 "profile '{}' has an invalid TLS 1.3 suite list: {}",
-                p.name, e
+                profile.name, e
             ))
         })?;
-        builder.set_sigalgs_list(p.tls.sigalgs).map_err(|e| {
+    }
+    if let Some(sigalgs) = profile.tls.sigalgs {
+        builder.set_sigalgs_list(sigalgs).map_err(|e| {
             ClientError::tls(format!(
                 "profile '{}' has an invalid signature algorithm list: {}",
-                p.name, e
+                profile.name, e
             ))
         })?;
-        builder.set_groups_list(p.tls.groups).map_err(|e| {
+    }
+    if let Some(groups) = profile.tls.groups {
+        builder.set_groups_list(groups).map_err(|e| {
             ClientError::tls(format!(
                 "profile '{}' has an invalid group list: {}",
-                p.name, e
+                profile.name, e
             ))
         })?;
-
-        add_browser_extensions(builder)?;
     }
 
+    // Always set the cipher list, never leave it alone.
+    //
+    // `SslConnector::builder` installs its own first,
+    // `DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:...`, and
+    // `set_security_level(0)` does NOT undo it. The security level governs how
+    // weak a negotiated cipher may be; the cipher list governs which ones are
+    // offered at all. Leaving it alone means RC4, DES, 3DES and SEED never
+    // reach the wire whatever the security level says, so a server speaking
+    // only one of them is unreachable, which is the whole reason this project
+    // builds its own OpenSSL.
     let ciphers = config
         .cipher_string
         .as_deref()
-        .or(profile.map(|p| p.tls.cipher_list))
-        .unwrap_or(DEFAULT_CIPHER_LIST);
+        .unwrap_or(profile.tls.cipher_list);
     builder
         .set_cipher_list(ciphers)
         .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
@@ -283,7 +285,7 @@ fn apply_tls_settings(
     let min_version = config
         .min_tls_version
         .as_deref()
-        .or(profile.map(|p| p.tls.min_version));
+        .or(profile.tls.min_version);
     if let Some(v) = min_version {
         let version = parse_tls_version(v)?;
         builder
@@ -294,7 +296,7 @@ fn apply_tls_settings(
     let max_version = config
         .max_tls_version
         .as_deref()
-        .or(profile.map(|p| p.tls.max_version));
+        .or(profile.tls.max_version);
     if let Some(v) = max_version {
         let version = parse_tls_version(v)?;
         builder
@@ -638,7 +640,7 @@ impl OpenSslConnector {
         http.set_connect_timeout(Some(connect_timeout));
 
         Ok(OpenSslConnector {
-            profile_active: config.resolved_profile().is_some(),
+            profile_active: config.resolved_profile().tls.browser_extensions,
             http,
             ssl,
             cert_slot,
@@ -1025,18 +1027,20 @@ impl HyperClient {
         // everything hyper-util exposes of the Akamai HTTP/2 fingerprint; the
         // remaining two fields, HEADER_TABLE_SIZE and the pseudo-header order,
         // need forks of hyper-util and h2 respectively and are left alone.
-        if let Some(p) = config
-            .resolved_profile()
-            .filter(|_| !bisect_disabled("http2"))
-        {
+        let http2_profile = if bisect_disabled("http2") {
+            None
+        } else {
+            config.resolved_profile().http2
+        };
+        if let Some(p) = http2_profile {
             builder
-                .http2_initial_stream_window_size(p.http2.initial_stream_window)
-                .http2_initial_connection_window_size(p.http2.initial_connection_window)
-                .http2_max_header_list_size(p.http2.max_header_list_size)
+                .http2_initial_stream_window_size(p.initial_stream_window)
+                .http2_initial_connection_window_size(p.initial_connection_window)
+                .http2_max_header_list_size(p.max_header_list_size)
                 // `None` omits MAX_FRAME_SIZE entirely, which is what Chrome
                 // does. Our default announces 16384 and Chrome announces
                 // nothing, so leaving this unset is itself part of the match.
-                .http2_max_frame_size(p.http2.max_frame_size)
+                .http2_max_frame_size(p.max_frame_size)
                 // Adaptive window resizes the connection window as traffic
                 // flows, which would emit WINDOW_UPDATE frames no browser
                 // sends and undo the fixed value set above.
@@ -1360,7 +1364,7 @@ pub(crate) async fn connect_stream(
         .map_err(|e| ClientError::tls(format!("failed to set ALPN: {}", e)))?;
 
     let ssl_connector = ssl_builder.build();
-    let profile_active = config.resolved_profile().is_some();
+    let profile_active = config.resolved_profile().tls.browser_extensions;
     let mut ssl_conf = openssl::ssl::Ssl::new(ssl_connector.context())
         .map_err(|e| ClientError::tls(format!("SSL conf failed: {}", e)))?;
 
@@ -1690,11 +1694,15 @@ fn build_request(
     //
     // A caller's own header still wins over the profile's, since naming one
     // explicitly means it.
-    if let Some(p) = config
-        .resolved_profile()
-        .filter(|_| !bisect_disabled("headers"))
-    {
-        for (name, value) in p.headers {
+    // A profile with no headers of its own means the client's own minimal
+    // defaults, which is what `compatibility` and `modern` both want.
+    let header_profile = if bisect_disabled("headers") {
+        crate::profile::default_profile()
+    } else {
+        config.resolved_profile()
+    };
+    if !header_profile.headers.is_empty() {
+        for (name, value) in header_profile.headers {
             let already_set = custom.iter().any(|(k, _)| k.eq_ignore_ascii_case(name));
             if !already_set {
                 builder = builder.header(*name, *value);
diff --git a/src/config.rs b/src/config.rs
index d3f243b..4d597a0 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -96,13 +96,29 @@ impl RequestConfig {
         }
     }
 
-    /// The browser profile this request should imitate, if it named one.
+    /// The connection profile this request should use.
     ///
-    /// An unknown name resolves to `None` rather than erroring, so a profile
-    /// that has not been written yet degrades to ordinary behaviour instead of
-    /// failing the request.
-    pub fn resolved_profile(&self) -> Option<&'static crate::profile::BrowserProfile> {
-        self.profile.as_deref().and_then(crate::profile::by_name)
+    /// Infallible, because every request has a profile now: naming none means
+    /// the default rather than "no profile", and the code that applies one no
+    /// longer has an absent case to branch on. An unrecognised name falls back
+    /// to the default here; `validate_profile` is what turns it into an error,
+    /// so a typo is caught once at the edge rather than silently changing
+    /// behaviour deep in the connector.
+    pub fn resolved_profile(&self) -> &'static crate::profile::ConnectionProfile {
+        match self.profile.as_deref() {
+            Some(name) => {
+                crate::profile::by_name(name).unwrap_or_else(|_| crate::profile::default_profile())
+            }
+            None => crate::profile::default_profile(),
+        }
+    }
+
+    /// Reject an unknown profile name.
+    pub fn validate_profile(&self) -> Result<(), String> {
+        match self.profile.as_deref() {
+            Some(name) => crate::profile::by_name(name).map(|_| ()),
+            None => Ok(()),
+        }
     }
 
     pub fn method(&self) -> &str {
diff --git a/src/profile.rs b/src/profile.rs
index e428a42..0d3c61b 100644
--- a/src/profile.rs
+++ b/src/profile.rs
@@ -1,15 +1,33 @@
-//! Browser profiles: what a real browser puts on the wire.
+//! Connection profiles: what we put on the wire, and what we claim to be.
 //!
-//! A profile is data, not code, because browsers ship every few weeks and a
-//! profile that has to be rewritten each time will not be kept current.
+//! A profile is data, not code, because browsers ship every few weeks and one
+//! that has to be rewritten each time will not be kept current.
 //!
-//! The rule that shapes all of this: a profile has to be applied whole or not
-//! at all. Sending browser headers over a non-browser handshake is worse than
-//! sending honest ones, because a client claiming to be Chrome whose first
-//! packet says otherwise has stated something checkably false, and that is a
-//! stronger signal than merely being unfamiliar. Measured: `udemy.com` returns
-//! 200 to an honest curl and 403 with `cf-mitigated: challenge` to the same
-//! client wearing only a Chrome User-Agent.
+//! Profiles differ on two independent axes, which is why a single ladder from
+//! "safe" to "aggressive" does not describe them:
+//!
+//!   breadth  narrow current ciphers  <->  everything, SSLv3 upward
+//!   claim    no identity claim       <->  claiming to be a browser
+//!
+//! | profile | breadth | claim | for |
+//! |---|---|---|---|
+//! | [`COMPATIBILITY`] | everything | honest | ancient servers, TLS enumeration |
+//! | [`MODERN`] | narrow | honest | ordinary traffic |
+//! | [`CHROME_131`] | Chrome's list | Chrome | hosts that reward imitation |
+//!
+//! Two measured findings shape this.
+//!
+//! A profile must be applied whole or not at all. Browser headers over a
+//! non-browser handshake are worse than honest ones: `udemy.com` returns 200
+//! to an honest curl and 403 with `cf-mitigated: challenge` to the same client
+//! wearing only a Chrome User-Agent.
+//!
+//! And claiming to be a browser is not free. The User-Agent is not what gets
+//! scored, it selects which scoring applies: claim a browser and the client is
+//! checked against the claim. On `priceline.com`, plain blasthttp gets through
+//! 3/3 and our Chrome profile 0/3, while curl_cffi, whose imitation survives
+//! the check, gets 3/3. So `MODERN` exists to be unremarkable without
+//! inviting that inspection.
 
 /// The TLS half of a browser profile.
 ///
@@ -20,20 +38,34 @@
 /// suites at OpenSSL's defaults in OpenSSL's order.
 #[derive(Debug, Clone, Copy)]
 pub struct TlsProfile {
-    /// TLS 1.3 suites, in order, colon separated (`SSL_CTX_set_ciphersuites`).
-    pub ciphersuites: &'static str,
+    /// TLS 1.3 suites, in order, colon separated
+    /// (`SSL_CTX_set_ciphersuites`). `None` leaves OpenSSL's own list and
+    /// order, which is what a profile wanting to look like an ordinary
+    /// OpenSSL client should do.
+    pub ciphersuites: Option<&'static str>,
     /// TLS 1.2 and below, in order, colon separated (`SSL_CTX_set_cipher_list`).
     pub cipher_list: &'static str,
     /// Signature algorithms, in order. JA4 hashes this list as sent rather
-    /// than sorted, so the order is part of the fingerprint.
-    pub sigalgs: &'static str,
-    /// Supported groups, in order. Note these do not affect JA4: only the
-    /// presence of the extension is counted, not its contents. They are still
-    /// visible to anything reading the raw hello.
-    pub groups: &'static str,
-    pub min_version: &'static str,
-    pub max_version: &'static str,
+    /// than sorted, so the order is part of the fingerprint. `None` leaves
+    /// OpenSSL's.
+    pub sigalgs: Option<&'static str>,
+    /// Supported groups, in order. These do not affect JA4, which counts only
+    /// the presence of the extension and not its contents, but they are
+    /// visible to anything reading the raw hello. `None` leaves OpenSSL's.
+    pub groups: Option<&'static str>,
+    /// `None` leaves the floor and ceiling entirely to OpenSSL, which at
+    /// security level 0 with NO_SSLV3 cleared means SSLv3 through TLS 1.3.
+    pub min_version: Option<&'static str>,
+    pub max_version: Option<&'static str>,
     pub alpn: &'static [&'static str],
+    /// Send the extensions a browser sends and suppress the two it does not.
+    ///
+    /// An explicit field rather than "is a profile set", which is how this
+    /// worked before there was more than one profile. Left implicit, naming
+    /// today's behaviour as a profile would have switched certificate-status
+    /// requests and SCT on, and padding and encrypt_then_mac off, for the very
+    /// configuration that is supposed to be byte-identical to what shipped.
+    pub browser_extensions: bool,
 }
 
 /// The HTTP/2 half of a profile.
@@ -69,14 +101,23 @@ pub struct Http2Profile {
 
 /// A complete profile: TLS, plus the HTTP layers that have to agree with it.
 #[derive(Debug, Clone, Copy)]
-pub struct BrowserProfile {
+pub struct ConnectionProfile {
     pub name: &'static str,
     pub tls: TlsProfile,
-    pub http2: Http2Profile,
-    /// Default request headers in the order the browser sends them. HTTP/1.1
-    /// header order is preserved by this client, so this is the order that
-    /// goes on the wire.
+    /// `None` leaves hyper-util's HTTP/2 defaults, which match no browser but
+    /// are the right answer for a profile not pretending to be one.
+    pub http2: Option<Http2Profile>,
+    /// Default request headers, in the order they go on the wire. HTTP/1.1
+    /// header order is preserved by this client, so this ordering is real.
+    /// Empty means the client's own minimal defaults.
     pub headers: &'static [(&'static str, &'static str)],
+    /// Whether this profile claims to be a browser.
+    ///
+    /// The axis the ladder moves along on a policy block, and the reason it is
+    /// a field rather than something inferred from the headers: what matters
+    /// is whether we invited a browser-shaped inspection, not which browser we
+    /// named.
+    pub claims_browser: bool,
 }
 
 /// Chrome 131 on Windows.
@@ -90,11 +131,13 @@ pub struct BrowserProfile {
 /// this is not a stale profile in the way the version number suggests; only
 /// the signature algorithms moved. Both Chrome 131 and 124 produce
 /// `t13d1516h2_8daaf6152771_02713d6af862`.
-pub const CHROME_131: BrowserProfile = BrowserProfile {
+pub const CHROME_131: ConnectionProfile = ConnectionProfile {
     name: "chrome131",
     tls: TlsProfile {
         // 1301, 1302, 1303.
-        ciphersuites: "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256",
+        ciphersuites: Some(
+            "TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256",
+        ),
         // c02b, c02f, c02c, c030, cca9, cca8, c013, c014, 009c, 009d, 002f, 0035.
         cipher_list: "ECDHE-ECDSA-AES128-GCM-SHA256:\
                       ECDHE-RSA-AES128-GCM-SHA256:\
@@ -109,30 +152,33 @@ pub const CHROME_131: BrowserProfile = BrowserProfile {
                       AES128-SHA:\
                       AES256-SHA",
         // 0403, 0804, 0401, 0503, 0805, 0501, 0806, 0601, in that order.
-        sigalgs: "ECDSA+SHA256:RSA-PSS+SHA256:RSA+SHA256:\
-                  ECDSA+SHA384:RSA-PSS+SHA384:RSA+SHA384:\
-                  RSA-PSS+SHA512:RSA+SHA512",
+        sigalgs: Some(
+            "ECDSA+SHA256:RSA-PSS+SHA256:RSA+SHA256:\
+             ECDSA+SHA384:RSA-PSS+SHA384:RSA+SHA384:\
+             RSA-PSS+SHA512:RSA+SHA512",
+        ),
         // Chrome also offers X25519MLKEM768 (4588) ahead of these, which
         // OpenSSL gained in 3.5 and ours cannot send. It does not affect JA4,
         // but it is visible in the raw hello, and a client claiming to be a
         // current Chrome while offering no post-quantum share is itself worth
         // noticing.
-        groups: "X25519:P-256:P-384",
-        min_version: "1.2",
-        max_version: "1.3",
+        groups: Some("X25519:P-256:P-384"),
+        min_version: Some("1.2"),
+        max_version: Some("1.3"),
         alpn: &["h2", "http/1.1"],
+        browser_extensions: true,
     },
     // Chrome's Akamai fingerprint is
     // 1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p. ENABLE_PUSH is
     // already 0 because hyper always disables push.
-    http2: Http2Profile {
+    http2: Some(Http2Profile {
         initial_stream_window: 6_291_456,
         // 15 MiB. The WINDOW_UPDATE that results is 15728640 - 65535 =
         // 15663105, which is what Chrome sends.
         initial_connection_window: 15_728_640,
         max_header_list_size: 262_144,
         max_frame_size: None,
-    },
+    }),
     headers: &[
         (
             "sec-ch-ua",
@@ -160,46 +206,226 @@ pub const CHROME_131: BrowserProfile = BrowserProfile {
         ("accept-language", "en-US,en;q=0.9"),
         ("priority", "u=0, i"),
     ],
+    claims_browser: true,
+};
+
+/// Everything, SSLv3 upward, with an honest User-Agent.
+///
+/// A faithful transcription of what blasthttp sent before profiles existed,
+/// and it has to stay that way: `tests/legacy_default.rs` and
+/// `test_compat_path_keeps_the_scsv` are its specification. Every `None` here
+/// is deliberate, meaning "leave OpenSSL alone", because that is what the
+/// unnamed default did.
+///
+/// Honest but conspicuous. The cipher list resolves to 105 suites, and the
+/// resulting hello has never been observed in roughly 17 billion connections
+/// recorded by tlsfingerprint.io. That is the cost of reaching a server from
+/// 2003, and it is why it is no longer the thing you get by accident.
+pub const COMPATIBILITY: ConnectionProfile = ConnectionProfile {
+    name: "compatibility",
+    tls: TlsProfile {
+        ciphersuites: None,
+        // Every suite the build provides except the eNULL ones. Includes the
+        // aNULL suites, which skip authentication: worth reaching for a
+        // scanner, and this client already does not verify certificates by
+        // default, so they give up nothing that was being enforced.
+        cipher_list: "ALL",
+        sigalgs: None,
+        groups: None,
+        min_version: None,
+        max_version: None,
+        alpn: &["h2", "http/1.1"],
+        browser_extensions: false,
+    },
+    http2: None,
+    headers: &[],
+    claims_browser: false,
+};
+
+/// Narrow current ciphers, no identity claim.
+///
+/// Sits between the other two: `COMPATIBILITY` is honest but offers 105 suites
+/// and every protocol version back to SSLv3; `CHROME_131` looks ordinary but
+/// makes a claim it cannot fully back. This offers a current, boring cipher
+/// list and claims nothing.
+///
+/// What it buys, measured:
+///
+///   JA4 goes from `t13d10512h2` to `t13d1113h2`. Eleven suites instead of a
+///   hundred and five, which is a shape plenty of software has, where 105 is
+///   a shape essentially nothing has.
+///
+///   SSLv3, TLS 1.0 and TLS 1.1 stop being offered. A client advertising
+///   SSLv3 in 2026 is conspicuous on its own.
+///
+/// What it does NOT buy, also measured, and worth recording because it was the
+/// original argument for this profile: it is not "unremarkable in real
+/// traffic". Checked against tlsfingerprint.io, which indexes roughly 17
+/// billion observed connections, all three of our profiles come back never
+/// seen, `MODERN` included. Narrowing the cipher list fixes the JA4 and does
+/// nothing for the rest of the hello, which stays distinctive for reasons
+/// OpenSSL gives us no way to change: no GREASE anywhere, and extensions in
+/// OpenSSL's fixed order rather than a browser's shuffled one.
+///
+/// So the case for it rests on the JA4 and the version floor, not on blending
+/// in. Blending in is not available on this TLS stack.
+///
+/// It cannot reach a server that speaks only deprecated ciphers. That is what
+/// falling back to `COMPATIBILITY` is for.
+pub const MODERN: ConnectionProfile = ConnectionProfile {
+    name: "modern",
+    tls: TlsProfile {
+        ciphersuites: Some(
+            "TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256",
+        ),
+        // ECDHE with AEAD, which is what current guidance and every modern
+        // library converge on. Twelve suites rather than a hundred and five.
+        cipher_list: "ECDHE-ECDSA-AES128-GCM-SHA256:\
+                      ECDHE-RSA-AES128-GCM-SHA256:\
+                      ECDHE-ECDSA-AES256-GCM-SHA384:\
+                      ECDHE-RSA-AES256-GCM-SHA384:\
+                      ECDHE-ECDSA-CHACHA20-POLY1305:\
+                      ECDHE-RSA-CHACHA20-POLY1305:\
+                      DHE-RSA-AES128-GCM-SHA256:\
+                      DHE-RSA-AES256-GCM-SHA384",
+        sigalgs: None,
+        groups: Some("X25519:P-256:P-384:P-521"),
+        // No SSLv3, TLS 1.0 or 1.1. Offering them is one of the louder things
+        // the old default did.
+        min_version: Some("1.2"),
+        max_version: Some("1.3"),
+        alpn: &["h2", "http/1.1"],
+        browser_extensions: false,
+    },
+    http2: None,
+    headers: &[],
+    claims_browser: false,
 };
 
 /// Look a profile up by name.
-pub fn by_name(name: &str) -> Option<&'static BrowserProfile> {
+///
+/// Unknown names are an error rather than silently resolving to the default.
+/// They used to: a typo bought you whatever the default was, with nothing said
+/// about it, which is a poor trade once the choice of profile decides whether
+/// a request is inspected.
+pub fn by_name(name: &str) -> Result<&'static ConnectionProfile, String> {
     match name.to_ascii_lowercase().as_str() {
-        // "chrome" tracks the newest profile we can actually reproduce, which
+        "compatibility" | "compat" => Ok(&COMPATIBILITY),
+        "modern" => Ok(&MODERN),
+        // "chrome" tracks the newest Chrome we can actually reproduce, which
         // is what a caller asking for "chrome" wants.
-        "chrome" | "chrome131" => Some(&CHROME_131),
-        _ => None,
+        "chrome" | "chrome131" | "browser" => Ok(&CHROME_131),
+        other => Err(format!(
+            "unknown profile '{}' (known: compatibility, modern, chrome)",
+            other
+        )),
     }
 }
 
+/// The profile a request gets when it does not name one.
+pub fn default_profile() -> &'static ConnectionProfile {
+    &COMPATIBILITY
+}
+
 #[cfg(test)]
+// These assertions are deliberately about constants: the profiles are data,
+// and the point of the tests is that editing that data fails loudly. Clippy
+// is right that they fold at compile time and wrong that this makes them
+// pointless.
+#[allow(clippy::assertions_on_constants)]
 mod tests {
     use super::*;
 
     #[test]
-    fn test_chrome_alias_resolves() {
+    fn test_lookup_resolves_and_rejects() {
         assert_eq!(by_name("chrome").unwrap().name, "chrome131");
         assert_eq!(by_name("CHROME131").unwrap().name, "chrome131");
-        assert!(by_name("netscape").is_none());
+        assert_eq!(by_name("modern").unwrap().name, "modern");
+        assert_eq!(by_name("compat").unwrap().name, "compatibility");
+
+        // An unknown name is an error, not a silent fallback. It used to be a
+        // fallback, which meant a typo bought you the default with nothing
+        // said about it.
+        let err = by_name("netscape").unwrap_err();
+        assert!(
+            err.contains("netscape"),
+            "error should name the input: {err}"
+        );
+        assert!(
+            err.contains("compatibility"),
+            "error should list the options"
+        );
+    }
+
+    #[test]
+    fn test_compatibility_leaves_openssl_alone() {
+        // This profile has to stay byte-identical to what shipped before
+        // profiles existed, and every `None` is how that is expressed.
+        // `tests/legacy_default.rs` is the end-to-end proof; this is the
+        // cheap guard that catches an accidental edit.
+        let c = COMPATIBILITY.tls;
+        assert_eq!(c.cipher_list, "ALL");
+        assert!(c.ciphersuites.is_none(), "would reorder the TLS 1.3 suites");
+        assert!(c.sigalgs.is_none());
+        assert!(c.groups.is_none());
+        assert!(
+            c.min_version.is_none() && c.max_version.is_none(),
+            "a version floor would put SSLv3 and TLS 1.0 out of reach"
+        );
+        assert!(
+            !c.browser_extensions,
+            "browser extensions would add SCT and status_request and drop \
+             padding and encrypt_then_mac, none of which the old default did"
+        );
+        assert!(
+            COMPATIBILITY.http2.is_none(),
+            "would change the SETTINGS frame"
+        );
+        assert!(COMPATIBILITY.headers.is_empty());
+        assert!(!COMPATIBILITY.claims_browser);
     }
 
     #[test]
-    fn test_cipher_counts_match_the_target() {
-        // 12 TLS 1.2 suites plus 3 TLS 1.3 suites is the 15 that JA4 counts
-        // for Chrome. Getting this wrong moves the JA4 header, so it is worth
-        // asserting next to the data rather than only end to end.
-        let p = CHROME_131.tls;
-        assert_eq!(p.cipher_list.split(':').count(), 12);
-        assert_eq!(p.ciphersuites.split(':').count(), 3);
-        assert_eq!(p.sigalgs.split(':').count(), 8);
+    fn test_modern_is_narrow_and_honest() {
+        let m = MODERN.tls;
+        // The point of this profile: a short list instead of 105 suites.
+        assert_eq!(m.cipher_list.split(':').count(), 8);
+        assert_eq!(m.ciphersuites.unwrap().split(':').count(), 3);
+        assert_eq!(m.min_version, Some("1.2"), "no SSLv3, TLS 1.0 or 1.1");
+        assert!(
+            !m.browser_extensions && !MODERN.claims_browser && MODERN.headers.is_empty(),
+            "modern must not claim to be a browser; that is the whole \
+             difference between it and chrome131"
+        );
     }
 
     #[test]
-    fn test_headers_are_in_browser_order() {
-        // Order is part of the fingerprint, so guard the two ends of it.
+    fn test_chrome_matches_the_target_shape() {
+        // 12 TLS 1.2 suites plus 3 TLS 1.3 suites is the 15 JA4 counts for
+        // Chrome. Getting this wrong moves the JA4 header.
+        let c = CHROME_131.tls;
+        assert_eq!(c.cipher_list.split(':').count(), 12);
+        assert_eq!(c.ciphersuites.unwrap().split(':').count(), 3);
+        assert_eq!(c.sigalgs.unwrap().split(':').count(), 8);
+        assert!(c.browser_extensions);
+        assert!(CHROME_131.claims_browser);
+        assert!(CHROME_131.http2.is_some());
+    }
+
+    #[test]
+    fn test_chrome_headers_are_in_browser_order() {
+        // Order is part of the fingerprint, so guard both ends of it.
         let names: Vec<&str> = CHROME_131.headers.iter().map(|(n, _)| *n).collect();
         assert_eq!(names.first(), Some(&"sec-ch-ua"));
         assert_eq!(names.last(), Some(&"priority"));
         assert!(names.contains(&"sec-fetch-dest"));
     }
+
+    #[test]
+    fn test_only_chrome_claims_to_be_a_browser() {
+        // The axis the ladder moves along on a policy block.
+        assert!(!COMPATIBILITY.claims_browser);
+        assert!(!MODERN.claims_browser);
+        assert!(CHROME_131.claims_browser);
+    }
 }

From 6ceb57fca67baafe0149a434512a8d1b1749483b Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 15:50:37 -0400
Subject: [PATCH 13/44] Add the profile ladder

Phase 4. A hop that fails now retries with a different profile, per hop
rather than per request, because a redirect can land on a differently
protected host.

Two axes with separate triggers, which is why this is a decision function
and not a chain. A handshake failure that suggests a wider offer widens
to compatibility; a refusal changes what we claim to be. Conflating them
would answer a certificate rejection by offering weaker crypto.

Which direction a refusal moves in is empirical, not a rule keyed on
vendor. The evidence points both ways: Akamai, Kasada and Cloudflare
deployments reward a browser claim, PerimeterX punishes it on every
target tested, and two deployments of the same product routinely
disagree. So try the other side once and let the result decide.

Three things the tests and the wire caught.

next_rung cycled. modern refused moves to chrome131, chrome131 refused
moves back to modern, forever. The termination test was written before
the function and failed immediately. Fixed by passing what has already
been tried, so termination is a property of the function rather than of
the cap at the call site. MAX_RUNGS stays as a belt to that braces.

The ladder fired on any 403. Most 403s are ordinary authorization
failures, not bot blocks, and retrying each one with different TLS
settings doubles the request count of any scan that touches one. It now
moves only on positive evidence that a product intervened, meaning a
named vendor. The cost is missing a product we cannot fingerprint; the
alternative cost is much larger and falls on every scan.

And one such miss, found immediately: wizzair.com answers 405 with
`x-amzn-waf-action: captcha`, and the classifier only knew `challenge`,
so a genuine interception read as an ordinary refusal. AWS WAF is now a
vendor the classifier knows and the header's presence is the signal
rather than one of its values.

A profile is remembered per host only after it has actually got through,
so a scan does not re-walk the ladder on every request. Only on success:
a failure could be the host having a bad minute, and recording those
would let one flake pin a host to a worse profile for the rest of a run.
A pinned config, meaning an explicit profile, cipher string or TLS
version, never moves at all.

On the wire, with the default still compatibility:

  akamai.com    403 -> 200   walks compatibility, modern, chrome131
  wizzair.com   405 -> 302   after the AWS WAF fix
  priceline.com 200         unchanged; never claims a browser, so never
  zillow.com    200         walks into the PerimeterX trap
---
 src/antibot.rs            |  34 ++++++
 src/client/hyper.rs       | 239 +++++++++++++++++++++++++++++++++-----
 src/config.rs             |  26 +++++
 src/profile.rs            | 163 ++++++++++++++++++++++++++
 tests/antibot_classify.rs |  31 +++++
 5 files changed, 464 insertions(+), 29 deletions(-)

diff --git a/src/antibot.rs b/src/antibot.rs
index 35bd27b..d5e664f 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -30,6 +30,7 @@ pub enum Vendor {
     Imperva,
     F5,
     Kasada,
+    AwsWaf,
     Unknown,
 }
 
@@ -43,6 +44,7 @@ impl fmt::Display for Vendor {
             Vendor::Imperva => "imperva",
             Vendor::F5 => "f5",
             Vendor::Kasada => "kasada",
+            Vendor::AwsWaf => "awswaf",
             Vendor::Unknown => "unknown",
         };
         f.write_str(s)
@@ -72,6 +74,27 @@ impl Outcome {
         matches!(self, Outcome::Ok | Outcome::Present(_))
     }
 
+    /// Is there positive evidence that a protection product intervened?
+    ///
+    /// The distinction between this and "we did not get through" matters, and
+    /// getting it wrong costs real requests. Most 403s are ordinary
+    /// authorization failures: an endpoint needing a token, a path that is not
+    /// public, a method that is not allowed. Those classify as
+    /// `Blocked(Unknown)` because they are refusals with no product signature,
+    /// and retrying them with different TLS settings is pure waste against a
+    /// server that will say 403 however it is asked.
+    ///
+    /// So a named vendor is the bar. It means something recognisable answered
+    /// instead of the origin, which is the only case where trying a different
+    /// profile is worth a second request.
+    ///
+    /// The cost of that conservatism is missing a product we cannot
+    /// fingerprint. The cost of the alternative is doubling the request count
+    /// of every scan that touches a 403, which is most of them.
+    pub fn indicates_protection(&self) -> bool {
+        !matches!(self.vendor(), None | Some(Vendor::Unknown))
+    }
+
     pub fn vendor(&self) -> Option<Vendor> {
         match self {
             Outcome::Present(v) | Outcome::Challenge(v) | Outcome::Blocked(v) => Some(*v),
@@ -189,6 +212,17 @@ pub fn classify(facts: &ResponseFacts) -> Outcome {
         return Outcome::Challenge(Vendor::Imperva);
     }
 
+    // ── AWS WAF ───────────────────────────────────────────────────
+    // The header carries the action, and there is more than one: `challenge`
+    // for the silent JavaScript kind, `captcha` for the kind wanting a person.
+    // Matching only `challenge` missed wizzair.com, which answers 405 with
+    // `captcha`, so the ladder read a genuine interception as an ordinary
+    // refusal and declined to try anything else. Match the header's presence
+    // and let the value be detail.
+    if facts.has_header("x-amzn-waf-action") {
+        return Outcome::Challenge(Vendor::AwsWaf);
+    }
+
     // ── Kasada ────────────────────────────────────────────────────
     // Kasada answers 429 rather than 403 for a refusal.
     if facts.has_header("x-kpsdk-ct") || facts.body_contains("KPSDK") {
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 69d84a4..c3829b4 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1,6 +1,7 @@
 use super::{ClientError, HttpClient, TlsFailure};
 use crate::config::RequestConfig;
 use crate::debug::{DebugLog, debug_record, new_debug_log};
+use crate::profile::RungFailure;
 use crate::response::{CertInfo, RedirectHop, Response};
 
 use http_body_util::BodyExt;
@@ -935,6 +936,10 @@ pub struct HyperClient {
     // both a ForwardProxy client (for HTTP targets) and a Tunnel client (for
     // HTTPS targets), so we cache per-mode rather than a single client.
     cached: Mutex<std::collections::HashMap<ConnMode, CachedClient>>,
+    // Which profile last got through to each host, so a scan does not re-walk
+    // the ladder on every request. Written only on success; see
+    // `remember_profile`.
+    host_profiles: Mutex<HashMap<String, &'static crate::profile::ConnectionProfile>>,
     // Why the last handshake to each host failed.
     //
     // Deliberately on the client rather than on `CachedClient`: a retry with
@@ -954,6 +959,7 @@ impl HyperClient {
     pub fn new() -> Self {
         HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
+            host_profiles: Mutex::new(HashMap::new()),
             tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
@@ -2037,6 +2043,122 @@ impl HttpClient for HyperClient {
 }
 
 impl HyperClient {
+    /// One attempt at one hop with one profile.
+    ///
+    /// Everything that used to sit inline in the redirect loop, extracted so
+    /// the ladder can call it more than once. The `?` operators that used to
+    /// short-circuit out of `send_inner` now surface here as an `Err` the
+    /// ladder can decide about, which is the substance of the change rather
+    /// than the extraction itself.
+    async fn attempt_hop(
+        &self,
+        config: &RequestConfig,
+        uri: &http::Uri,
+        log: &DebugLog,
+        hop_cookies: Option<&str>,
+    ) -> Result<(SingleResponse, Option<CachedClient>), ClientError> {
+        // Decide the connection mode for the *current* target host on every
+        // hop, not just the first. A redirect can send the request to a
+        // different host, and the proxy / no_proxy decision has to follow it.
+        // Freezing the first hop's choice would otherwise let a request that
+        // started direct keep connecting directly after a redirect onto a
+        // proxied host (leaking traffic past the proxy), and let a request
+        // that started proxied keep using the proxy after a redirect onto a
+        // no_proxy host. Clients are cached by mode, so hops that share a mode
+        // reuse the same client.
+        let mode = Self::conn_mode(config, uri)?;
+
+        // Forward proxy: dispatch directly via TCP + http1::SendRequest
+        // (bypasses hyper Client's URI normalization to preserve absolute-form)
+        let proxy_url_for_fwd = if let ConnMode::ForwardProxy(ref url) = mode {
+            Some(url.clone())
+        } else {
+            None
+        };
+
+        let cached = if proxy_url_for_fwd.is_some() {
+            None
+        } else {
+            Some(self.get_or_build(config, &mode)?)
+        };
+
+        let resp = if let Some(ref proxy_url) = proxy_url_for_fwd {
+            dispatch_forward_proxy(proxy_url, uri, config, log, hop_cookies).await?
+        } else {
+            dispatch_request(
+                &cached.as_ref().unwrap().inner,
+                uri,
+                config,
+                log,
+                hop_cookies,
+                &self.tls_failures,
+            )
+            .await?
+        };
+
+        Ok((resp, cached))
+    }
+
+    /// Which profile this hop starts on.
+    ///
+    /// A pinned config starts and ends on what the caller asked for. Otherwise
+    /// a host we have succeeded against before starts on whatever worked,
+    /// which is the whole point of remembering: the ladder costs a wasted
+    /// request every time it walks, and a scan hits the same host repeatedly.
+    fn starting_profile(
+        &self,
+        config: &RequestConfig,
+        uri: &http::Uri,
+    ) -> &'static crate::profile::ConnectionProfile {
+        if config.tls_is_pinned() {
+            return config.resolved_profile();
+        }
+        if let Some(key) = peer_slot_key(uri)
+            && let Ok(map) = self.host_profiles.lock()
+            && let Some(p) = map.get(&key)
+        {
+            return p;
+        }
+        config.resolved_profile()
+    }
+
+    /// The next rung, or `None` to stop.
+    ///
+    /// Returns `None` immediately for a pinned config: naming a cipher string,
+    /// a TLS version or a profile means the caller meant it, and quietly
+    /// substituting another profile would put something on the wire they did
+    /// not ask for.
+    fn next_rung_for(
+        &self,
+        config: &RequestConfig,
+        current: &crate::profile::ConnectionProfile,
+        failure: RungFailure,
+        tried: &[&'static str],
+    ) -> Option<&'static crate::profile::ConnectionProfile> {
+        if config.tls_is_pinned() || tried.len() >= crate::profile::MAX_RUNGS {
+            return None;
+        }
+        crate::profile::next_rung(current, failure, tried)
+    }
+
+    /// Record that a profile worked against a host.
+    ///
+    /// Only ever called after an attempt got through. A failure alone is a
+    /// hypothesis and could be the host having a bad minute; a success is
+    /// evidence. Recording failures would let one flake pin a host to a worse
+    /// profile for the rest of a scan.
+    fn remember_profile(
+        &self,
+        uri: &http::Uri,
+        profile: &'static crate::profile::ConnectionProfile,
+    ) {
+        if let Some(key) = peer_slot_key(uri)
+            && let Ok(mut map) = self.host_profiles.lock()
+        {
+            map.insert(key, profile);
+        }
+    }
+
     async fn send_inner(
         &self,
         config: &RequestConfig,
@@ -2134,36 +2256,95 @@ impl HyperClient {
             // that started proxied keep using the proxy after a redirect onto a
             // no_proxy host. Clients are cached by mode, so hops that share a
             // mode reuse the same client.
-            let mode = Self::conn_mode(config, &uri)?;
-
-            // Forward proxy: dispatch directly via TCP + http1::SendRequest
-            // (bypasses hyper Client's URI normalization to preserve absolute-form)
-            let is_forward_proxy = matches!(&mode, ConnMode::ForwardProxy(_));
-            let proxy_url_for_fwd = if let ConnMode::ForwardProxy(ref url) = mode {
-                Some(url.clone())
-            } else {
-                None
-            };
-
-            // For non-forward-proxy modes, get the cached hyper Client.
-            let cached = if is_forward_proxy {
-                None
-            } else {
-                Some(self.get_or_build(config, &mode)?)
-            };
+            // Walk the profile ladder for this hop.
+            //
+            // Per hop rather than per request, because a redirect can land on
+            // a differently protected host. Inside `send_inner` rather than in
+            // `send`, so it does not spend the caller's retry budget: those are
+            // different questions, one being "did this flake" and the other
+            // "is this the wrong way to talk to this host".
+            let mut rung = self.starting_profile(config, &uri);
+            let mut tried: Vec<&'static str> = Vec::new();
+            let (resp, cached) = loop {
+                tried.push(rung.name);
+                let rung_config = config.with_profile(rung.name);
+
+                match self
+                    .attempt_hop(&rung_config, &uri, log, hop_cookies.as_deref())
+                    .await
+                {
+                    Ok((resp, cached)) => {
+                        let outcome = crate::antibot::classify(&crate::antibot::ResponseFacts {
+                            status: resp.status,
+                            headers: &resp.headers,
+                            body: &String::from_utf8_lossy(&resp.body_bytes),
+                        });
+
+                        // Only walk the ladder when something recognisable
+                        // refused us. A bare 403 with no product signature is
+                        // an ordinary refusal, and retrying it differently
+                        // just spends another request to be told the same
+                        // thing.
+                        if outcome.got_through() || !outcome.indicates_protection() {
+                            // Remember what worked, but only on a real
+                            // success. A failure is a hypothesis; this is
+                            // evidence.
+                            if outcome.got_through() {
+                                self.remember_profile(&uri, rung);
+                            }
+                            break (resp, cached);
+                        }
 
-            let resp = if let Some(ref proxy_url) = proxy_url_for_fwd {
-                dispatch_forward_proxy(proxy_url, &uri, config, log, hop_cookies.as_deref()).await?
-            } else {
-                dispatch_request(
-                    &cached.as_ref().unwrap().inner,
-                    &uri,
-                    config,
-                    log,
-                    hop_cookies.as_deref(),
-                    &self.tls_failures,
-                )
-                .await?
+                        match self.next_rung_for(config, rung, RungFailure::Refused, &tried) {
+                            Some(next) => {
+                                debug_record(
+                                    log,
+                                    v,
+                                    1,
+                                    &format!(
+                                        "   {} refused under '{}', trying '{}'",
+                                        resp.status, rung.name, next.name
+                                    ),
+                                );
+                                rung = next;
+                            }
+                            // Out of rungs: the refusal is the answer. Hand it
+                            // back as a response rather than an error, so the
+                            // caller can read the status, the headers and
+                            // whatever the challenge page said.
+                            None => break (resp, cached),
+                        }
+                    }
+                    Err(e) => {
+                        let widens = e
+                            .tls_failure
+                            .as_ref()
+                            .is_some_and(|f| f.suggests_wider_offer());
+                        let next = if widens {
+                            self.next_rung_for(config, rung, RungFailure::CouldNotNegotiate, &tried)
+                        } else {
+                            None
+                        };
+                        match next {
+                            Some(next) => {
+                                debug_record(
+                                    log,
+                                    v,
+                                    1,
+                                    &format!(
+                                        "   handshake failed under '{}', widening to '{}'",
+                                        rung.name, next.name
+                                    ),
+                                );
+                                rung = next;
+                            }
+                            // Nothing else to try, or the failure says nothing
+                            // about our offer. Report the error we actually
+                            // got.
+                            None => return Err(e),
+                        }
+                    }
+                }
             };
             let hop_ms = start.elapsed().as_millis();
             debug_record(log, v, 1, &format!("<- {} ({}ms)", resp.status, hop_ms));
diff --git a/src/config.rs b/src/config.rs
index 4d597a0..c6a9d92 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -113,6 +113,32 @@ impl RequestConfig {
         }
     }
 
+    /// This config with a different profile selected.
+    ///
+    /// Used by the ladder to retry a hop differently. A clone rather than a
+    /// parameter threaded through the connector because `TlsKey`, and so the
+    /// cached-client lookup, is derived from the config: changing the profile
+    /// anywhere else would key the cache on one profile while building the
+    /// connector from another.
+    pub fn with_profile(&self, name: &str) -> RequestConfig {
+        let mut c = self.clone();
+        c.profile = Some(name.to_string());
+        c
+    }
+
+    /// Whether the caller pinned TLS settings themselves.
+    ///
+    /// When they did, the ladder must not move: naming a cipher string, a TLS
+    /// version or a profile means they meant it, and silently substituting
+    /// another profile would send something they did not ask for.
+    /// `tests/tls_integration.rs` depends on this.
+    pub fn tls_is_pinned(&self) -> bool {
+        self.profile.is_some()
+            || self.cipher_string.is_some()
+            || self.min_tls_version.is_some()
+            || self.max_tls_version.is_some()
+    }
+
     /// Reject an unknown profile name.
     pub fn validate_profile(&self) -> Result<(), String> {
         match self.profile.as_deref() {
diff --git a/src/profile.rs b/src/profile.rs
index 0d3c61b..897ae24 100644
--- a/src/profile.rs
+++ b/src/profile.rs
@@ -429,3 +429,166 @@ mod tests {
         assert!(CHROME_131.claims_browser);
     }
 }
+
+// ── The ladder ────────────────────────────────────────────────────
+
+/// Why an attempt did not produce a usable response.
+///
+/// The two variants move along different axes and must not be conflated. A
+/// transport failure says the peer could not negotiate with us, which is about
+/// breadth. A policy block says it negotiated fine and then refused, which is
+/// about what we claimed to be.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub enum RungFailure {
+    /// The handshake failed in a way suggesting a wider offer might work.
+    /// Retrying is free: no application data was sent.
+    CouldNotNegotiate,
+    /// A clean handshake, then a refusal or a challenge.
+    Refused,
+}
+
+/// Which profile to try next, if any.
+///
+/// Deliberately a decision rather than a fixed list, because the two axes have
+/// different triggers and a linear chain cannot express that.
+///
+/// On a negotiation failure, widen: `modern` and `chrome131` both offer narrow
+/// modern ciphers, so the answer is `compatibility`, which offers everything
+/// back to SSLv3. From `compatibility` there is nowhere wider to go.
+///
+/// On a refusal, change what we claim. Empirically rather than by rule: the
+/// evidence points both ways depending on the host, so try the other side once
+/// and let the result decide. Akamai, Kasada and Cloudflare deployments have
+/// rewarded the browser claim; PerimeterX has punished it, on every target
+/// tested. Nothing here inspects the vendor, because two deployments of one
+/// product routinely disagree.
+/// `tried` is every profile already attempted for this hop, and it is not
+/// optional bookkeeping. Without it the honest and browser profiles oscillate:
+/// a refusal moves `modern` to `chrome131`, and a refusal there moves it back
+/// to `modern`, forever. Termination is a property of this function rather
+/// than of the cap at the call site.
+pub fn next_rung(
+    current: &ConnectionProfile,
+    failure: RungFailure,
+    tried: &[&str],
+) -> Option<&'static ConnectionProfile> {
+    let candidate = match failure {
+        RungFailure::CouldNotNegotiate => {
+            if current.name == COMPATIBILITY.name {
+                // Already offering everything; the same bytes would go out
+                // again and waste a handshake on a server that just refused
+                // them.
+                None
+            } else {
+                Some(&COMPATIBILITY)
+            }
+        }
+        RungFailure::Refused => {
+            if current.claims_browser {
+                // Drop the claim. A client that says it is Chrome gets checked
+                // against that; one that says nothing does not.
+                Some(&MODERN)
+            } else if current.name == COMPATIBILITY.name {
+                // Both honest and conspicuous, so there are two things to
+                // change. Narrowing is the smaller step and leaves the browser
+                // claim available afterwards.
+                Some(&MODERN)
+            } else {
+                Some(&CHROME_131)
+            }
+        }
+    }?;
+
+    if tried.contains(&candidate.name) {
+        None
+    } else {
+        Some(candidate)
+    }
+}
+
+/// How many attempts one hop may make.
+///
+/// Three is the longest sensible walk: compatibility narrows to modern, modern
+/// claims a browser, and that is the end of it. `next_rung` already terminates
+/// on its own by refusing to revisit; this is the belt to that braces, because
+/// the cost of being wrong is an unbounded request loop against someone else's
+/// server.
+pub const MAX_RUNGS: usize = 3;
+
+#[cfg(test)]
+mod ladder_tests {
+    use super::*;
+
+    #[test]
+    fn test_negotiation_failure_widens_to_compatibility() {
+        assert_eq!(
+            next_rung(&MODERN, RungFailure::CouldNotNegotiate, &["modern"]).map(|p| p.name),
+            Some("compatibility")
+        );
+        assert_eq!(
+            next_rung(&CHROME_131, RungFailure::CouldNotNegotiate, &["chrome131"]).map(|p| p.name),
+            Some("compatibility")
+        );
+    }
+
+    #[test]
+    fn test_nowhere_wider_than_compatibility() {
+        // Offering everything already; a second attempt would send the same
+        // bytes and waste a handshake on a server that just refused them.
+        assert!(
+            next_rung(
+                &COMPATIBILITY,
+                RungFailure::CouldNotNegotiate,
+                &["compatibility"]
+            )
+            .is_none()
+        );
+    }
+
+    #[test]
+    fn test_refusal_drops_a_browser_claim() {
+        // The PerimeterX case: claiming Chrome invited an inspection we
+        // failed, so stop claiming it.
+        let next = next_rung(&CHROME_131, RungFailure::Refused, &["chrome131"]).unwrap();
+        assert_eq!(next.name, "modern");
+        assert!(!next.claims_browser);
+    }
+
+    #[test]
+    fn test_refusal_from_an_honest_profile_tries_the_claim() {
+        // The Akamai case: honest gets refused, a browser claim gets through.
+        assert_eq!(
+            next_rung(&MODERN, RungFailure::Refused, &["modern"]).map(|p| p.name),
+            Some("chrome131")
+        );
+    }
+
+    #[test]
+    fn test_the_walk_terminates_within_the_cap() {
+        // Guards the cap against a future edit to next_rung that cycles.
+        for start in [&COMPATIBILITY, &MODERN, &CHROME_131] {
+            for failure in [RungFailure::CouldNotNegotiate, RungFailure::Refused] {
+                let mut seen = vec![start.name];
+                let mut current = start;
+                while let Some(next) = next_rung(current, failure, &seen) {
+                    assert!(
+                        !seen.contains(&next.name),
+                        "ladder revisited {} starting from {} on {:?}",
+                        next.name,
+                        start.name,
+                        failure
+                    );
+                    seen.push(next.name);
+                    current = next;
+                    assert!(
+                        seen.len() <= MAX_RUNGS,
+                        "walk from {} on {:?} exceeded MAX_RUNGS: {:?}",
+                        start.name,
+                        failure,
+                        seen
+                    );
+                }
+            }
+        }
+    }
+}
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 6ac0a9b..8d315e0 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -257,3 +257,34 @@ fn test_product_cookie_with_refusal_still_counts_as_blocked() {
     assert_eq!(outcome, Outcome::Blocked(Vendor::Cloudflare));
     assert!(!outcome.got_through());
 }
+
+#[test]
+fn test_aws_waf_captcha_is_a_challenge() {
+    // wizzair.com answers 405 with `captcha` rather than `challenge`. A rule
+    // matching only the latter classified this as an ordinary refusal, which
+    // meant the profile ladder saw no reason to try anything else against a
+    // host that a browser profile does in fact reach.
+    let h = hdrs(&[
+        ("server", "CloudFront"),
+        ("x-amzn-waf-action", "captcha"),
+        ("x-cache", "Error from cloudfront"),
+    ]);
+    let outcome = classify(&facts(405, &h, ""));
+    assert_eq!(outcome, Outcome::Challenge(Vendor::AwsWaf));
+    assert!(
+        outcome.indicates_protection(),
+        "the ladder only moves on positive evidence of a product"
+    );
+}
+
+#[test]
+fn test_a_bare_403_is_not_evidence_of_protection() {
+    // The counterpart, and the reason `indicates_protection` exists at all.
+    // Most 403s are ordinary authorization failures, and retrying each one
+    // with different TLS settings would double the request count of any scan
+    // that touches one.
+    let h = hdrs(&[("server", "nginx")]);
+    let outcome = classify(&facts(403, &h, "forbidden"));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+    assert!(!outcome.indicates_protection());
+}

From bf1bad9439eec9957d06d9d970682bfe875539c4 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 16:02:50 -0400
Subject: [PATCH 14/44] Report what was tried, and ship 1.0

Phase 5, and the breaking change the rest was building toward.

A request that names no profile now sends `modern`: 11 cipher suites,
TLS 1.2 and 1.3. It used to send 105 suites and every version back to
SSLv3, to every host on the internet, so that the small number needing
that stayed reachable. That is now paid for only where it is wanted: a
server too old for `modern` is still reached, by the ladder widening to
`compatibility` on the second handshake.

Response carries what the ladder tried and what it amounts to:

  attempts    [(profile, outcome, status)] in order
  protection  (outcome, vendor) from the classifier
  conclusion  reached | blocked | needs_browser | needs_legacy_tls |
              unreachable

needs_browser is the one worth acting on. It means a JavaScript
challenge, which no HTTP client passes, so retrying with different
settings cannot help and the URL should go to a real browser. Blocked is
the opposite: a different approach might work. A consumer should not have
to tell those apart by grepping our response body for cf-mitigated.

TransportError carries the same verdict plus kind, tls_failure and
retryable, so a failure that never produced a response is just as legible
as one that did.

Verified from the 1.0 wheel:

  akamai.com    200  reached                   modern refused, chrome131 reached
  priceline.com 200  reached                   modern reached
  cf challenge  403  needs_browser cloudflare  modern and chrome131 challenged
  google.com    200  reached                   modern reached

Two things the default change turned up.

The test server accepted one connection then exited, which was fine while
a failed handshake ended the story. It does not any more: the ladder
retries, so every legacy-cipher test needs two connections, and with a
single-shot server the second gets ECONNREFUSED and the test fails for a
reason unrelated to what it tests. It now serves until shutdown.

And asking for SSLv3 without naming a profile failed outright with "no
ciphers available", because the default offers nothing that exists in
SSLv3, and naming a version pins the configuration, which is exactly what
stops the ladder rescuing it. A pre-TLS1.2 floor now selects
`compatibility` on its own.

The ten legacy_default.rs tests pass unchanged and now mean something
stronger: they assert RC4, 3DES, SEED, Camellia, anonymous DH and SSLv3
are reachable through the ladder rather than on the first hello.
fingerprint_baseline.rs is re-pinned, with compatibility's old values
kept under their own test so the transcription stays honest.
---
 CHANGELOG.md                  |  48 +++++++++
 Cargo.lock                    |   2 +-
 Cargo.toml                    |   2 +-
 README.md                     |   9 +-
 pyproject.toml                |   2 +-
 src/client/hyper.rs           |  30 +++++-
 src/client/mock.rs            |   4 +
 src/client/mod.rs             |  30 ++++++
 src/config.rs                 |  42 +++++++-
 src/lib.rs                    |   1 +
 src/mock.rs                   |   1 +
 src/profile.rs                |  12 ++-
 src/python.rs                 |  64 ++++++++++++
 src/report.rs                 | 184 ++++++++++++++++++++++++++++++++++
 src/response.rs               |  21 +++-
 tests/fingerprint_baseline.rs |  80 ++++++++++-----
 tests/support/tls_server.rs   |  48 +++++----
 17 files changed, 519 insertions(+), 61 deletions(-)
 create mode 100644 src/report.rs

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d408874..5911c76 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,5 +1,53 @@
 # Changelog
 
+## 1.0.0
+
+Breaking: a request that names no profile no longer offers 105 cipher suites
+and every protocol version back to SSLv3. It sends `modern` instead, which is
+11 suites and TLS 1.2 or 1.3. A server too old for that is still reached, by
+the ladder falling back to `compatibility` on the second handshake.
+
+- **Connection profiles.** Three of them. `compatibility` is a faithful
+  transcription of what 0.10 sent by default, kept byte-identical and verified
+  by the existing tests. `modern` is new. `chrome131` imitates Chrome's TLS,
+  HTTP/2 settings and headers together. Select one with `profile=`; an unknown
+  name is now an error rather than silently falling back
+- **Automatic shifting between them.** A handshake failure that suggests the
+  peer could not negotiate widens the offer; a refusal from a recognised
+  protection product changes what the client claims to be. Per redirect hop,
+  since a redirect can land on a differently protected host, and without
+  spending the caller's retry budget, which is a different question. A profile
+  that gets through is remembered per host so a scan does not re-walk the
+  ladder on every request. Naming a `profile`, `cipher_string`,
+  `min_tls_version` or `max_tls_version` pins the configuration and the ladder
+  does not move
+- The ladder only moves on positive evidence that a product intervened, not on
+  any refusal. Most 403s are ordinary authorization failures, and retrying each
+  one with different TLS settings would double the request count of any scan
+  that touches one
+- **`Response.protection`** reports what a bot-management product did:
+  `(outcome, vendor)` where outcome is ok, present, challenge, blocked or error
+- **`Response.conclusion`** reduces that to one verdict: reached, blocked,
+  needs_browser, needs_legacy_tls or unreachable. `needs_browser` means a
+  JavaScript challenge, which no HTTP client passes, so the URL should go to a
+  real browser rather than be retried
+- **`Response.attempts`** lists what the ladder tried, in order
+- **TLS failures now say why.** The reason used to be destroyed in transit:
+  hyper-util's error displays as the literal "client error (Connect)" for every
+  connect-time failure, so a cipher mismatch, a rejected certificate, a DNS
+  failure and a refused connection were indistinguishable. Failures now raise
+  `TransportError` carrying `kind`, `tls_failure`, `retryable`, `conclusion`
+  and `attempts`. `TransportError` subclasses `RuntimeError`, so existing
+  handlers keep working
+- Fixes a consequence of that: TLS failures were being retried, because they
+  were misclassified as connection errors, contradicting the rule that says
+  they should not be
+- Asking for a pre-TLS1.2 version without naming a profile now selects
+  `compatibility`, rather than failing with "no ciphers available" because the
+  default offers nothing that exists in SSLv3
+- `BLASTHTTP_BISECT=tls,headers,http2` disables profile layers independently,
+  for finding out which one a detector is reacting to
+
 ## 0.10.1
 
 - Legacy ciphers are offered by default, which is what the custom OpenSSL build has always been for. A server speaking only RC4, RC4-MD5, 3DES, SEED, Camellia or anonymous DH is now reachable without passing `cipher_string` yourself. Previously `SslConnector::builder` installed its own list (`DEFAULT:!aNULL:!eNULL:!MD5:!3DES:!DES:!RC4:!IDEA:!SEED:...`) and nothing replaced it, so those suites never reached the wire and such a server was unreachable at any setting. `set_security_level(0)` did not help: the security level governs how weak a negotiated cipher may be, not which ones are offered
diff --git a/Cargo.lock b/Cargo.lock
index 0ac2e7f..1f36bdc 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -99,7 +99,7 @@ checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
 
 [[package]]
 name = "blasthttp"
-version = "0.10.0"
+version = "1.0.0"
 dependencies = [
  "brotli",
  "bytes",
diff --git a/Cargo.toml b/Cargo.toml
index ca98715..861c18f 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "blasthttp"
-version = "0.10.0"
+version = "1.0.0"
 edition = "2024"
 description = "Offensive-first HTTP library with Python bindings"
 license = "GPL-3.0"
diff --git a/README.md b/README.md
index b55ca1e..190fadb 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,8 @@ cargo add blasthttp
 - **Batch connection reuse** — connections are pooled and reused within a batch, dramatically reducing overhead when scanning many URLs on the same hosts
 - **Rust performance** — async I/O, zero-copy where possible, and native concurrency give significant speed improvements over pure Python HTTP clients
 - **SSL cert info on every request** — extracts CN, SANs, emails, issuer, validity dates, and fingerprint during the TLS handshake that's already happening, eliminating the need for a separate connection to gather this information
-- **Legacy TLS offered by default**: custom-compiled OpenSSL 3.3.2 with the legacy provider baked in, so a server that speaks only RC4, 3DES, SEED, Camellia or anonymous DH is reachable without naming the cipher yourself. SSLv3, TLS 1.0 and TLS 1.1 are offered too. Null-encryption suites are the one exception, available through `cipher_string` but never offered by default
+- **Connection profiles, and automatic shifting between them**: `modern` by default (11 cipher suites, TLS 1.2+), `compatibility` for servers that speak only RC4, 3DES, SEED, Camellia, anonymous DH or SSLv3, and `chrome` to imitate a browser's TLS, HTTP/2 and headers together. A handshake the peer cannot negotiate widens the offer; a refusal from a recognised protection product changes what the client claims to be. What works is remembered per host
+- **Says why it failed**: a blocked request reports which product blocked it, and a JavaScript challenge reports `needs_browser`, because no HTTP client passes that tier and retrying will not change it
 - **No cert validation by default** — offensive-first: connects to self-signed, expired, and misconfigured TLS without extra config
 - **HTTP/2 support** — automatic via ALPN negotiation, falls back to HTTP/1.1
 - **Low-level primitives** — `RawConnection` for byte-level TCP/TLS I/O (bypasses HTTP framing entirely) and `blasthttp.h2` for manual H2 frame construction (includes a permissive HPACK encoder that emits bytes a strict encoder refuses, the building block for H2 smuggling / CRLF-injection tooling)
@@ -115,7 +116,8 @@ Output is JSON (one object per response), including status, headers, redirect ch
 | `--verify` | Enable TLS cert validation | off |
 | `-x, --proxy` | HTTP/SOCKS proxy URL | |
 | `--no-proxy` | Host that bypasses the proxy: `host`, `*.suffix`, IP, CIDR, or `*` (repeatable) | |
-| `--ciphers` | OpenSSL cipher string | `ALL` (every suite except null-encryption) |
+| `--profile` | Connection profile: `modern`, `compatibility`, `chrome` | `modern` |
+| `--ciphers` | OpenSSL cipher string | from the profile |
 | `--min-tls` | Minimum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
 | `--max-tls` | Maximum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
 | `-v, --verbose` | Verbose output (-vv includes body) | |
@@ -239,7 +241,8 @@ All parameters except `url` are optional:
 | `verify_certs` | `bool` | Enable TLS cert validation (default `False`) |
 | `proxy` | `str` | HTTP/SOCKS proxy URL |
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
-| `cipher_string` | `str` | OpenSSL cipher string (default `ALL`: every suite the build provides except null-encryption) |
+| `profile` | `str` | Connection profile: `modern` (default), `compatibility`, `chrome` |
+| `cipher_string` | `str` | OpenSSL cipher string. Overrides the profile's, and pins it so the ladder will not move |
 | `min_tls_version` | `str` | Minimum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `max_tls_version` | `str` | Maximum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `retries` | `int` | Number of retries on failure |
diff --git a/pyproject.toml b/pyproject.toml
index 18d6b4e..ca0fd92 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "blasthttp"
-version = "0.10.0"
+version = "1.0.0"
 description = "Offensive-first HTTP library"
 license = "GPL-3.0"
 requires-python = ">=3.10"
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index c3829b4..51bdfd6 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -2213,6 +2213,10 @@ impl HyperClient {
                 status: resp.status,
                 headers: resp.headers,
                 body_bytes: resp.body_bytes,
+                // This path never ladders: it is the resolve_ip /
+                // request_target branch, which dispatches once and follows no
+                // redirects.
+                attempts: Vec::new(),
                 elapsed_ms,
                 redirect_chain,
                 cert_info,
@@ -2265,6 +2269,7 @@ impl HyperClient {
             // "is this the wrong way to talk to this host".
             let mut rung = self.starting_profile(config, &uri);
             let mut tried: Vec<&'static str> = Vec::new();
+            let mut attempts: Vec<crate::report::Attempt> = Vec::new();
             let (resp, cached) = loop {
                 tried.push(rung.name);
                 let rung_config = config.with_profile(rung.name);
@@ -2285,6 +2290,12 @@ impl HyperClient {
                         // an ordinary refusal, and retrying it differently
                         // just spends another request to be told the same
                         // thing.
+                        attempts.push(crate::report::Attempt::from_outcome(
+                            rung.name,
+                            resp.status,
+                            &outcome,
+                        ));
+
                         if outcome.got_through() || !outcome.indicates_protection() {
                             // Remember what worked, but only on a real
                             // success. A failure is a hypothesis; this is
@@ -2315,7 +2326,14 @@ impl HyperClient {
                             None => break (resp, cached),
                         }
                     }
-                    Err(e) => {
+                    Err(mut e) => {
+                        attempts.push(crate::report::Attempt::new(
+                            rung.name,
+                            crate::report::AttemptOutcome::HandshakeFailed {
+                                reason: e.message.clone(),
+                            },
+                        ));
+
                         let widens = e
                             .tls_failure
                             .as_ref()
@@ -2340,8 +2358,12 @@ impl HyperClient {
                             }
                             // Nothing else to try, or the failure says nothing
                             // about our offer. Report the error we actually
-                            // got.
-                            None => return Err(e),
+                            // got, carrying what was tried so the caller can
+                            // see the ladder ran and where it stopped.
+                            None => {
+                                e.attempts = attempts;
+                                return Err(e);
+                            }
                         }
                     }
                 }
@@ -2460,6 +2482,8 @@ impl HyperClient {
                 status: resp.status,
                 headers: resp.headers,
                 body_bytes: resp.body_bytes,
+                // What the ladder tried on this, the final, hop.
+                attempts,
                 elapsed_ms,
                 redirect_chain,
                 cert_info,
diff --git a/src/client/mock.rs b/src/client/mock.rs
index 4916224..8ea18e9 100644
--- a/src/client/mock.rs
+++ b/src/client/mock.rs
@@ -76,6 +76,7 @@ impl HttpClient for ErrorKindMockClient {
             message: self.message.clone(),
             kind: self.kind.clone(),
             tls_failure: None,
+            attempts: Vec::new(),
         })
     }
 }
@@ -119,6 +120,7 @@ impl HttpClient for SequenceMockClient {
                 message: format!("mock error on call {}", call),
                 kind: self.fail_kind.clone(),
                 tls_failure: None,
+                attempts: Vec::new(),
             });
         }
 
@@ -128,6 +130,7 @@ impl HttpClient for SequenceMockClient {
             status: self.success_status,
             headers: Vec::new(),
             body_bytes,
+            attempts: Vec::new(),
             elapsed_ms: 0,
             redirect_chain: Vec::new(),
             cert_info: None,
@@ -180,6 +183,7 @@ impl HttpClient for MockClient {
             status: self.status,
             headers: self.headers.clone(),
             body_bytes,
+            attempts: Vec::new(),
             elapsed_ms: 0,
             redirect_chain: Vec::new(),
             cert_info: None,
diff --git a/src/client/mod.rs b/src/client/mod.rs
index 7695324..5a6e683 100644
--- a/src/client/mod.rs
+++ b/src/client/mod.rs
@@ -186,6 +186,12 @@ pub struct ClientError {
     /// Structured detail when the failure was a TLS handshake, so callers can
     /// branch on why rather than substring-matching `message`.
     pub tls_failure: Option<TlsFailure>,
+    /// What the profile ladder tried before giving up.
+    ///
+    /// Empty when nothing laddered, which is every error raised before a hop
+    /// begins. More than one entry means the client already tried the obvious
+    /// alternatives, so the caller need not.
+    pub attempts: Vec<crate::report::Attempt>,
 }
 
 impl ClientError {
@@ -194,6 +200,7 @@ impl ClientError {
             message,
             kind: ErrorKind::Connection,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -202,6 +209,7 @@ impl ClientError {
             message,
             kind: ErrorKind::Timeout,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -210,6 +218,7 @@ impl ClientError {
             message,
             kind: ErrorKind::Tls,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -218,6 +227,7 @@ impl ClientError {
             message,
             kind: ErrorKind::InvalidUrl,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -226,6 +236,7 @@ impl ClientError {
             message,
             kind: ErrorKind::TooManyRedirects,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -234,6 +245,7 @@ impl ClientError {
             message,
             kind: ErrorKind::Status(status),
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 
@@ -245,6 +257,23 @@ impl ClientError {
             message,
             kind: ErrorKind::Tls,
             tls_failure: Some(failure),
+            attempts: Vec::new(),
+        }
+    }
+
+    /// What this failure amounts to for a caller deciding what to do next.
+    ///
+    /// A peer that could not negotiate with us is reported separately from one
+    /// we never reached: the first is alive and wants cryptography we declined
+    /// or cannot do, which is actionable, and the second is not.
+    pub fn conclusion(&self) -> crate::report::Conclusion {
+        match &self.tls_failure {
+            Some(f) if f.suggests_wider_offer() => crate::report::Conclusion::NeedsLegacyTls {
+                reason: self.message.clone(),
+            },
+            _ => crate::report::Conclusion::Unreachable {
+                reason: self.message.clone(),
+            },
         }
     }
 
@@ -253,6 +282,7 @@ impl ClientError {
             message,
             kind: ErrorKind::Other,
             tls_failure: None,
+            attempts: Vec::new(),
         }
     }
 }
diff --git a/src/config.rs b/src/config.rs
index c6a9d92..6275890 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -105,12 +105,44 @@ impl RequestConfig {
     /// so a typo is caught once at the edge rather than silently changing
     /// behaviour deep in the connector.
     pub fn resolved_profile(&self) -> &'static crate::profile::ConnectionProfile {
-        match self.profile.as_deref() {
-            Some(name) => {
-                crate::profile::by_name(name).unwrap_or_else(|_| crate::profile::default_profile())
-            }
-            None => crate::profile::default_profile(),
+        if let Some(name) = self.profile.as_deref() {
+            return crate::profile::by_name(name)
+                .unwrap_or_else(|_| crate::profile::default_profile());
+        }
+        // Asking for a pre-TLS1.2 floor is asking for the legacy profile,
+        // whether or not the caller said so. The default offers only modern
+        // ciphers, none of which exist in SSLv3 or TLS 1.0, so honouring the
+        // version alone produces "no ciphers available": a request that names
+        // a protocol, and is told that protocol is impossible.
+        //
+        // The ladder cannot rescue this, because naming a version pins the
+        // configuration and pinning is what stops the ladder moving.
+        if self.wants_legacy_protocol() {
+            return crate::profile::by_name("compatibility")
+                .unwrap_or_else(|_| crate::profile::default_profile());
         }
+        crate::profile::default_profile()
+    }
+
+    /// Did the caller ask to speak something older than TLS 1.2?
+    fn wants_legacy_protocol(&self) -> bool {
+        let is_legacy = |v: &str| {
+            matches!(
+                v.to_ascii_lowercase().as_str(),
+                "3.0"
+                    | "ssl3"
+                    | "sslv3"
+                    | "ssl3.0"
+                    | "1.0"
+                    | "tls1.0"
+                    | "tlsv1.0"
+                    | "1.1"
+                    | "tls1.1"
+                    | "tlsv1.1"
+            )
+        };
+        self.min_tls_version.as_deref().is_some_and(is_legacy)
+            || self.max_tls_version.as_deref().is_some_and(is_legacy)
     }
 
     /// This config with a different profile selected.
diff --git a/src/lib.rs b/src/lib.rs
index d2549dc..733c6c8 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -6,6 +6,7 @@ pub mod cookies;
 pub mod debug;
 pub mod h2;
 pub mod profile;
+pub mod report;
 pub mod response;
 
 #[cfg(feature = "python")]
diff --git a/src/mock.rs b/src/mock.rs
index 1552c27..42aef54 100644
--- a/src/mock.rs
+++ b/src/mock.rs
@@ -276,6 +276,7 @@ fn make_response(
         status: status_code,
         headers,
         body_bytes: body,
+        attempts: Vec::new(),
         elapsed_ms: 0,
         redirect_chain: Vec::new(),
         cert_info: None,
diff --git a/src/profile.rs b/src/profile.rs
index 897ae24..5542141 100644
--- a/src/profile.rs
+++ b/src/profile.rs
@@ -323,8 +323,18 @@ pub fn by_name(name: &str) -> Result<&'static ConnectionProfile, String> {
 }
 
 /// The profile a request gets when it does not name one.
+///
+/// `MODERN` rather than `COMPATIBILITY`, which is the breaking change in 1.0.
+/// A request that meets a server too old for it does not fail: the ladder
+/// widens to `COMPATIBILITY` on a negotiation failure, at the cost of one
+/// extra handshake against such a server and nothing at all against the rest.
+///
+/// The old default offered 105 cipher suites and every protocol version back
+/// to SSLv3 to every host on the internet, to stay reachable for the small
+/// number that need it. This inverts that: be ordinary by default, and pay
+/// for breadth only where breadth is actually wanted.
 pub fn default_profile() -> &'static ConnectionProfile {
-    &COMPATIBILITY
+    &MODERN
 }
 
 #[cfg(test)]
diff --git a/src/python.rs b/src/python.rs
index ae1e54f..8f83161 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -406,6 +406,19 @@ pyo3::create_exception!(
 ///                  unsupported_protocol, alert, certificate_verify, reset,
 ///                  timeout, other. `None` otherwise.
 ///   `retryable`    whether trying the same request again might help.
+/// Render a `Conclusion` as the `(name, vendor)` pair Python sees.
+fn conclusion_to_py(c: &crate::report::Conclusion) -> (String, String) {
+    use crate::report::Conclusion;
+    let (name, vendor) = match c {
+        Conclusion::Reached => ("reached", None),
+        Conclusion::Blocked { vendor } => ("blocked", vendor.clone()),
+        Conclusion::NeedsBrowser { vendor } => ("needs_browser", vendor.clone()),
+        Conclusion::NeedsLegacyTls { .. } => ("needs_legacy_tls", None),
+        Conclusion::Unreachable { .. } => ("unreachable", None),
+    };
+    (name.to_string(), vendor.unwrap_or_else(|| "-".to_string()))
+}
+
 fn client_error_to_py(e: ClientError) -> PyErr {
     let kind = match &e.kind {
         ErrorKind::Connection => "connection",
@@ -427,12 +440,22 @@ fn client_error_to_py(e: ClientError) -> PyErr {
     });
     let retryable = e.kind.is_retryable();
 
+    let (conclusion, conclusion_vendor) = conclusion_to_py(&e.conclusion());
+    let attempts: Vec<(String, String)> = e
+        .attempts
+        .iter()
+        .map(|a| (a.profile.clone(), format!("{:?}", a.outcome)))
+        .collect();
+
     let err = TransportError::new_err(e.message);
     Python::attach(|py| {
         let v = err.value(py);
         let _ = v.setattr("kind", kind);
         let _ = v.setattr("tls_failure", tls_failure);
         let _ = v.setattr("retryable", retryable);
+        let _ = v.setattr("conclusion", conclusion);
+        let _ = v.setattr("conclusion_vendor", conclusion_vendor);
+        let _ = v.setattr("attempts", attempts);
     });
     err
 }
@@ -525,6 +548,9 @@ impl PyResponse {
             status,
             headers: headers.unwrap_or_default(),
             body_bytes,
+            // A response built from Python was not produced by the ladder, so
+            // it has nothing to report about what was tried.
+            attempts: Vec::new(),
             elapsed_ms,
             redirect_chain: redirect_chain
                 .map(|hops| hops.into_iter().map(|h| h.inner.clone()).collect())
@@ -722,6 +748,44 @@ impl PyResponse {
     /// with different settings; a challenge will not, because answering it
     /// means running the page's JavaScript. It is the signal to hand the URL
     /// to a real browser rather than retry.
+    /// What this request amounts to, as a single verdict a caller can branch
+    /// on. Returns `(conclusion, vendor)`:
+    ///
+    ///   reached          got the page
+    ///   blocked          a product refused us
+    ///   needs_browser    a JavaScript challenge; hand this URL to a real
+    ///                    browser, because no HTTP client passes this tier
+    ///   needs_legacy_tls the peer wants cryptography we declined or cannot do
+    ///   unreachable      never got a response
+    ///
+    /// `needs_browser` is the one worth acting on. It means retrying with
+    /// different settings cannot help, which is the opposite of `blocked`.
+    #[getter]
+    fn conclusion(&self) -> (String, String) {
+        conclusion_to_py(&self.inner.conclusion())
+    }
+
+    /// What the profile ladder tried, in order, as `(profile, outcome, status)`
+    /// tuples. Usually one entry; more than one means the first approach did
+    /// not work and the client moved.
+    #[getter]
+    fn attempts(&self) -> Vec<(String, String, Option<u16>)> {
+        use crate::report::AttemptOutcome;
+        self.inner
+            .attempts
+            .iter()
+            .map(|a| {
+                let (kind, status) = match &a.outcome {
+                    AttemptOutcome::Reached { status } => ("reached", Some(*status)),
+                    AttemptOutcome::Refused { status, .. } => ("refused", Some(*status)),
+                    AttemptOutcome::Challenged { status, .. } => ("challenged", Some(*status)),
+                    AttemptOutcome::HandshakeFailed { .. } => ("handshake_failed", None),
+                };
+                (a.profile.clone(), kind.to_string(), status)
+            })
+            .collect()
+    }
+
     #[getter]
     fn protection(&self) -> (String, String) {
         let outcome = self.inner.protection();
diff --git a/src/report.rs b/src/report.rs
new file mode 100644
index 0000000..bb7ed42
--- /dev/null
+++ b/src/report.rs
@@ -0,0 +1,184 @@
+//! What was tried, and what it means.
+//!
+//! A request that walks the profile ladder makes several attempts, and the
+//! caller sees only the last one. That is fine when it worked and useless when
+//! it did not: "403" says nothing about whether a different approach exists,
+//! whether one was already tried, or whether anything would have helped.
+//!
+//! The consumer this is for is BBOT, which has to tell four outcomes apart
+//! that all look like "no page":
+//!
+//!   - the host is dead
+//!   - the host needs legacy TLS we declined to offer
+//!   - a product blocked us, and here is which
+//!   - a product served a JavaScript challenge, so no HTTP client will ever
+//!     get through and the URL should go to a real browser
+//!
+//! Only the last of those justifies launching a browser, and it must be a
+//! field rather than something inferred by grepping our response body for
+//! `cf-mitigated`.
+
+use serde::{Deserialize, Serialize};
+
+use crate::antibot::{Outcome, Vendor};
+
+/// What one rung of the ladder produced.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub enum AttemptOutcome {
+    /// Got a response the classifier was happy with.
+    Reached { status: u16 },
+    /// A clean handshake, then a refusal.
+    Refused {
+        status: u16,
+        #[serde(skip_serializing_if = "Option::is_none")]
+        vendor: Option<String>,
+    },
+    /// A challenge: answering it needs JavaScript.
+    Challenged {
+        status: u16,
+        #[serde(skip_serializing_if = "Option::is_none")]
+        vendor: Option<String>,
+    },
+    /// Never got as far as a response.
+    HandshakeFailed { reason: String },
+}
+
+/// One rung: which profile, and what happened under it.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub struct Attempt {
+    pub profile: String,
+    pub outcome: AttemptOutcome,
+}
+
+impl Attempt {
+    pub fn new(profile: &str, outcome: AttemptOutcome) -> Self {
+        Attempt {
+            profile: profile.to_string(),
+            outcome,
+        }
+    }
+
+    /// Build from a classifier verdict.
+    pub fn from_outcome(profile: &str, status: u16, outcome: &Outcome) -> Self {
+        let vendor = outcome
+            .vendor()
+            .filter(|v| !matches!(v, Vendor::Unknown))
+            .map(|v| v.to_string());
+        let o = match outcome {
+            Outcome::Ok | Outcome::Present(_) => AttemptOutcome::Reached { status },
+            Outcome::Challenge(_) => AttemptOutcome::Challenged { status, vendor },
+            Outcome::Blocked(_) | Outcome::Error => AttemptOutcome::Refused { status, vendor },
+        };
+        Attempt::new(profile, o)
+    }
+}
+
+/// What the whole request amounts to, once every rung has been tried.
+#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
+pub enum Conclusion {
+    /// Got the page.
+    Reached,
+    /// A product refused us and nothing we can change is likely to help.
+    Blocked {
+        #[serde(skip_serializing_if = "Option::is_none")]
+        vendor: Option<String>,
+    },
+    /// A JavaScript challenge. **The signal to hand this URL to a real
+    /// browser.** No HTTP client passes this tier, curl_cffi included, and
+    /// retrying with different connection settings will not change it.
+    NeedsBrowser {
+        #[serde(skip_serializing_if = "Option::is_none")]
+        vendor: Option<String>,
+    },
+    /// The peer could not negotiate with us and we had nothing wider to offer.
+    /// Distinct from `Unreachable`: the host is alive and talking, it just
+    /// wants cryptography we declined or cannot do.
+    NeedsLegacyTls { reason: String },
+    /// Never got a response at all.
+    Unreachable { reason: String },
+}
+
+impl Conclusion {
+    /// Should the caller hand this to a browser?
+    ///
+    /// The single question BBOT needs answered, exposed as a predicate so it
+    /// does not have to match on the variant.
+    pub fn needs_browser(&self) -> bool {
+        matches!(self, Conclusion::NeedsBrowser { .. })
+    }
+
+    /// Work out the conclusion from the last outcome and what was tried.
+    pub fn from_attempts(last: &Outcome, attempts: &[Attempt]) -> Self {
+        let vendor = last
+            .vendor()
+            .filter(|v| !matches!(v, Vendor::Unknown))
+            .map(|v| v.to_string());
+        match last {
+            Outcome::Ok | Outcome::Present(_) => Conclusion::Reached,
+            Outcome::Challenge(_) => Conclusion::NeedsBrowser { vendor },
+            Outcome::Blocked(_) => Conclusion::Blocked { vendor },
+            // A classifier error with attempts behind it means something went
+            // wrong that was not a refusal.
+            Outcome::Error => Conclusion::Unreachable {
+                reason: attempts
+                    .last()
+                    .map(|a| format!("{:?}", a.outcome))
+                    .unwrap_or_else(|| "no response".to_string()),
+            },
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_a_challenge_asks_for_a_browser_and_a_block_does_not() {
+        // The distinction the whole escalation path rests on. A block might
+        // yield to different settings; a challenge needs JavaScript and never
+        // will, so only one of these justifies starting a browser.
+        let challenge = Conclusion::from_attempts(&Outcome::Challenge(Vendor::Cloudflare), &[]);
+        assert!(challenge.needs_browser());
+        assert_eq!(
+            challenge,
+            Conclusion::NeedsBrowser {
+                vendor: Some("cloudflare".to_string())
+            }
+        );
+
+        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::Akamai), &[]);
+        assert!(!blocked.needs_browser());
+    }
+
+    #[test]
+    fn test_an_unnamed_vendor_is_left_unnamed() {
+        // Reporting "unknown" as though it were a product would invite a
+        // consumer to branch on it. An absent vendor is the honest answer.
+        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::Unknown), &[]);
+        assert_eq!(blocked, Conclusion::Blocked { vendor: None });
+    }
+
+    #[test]
+    fn test_reaching_it_is_not_an_escalation() {
+        for outcome in [Outcome::Ok, Outcome::Present(Vendor::Akamai)] {
+            let c = Conclusion::from_attempts(&outcome, &[]);
+            assert_eq!(c, Conclusion::Reached);
+            assert!(!c.needs_browser());
+        }
+    }
+
+    #[test]
+    fn test_attempts_record_the_profile_that_produced_them() {
+        // The log is only useful if it says which rung did what.
+        let a = Attempt::from_outcome("modern", 403, &Outcome::Challenge(Vendor::PerimeterX));
+        assert_eq!(a.profile, "modern");
+        assert_eq!(
+            a.outcome,
+            AttemptOutcome::Challenged {
+                status: 403,
+                vendor: Some("perimeterx".to_string()),
+            }
+        );
+    }
+}
diff --git a/src/response.rs b/src/response.rs
index c2a1ed1..5f25b5c 100644
--- a/src/response.rs
+++ b/src/response.rs
@@ -32,6 +32,13 @@ pub struct Response {
     /// Debug messages collected during the request (for Python-side inspection)
     #[serde(skip_serializing)]
     pub debug_log: Vec<String>,
+    /// What the profile ladder tried on the final hop, in order.
+    ///
+    /// Usually one entry. More than one means the first approach did not work
+    /// and the client moved, which is worth knowing even on success: it says
+    /// this host needs something other than the default.
+    #[serde(default)]
+    pub attempts: Vec<crate::report::Attempt>,
     /// Why `body_bytes` is not decoded content, when it isn't.
     ///
     /// `None` on any ordinary response, including one with no
@@ -79,7 +86,7 @@ impl Serialize for Response {
     /// callers that just want lazy memory behavior should access
     /// fields directly instead of serializing.
     fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
-        let mut out = s.serialize_struct("Response", 13)?;
+        let mut out = s.serialize_struct("Response", 14)?;
         out.serialize_field("url", &self.url)?;
         out.serialize_field("status", &self.status)?;
         out.serialize_field("headers", &self.headers)?;
@@ -89,6 +96,7 @@ impl Serialize for Response {
         out.serialize_field("hash", self.hash())?;
         out.serialize_field("elapsed_ms", &self.elapsed_ms)?;
         out.serialize_field("redirect_chain", &self.redirect_chain)?;
+        out.serialize_field("attempts", &self.attempts)?;
         if let Some(ref c) = self.cert_info {
             out.serialize_field("cert_info", c)?;
         } else {
@@ -121,6 +129,7 @@ impl Clone for Response {
             status: self.status,
             headers: self.headers.clone(),
             body_bytes: self.body_bytes.clone(),
+            attempts: self.attempts.clone(),
             elapsed_ms: self.elapsed_ms,
             redirect_chain: self.redirect_chain.clone(),
             cert_info: self.cert_info.clone(),
@@ -157,6 +166,15 @@ impl Response {
     /// succeed with different connection settings; a challenge will not,
     /// because answering it means executing the page's JavaScript. That is the
     /// signal to hand the URL to a real browser instead of retrying.
+    /// What this request amounts to, as a single verdict.
+    ///
+    /// `Conclusion::NeedsBrowser` is the one to branch on: it means a
+    /// JavaScript challenge, which no HTTP client passes, so the URL should go
+    /// to a real browser rather than be retried.
+    pub fn conclusion(&self) -> crate::report::Conclusion {
+        crate::report::Conclusion::from_attempts(&self.protection(), &self.attempts)
+    }
+
     pub fn protection(&self) -> crate::antibot::Outcome {
         crate::antibot::classify(&crate::antibot::ResponseFacts {
             status: self.status,
@@ -485,6 +503,7 @@ mod protection_tests {
                 .map(|(k, v)| (k.to_string(), v.to_string()))
                 .collect(),
             body_bytes: body.as_bytes().to_vec(),
+            attempts: Vec::new(),
             elapsed_ms: 1,
             redirect_chain: Vec::new(),
             cert_info: None,
diff --git a/tests/fingerprint_baseline.rs b/tests/fingerprint_baseline.rs
index 672c45c..de7f3f2 100644
--- a/tests/fingerprint_baseline.rs
+++ b/tests/fingerprint_baseline.rs
@@ -81,32 +81,55 @@ async fn test_default_ja4_is_pinned() {
     let hello = capture_default_hello().await;
     let actual = support::ja4::ja4(&hello);
 
-    // Current default: the broad legacy cipher list, which is 105 suites.
-    // JA4 renders that as `99`, because the count field is two digits wide and
-    // the spec caps it. Worth knowing on its own: past 99 suites, JA4 cannot
-    // tell 105 from 200, so the header alone stops distinguishing us from any
-    // other client with an unreasonable cipher list.
+    // The default is now `modern`: 11 cipher suites, TLS 1.2 and 1.3 only.
     //
-    // The extension count is 11 rather than 12 because this server is
-    // addressed by IP, so no SNI is sent. Against a hostname it reads
-    // t13d9912h2.
+    // 12 extensions rather than 11, because a TLS 1.2 floor trips our OpenSSL
+    // patch into sending renegotiation_info (0xff01) instead of the SCSV. That
+    // is the intended behaviour of the patch and not incidental: a client
+    // whose floor is 1.2 signals the modern way, and one willing to speak
+    // older protocols keeps the SCSV.
     //
-    // Chrome 150 for contrast is t13d1516h2_8daaf6152771_806a8c22fdea: 15
-    // ciphers, 16 extensions. Closing that gap is the point of the stealth
-    // work, and this pin is how we will see it move.
-    const EXPECTED: &str = "t13i9911h2";
+    // Reads `t13i` rather than `t13d` because this server is addressed by IP,
+    // so no SNI is sent. Chrome 131 for contrast is t13d1516h2.
+    const EXPECTED: &str = "t13i1112h2";
 
     let header = actual.split('_').next().unwrap_or_default();
     assert_eq!(
         header,
         EXPECTED,
-        "JA4 header moved.\n  was: {}\n  now: {}\n  raw: {}",
+        "default JA4 header moved.\n  was: {}\n  now: {}\n  raw: {}",
         EXPECTED,
         header,
         support::ja4::ja4_raw(&hello),
     );
 }
 
+#[tokio::test]
+async fn test_compatibility_ja4_is_pinned() {
+    // What the default used to be, and still is when asked for by name. This
+    // pin is `compatibility`'s specification along with legacy_default.rs: if
+    // it moves, the profile has stopped being a faithful transcription of what
+    // shipped before profiles existed.
+    //
+    // 105 suites renders as 99 because the JA4 count field is two digits and
+    // the spec caps it. Worth knowing on its own: past 99, JA4 cannot tell 105
+    // from 200.
+    //
+    // Reads `t13d` and 12 extensions, not `t13i` and 11, because this capture
+    // addresses the server by hostname so SNI is sent. The IP-addressed
+    // equivalent is t13i9911h2.
+    let hello = capture_profile_hello("compatibility").await;
+    let header = support::ja4::ja4(&hello);
+    let header = header.split('_').next().unwrap_or_default();
+
+    assert_eq!(
+        header,
+        "t13d9912h2",
+        "compatibility JA4 moved.\n  raw: {}",
+        support::ja4::ja4_raw(&hello),
+    );
+}
+
 #[tokio::test]
 async fn test_grease_is_absent() {
     // We send no GREASE anywhere, which JA4 ignores but a detector checking
@@ -135,11 +158,12 @@ async fn test_grease_is_absent() {
 }
 
 #[tokio::test]
-async fn test_offers_legacy_protocol_versions() {
-    // The companion to the cipher work: SSLv3 through TLS 1.3 are all offered,
-    // which is what makes an ancient server reachable without configuration.
-    // It is also conspicuous, since a browser offers 1.2 and 1.3 only.
-    let hello = capture_default_hello().await;
+async fn test_compatibility_offers_legacy_protocol_versions() {
+    // SSLv3 through TLS 1.3 all offered, which is what makes an ancient server
+    // reachable. No longer what an unconfigured request sends: the default is
+    // `modern`, and a legacy server is reached by the ladder falling back to
+    // this profile instead.
+    let hello = capture_profile_hello("compatibility").await;
 
     for (version, label) in [
         (0x0300u16, "SSLv3"),
@@ -159,9 +183,13 @@ async fn test_offers_legacy_protocol_versions() {
 
 #[tokio::test]
 async fn test_no_browser_only_extensions_yet() {
-    // The six extensions a JA4 match against Chrome needs, none of which we
-    // send today. Named here so the gap is legible and so each one flips this
-    // test as it lands.
+    // Extensions the browser profile sends and the default does not. Named
+    // here so the gap stays legible.
+    //
+    // renegotiation_info (0xff01) is deliberately absent from this list: the
+    // default now sends it, because `modern` sets a TLS 1.2 floor and the
+    // OpenSSL patch keys on exactly that. It is not a browser-only extension,
+    // it is what any client with a modern floor should send.
     let hello = capture_default_hello().await;
 
     for (ext, name) in [
@@ -170,7 +198,6 @@ async fn test_no_browser_only_extensions_yet() {
         (0x001b, "compress_certificate"),
         (0x44cd, "application_settings (ALPS)"),
         (0xfe0d, "encrypted_client_hello"),
-        (0xff01, "renegotiation_info"),
     ] {
         assert!(
             !hello.extensions.contains(&ext),
@@ -283,10 +310,11 @@ async fn test_chrome_profile_drops_non_browser_extensions() {
 
 #[tokio::test]
 async fn test_compat_path_keeps_the_scsv() {
-    // The patch is conditioned on the TLS floor so the compatibility path is
-    // byte-identical to stock OpenSSL. Without a profile we should still send
-    // the SCSV and still not send renegotiation_info.
-    let hello = capture_default_hello().await;
+    // The OpenSSL patch is conditioned on the TLS floor, so `compatibility`,
+    // which sets no floor, stays byte-identical to stock OpenSSL: the SCSV
+    // goes out and renegotiation_info does not. `modern` is the other side of
+    // that same condition, and test_default_ja4_is_pinned covers it.
+    let hello = capture_profile_hello("compatibility").await;
     assert!(
         hello.ciphers.contains(&0x00ff),
         "compatibility path lost the SCSV, so the patch is too broad"
diff --git a/tests/support/tls_server.rs b/tests/support/tls_server.rs
index 85b3c53..a8fd124 100644
--- a/tests/support/tls_server.rs
+++ b/tests/support/tls_server.rs
@@ -358,14 +358,28 @@ impl TlsTestServer {
                 config.response_body,
             );
 
-            tokio::select! {
-                result = listener.accept() => {
-                    if let Ok((tcp_stream, _)) = result {
-                        // Read the ClientHello off the wire and parse it before
-                        // TLS touches it, then hand the same bytes to the
-                        // handshake. Peeking rather than consuming keeps the
-                        // stream intact, so no replay buffer is needed and the
-                        // handshake proceeds exactly as it would have.
+            // Serve connections until shutdown, rather than exactly one.
+            //
+            // One was enough while a failed handshake was the end of the
+            // story. It is not any more: the profile ladder retries a refused
+            // handshake with a wider offer, so every legacy-cipher test needs
+            // at least two connections. With a single-shot server the second
+            // attempt gets ECONNREFUSED and the test fails for a reason that
+            // has nothing to do with what it is testing.
+            let mut shutdown_rx = shutdown_rx;
+            loop {
+                tokio::select! {
+                    result = listener.accept() => {
+                        let Ok((tcp_stream, _)) = result else { continue };
+
+                        // Read the ClientHello off the wire and parse it
+                        // before TLS touches it. Peeking rather than consuming
+                        // keeps the stream intact, so no replay buffer is
+                        // needed and the handshake proceeds as it would have.
+                        //
+                        // Overwritten per connection, so after a ladder walk
+                        // the slot holds the LAST hello, which is the one that
+                        // succeeded.
                         let mut peek = vec![0u8; 8192];
                         if let Ok(n) = tcp_stream.peek(&mut peek).await
                             && let Some(parsed) = parse_client_hello(&peek[..n])
@@ -374,30 +388,26 @@ impl TlsTestServer {
                             *slot = Some(parsed);
                         }
 
-                        // Async TLS handshake via tokio-openssl
                         let ssl = openssl::ssl::Ssl::new(acceptor.context()).unwrap();
                         let mut tls_stream = match tokio_openssl::SslStream::new(ssl, tcp_stream) {
                             Ok(s) => s,
-                            Err(_) => return,
+                            Err(_) => continue,
                         };
 
-                        // Perform the TLS accept handshake
+                        // A failed handshake is an ordinary event here, not
+                        // the end: it is what the cipher and version mismatch
+                        // tests are producing on purpose, and what the ladder
+                        // then retries.
                         if std::pin::Pin::new(&mut tls_stream).accept().await.is_err() {
-                            // Handshake failed — expected for cipher/version mismatch tests
-                            return;
+                            continue;
                         }
 
-                        // Read the HTTP request (we don't care about contents)
                         let mut buf = [0u8; 4096];
                         let _ = tls_stream.read(&mut buf).await;
-
-                        // Send response
                         let _ = tls_stream.write_all(response.as_bytes()).await;
                         let _ = tls_stream.shutdown().await;
                     }
-                }
-                _ = shutdown_rx => {
-                    // Shutdown requested
+                    _ = &mut shutdown_rx => break,
                 }
             }
         });

From 4ecc6788bb5869c68cebbfd682b1f9cadc758145 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 17:05:25 -0400
Subject: [PATCH 15/44] Test the 1.0 Python surface, and make an unknown
 profile actually error

The Python API is what BBOT consumes and it had no pytest coverage at
all: not profiles, not conclusion, not attempts, not protection, not
TransportError. That mattered more than the usual reason, because the
interesting half of this release is whether a *failure* is legible
enough to act on, and that is exactly the path nothing exercised.

Twelve tests against a local server, so they are deterministic and need
no network. They cover which profile a request gets and what it puts on
the wire, header precedence, the reached / blocked / needs_browser
distinction, a bare 403 naming no vendor, and the attributes on a
transport failure.

Writing them found a real gap. validate_profile() was added in the
profile work and never called, so an unknown name still resolved
silently to the default: precisely the behaviour I had described as
fixed. resolved_profile falls back on purpose, because it runs deep
inside the connector where a failure has nowhere to go, so the check
belongs at the edge next to validate_proxy. A typo is now a message
rather than a profile the caller did not ask for.

174 pytest and 326 cargo tests.
---
 src/client/hyper.rs                      |   6 +
 tests/python/test_profiles_and_report.py | 230 +++++++++++++++++++++++
 2 files changed, 236 insertions(+)
 create mode 100644 tests/python/test_profiles_and_report.py

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 51bdfd6..9ccface 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1973,6 +1973,12 @@ fn retry_backoff(attempt: u32, min_wait: Duration, max_wait: Duration) -> Durati
 impl HttpClient for HyperClient {
     async fn send(&self, config: &RequestConfig) -> Result<Response, ClientError> {
         config.validate_proxy().map_err(ClientError::other)?;
+        // Checked once, at the edge. `resolved_profile` deliberately falls
+        // back rather than erroring, because it is called from deep inside the
+        // connector where there is nothing useful to do with a failure; this
+        // is the place that turns a typo into a message instead of silently
+        // giving the caller a profile they did not ask for.
+        config.validate_profile().map_err(ClientError::other)?;
         let timeout_duration = Duration::from_secs(config.timeout());
         let max_retries = config.max_retries();
         let min_wait = config.retry_wait_min();
diff --git a/tests/python/test_profiles_and_report.py b/tests/python/test_profiles_and_report.py
new file mode 100644
index 0000000..2db2deb
--- /dev/null
+++ b/tests/python/test_profiles_and_report.py
@@ -0,0 +1,230 @@
+"""The 1.0 Python surface: connection profiles, and the report a caller reads
+to decide what to do next.
+
+This is the contract BBOT consumes. It had no test coverage at all until this
+file, which mattered more than usual because the interesting part is not
+whether a request succeeds but whether a *failure* is legible enough to act
+on, and that is exactly the path nothing exercised.
+
+Everything here runs against a local server, so it is deterministic and needs
+no network. What it cannot cover locally is the ladder actually shifting
+profiles against a real protection product; the Rust integration tests do that
+against a TLS server that can refuse a handshake, and `bench/` does it against
+real sites.
+"""
+
+import asyncio
+
+import pytest
+import pytest_asyncio
+
+import blasthttp
+
+
+# ── Local server ─────────────────────────────────────────────────
+
+
+async def _handler(reader, writer):
+    """Serves paths that stand in for the things a caller has to tell apart.
+
+    /              ordinary 200
+    /echo-headers  200 whose body is the request headers it received
+    /cf-challenge  403 shaped like a Cloudflare managed challenge
+    /akamai-block  403 shaped like an Akamai edge refusal
+    /plain-403     403 with no product signature at all
+    """
+    try:
+        request_line = await reader.readuntil(b"\r\n")
+        received = []
+        while True:
+            line = await reader.readuntil(b"\r\n")
+            if line == b"\r\n":
+                break
+            received.append(line.decode("latin1").rstrip("\r\n"))
+
+        try:
+            _method, path, _ver = request_line.decode().split(" ", 2)
+        except ValueError:
+            path = "/"
+
+        def respond(status, headers, body):
+            raw = f"HTTP/1.1 {status} X\r\n".encode()
+            for k, v in headers:
+                raw += f"{k}: {v}\r\n".encode()
+            raw += b"Content-Length: " + str(len(body)).encode() + b"\r\n"
+            raw += b"Connection: close\r\n\r\n" + body
+            writer.write(raw)
+
+        if path == "/echo-headers":
+            body = "\n".join(received).encode()
+            respond(200, [("Content-Type", "text/plain")], body)
+        elif path == "/cf-challenge":
+            # cf-mitigated is the header Cloudflare sets specifically to say it
+            # intervened, and is what separates a challenge from a WAF block.
+            respond(
+                403,
+                [("server", "cloudflare"), ("cf-mitigated", "challenge")],
+                b"<html>Just a moment...</html>",
+            )
+        elif path == "/akamai-block":
+            # No `server` header at all, which is what www.akamai.com's own 403
+            # actually sends, and the reason a rule keyed on it matched nothing.
+            respond(
+                403,
+                [("akamai-grn", "0.6d551702.1790644996.315f62a4")],
+                b"<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD></HTML>",
+            )
+        elif path == "/plain-403":
+            respond(403, [("server", "nginx")], b"forbidden")
+        else:
+            respond(200, [("Content-Type", "text/plain")], b"hello")
+
+        await writer.drain()
+    except Exception:
+        pass
+    finally:
+        writer.close()
+
+
+@pytest_asyncio.fixture
+async def server():
+    srv = await asyncio.start_server(_handler, "127.0.0.1", 0)
+    port = srv.sockets[0].getsockname()[1]
+    async with srv:
+        yield f"http://127.0.0.1:{port}"
+
+
+@pytest.fixture
+def client():
+    return blasthttp.BlastHTTP()
+
+
+# ── Profiles ─────────────────────────────────────────────────────
+
+
+async def test_default_profile_sends_an_honest_user_agent(server, client):
+    """The default claims nothing. That is the whole point of it: claiming to
+    be a browser selects which scrutiny applies, and a client that cannot back
+    the claim is worse off than one that never made it."""
+    r = await client.request(f"{server}/echo-headers")
+    assert "blasthttp/" in r.text
+    assert "Mozilla/5.0" not in r.text
+
+
+async def test_chrome_profile_sends_browser_headers_in_order(server, client):
+    r = await client.request(f"{server}/echo-headers", profile="chrome")
+    sent = [line.split(":", 1)[0].lower() for line in r.text.splitlines() if ":" in line]
+
+    assert "Mozilla/5.0" in r.text
+    for expected in ("sec-ch-ua", "sec-fetch-dest", "accept-language", "priority"):
+        assert expected in sent, f"{expected} missing: {sent}"
+
+    # Order is part of the fingerprint, not decoration, so a profile whose
+    # headers arrive shuffled has not been applied properly.
+    assert sent.index("sec-ch-ua") < sent.index("user-agent") < sent.index("accept")
+
+
+async def test_compatibility_profile_is_also_honest(server, client):
+    r = await client.request(f"{server}/echo-headers", profile="compatibility")
+    assert "blasthttp/" in r.text
+    assert "Mozilla/5.0" not in r.text
+
+
+async def test_unknown_profile_is_an_error(server, client):
+    """It used to resolve silently to the default, so a typo bought you
+    whatever that was with nothing said about it. Once the choice of profile
+    decides whether a request gets inspected, that is not a good trade."""
+    with pytest.raises(Exception) as exc:
+        await client.request(f"{server}/", profile="chrmoe")
+    assert "chrmoe" in str(exc.value).lower()
+
+
+async def test_an_explicit_header_beats_the_profile(server, client):
+    r = await client.request(
+        f"{server}/echo-headers",
+        profile="chrome",
+        headers=[("user-agent", "mine/1.0")],
+    )
+    assert "mine/1.0" in r.text
+    assert "Chrome/131" not in r.text
+
+
+# ── The report ───────────────────────────────────────────────────
+
+
+async def test_an_ordinary_response_reports_reached(server, client):
+    r = await client.request(f"{server}/")
+    assert r.conclusion == ("reached", "-")
+    assert r.protection == ("ok", "-")
+    assert [a[0] for a in r.attempts] == ["modern"]
+    assert r.attempts[0][1] == "reached"
+    assert r.attempts[0][2] == 200
+
+
+async def test_a_challenge_asks_for_a_browser(server, client):
+    """The signal BBOT keys on. A challenge needs JavaScript, so no amount of
+    retrying with different connection settings will help and the URL should
+    go to a real browser instead."""
+    r = await client.request(f"{server}/cf-challenge")
+    assert r.status_code == 403
+    assert r.protection == ("challenge", "cloudflare")
+    assert r.conclusion == ("needs_browser", "cloudflare")
+
+
+async def test_a_block_is_not_a_challenge(server, client):
+    """The distinction that decides whether to launch a browser. A block might
+    yield to a different approach; a challenge will not."""
+    r = await client.request(f"{server}/akamai-block")
+    assert r.protection == ("blocked", "akamai")
+    conclusion, vendor = r.conclusion
+    assert conclusion == "blocked"
+    assert vendor == "akamai"
+    assert conclusion != "needs_browser"
+
+
+async def test_a_bare_403_names_no_vendor(server, client):
+    """Most 403s are ordinary authorization failures. Reporting one as though a
+    product refused us would invite a caller to act on nothing, and internally
+    it is what stops the ladder spending a second request on every one."""
+    r = await client.request(f"{server}/plain-403")
+    assert r.protection == ("blocked", "unknown")
+    assert r.conclusion == ("blocked", "-")
+
+
+# ── Failures ─────────────────────────────────────────────────────
+
+
+async def test_a_refused_connection_is_legible(client):
+    """Before 1.0 every transport failure arrived as a RuntimeError whose only
+    content was a sentence, so a caller could not tell a dead host from a
+    refused cipher without parsing English."""
+    with pytest.raises(blasthttp.TransportError) as exc:
+        await client.request("http://127.0.0.1:1/", timeout=5)
+
+    e = exc.value
+    assert e.kind == "connection"
+    assert e.tls_failure is None
+    assert e.retryable is True
+    assert e.conclusion == "unreachable"
+
+
+async def test_transport_error_is_still_a_runtime_error(client):
+    """Subclassing RuntimeError on purpose, so callers written against the old
+    behaviour keep working."""
+    with pytest.raises(RuntimeError):
+        await client.request("http://127.0.0.1:1/", timeout=5)
+
+
+async def test_batch_results_carry_the_report(server, client):
+    cfgs = [
+        blasthttp.BatchConfig(f"{server}/"),
+        blasthttp.BatchConfig(f"{server}/cf-challenge"),
+    ]
+    results = await client.request_batch(cfgs, concurrency=2)
+    by_url = {r.url: r for r in results}
+
+    assert by_url[f"{server}/"].response.conclusion == ("reached", "-")
+    assert by_url[f"{server}/cf-challenge"].response.conclusion == (
+        "needs_browser",
+        "cloudflare",
+    )

From 614a3c3fdc20f4189a5927978cec0fa642f7736e Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 18:01:27 -0400
Subject: [PATCH 16/44] Share ladder discovery across a concurrent burst

Every request in an opening burst walked the ladder independently,
because the per-host memory is only written once something succeeds and
nothing had succeeded yet. Measured against akamai.com: twelve
concurrent requests, twenty-four handshake attempts.

The overshoot is bounded by concurrency rather than scan size, so a
thousand-path scan at fifty concurrent pays about fifty extra
handshakes, not a thousand. They all land in the same burst though,
against a host that is by definition already watching, which is the
worst possible moment to look like a pile of odd clients.

One request now discovers and the rest wait for the answer, via a
per-host semaphore with the usual double-check after acquiring: the
leader often finishes while a waiter is queued, which is the whole point
of having queued.

Bounded on purpose. A waiter gives up after the request timeout and
walks the ladder itself rather than inheriting a stalled leader; a
wasted handshake is a far smaller problem than a request that never
returns. The semaphore is dropped once a host's profile is known, so the
map holds hosts under discovery rather than every host ever seen.

The permit is released the moment the profile is known rather than at
the end of the hop, and that detail is most of the win. Held to the end,
waiters also sat through redirect handling and cookie bookkeeping they
had no use for: 13 attempts but 4.0s. Released early, the same 13
attempts in 1.6s.

  before                     24 attempts  1.4s
  after, released late       13 attempts  4.0s
  after, released early      13 attempts  1.6s
  floor (one attempt each)   12 attempts

Half the handshakes for 0.2s, and nothing at all on a host that does not
need the ladder: google.com is 12 attempts in 1.0s either way.
---
 src/client/hyper.rs     | 114 +++++++++++++++++++++++++++++++++++-----
 tests/legacy_default.rs |  61 +++++++++++++++++++++
 2 files changed, 161 insertions(+), 14 deletions(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 9ccface..92cfd30 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -940,6 +940,10 @@ pub struct HyperClient {
     // the ladder on every request. Written only on success; see
     // `remember_profile`.
     host_profiles: Mutex<HashMap<String, &'static crate::profile::ConnectionProfile>>,
+    // One-at-a-time ladder discovery per host. See `discovery_permit`.
+    // Entries are removed once a host's profile is known, so this holds only
+    // hosts currently being discovered rather than every host ever seen.
+    host_discovery: Mutex<HashMap<String, Arc<tokio::sync::Semaphore>>>,
     // Why the last handshake to each host failed.
     //
     // Deliberately on the client rather than on `CachedClient`: a retry with
@@ -960,6 +964,7 @@ impl HyperClient {
         HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
             host_profiles: Mutex::new(HashMap::new()),
+            host_discovery: Mutex::new(HashMap::new()),
             tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
@@ -2111,21 +2116,90 @@ impl HyperClient {
     /// a host we have succeeded against before starts on whatever worked,
     /// which is the whole point of remembering: the ladder costs a wasted
     /// request every time it walks, and a scan hits the same host repeatedly.
-    fn starting_profile(
+    /// What we already know about this host, if anything.
+    fn remembered_profile(
+        &self,
+        uri: &http::Uri,
+    ) -> Option<&'static crate::profile::ConnectionProfile> {
+        let key = peer_slot_key(uri)?;
+        let map = self.host_profiles.lock().ok()?;
+        map.get(&key).copied()
+    }
+
+    /// Hold the right to be the one that walks the ladder for this host.
+    ///
+    /// Without it, every request in an opening burst walks independently,
+    /// because the per-host memory is only written once something succeeds and
+    /// nothing has succeeded yet. Measured: twelve concurrent requests to a
+    /// host needing one shift produced twenty-four handshake attempts instead
+    /// of twelve. The overshoot is bounded by concurrency rather than by scan
+    /// size, so a thousand-path scan at fifty concurrent pays about fifty
+    /// extra handshakes, not a thousand. They all land in the same burst
+    /// though, against a host that is by definition already watching, which is
+    /// the worst moment to look like a pile of odd clients.
+    ///
+    /// So one request discovers and the rest wait for the answer. That costs
+    /// the followers the leader's walk in latency, once per host, in exchange
+    /// for roughly a third of the handshakes.
+    ///
+    /// Bounded: if the leader is slow or fails outright, waiters give up and
+    /// walk it themselves rather than inheriting a stall. A wasted handshake
+    /// is a much smaller problem than a request that never returns.
+    async fn discovery_permit(
+        &self,
+        uri: &http::Uri,
+        timeout: Duration,
+    ) -> Option<tokio::sync::OwnedSemaphorePermit> {
+        let key = peer_slot_key(uri)?;
+        let sem = {
+            // Clone the Arc and drop the std guard before awaiting: holding a
+            // std::sync::Mutex across an await point is how this deadlocks.
+            let mut map = self.host_discovery.lock().ok()?;
+            map.entry(key)
+                .or_insert_with(|| Arc::new(tokio::sync::Semaphore::new(1)))
+                .clone()
+        };
+        tokio::time::timeout(timeout, sem.acquire_owned())
+            .await
+            .ok()?
+            .ok()
+    }
+
+    /// Which profile this hop starts on.
+    ///
+    /// A pinned config starts and ends on what the caller asked for. Otherwise
+    /// a host we have succeeded against before starts on whatever worked,
+    /// which is the point of remembering: the ladder costs a wasted request
+    /// every time it walks, and a scan hits the same host repeatedly.
+    ///
+    /// Returns a permit alongside the profile. Holding it is what stops the
+    /// rest of a concurrent burst walking the same ladder; it is released when
+    /// the hop finishes, whether or not anything worked.
+    async fn starting_profile(
         &self,
         config: &RequestConfig,
         uri: &http::Uri,
-    ) -> &'static crate::profile::ConnectionProfile {
+    ) -> (
+        &'static crate::profile::ConnectionProfile,
+        Option<tokio::sync::OwnedSemaphorePermit>,
+    ) {
         if config.tls_is_pinned() {
-            return config.resolved_profile();
+            return (config.resolved_profile(), None);
         }
-        if let Some(key) = peer_slot_key(uri)
-            && let Ok(map) = self.host_profiles.lock()
-            && let Some(p) = map.get(&key)
-        {
-            return p;
+        if let Some(p) = self.remembered_profile(uri) {
+            return (p, None);
         }
-        config.resolved_profile()
+
+        let permit = self
+            .discovery_permit(uri, Duration::from_secs(config.timeout()))
+            .await;
+
+        // Check again now we hold it. The leader may have finished while we
+        // were waiting, which is the whole point of having waited.
+        if let Some(p) = self.remembered_profile(uri) {
+            return (p, None);
+        }
+        (config.resolved_profile(), permit)
     }
 
     /// The next rung, or `None` to stop.
@@ -2158,10 +2232,16 @@ impl HyperClient {
         uri: &http::Uri,
         profile: &'static crate::profile::ConnectionProfile,
     ) {
-        if let Some(key) = peer_slot_key(uri)
-            && let Ok(mut map) = self.host_profiles.lock()
-        {
-            map.insert(key, profile);
+        let Some(key) = peer_slot_key(uri) else {
+            return;
+        };
+        if let Ok(mut map) = self.host_profiles.lock() {
+            map.insert(key.clone(), profile);
+        }
+        // Discovery is over for this host, so drop its semaphore rather than
+        // keeping one per host for the life of the client.
+        if let Ok(mut map) = self.host_discovery.lock() {
+            map.remove(&key);
         }
     }
 
@@ -2273,7 +2353,7 @@ impl HyperClient {
             // `send`, so it does not spend the caller's retry budget: those are
             // different questions, one being "did this flake" and the other
             // "is this the wrong way to talk to this host".
-            let mut rung = self.starting_profile(config, &uri);
+            let (mut rung, mut discovery_permit) = self.starting_profile(config, &uri).await;
             let mut tried: Vec<&'static str> = Vec::new();
             let mut attempts: Vec<crate::report::Attempt> = Vec::new();
             let (resp, cached) = loop {
@@ -2308,6 +2388,12 @@ impl HyperClient {
                             // evidence.
                             if outcome.got_through() {
                                 self.remember_profile(&uri, rung);
+                                // Release the moment the answer is known,
+                                // rather than at the end of the hop. Waiters
+                                // only ever needed the profile; making them
+                                // also wait out redirect handling and cookie
+                                // bookkeeping would be latency for nothing.
+                                drop(discovery_permit.take());
                             }
                             break (resp, cached);
                         }
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index ddc7869..637811e 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -236,3 +236,64 @@ async fn test_modern_still_reachable_by_default() {
 
     server.shutdown().await;
 }
+
+// ── Concurrent discovery ──────────────────────────────────────────
+
+#[tokio::test]
+async fn test_a_burst_does_not_all_walk_the_ladder() {
+    // Every request in an opening burst used to walk the ladder
+    // independently, because the per-host memory is only written once
+    // something succeeds and nothing has succeeded yet. Measured against a
+    // real host before this was fixed: twelve concurrent requests produced
+    // twenty-four handshake attempts.
+    //
+    // That overshoot is bounded by concurrency rather than by scan size, so a
+    // thousand-path scan at fifty concurrent pays about fifty extra
+    // handshakes. They all land in the same burst though, against a host that
+    // is by definition already watching, which is the worst moment to look
+    // like a pile of odd clients.
+    //
+    // One request now discovers and the rest wait for the answer.
+    const N: usize = 8;
+
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let client = std::sync::Arc::new(HyperClient::new());
+    let url = server.url();
+
+    let mut tasks = Vec::new();
+    for _ in 0..N {
+        let client = client.clone();
+        let config = default_config(&url);
+        tasks.push(tokio::spawn(async move { client.send(&config).await }));
+    }
+
+    let mut attempts = 0usize;
+    let mut reached = 0usize;
+    for t in tasks {
+        if let Ok(Ok(resp)) = t.await {
+            attempts += resp.attempts.len();
+            if resp.status == 200 {
+                reached += 1;
+            }
+        }
+    }
+
+    assert_eq!(reached, N, "every request should have got through");
+    // The floor is N, one attempt each. The leader spends one extra walking
+    // from `modern` to `compatibility`. Anything approaching 2N means the
+    // burst is re-discovering per request.
+    assert!(
+        attempts <= N + 2,
+        "burst made {attempts} attempts for {N} requests; expected about {} \
+         (one each, plus the leader's walk). Discovery is not being shared.",
+        N + 1
+    );
+
+    server.shutdown().await;
+}

From d1d01809a1488b7fc07653bce84e8a3aff38c011 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 18:08:03 -0400
Subject: [PATCH 17/44] Back out the burst fix: it trades a worse property than
 it buys

Reverts 614a3c3, which added a per-host semaphore so that in an opening
burst one request walks the ladder and the rest wait for its answer.

It worked. Twelve concurrent requests to akamai.com went from 24
handshake attempts to 13, at 1.6s against 1.4s.

But a waiter can only be released once the leader has classified its
response, and classification needs the body, so the wait is a whole
request long. That is a head-of-line stall, and not blocking a fast
request behind a slow one is a property request_batch_stream exists to
provide and has a test guarding it. The fix broke that test.

Bounding the wait to 250ms put the test back and removed the entire
saving, 13 attempts back to 24, because no bound short enough to keep
the latency property is long enough for the leader to have learned
anything. The two goals are in direct conflict and latency wins: the
extra handshakes are bounded by concurrency, happen only on the first
burst against a host that needs a shift, and cost nobody a response.

The test the fix came with is kept, rewritten to record the overshoot
and what was tried, so the cost stays visible instead of being
rediscovered later.
---
 src/client/hyper.rs     | 114 +++++-----------------------------------
 tests/legacy_default.rs |  53 ++++++++++---------
 2 files changed, 43 insertions(+), 124 deletions(-)

diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 92cfd30..9ccface 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -940,10 +940,6 @@ pub struct HyperClient {
     // the ladder on every request. Written only on success; see
     // `remember_profile`.
     host_profiles: Mutex<HashMap<String, &'static crate::profile::ConnectionProfile>>,
-    // One-at-a-time ladder discovery per host. See `discovery_permit`.
-    // Entries are removed once a host's profile is known, so this holds only
-    // hosts currently being discovered rather than every host ever seen.
-    host_discovery: Mutex<HashMap<String, Arc<tokio::sync::Semaphore>>>,
     // Why the last handshake to each host failed.
     //
     // Deliberately on the client rather than on `CachedClient`: a retry with
@@ -964,7 +960,6 @@ impl HyperClient {
         HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
             host_profiles: Mutex::new(HashMap::new()),
-            host_discovery: Mutex::new(HashMap::new()),
             tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
@@ -2116,90 +2111,21 @@ impl HyperClient {
     /// a host we have succeeded against before starts on whatever worked,
     /// which is the whole point of remembering: the ladder costs a wasted
     /// request every time it walks, and a scan hits the same host repeatedly.
-    /// What we already know about this host, if anything.
-    fn remembered_profile(
-        &self,
-        uri: &http::Uri,
-    ) -> Option<&'static crate::profile::ConnectionProfile> {
-        let key = peer_slot_key(uri)?;
-        let map = self.host_profiles.lock().ok()?;
-        map.get(&key).copied()
-    }
-
-    /// Hold the right to be the one that walks the ladder for this host.
-    ///
-    /// Without it, every request in an opening burst walks independently,
-    /// because the per-host memory is only written once something succeeds and
-    /// nothing has succeeded yet. Measured: twelve concurrent requests to a
-    /// host needing one shift produced twenty-four handshake attempts instead
-    /// of twelve. The overshoot is bounded by concurrency rather than by scan
-    /// size, so a thousand-path scan at fifty concurrent pays about fifty
-    /// extra handshakes, not a thousand. They all land in the same burst
-    /// though, against a host that is by definition already watching, which is
-    /// the worst moment to look like a pile of odd clients.
-    ///
-    /// So one request discovers and the rest wait for the answer. That costs
-    /// the followers the leader's walk in latency, once per host, in exchange
-    /// for roughly a third of the handshakes.
-    ///
-    /// Bounded: if the leader is slow or fails outright, waiters give up and
-    /// walk it themselves rather than inheriting a stall. A wasted handshake
-    /// is a much smaller problem than a request that never returns.
-    async fn discovery_permit(
-        &self,
-        uri: &http::Uri,
-        timeout: Duration,
-    ) -> Option<tokio::sync::OwnedSemaphorePermit> {
-        let key = peer_slot_key(uri)?;
-        let sem = {
-            // Clone the Arc and drop the std guard before awaiting: holding a
-            // std::sync::Mutex across an await point is how this deadlocks.
-            let mut map = self.host_discovery.lock().ok()?;
-            map.entry(key)
-                .or_insert_with(|| Arc::new(tokio::sync::Semaphore::new(1)))
-                .clone()
-        };
-        tokio::time::timeout(timeout, sem.acquire_owned())
-            .await
-            .ok()?
-            .ok()
-    }
-
-    /// Which profile this hop starts on.
-    ///
-    /// A pinned config starts and ends on what the caller asked for. Otherwise
-    /// a host we have succeeded against before starts on whatever worked,
-    /// which is the point of remembering: the ladder costs a wasted request
-    /// every time it walks, and a scan hits the same host repeatedly.
-    ///
-    /// Returns a permit alongside the profile. Holding it is what stops the
-    /// rest of a concurrent burst walking the same ladder; it is released when
-    /// the hop finishes, whether or not anything worked.
-    async fn starting_profile(
+    fn starting_profile(
         &self,
         config: &RequestConfig,
         uri: &http::Uri,
-    ) -> (
-        &'static crate::profile::ConnectionProfile,
-        Option<tokio::sync::OwnedSemaphorePermit>,
-    ) {
+    ) -> &'static crate::profile::ConnectionProfile {
         if config.tls_is_pinned() {
-            return (config.resolved_profile(), None);
-        }
-        if let Some(p) = self.remembered_profile(uri) {
-            return (p, None);
+            return config.resolved_profile();
         }
-
-        let permit = self
-            .discovery_permit(uri, Duration::from_secs(config.timeout()))
-            .await;
-
-        // Check again now we hold it. The leader may have finished while we
-        // were waiting, which is the whole point of having waited.
-        if let Some(p) = self.remembered_profile(uri) {
-            return (p, None);
+        if let Some(key) = peer_slot_key(uri)
+            && let Ok(map) = self.host_profiles.lock()
+            && let Some(p) = map.get(&key)
+        {
+            return p;
         }
-        (config.resolved_profile(), permit)
+        config.resolved_profile()
     }
 
     /// The next rung, or `None` to stop.
@@ -2232,16 +2158,10 @@ impl HyperClient {
         uri: &http::Uri,
         profile: &'static crate::profile::ConnectionProfile,
     ) {
-        let Some(key) = peer_slot_key(uri) else {
-            return;
-        };
-        if let Ok(mut map) = self.host_profiles.lock() {
-            map.insert(key.clone(), profile);
-        }
-        // Discovery is over for this host, so drop its semaphore rather than
-        // keeping one per host for the life of the client.
-        if let Ok(mut map) = self.host_discovery.lock() {
-            map.remove(&key);
+        if let Some(key) = peer_slot_key(uri)
+            && let Ok(mut map) = self.host_profiles.lock()
+        {
+            map.insert(key, profile);
         }
     }
 
@@ -2353,7 +2273,7 @@ impl HyperClient {
             // `send`, so it does not spend the caller's retry budget: those are
             // different questions, one being "did this flake" and the other
             // "is this the wrong way to talk to this host".
-            let (mut rung, mut discovery_permit) = self.starting_profile(config, &uri).await;
+            let mut rung = self.starting_profile(config, &uri);
             let mut tried: Vec<&'static str> = Vec::new();
             let mut attempts: Vec<crate::report::Attempt> = Vec::new();
             let (resp, cached) = loop {
@@ -2388,12 +2308,6 @@ impl HyperClient {
                             // evidence.
                             if outcome.got_through() {
                                 self.remember_profile(&uri, rung);
-                                // Release the moment the answer is known,
-                                // rather than at the end of the hop. Waiters
-                                // only ever needed the profile; making them
-                                // also wait out redirect handling and cookie
-                                // bookkeeping would be latency for nothing.
-                                drop(discovery_permit.take());
                             }
                             break (resp, cached);
                         }
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index 637811e..16ce74c 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -237,24 +237,33 @@ async fn test_modern_still_reachable_by_default() {
     server.shutdown().await;
 }
 
-// ── Concurrent discovery ──────────────────────────────────────────
+// ── A known cost, deliberately not fixed ──────────────────────────
 
 #[tokio::test]
-async fn test_a_burst_does_not_all_walk_the_ladder() {
-    // Every request in an opening burst used to walk the ladder
-    // independently, because the per-host memory is only written once
-    // something succeeds and nothing has succeeded yet. Measured against a
-    // real host before this was fixed: twelve concurrent requests produced
-    // twenty-four handshake attempts.
+async fn test_a_cold_burst_rediscovers_per_request() {
+    // Every request in an opening burst walks the ladder independently,
+    // because the per-host memory is only written once something succeeds and
+    // nothing has succeeded yet. This test records that rather than asserting
+    // it away, because the obvious fix turned out to cost more than it saved.
     //
-    // That overshoot is bounded by concurrency rather than by scan size, so a
-    // thousand-path scan at fifty concurrent pays about fifty extra
-    // handshakes. They all land in the same burst though, against a host that
-    // is by definition already watching, which is the worst moment to look
-    // like a pile of odd clients.
+    // The cost: roughly double the handshakes in the first burst against a
+    // host needing a shift. Measured against a real host, twelve concurrent
+    // requests produced twenty-four attempts. It is bounded by concurrency
+    // and not by scan size, so a thousand-path scan at fifty concurrent pays
+    // about fifty extra handshakes, and only on the first burst; afterwards
+    // the memory answers.
     //
-    // One request now discovers and the rest wait for the answer.
-    const N: usize = 8;
+    // The attempted fix was a per-host semaphore, so one request discovers
+    // and the rest wait. It worked, 24 attempts down to 13. But a waiter
+    // cannot be released until the leader has classified its response, and
+    // classification needs the body, so the wait is a full request long. That
+    // is a head-of-line stall, which is exactly what request_batch_stream
+    // exists to avoid, and it broke the test that guards that property.
+    // Shortening the wait enough to preserve it removed the entire saving.
+    //
+    // So: leave it, and revisit if scan telemetry ever shows the burst
+    // mattering more than the latency.
+    const N: usize = 6;
 
     let server = TlsTestServer::start(TlsServerConfig {
         cipher_list: Some("RC4-SHA".to_string()),
@@ -273,8 +282,7 @@ async fn test_a_burst_does_not_all_walk_the_ladder() {
         tasks.push(tokio::spawn(async move { client.send(&config).await }));
     }
 
-    let mut attempts = 0usize;
-    let mut reached = 0usize;
+    let (mut attempts, mut reached) = (0usize, 0usize);
     for t in tasks {
         if let Ok(Ok(resp)) = t.await {
             attempts += resp.attempts.len();
@@ -284,15 +292,12 @@ async fn test_a_burst_does_not_all_walk_the_ladder() {
         }
     }
 
-    assert_eq!(reached, N, "every request should have got through");
-    // The floor is N, one attempt each. The leader spends one extra walking
-    // from `modern` to `compatibility`. Anything approaching 2N means the
-    // burst is re-discovering per request.
+    // Correctness is what matters and it holds: every request gets through,
+    // the ladder just runs more often than it strictly needs to.
+    assert_eq!(reached, N, "every request should get through");
     assert!(
-        attempts <= N + 2,
-        "burst made {attempts} attempts for {N} requests; expected about {} \
-         (one each, plus the leader's walk). Discovery is not being shared.",
-        N + 1
+        attempts >= N,
+        "expected at least one attempt each, got {attempts}"
     );
 
     server.shutdown().await;

From 021a6a47a1860b6461513e39fd154625621c45f3 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 29 Sep 2026 18:30:01 -0400
Subject: [PATCH 18/44] Give the other three request paths the same connection
 handling

Three places where the 1.0 work stopped short of the paths that do not
go through the pooled client.

raw_connect, and requests using resolve_ip or request_target, now widen
their cipher offer and retry when a handshake fails in a way that says
the peer could not negotiate. They bypass the pooled client, so they
bypassed the ladder too, and when the default narrowed from 105 suites
to 11 they quietly lost the legacy reach the custom OpenSSL build
exists to give them. A virtualhost sweep is exactly where an old
appliance turns up and exactly where it would have gone unseen.

They stop at that rung. The other one reacts to a refusal by changing
what the client claims to be, and that would be wrong here: these are
the callers who asked for exact control over one request, so re-sending
a probe dressed differently answers a question nobody asked. A raw
connection has no response to classify at all. A named profile, cipher
string or TLS version still pins everything, and there is a test for
that, because widening past a pin would make a TLS enumeration sweep
report every server as speaking everything.

connect_stream returns the profile it settled on so dispatch_direct
builds its request under the same one, rather than letting the headers
drift away from the TLS underneath them.

download() takes profile, cipher_string, min_tls_version,
max_tls_version and redirect_cookies. It hardcoded no profile with no
way to pass one, so a file behind a host that only answers a browser
was unreachable through download while the same URL through request
was fine.

blasthttp.mock forwards every kwarg to the real client on a passthrough
request instead of naming six and dropping the rest. Ignoring them is
right when the URL is intercepted, since nothing is dialled, but a
passed-through request is real and was being given different TLS, a
different profile and no timeout from what the caller asked for. BBOT
runs its local-target suite through this path.

That last test then found a fourth thing: a request that named a
profile was writing it into the per-host memory, although reads already
skip the memory when the caller pinned something. So one deliberate
profile="chrome" request converted the host for every later request,
which is the worst direction for it to drift in, since claiming to be a
browser is what invites a detector to check the claim. It also made the
result depend on the order two unrelated requests ran in. Writes now
follow the same rule as reads.
---
 CHANGELOG.md                             |  21 +++
 README.md                                |   9 +-
 src/client/hyper.rs                      | 168 ++++++++++++++++++--
 src/client/raw.rs                        |  11 +-
 src/mock.rs                              |  23 ++-
 src/python.rs                            |  20 ++-
 tests/legacy_default.rs                  | 190 +++++++++++++++++++++++
 tests/python/test_mock.py                |  54 +++++++
 tests/python/test_profiles_and_report.py |  28 ++++
 9 files changed, 497 insertions(+), 27 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5911c76..c2e7e68 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -47,6 +47,27 @@ the ladder falling back to `compatibility` on the second handshake.
   default offers nothing that exists in SSLv3
 - `BLASTHTTP_BISECT=tls,headers,http2` disables profile layers independently,
   for finding out which one a detector is reacting to
+- `raw_connect`, and requests using `resolve_ip` or `request_target`, widen
+  their cipher offer and retry on a handshake the peer could not negotiate,
+  the same as an ordinary request. These bypass the pooled client, so they
+  bypassed the ladder too, and when the default narrowed from 105 suites to 11
+  they quietly lost the legacy reach the custom OpenSSL build exists to give
+  them. They stop at that rung: the other one reacts to a refusal by changing
+  what the client claims to be, and these are the callers who asked for exact
+  control over one request
+- `download()` takes `profile`, `cipher_string`, `min_tls_version`,
+  `max_tls_version` and `redirect_cookies`. It hardcoded no profile with no
+  way to pass one, so a file behind a host that only answers a browser was
+  unreachable through `download` while the same URL through `request` was fine
+- `blasthttp.mock` forwards every kwarg to the real client on a passthrough
+  request. It named six and dropped the rest, so a URL excluded from mocking
+  was dialled with different TLS, a different profile and no timeout from what
+  the caller asked for
+- A request that names a profile no longer writes it into the per-host memory.
+  Reads already skipped the memory when the caller pinned something; writes
+  did not, so one deliberate `profile="chrome"` request turned every later
+  request to that host into a browser claim, and made the outcome depend on
+  the order two unrelated requests ran in
 
 ## 0.10.1
 
diff --git a/README.md b/README.md
index 190fadb..aa3a7fa 100644
--- a/README.md
+++ b/README.md
@@ -299,6 +299,11 @@ r = await client.request(
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
 | `headers` | `list[tuple[str, str]]` | Request headers |
 | `retries` | `int` | Number of retries on failure |
+| `redirect_cookies` | `bool` | Carry `Set-Cookie` across redirect hops |
+| `profile` | `str` | Connection profile, as on `request()` |
+| `cipher_string` | `str` | OpenSSL cipher string |
+| `min_tls_version` | `str` | Minimum TLS version |
+| `max_tls_version` | `str` | Maximum TLS version |
 
 ### DNS Pinning & Request-Line Control
 
@@ -443,7 +448,9 @@ async def main():
 asyncio.run(main())
 ```
 
-`raw_connect()` takes all the same TLS knobs as `request()` — `verify_certs`, `cipher_string`, `min_tls_version`, `max_tls_version`, `resolve_ip`, `proxy`, `no_proxy`. `alpn_protocols` is a list of byte-strings (commonly `["h2", "http/1.1"]`) used in the TLS ALPN extension. After the handshake, `conn.negotiated_alpn` reports which one the server picked (or `None` if no ALPN was negotiated, including all plain-HTTP connections).
+`raw_connect()` takes all the same TLS knobs as `request()` — `verify_certs`, `profile`, `cipher_string`, `min_tls_version`, `max_tls_version`, `resolve_ip`, `proxy`, `no_proxy`. `alpn_protocols` is a list of byte-strings (commonly `["h2", "http/1.1"]`) used in the TLS ALPN extension. After the handshake, `conn.negotiated_alpn` reports which one the server picked (or `None` if no ALPN was negotiated, including all plain-HTTP connections).
+
+`raw_connect`, and requests using `resolve_ip` or `request_target`, widen their cipher offer and retry when a handshake fails in a way that says the peer could not negotiate, the same as an ordinary request. They stop there: the other half of the ladder reacts to a *refusal* by changing what the client claims to be, and these are the callers who asked for exact control over one request, so re-sending it dressed differently would answer a question they did not ask. Naming a `profile`, `cipher_string` or TLS version pins the configuration and nothing widens at all.
 
 `conn.peer_ip` returns the actual IP address the OS connected to, same as `Response.peer_ip`. `None` when the connection went through a proxy.
 
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 9ccface..749debc 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1175,18 +1175,117 @@ fn describe_tls_failure(failure: &TlsFailure) -> String {
     }
 }
 
+/// A connection that was established, and what it took to establish it.
+pub(crate) struct Connected {
+    pub stream: Box<dyn IoReadWrite + Send + Unpin>,
+    pub cert_info: Option<CertInfo>,
+    pub alpn: Option<String>,
+    pub peer_ip: Option<IpAddr>,
+    /// The profile the handshake finally succeeded under, which is not
+    /// necessarily the one asked for. `dispatch_direct` builds its request
+    /// from this so the headers cannot drift away from the TLS underneath
+    /// them, which is the failure mode a half-applied profile produces.
+    pub profile: &'static crate::profile::ConnectionProfile,
+    pub attempts: Vec<crate::report::Attempt>,
+}
+
 /// Opens a fresh TCP connection (optionally to a resolved IP instead of DNS)
 /// and performs TLS if HTTPS (with SNI set to the original hostname).
-/// Returns the connected stream and any certificate info collected during
-/// the TLS handshake.
 ///
 /// Shared setup used by `dispatch_direct` (one-shot hyper requests over an
 /// un-pooled socket) and by callers that need a long-lived, unframed handle
 /// to a TCP or TLS stream.
+///
+/// Widens the offer and tries again when a handshake fails in a way that says
+/// the peer could not negotiate, which is the same breadth rung the pooled
+/// path climbs. Reaching a server too old for the default is the reason this
+/// library carries its own OpenSSL, and without this these callers lost it
+/// when the default narrowed: a virtualhost sweep across an estate with one
+/// legacy appliance in it would simply not see the appliance.
+///
+/// It stops there. The pooled ladder has a second rung that reacts to a
+/// *refusal* by changing what the client claims to be, and that one would be
+/// wrong here. `resolve_ip` and `request_target` exist for probes whose point
+/// is how one exact request is answered, so re-sending a probe dressed
+/// differently answers a question nobody asked, doubles the requests, and
+/// muddies the result. A raw connection has no response to classify at all.
 pub(crate) async fn connect_stream(
     target_uri: &http::Uri,
     config: &RequestConfig,
     log: &DebugLog,
+) -> Result<Connected, ClientError> {
+    let mut rung = config.resolved_profile();
+    let mut tried: Vec<&'static str> = Vec::new();
+    let mut attempts: Vec<crate::report::Attempt> = Vec::new();
+
+    loop {
+        tried.push(rung.name);
+        let rung_config = config.with_profile(rung.name);
+
+        match connect_once(target_uri, &rung_config, log).await {
+            Ok((stream, cert_info, alpn, peer_ip)) => {
+                return Ok(Connected {
+                    stream,
+                    cert_info,
+                    alpn,
+                    peer_ip,
+                    profile: rung,
+                    attempts,
+                });
+            }
+            Err(e) => {
+                attempts.push(crate::report::Attempt::new(
+                    rung.name,
+                    crate::report::AttemptOutcome::HandshakeFailed {
+                        reason: e.message.clone(),
+                    },
+                ));
+
+                // A pinned config ends where the caller put it. Naming a
+                // cipher string, a TLS version or a profile means they meant
+                // it, and on this path more than any other: these are the
+                // callers who asked for exact control.
+                let widen = !config.tls_is_pinned()
+                    && tried.len() < crate::profile::MAX_RUNGS
+                    && e.tls_failure
+                        .as_ref()
+                        .is_some_and(|f| f.suggests_wider_offer());
+
+                match widen
+                    .then(|| {
+                        crate::profile::next_rung(rung, RungFailure::CouldNotNegotiate, &tried)
+                    })
+                    .flatten()
+                {
+                    Some(next) => {
+                        debug_record(
+                            log,
+                            config.verbosity,
+                            1,
+                            &format!(
+                                "   handshake failed under '{}', widening to '{}'",
+                                rung.name, next.name
+                            ),
+                        );
+                        rung = next;
+                    }
+                    None => {
+                        let mut e = e;
+                        e.attempts = attempts;
+                        return Err(e);
+                    }
+                }
+            }
+        }
+    }
+}
+
+/// One connection attempt under one profile. The retrying lives in
+/// `connect_stream`; this just does as it is told.
+async fn connect_once(
+    target_uri: &http::Uri,
+    config: &RequestConfig,
+    log: &DebugLog,
 ) -> Result<
     (
         Box<dyn IoReadWrite + Send + Unpin>,
@@ -1443,11 +1542,27 @@ async fn dispatch_direct(
     target_uri: &http::Uri,
     config: &RequestConfig,
     log: &DebugLog,
-) -> Result<(SingleResponse, Option<CertInfo>, Option<IpAddr>), ClientError> {
+) -> Result<
+    (
+        SingleResponse,
+        Option<CertInfo>,
+        Option<IpAddr>,
+        Vec<crate::report::Attempt>,
+    ),
+    ClientError,
+> {
     let v = config.verbosity;
-    let (stream, cert_info, alpn, peer_ip) = connect_stream(target_uri, config, log).await?;
-    let io = hyper_util::rt::TokioIo::new(stream);
-    let h2 = alpn.as_deref() == Some("h2");
+    let conn = connect_stream(target_uri, config, log).await?;
+    let (cert_info, peer_ip) = (conn.cert_info, conn.peer_ip);
+
+    // Build the request under whatever profile the handshake settled on, not
+    // whatever was asked for. If `connect_stream` widened to `compatibility`
+    // to reach an old server, the headers have to widen with it, or the
+    // request states something the connection underneath it contradicts.
+    let config = &config.with_profile(conn.profile.name);
+
+    let io = hyper_util::rt::TokioIo::new(conn.stream);
+    let h2 = conn.alpn.as_deref() == Some("h2");
 
     // HTTP/2 carries the target in `:path`, which hyper derives from the
     // request URI, so there is no request-line for `request_target` to
@@ -1530,7 +1645,23 @@ async fn dispatch_direct(
     })?;
 
     let resp = parse_response(hyper_response, config, log).await?;
-    Ok((resp, cert_info, peer_ip))
+
+    // `conn.attempts` holds the handshakes that failed on the way here. The
+    // one that worked is this response, and it can only be classified now
+    // that there is a body, so it is recorded here rather than in
+    // `connect_stream`, which never sees one.
+    let mut attempts = conn.attempts;
+    attempts.push(crate::report::Attempt::from_outcome(
+        conn.profile.name,
+        resp.status,
+        &crate::antibot::classify(&crate::antibot::ResponseFacts {
+            status: resp.status,
+            headers: &resp.headers,
+            body: &String::from_utf8_lossy(&resp.body_bytes),
+        }),
+    ));
+
+    Ok((resp, cert_info, peer_ip, attempts))
 }
 
 /// The two one-shot senders `dispatch_direct` can end up holding, picked by
@@ -2153,11 +2284,25 @@ impl HyperClient {
     /// hypothesis and could be the host having a bad minute; a success is
     /// evidence. Recording failures would let one flake pin a host to a worse
     /// profile for the rest of a scan.
+    ///
+    /// A pinned request records nothing, which matches `starting_profile`
+    /// refusing to read the memory when pinned. The memory is a cache of what
+    /// the ladder found out, and a caller naming a profile is not the ladder
+    /// finding anything out. Recording it would mean one deliberate
+    /// `profile="chrome"` request quietly turned every later request to that
+    /// host into a browser claim, which is the change most likely to make
+    /// things worse: PerimeterX passes an honest client and refuses a
+    /// half-convincing browser. It would also make the result depend on the
+    /// order two independent requests happened to run in.
     fn remember_profile(
         &self,
+        config: &RequestConfig,
         uri: &http::Uri,
         profile: &'static crate::profile::ConnectionProfile,
     ) {
+        if config.tls_is_pinned() {
+            return;
+        }
         if let Some(key) = peer_slot_key(uri)
             && let Ok(mut map) = self.host_profiles.lock()
         {
@@ -2205,7 +2350,7 @@ impl HyperClient {
         // Bypasses the cached connection pool — opens a fresh TCP connection.
         if config.resolve_ip.is_some() || config.request_target.is_some() {
             let redirect_chain: Vec<RedirectHop> = Vec::new();
-            let (resp, cert_info, peer_ip) = dispatch_direct(&uri, config, log).await?;
+            let (resp, cert_info, peer_ip, attempts) = dispatch_direct(&uri, config, log).await?;
             let hop_ms = start.elapsed().as_millis();
             debug_record(log, v, 1, &format!("<- {} ({}ms)", resp.status, hop_ms));
 
@@ -2219,10 +2364,7 @@ impl HyperClient {
                 status: resp.status,
                 headers: resp.headers,
                 body_bytes: resp.body_bytes,
-                // This path never ladders: it is the resolve_ip /
-                // request_target branch, which dispatches once and follows no
-                // redirects.
-                attempts: Vec::new(),
+                attempts,
                 elapsed_ms,
                 redirect_chain,
                 cert_info,
@@ -2307,7 +2449,7 @@ impl HyperClient {
                             // success. A failure is a hypothesis; this is
                             // evidence.
                             if outcome.got_through() {
-                                self.remember_profile(&uri, rung);
+                                self.remember_profile(config, &uri, rung);
                             }
                             break (resp, cached);
                         }
diff --git a/src/client/raw.rs b/src/client/raw.rs
index fad3a85..05f7177 100644
--- a/src/client/raw.rs
+++ b/src/client/raw.rs
@@ -36,13 +36,12 @@ impl RawConnection {
             ClientError::invalid_url(format!("invalid URL '{}': {}", uri_str, e))
         })?;
         let log = new_debug_log();
-        let (stream, cert_info, negotiated_alpn, peer_ip) =
-            connect_stream(&uri, config, &log).await?;
+        let conn = connect_stream(&uri, config, &log).await?;
         Ok(Self {
-            stream: Arc::new(Mutex::new(Some(stream))),
-            cert_info,
-            negotiated_alpn,
-            peer_ip,
+            stream: Arc::new(Mutex::new(Some(conn.stream))),
+            cert_info: conn.cert_info,
+            negotiated_alpn: conn.alpn,
+            peer_ip: conn.peer_ip,
         })
     }
 
diff --git a/src/mock.rs b/src/mock.rs
index 42aef54..bd270ea 100644
--- a/src/mock.rs
+++ b/src/mock.rs
@@ -669,9 +669,12 @@ impl PyBlasthttpMock {
     }
 
     /// Dispatch a single request through the mock. Mirrors the
-    /// `BlastHTTP.request` shape — most kwargs (timeout, verify_certs,
-    /// retries, etc.) are accepted but ignored, since they don't apply
-    /// to a mock.
+    /// `BlastHTTP.request` shape. Kwargs the mock has no use for (timeout,
+    /// verify_certs, profile, retries and the rest) are accepted and ignored
+    /// when the URL is intercepted, since nothing is dialled. When the URL is
+    /// excluded and the call goes to the real client they are forwarded
+    /// verbatim, because then they describe a connection that actually
+    /// happens.
     #[pyo3(signature = (
         url,
         method=None,
@@ -705,6 +708,7 @@ impl PyBlasthttpMock {
                 files,
                 follow_redirects,
                 max_redirects,
+                _kwargs,
             );
         }
 
@@ -951,6 +955,7 @@ impl PyBlasthttpMock {
         files: Option<Bound<'py, PyAny>>,
         follow_redirects: Option<bool>,
         max_redirects: Option<u32>,
+        rest: Option<Bound<'py, PyDict>>,
     ) -> PyResult<Bound<'py, PyAny>> {
         let Some(ref client) = self.real_client else {
             return Err(PyRuntimeError::new_err(format!(
@@ -958,6 +963,18 @@ impl PyBlasthttpMock {
             )));
         };
         let kwargs = PyDict::new(py);
+        // Everything the mock does not name itself goes through untouched.
+        // A mocked URL can ignore `timeout`, `profile` or `verify_certs`
+        // because nothing is dialled, but a passed-through one is a real
+        // request and dropping them here would quietly give it different
+        // TLS, a different profile and no timeout from what the caller
+        // asked for. Set first, so the named arguments below win on the
+        // (impossible from Python, but cheap to guarantee) chance of overlap.
+        if let Some(ref extra) = rest {
+            for (k, val) in extra.iter() {
+                kwargs.set_item(k, val)?;
+            }
+        }
         if let Some(m) = method {
             kwargs.set_item("method", m)?;
         }
diff --git a/src/python.rs b/src/python.rs
index 8f83161..39c355a 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -1280,6 +1280,10 @@ impl BlastHTTP {
     /// Download a URL directly to a local file.
     /// Returns the file path on success.
     /// max_size: maximum bytes to download (None = no limit, uses default 10MB)
+    ///
+    /// Takes the same connection knobs as `request`. Without them the profile
+    /// ladder runs as usual, so a download from a host that needs legacy TLS
+    /// or a browser profile reaches it the same way an ordinary request does.
     #[pyo3(signature = (
         url,
         path,
@@ -1291,6 +1295,10 @@ impl BlastHTTP {
         headers=None,
         retries=None,
         redirect_cookies=None,
+        profile=None,
+        cipher_string=None,
+        min_tls_version=None,
+        max_tls_version=None,
     ))]
     #[allow(clippy::too_many_arguments)]
     fn download<'py>(
@@ -1306,6 +1314,10 @@ impl BlastHTTP {
         headers: Option<Vec<(String, String)>>,
         retries: Option<u32>,
         redirect_cookies: Option<bool>,
+        profile: Option<String>,
+        cipher_string: Option<String>,
+        min_tls_version: Option<String>,
+        max_tls_version: Option<String>,
     ) -> PyResult<Bound<'py, PyAny>> {
         let config = RequestConfig {
             url,
@@ -1320,10 +1332,10 @@ impl BlastHTTP {
             verify_certs,
             proxy,
             no_proxy: no_proxy.unwrap_or_default(),
-            profile: None,
-            cipher_string: None,
-            min_tls_version: None,
-            max_tls_version: None,
+            profile,
+            cipher_string,
+            min_tls_version,
+            max_tls_version,
             retries,
             retry_wait_min_ms: None,
             retry_wait_max_ms: None,
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index 16ce74c..ad2e2d2 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -302,3 +302,193 @@ async fn test_a_cold_burst_rediscovers_per_request() {
 
     server.shutdown().await;
 }
+
+// ── The same promise on the direct and raw paths ──────────────────
+//
+// `resolve_ip`, `request_target` and `raw_connect` do not go through the
+// pooled client, so they did not go through the ladder either. That was fine
+// while the default offered every cipher OpenSSL has; once it narrowed to
+// eleven, these callers quietly lost the legacy reach the custom build exists
+// to give them. A virtualhost sweep is exactly where an old appliance turns
+// up, and exactly where it would have gone unseen.
+
+#[tokio::test]
+async fn test_resolve_ip_reaches_a_legacy_server() {
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let mut config = default_config(&server.url());
+    config.resolve_ip = Some("127.0.0.1".to_string());
+
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "resolve_ip should reach an RC4-only server the same way the pooled \
+         path does: {:?}",
+        result.as_ref().err()
+    );
+    let resp = result.unwrap();
+    assert_eq!(resp.status, 200);
+
+    // And it should say how it got there. Two rungs: the default could not
+    // negotiate, the wider one could.
+    let profiles: Vec<&str> = resp.attempts.iter().map(|a| a.profile.as_str()).collect();
+    assert_eq!(
+        profiles,
+        vec!["modern", "compatibility"],
+        "expected the report to show the widening, got {profiles:?}"
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_request_target_reaches_a_legacy_server() {
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let mut config = default_config(&server.url());
+    config.request_target = Some("/".to_string());
+
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_ok(),
+        "request_target takes the same un-pooled path and needs the same \
+         reach: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_raw_connect_reaches_a_legacy_server() {
+    // A raw connection has no response to classify, so only the cipher half
+    // of the ladder can apply to it. That half is the half it needs: byte-
+    // level work against an old appliance starts with reaching the appliance.
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let url = server.url();
+    let config = default_config(&url);
+    let result = blasthttp::client::raw::RawConnection::connect(&url, &config).await;
+
+    assert!(
+        result.is_ok(),
+        "raw_connect should reach an RC4-only server with no cipher_string: \
+         {:?}",
+        result.as_ref().err()
+    );
+
+    // Write a request and read the answer, rather than stopping at "the
+    // handshake returned Ok". A connection that cannot carry bytes is not a
+    // connection, and the harness serves each connection on its own accept
+    // loop iteration, so leaving one open with nothing written parks the
+    // server in a read that never returns and the shutdown below deadlocks.
+    let conn = result.unwrap();
+    conn.send_bytes(b"GET / HTTP/1.1\r\nHost: 127.0.0.1\r\nConnection: close\r\n\r\n")
+        .await
+        .expect("write over the RC4 connection");
+    let data = conn
+        .read_raw(4096, Some(5000))
+        .await
+        .expect("read back over the RC4 connection");
+    assert!(
+        data.starts_with(b"HTTP/1.1 200"),
+        "expected a response over the raw connection, got {:?}",
+        String::from_utf8_lossy(&data[..data.len().min(80)])
+    );
+    let _ = conn.close().await;
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_the_direct_path_does_not_widen_past_a_pin() {
+    // The counterpart, and the more important one on this path. These are the
+    // callers who asked for exact control, so a named cipher string has to be
+    // the last word: widening past it would put suites on the wire the caller
+    // deliberately left off, and a TLS enumeration sweep would report every
+    // server as speaking everything.
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let mut config = default_config(&server.url());
+    config.resolve_ip = Some("127.0.0.1".to_string());
+    config.cipher_string = Some("AES128-SHA".to_string());
+
+    let client = HyperClient::new();
+    let result = client.send(&config).await;
+
+    assert!(
+        result.is_err(),
+        "a pinned cipher string that the server does not speak must fail \
+         rather than be widened behind the caller's back"
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_a_pinned_request_does_not_teach_the_host_memory() {
+    // The memory is a cache of what the ladder worked out, so a request that
+    // never laddered has nothing to contribute to it.
+    //
+    // Found by a mock test rather than reasoning: a passthrough request with
+    // profile="chrome" was followed by one with no profile, and the second
+    // sent Chrome's headers. One deliberate browser request had silently
+    // converted the host for the rest of the client's life. That is the worst
+    // direction for it to drift in, because claiming to be a browser is what
+    // invites a detector to check the claim, and it made the outcome depend
+    // on the order two unrelated requests ran in.
+    let server = TlsTestServer::start(TlsServerConfig {
+        min_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let client = HyperClient::new();
+
+    let mut pinned = default_config(&server.url());
+    pinned.profile = Some("chrome".to_string());
+    let first = client.send(&pinned).await.expect("pinned request");
+    assert_eq!(
+        first.attempts.first().map(|a| a.profile.as_str()),
+        Some("chrome131"),
+        "the pinned request should have used what it named"
+    );
+
+    let second = client
+        .send(&default_config(&server.url()))
+        .await
+        .expect("unpinned request");
+    assert_eq!(
+        second.attempts.first().map(|a| a.profile.as_str()),
+        Some("modern"),
+        "an unpinned request should start at the default, not inherit the \
+         profile a previous pinned request named"
+    );
+
+    server.shutdown().await;
+}
diff --git a/tests/python/test_mock.py b/tests/python/test_mock.py
index 6e40b49..1b0d010 100644
--- a/tests/python/test_mock.py
+++ b/tests/python/test_mock.py
@@ -388,3 +388,57 @@ async def handler(reader, writer):
     finally:
         server.close()
         await server.wait_closed()
+
+
+@pytest.mark.asyncio
+async def test_passthrough_forwards_the_kwargs_it_ignores():
+    """A mocked URL can ignore `profile` because nothing is dialled. A
+    passed-through one cannot: it is a real connection, and dropping the
+    kwarg silently gives it different TLS and different headers from what
+    the caller asked for. BBOT runs its whole local-target suite through
+    this path, so a dropped kwarg there is a test that passes while
+    exercising the wrong thing."""
+
+    async def handler(reader, writer):
+        received = []
+        await reader.readuntil(b"\r\n")
+        while (line := await reader.readuntil(b"\r\n")) != b"\r\n":
+            received.append(line.decode("latin1").rstrip("\r\n"))
+        body = "\n".join(received).encode()
+        writer.write(
+            b"HTTP/1.1 200 OK\r\nContent-Length: "
+            + str(len(body)).encode()
+            + b"\r\n\r\n"
+            + body
+        )
+        await writer.drain()
+        writer.close()
+
+    server = await asyncio.start_server(handler, "127.0.0.1", 0)
+    addr = server.sockets[0].getsockname()
+    base = f"http://{addr[0]}:{addr[1]}/"
+    try:
+        real = blasthttp.BlastHTTP()
+        mock = BlasthttpMock(real_client=real, should_mock_fn=lambda host: host != "127.0.0.1")
+
+        r = await mock.request(base, profile="chrome")
+        assert "Mozilla/5.0" in r.text, f"profile was dropped on the way through: {r.text}"
+
+        # And the default still arrives as itself, so the test above is
+        # measuring the kwarg rather than a client that always sends Chrome.
+        r2 = await mock.request(base)
+        assert "blasthttp/" in r2.text
+    finally:
+        server.close()
+        await server.wait_closed()
+
+
+@pytest.mark.asyncio
+async def test_passthrough_rejects_a_kwarg_the_real_client_does_not_take():
+    """The flip side of forwarding. An intercepted URL swallows anything,
+    which is right when nothing is dialled, but on the way through the real
+    client's signature is the one that counts and a typo should be audible."""
+    real = blasthttp.BlastHTTP()
+    mock = BlasthttpMock(real_client=real, should_mock_fn=lambda host: host != "127.0.0.1")
+    with pytest.raises(TypeError):
+        await mock.request("http://127.0.0.1:1/", not_a_real_kwarg=1)
diff --git a/tests/python/test_profiles_and_report.py b/tests/python/test_profiles_and_report.py
index 2db2deb..c47bc8b 100644
--- a/tests/python/test_profiles_and_report.py
+++ b/tests/python/test_profiles_and_report.py
@@ -228,3 +228,31 @@ async def test_batch_results_carry_the_report(server, client):
         "needs_browser",
         "cloudflare",
     )
+
+
+# ── download ─────────────────────────────────────────────────────
+
+
+async def test_download_takes_a_profile(server, client, tmp_path):
+    """`download` used to hardcode no profile with no way to pass one, so a
+    file behind a host that only answers a browser was unreachable through it
+    while the identical URL through `request` was fine."""
+    dest = tmp_path / "headers.txt"
+    await client.download(f"{server}/echo-headers", str(dest), profile="chrome")
+    assert "Mozilla/5.0" in dest.read_text()
+
+
+async def test_download_defaults_to_the_honest_profile(server, client, tmp_path):
+    """Which makes the test above about the argument rather than about a
+    client that always claims to be Chrome."""
+    dest = tmp_path / "headers.txt"
+    await client.download(f"{server}/echo-headers", str(dest))
+    text = dest.read_text()
+    assert "blasthttp/" in text
+    assert "Mozilla/5.0" not in text
+
+
+async def test_download_rejects_an_unknown_profile(server, client, tmp_path):
+    with pytest.raises(Exception) as exc:
+        await client.download(f"{server}/", str(tmp_path / "x"), profile="chrmoe")
+    assert "chrmoe" in str(exc.value).lower()

From 71e9b18cc4006a636592ffa1856eececa6272e8b Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Wed, 30 Sep 2026 09:34:21 -0400
Subject: [PATCH 19/44] Share host discovery across a burst by stepping aside,
 not waiting

Second attempt at the burst cost, after 614a3c3 was backed out in
d1d0180. The problem is unchanged: at the start of a batch the per-host
memory is empty, so every concurrent request to one host separately
finds the default refused and separately shifts. Six requests, twelve
handshakes, where seven would do.

The first attempt made the others wait on a per-host lock. It cut the
handshakes and was slower anyway, because a waiter cannot be released
until the leader has classified its response, classification needs the
body, and so the wait is a whole request long. It also broke the rule
that a slow request never holds up faster ones, which is the entire
promise of send_batch_stream.

This does not wait. A request whose host is already being discovered
gives up its slot, lets a request for a different host have it, and
goes once there is an answer. Nothing idles, so the saving costs no
time.

The condition is the whole design and it comes from the test that
killed the lock: six requests, one host, concurrency six. Step aside
there and you are standing in the street, because every other request
wants the same host. So a request only steps aside when there is
undispatched work for a different host. When there is not, it goes and
discovers the host again, which is the cheaper mistake. That makes this
strictly better or equal to having no scheduler, never worse, which is
what the lock failed to be.

Measured against www.akamai.com, six concurrent requests:

  alone                    12 handshakes   (unchanged, correctly)
  mixed with another host   7 handshakes

Seven is the floor: one request discovers, five are told.

Details worth keeping:

The decision and the commit happen under one lock. Deciding and then
committing separately lets two requests both look, both see nobody
discovering, and both go.

The gate is a watch rather than a Notify, because the answer can land
between a request reading the state and awaiting on it, and a watch
carries the value so that request sees it instead of sleeping until its
timeout. It sends with send_replace, since send refuses when nothing is
subscribed and leaves the value alone, which is the common case: the
leader usually finishes before anyone has stepped aside. A unit test
caught that one.

The leader's guard marks the host answered on drop rather than on
success, so a request that fails, times out or panics releases whoever
stepped aside for it.

There is a ceiling on how many requests may be standing aside at once,
set to the concurrency limit. A request standing aside holds no permit,
so the stream driver spawns another in its place; without a ceiling a
batch alternating between two slow hosts would spawn task after task,
each handing its permit straight back, until the whole batch was
resident.

send_batch steps aside before taking a permit. The stream driver hands
the permit over with the request, so there it is given back for the
wait and retaken after.
---
 CHANGELOG.md             |   8 +
 src/batch.rs             | 441 +++++++++++++++++++++++++++++++++++++++
 tests/burst_scheduler.rs | 156 ++++++++++++++
 tests/legacy_default.rs  |  38 ++--
 4 files changed, 621 insertions(+), 22 deletions(-)
 create mode 100644 tests/burst_scheduler.rs

diff --git a/CHANGELOG.md b/CHANGELOG.md
index c2e7e68..23ea62f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -47,6 +47,14 @@ the ladder falling back to `compatibility` on the second handshake.
   default offers nothing that exists in SSLv3
 - `BLASTHTTP_BISECT=tls,headers,http2` disables profile layers independently,
   for finding out which one a detector is reacting to
+- Batches no longer have every request in an opening burst work out the same
+  host's profile separately. A request whose host is already being discovered
+  stands aside and lets a request for a *different* host take its slot, then
+  goes once there is an answer. Six requests to a host needing a shift went
+  from twelve handshakes to seven, with nothing idling: standing aside only
+  happens when there is undispatched work elsewhere, so when a batch is all
+  one host every request goes immediately and discovers it again, which is the
+  cheaper mistake. A slow request still never holds up faster ones behind it
 - `raw_connect`, and requests using `resolve_ip` or `request_target`, widen
   their cipher offer and retry on a handshake the peer could not negotiate,
   the same as an ordinary request. These bypass the pooled client, so they
diff --git a/src/batch.rs b/src/batch.rs
index 25a0b55..38ebb2c 100644
--- a/src/batch.rs
+++ b/src/batch.rs
@@ -136,6 +136,262 @@ fn merge_limiters(
     }
 }
 
+// ── Host discovery scheduler ──────────────────────────────────────
+
+/// Stops a burst of requests to one host from each separately working out
+/// which connection profile that host wants.
+///
+/// The problem. The client remembers, per host, the profile that got through,
+/// so a scan pays for the ladder once rather than once per request. At the
+/// start of a batch that has not been paid yet: if fifty requests to one host
+/// go out together, all fifty find the default refused and all fifty shift,
+/// a hundred handshakes where fifty one would have done.
+///
+/// What does not work, and this was built and measured before being thrown
+/// away: a per-host lock, so the first request discovers and the rest wait on
+/// it. It does cut the handshakes, and it is slower anyway, because a waiter
+/// cannot be released until the leader has classified its response,
+/// classification needs the body, and so the wait is a whole request long. A
+/// protection product does not refuse the handshake; it takes the connection
+/// and then answers 403, which puts the one fact worth knowing in the last
+/// byte to arrive. Waiting also broke the rule that a slow request never
+/// holds up faster ones behind it, which is the entire promise of
+/// `send_batch_stream`.
+///
+/// What works instead is to not wait. A request whose host is already being
+/// worked out steps aside and lets a request for a *different* host have its
+/// slot, then goes once there is an answer. Nothing idles, so the saving
+/// costs no time.
+///
+/// The condition is the whole design, and it comes straight from the test
+/// that killed the lock: six requests, one host, concurrency six. Step aside
+/// there and you are standing in the street, because every other request
+/// wants the same host. So: **only step aside when there is undispatched work
+/// for a different host.** When there is not, go and discover it yourself and
+/// accept the duplicate. That makes this strictly better or equal to having
+/// no scheduler at all, never worse, which is what the lock failed to be.
+struct HostGate {
+    /// Someone is making the first request to this host, so an answer is
+    /// coming.
+    claimed: bool,
+    /// Flips once a request to this host finishes. A `watch` rather than a
+    /// `Notify` because the answer can land between a waiter reading the
+    /// state and awaiting on it, and a watch carries the value so that
+    /// waiter sees it instead of sleeping until its timeout.
+    answered: tokio::sync::watch::Sender<bool>,
+}
+
+struct SchedulerState {
+    gates: std::collections::HashMap<String, HostGate>,
+    /// Requests not yet dispatched, in total and per host. Together these
+    /// answer the only question the scheduler asks: is there work for a
+    /// different host that could use the slot I would otherwise sit in?
+    remaining_total: usize,
+    remaining_by_host: std::collections::HashMap<String, usize>,
+    /// How many requests are standing aside right now, and the ceiling on
+    /// that. A request that has stepped aside holds no permit, so it stops
+    /// counting against concurrency and the driver spawns another in its
+    /// place. Without a ceiling, a batch alternating between two slow hosts
+    /// would have the driver spawn task after task, each stepping aside and
+    /// freeing the permit for the next, until the whole batch was resident.
+    /// The ceiling keeps live tasks inside the envelope the stream already
+    /// had.
+    aside: usize,
+    aside_limit: usize,
+}
+
+pub(crate) struct Scheduler {
+    state: std::sync::Mutex<SchedulerState>,
+}
+
+/// What a request should do about its host.
+pub(crate) enum Dispatch {
+    /// Go, and you are the one discovering this host. Hold the guard for the
+    /// request.
+    Lead(LeaderGuard),
+    /// Go. Either the host is already answered, or there was nothing else to
+    /// do with the slot so discovering it twice is the cheaper mistake.
+    Go,
+    /// Stand aside until this says true, then ask again.
+    Aside(tokio::sync::watch::Receiver<bool>, AsideGuard),
+}
+
+/// Held for the duration of the first request to a host. Marks the host
+/// answered when dropped, so a request that fails, times out or panics
+/// releases whoever stepped aside for it instead of leaving them on the
+/// timeout.
+pub(crate) struct LeaderGuard {
+    sched: Arc<Scheduler>,
+    host: String,
+}
+
+impl Drop for LeaderGuard {
+    fn drop(&mut self) {
+        if let Ok(mut st) = self.sched.state.lock()
+            && let Some(gate) = st.gates.get_mut(&self.host)
+        {
+            // `send` refuses when nothing is subscribed and leaves the value
+            // alone, which is the common case here: usually the leader
+            // finishes with nobody yet standing aside. The answer would then
+            // be lost, and the next request to this host would stand aside
+            // waiting for news that had already been and gone.
+            gate.answered.send_replace(true);
+        }
+    }
+}
+
+/// Held while a request stands aside, so the ceiling is released however the
+/// wait ends.
+pub(crate) struct AsideGuard {
+    sched: Arc<Scheduler>,
+}
+
+impl Drop for AsideGuard {
+    fn drop(&mut self) {
+        if let Ok(mut st) = self.sched.state.lock() {
+            st.aside = st.aside.saturating_sub(1);
+        }
+    }
+}
+
+/// Host and port, matching how the client keys its per-host profile memory.
+/// A URL that will not parse gets a bucket under its own text, which just
+/// means it coordinates with nothing.
+fn host_key(url: &str) -> String {
+    let Ok(uri) = url.parse::<http::Uri>() else {
+        return url.to_string();
+    };
+    match uri.host() {
+        Some(h) => {
+            let default_port = if uri.scheme_str() == Some("https") {
+                443
+            } else {
+                80
+            };
+            format!("{}:{}", h, uri.port_u16().unwrap_or(default_port))
+        }
+        None => url.to_string(),
+    }
+}
+
+impl Scheduler {
+    fn new(configs: &[RequestConfig], concurrency: usize) -> Arc<Self> {
+        let mut remaining_by_host: std::collections::HashMap<String, usize> =
+            std::collections::HashMap::new();
+        for c in configs {
+            *remaining_by_host.entry(host_key(&c.url)).or_insert(0) += 1;
+        }
+        Arc::new(Scheduler {
+            state: std::sync::Mutex::new(SchedulerState {
+                gates: std::collections::HashMap::new(),
+                remaining_total: configs.len(),
+                remaining_by_host,
+                aside: 0,
+                aside_limit: concurrency.max(1),
+            }),
+        })
+    }
+
+    /// Decide and commit in one lock, because deciding and then committing
+    /// separately lets two requests both look, both see no one discovering,
+    /// and both go: the duplicate this exists to prevent.
+    ///
+    /// `may_stand_aside` is false on the second ask, after a wait, so a
+    /// request stands aside at most once and cannot be starved by a host
+    /// that keeps being re-claimed.
+    fn dispatch(self: &Arc<Self>, host: &str, may_stand_aside: bool) -> Dispatch {
+        let Ok(mut st) = self.state.lock() else {
+            return Dispatch::Go;
+        };
+
+        if may_stand_aside && st.aside < st.aside_limit {
+            let busy = st
+                .gates
+                .get(host)
+                .is_some_and(|g| g.claimed && !*g.answered.borrow());
+            let mine = st.remaining_by_host.get(host).copied().unwrap_or(0);
+            let elsewhere = st.remaining_total.saturating_sub(mine) > 0;
+            if busy && elsewhere {
+                let rx = st.gates[host].answered.subscribe();
+                st.aside += 1;
+                return Dispatch::Aside(
+                    rx,
+                    AsideGuard {
+                        sched: self.clone(),
+                    },
+                );
+            }
+        }
+
+        // Going, so take the slot.
+        st.remaining_total = st.remaining_total.saturating_sub(1);
+        if let Some(n) = st.remaining_by_host.get_mut(host) {
+            *n = n.saturating_sub(1);
+        }
+        let gate = st
+            .gates
+            .entry(host.to_string())
+            .or_insert_with(|| HostGate {
+                claimed: false,
+                answered: tokio::sync::watch::Sender::new(false),
+            });
+        if gate.claimed || *gate.answered.borrow() {
+            return Dispatch::Go;
+        }
+        gate.claimed = true;
+        Dispatch::Lead(LeaderGuard {
+            sched: self.clone(),
+            host: host.to_string(),
+        })
+    }
+}
+
+/// How long a request stands aside before giving up and going anyway.
+///
+/// Only a backstop. The leader's guard releases waiters the moment its
+/// request ends, errors included, so this fires when a leader is wedged in a
+/// way its own timeout did not catch. Sized off the request timeout, since a
+/// leader cannot legitimately outlive that.
+fn step_aside_limit(config: &RequestConfig) -> Duration {
+    Duration::from_secs(config.timeout().clamp(1, 60))
+}
+
+/// Stand aside if asked to, then report whether this request is the one
+/// discovering its host.
+///
+/// Takes the caller's concurrency permit, if it is already holding one, and
+/// gives it back only if it never stood aside. The two callers differ on
+/// exactly this: `send_batch` has not taken a permit yet and passes `None`,
+/// while the stream driver hands one over with the request and needs it
+/// released for the wait, since a request standing aside is not in flight
+/// and holding a slot as though it were is the lock all over again.
+async fn await_turn(
+    sched: &Arc<Scheduler>,
+    host: &str,
+    config: &RequestConfig,
+    permit: Option<tokio::sync::OwnedSemaphorePermit>,
+) -> (
+    Option<LeaderGuard>,
+    Option<tokio::sync::OwnedSemaphorePermit>,
+) {
+    match sched.dispatch(host, true) {
+        Dispatch::Lead(g) => (Some(g), permit),
+        Dispatch::Go => (None, permit),
+        Dispatch::Aside(mut rx, aside) => {
+            drop(permit);
+            if !*rx.borrow() {
+                let _ = tokio::time::timeout(step_aside_limit(config), rx.changed()).await;
+            }
+            drop(aside);
+            let lead = match sched.dispatch(host, false) {
+                Dispatch::Lead(g) => Some(g),
+                _ => None,
+            };
+            (lead, None)
+        }
+    }
+}
+
 pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
     client: Arc<C>,
     configs: Vec<RequestConfig>,
@@ -145,6 +401,7 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
 ) -> Vec<BatchResult> {
     let semaphore = Arc::new(tokio::sync::Semaphore::new(concurrency));
     let limiter = merge_limiters(shared_limiter, rate_limit);
+    let sched = Scheduler::new(&configs, concurrency);
     let mut handles = Vec::new();
 
     for config in configs {
@@ -156,8 +413,15 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
         let client = client.clone();
         let permit = semaphore.clone();
         let url = config.url.clone();
+        let sched = sched.clone();
 
         let handle = tokio::spawn(async move {
+            // Stand aside before taking a permit rather than after, so the
+            // slot is genuinely free while we are standing there. Waiting
+            // with a permit in hand would be the lock this replaces.
+            let host = host_key(&config.url);
+            let (_lead, _) = await_turn(&sched, &host, &config, None).await;
+
             let _permit = permit.acquire().await.unwrap();
             let result = client.send(&config).await;
             BatchResult { url, result }
@@ -217,6 +481,7 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
 ) -> impl Stream<Item = BatchResult> + Send + 'static {
     let limiter = merge_limiters(shared_limiter, rate_limit);
     let semaphore = Arc::new(tokio::sync::Semaphore::new(concurrency));
+    let sched = Scheduler::new(&configs, concurrency);
 
     stream::once(async move {
         let (tx, rx) = futures::channel::mpsc::unbounded::<BatchResult>();
@@ -232,7 +497,22 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
                 };
                 let client = client.clone();
                 let tx = tx.clone();
+                let sched = sched.clone();
+                let semaphore = semaphore.clone();
                 tokio::spawn(async move {
+                    let host = host_key(&config.url);
+                    // The driver holds the permit here, unlike `send_batch`,
+                    // so standing aside means handing it back. The driver is
+                    // parked on the next acquire and takes it immediately,
+                    // which is what sends the slot to another host. Dispatch
+                    // order is untouched; only this one request is late.
+                    let (_lead, permit) = await_turn(&sched, &host, &config, Some(permit)).await;
+                    // Re-take a slot only if we gave ours up.
+                    let permit = match permit {
+                        Some(p) => Some(p),
+                        None => semaphore.acquire_owned().await.ok(),
+                    };
+
                     let _permit = permit;
                     let url = config.url.clone();
                     let result = client.send(&config).await;
@@ -648,3 +928,164 @@ mod tests {
         );
     }
 }
+
+#[cfg(test)]
+mod scheduler_tests {
+    use super::*;
+
+    fn cfgs(urls: &[&str]) -> Vec<RequestConfig> {
+        urls.iter()
+            .map(|u| RequestConfig::new(u.to_string()))
+            .collect()
+    }
+
+    fn is_aside(d: &Dispatch) -> bool {
+        matches!(d, Dispatch::Aside(..))
+    }
+
+    #[test]
+    fn test_one_host_never_stands_aside() {
+        // The case that killed the previous attempt, as a unit test. Six
+        // requests to one host: standing aside means standing in the street,
+        // because there is no other host to hand the slot to. Every one of
+        // them has to go, even though that means discovering the host six
+        // times over.
+        let sched = Scheduler::new(&cfgs(&["https://a.test/"; 6]), 6);
+        let host = host_key("https://a.test/");
+
+        let first = sched.dispatch(&host, true);
+        assert!(matches!(first, Dispatch::Lead(_)), "first one leads");
+
+        for i in 0..5 {
+            let d = sched.dispatch(&host, true);
+            assert!(!is_aside(&d), "request {i} should have gone, not waited");
+        }
+    }
+
+    #[test]
+    fn test_another_host_waiting_is_what_makes_it_worth_stepping_aside() {
+        // Same shape, one difference: there is work for a different host, so
+        // the slot this request would have sat in has somewhere to go.
+        let sched = Scheduler::new(
+            &cfgs(&["https://a.test/", "https://a.test/", "https://b.test/"]),
+            4,
+        );
+        let a = host_key("https://a.test/");
+
+        // Bound, not matched in place: the guard marks the host answered when
+        // it drops, so letting it die at the end of the statement would mean
+        // the discovering request had already finished.
+        let lead = sched.dispatch(&a, true);
+        assert!(matches!(lead, Dispatch::Lead(_)));
+        assert!(
+            is_aside(&sched.dispatch(&a, true)),
+            "b.test is still queued"
+        );
+    }
+
+    #[test]
+    fn test_the_second_ask_always_goes() {
+        // After standing aside once a request goes regardless, so a host that
+        // keeps being re-claimed cannot starve the requests behind it.
+        let sched = Scheduler::new(
+            &cfgs(&["https://a.test/", "https://a.test/", "https://b.test/"]),
+            4,
+        );
+        let a = host_key("https://a.test/");
+
+        let _lead = sched.dispatch(&a, true);
+        assert!(is_aside(&sched.dispatch(&a, true)));
+        assert!(!is_aside(&sched.dispatch(&a, false)), "the second ask goes");
+    }
+
+    #[test]
+    fn test_an_answered_host_is_not_worth_waiting_for() {
+        let configs = cfgs(&["https://a.test/", "https://a.test/", "https://b.test/"]);
+        let sched = Scheduler::new(&configs, 4);
+        let a = host_key("https://a.test/");
+
+        let lead = sched.dispatch(&a, true);
+        assert!(matches!(lead, Dispatch::Lead(_)));
+        drop(lead); // the discovering request finished
+
+        assert!(
+            !is_aside(&sched.dispatch(&a, true)),
+            "there is an answer now, so nothing to wait for"
+        );
+    }
+
+    #[test]
+    fn test_a_leader_that_dies_releases_the_host() {
+        // The guard marks the host answered on drop rather than on success,
+        // so a request that errors or panics does not leave everyone who
+        // stepped aside for it sitting on the timeout.
+        let configs = cfgs(&["https://a.test/", "https://a.test/", "https://b.test/"]);
+        let sched = Scheduler::new(&configs, 4);
+        let a = host_key("https://a.test/");
+
+        let Dispatch::Lead(lead) = sched.dispatch(&a, true) else {
+            panic!("expected to lead");
+        };
+        let Dispatch::Aside(rx, _guard) = sched.dispatch(&a, true) else {
+            panic!("expected to stand aside");
+        };
+        assert!(!*rx.borrow());
+
+        drop(lead);
+        assert!(
+            *rx.borrow(),
+            "dropping the leader should release the waiter"
+        );
+    }
+
+    #[test]
+    fn test_the_ceiling_bounds_how_many_can_be_standing_aside() {
+        // Without this, a batch alternating between two slow hosts has the
+        // driver spawn a task, watch it hand the permit straight back, spawn
+        // another, and so on until the whole batch is resident.
+        let mut urls = vec!["https://a.test/"; 10];
+        urls.push("https://b.test/");
+        let sched = Scheduler::new(&cfgs(&urls), 2);
+        let a = host_key("https://a.test/");
+
+        let _lead = sched.dispatch(&a, true);
+        let _one = sched.dispatch(&a, true);
+        let _two = sched.dispatch(&a, true);
+        assert!(
+            is_aside(&_one) && is_aside(&_two),
+            "both fit under a ceiling of 2"
+        );
+        assert!(
+            !is_aside(&sched.dispatch(&a, true)),
+            "the third should go rather than pile up"
+        );
+    }
+
+    #[test]
+    fn test_a_freed_slot_lets_another_stand_aside() {
+        let mut urls = vec!["https://a.test/"; 10];
+        urls.push("https://b.test/");
+        let sched = Scheduler::new(&cfgs(&urls), 1);
+        let a = host_key("https://a.test/");
+
+        let _lead = sched.dispatch(&a, true);
+        let waiting = sched.dispatch(&a, true);
+        assert!(is_aside(&waiting));
+        assert!(!is_aside(&sched.dispatch(&a, true)), "ceiling of 1 is full");
+
+        drop(waiting);
+        assert!(is_aside(&sched.dispatch(&a, true)), "and free again after");
+    }
+
+    #[test]
+    fn test_host_key_ignores_the_path_and_fills_in_the_port() {
+        assert_eq!(
+            host_key("https://a.test/one"),
+            host_key("https://a.test/two")
+        );
+        assert_eq!(host_key("https://a.test/"), "a.test:443");
+        assert_eq!(host_key("http://a.test/"), "a.test:80");
+        assert_eq!(host_key("https://a.test:8443/"), "a.test:8443");
+        assert_ne!(host_key("https://a.test/"), host_key("https://b.test/"));
+    }
+}
diff --git a/tests/burst_scheduler.rs b/tests/burst_scheduler.rs
new file mode 100644
index 0000000..1f51676
--- /dev/null
+++ b/tests/burst_scheduler.rs
@@ -0,0 +1,156 @@
+// Does a burst against one host stop re-discovering that host once there is
+// other work to get on with?
+//
+// The ladder trigger here is an RC4-only server rather than a protection
+// product: a stock client offers `modern`, cannot negotiate, and widens to
+// `compatibility`. Two attempts, visible in `Response::attempts`, and no
+// vendor spoofing needed. What the scheduler does with them is the same
+// either way.
+
+mod support;
+
+use blasthttp::batch::send_batch;
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::config::RequestConfig;
+use openssl::ssl::SslVersion;
+use std::sync::Arc;
+use support::tls_server::{TlsServerConfig, TlsTestServer};
+
+fn config(url: &str) -> RequestConfig {
+    let mut c = RequestConfig::new(url.to_string());
+    c.verify_certs = Some(false);
+    c.timeout_seconds = Some(5);
+    c
+}
+
+async fn legacy_server() -> TlsTestServer {
+    TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await
+}
+
+async fn modern_server() -> TlsTestServer {
+    TlsTestServer::start(TlsServerConfig {
+        min_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await
+}
+
+fn attempts(results: &[blasthttp::batch::BatchResult], host: &str) -> usize {
+    results
+        .iter()
+        .filter(|r| r.url.contains(host))
+        .filter_map(|r| r.result.as_ref().ok())
+        .map(|r| r.attempts.len())
+        .sum()
+}
+
+fn reached(results: &[blasthttp::batch::BatchResult]) -> usize {
+    results
+        .iter()
+        .filter(|r| r.result.as_ref().is_ok_and(|resp| resp.status == 200))
+        .count()
+}
+
+#[tokio::test]
+async fn test_a_burst_discovers_a_host_once_when_there_is_other_work() {
+    let legacy = legacy_server().await;
+    let modern = modern_server().await;
+
+    let legacy_host = legacy.addr.port().to_string();
+    let mut configs = Vec::new();
+    for _ in 0..4 {
+        configs.push(config(&legacy.url()));
+    }
+    for _ in 0..4 {
+        configs.push(config(&modern.url()));
+    }
+
+    let client = Arc::new(HyperClient::new());
+    let results = send_batch(client, configs, 8, None, None).await;
+
+    assert_eq!(
+        reached(&results),
+        8,
+        "every request should still get through"
+    );
+
+    // Four requests to a host needing a shift. Without the scheduler each
+    // discovers it alone: 4 x 2 = 8 handshakes. With it, one discovers and
+    // the other three are told: 2 + 3 = 5. Asserting on "fewer than eight"
+    // rather than "exactly five" because the modern server's requests can
+    // finish and free the other host before all four legacy ones have even
+    // started, and how many get the news is a scheduling detail.
+    let a = attempts(&results, &legacy_host);
+    assert!(
+        a < 8,
+        "expected the burst to share what it learned, got {a} attempts for 4 requests"
+    );
+    assert!(
+        a >= 5,
+        "cannot be fewer than one discovery plus three, got {a}"
+    );
+
+    legacy.shutdown().await;
+    modern.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_a_burst_with_nothing_else_to_do_does_not_wait() {
+    // The case that killed the previous attempt. Every request wants the same
+    // host, so standing aside would be standing idle, and the right answer is
+    // to pay for the duplicate discovery instead. Four requests, two attempts
+    // each, nobody waiting on anybody.
+    let legacy = legacy_server().await;
+    let legacy_host = legacy.addr.port().to_string();
+
+    let configs: Vec<_> = (0..4).map(|_| config(&legacy.url())).collect();
+    let client = Arc::new(HyperClient::new());
+
+    let started = std::time::Instant::now();
+    let results = send_batch(client, configs, 4, None, None).await;
+    let elapsed = started.elapsed();
+
+    assert_eq!(reached(&results), 4);
+    assert_eq!(
+        attempts(&results, &legacy_host),
+        8,
+        "with no other host to hand the slot to, every request discovers"
+    );
+    // Four concurrent local requests. If any of them had stood aside waiting
+    // for another, this would be serial rather than parallel.
+    assert!(
+        elapsed < std::time::Duration::from_secs(2),
+        "took {elapsed:?}, which means something waited"
+    );
+
+    legacy.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_a_host_that_cannot_be_reached_does_not_strand_the_rest() {
+    // The leader guard marks the host answered when it drops, not when it
+    // succeeds. Without that, a burst against a dead host would have one
+    // request fail and the others sit on the stand-aside timeout.
+    let modern = modern_server().await;
+    let mut configs: Vec<_> = (0..4).map(|_| config("https://127.0.0.1:1/")).collect();
+    configs.push(config(&modern.url()));
+
+    let client = Arc::new(HyperClient::new());
+    let started = std::time::Instant::now();
+    let results = send_batch(client, configs, 5, None, None).await;
+    let elapsed = started.elapsed();
+
+    assert_eq!(results.len(), 5);
+    assert_eq!(reached(&results), 1, "only the live server answers");
+    assert!(
+        elapsed < std::time::Duration::from_secs(3),
+        "took {elapsed:?}: a failed leader should release the others at once"
+    );
+
+    modern.shutdown().await;
+}
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index ad2e2d2..738b073 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -237,32 +237,26 @@ async fn test_modern_still_reachable_by_default() {
     server.shutdown().await;
 }
 
-// ── A known cost, deliberately not fixed ──────────────────────────
+// ── What a cold burst costs when there is nothing else to do ──────
 
 #[tokio::test]
-async fn test_a_cold_burst_rediscovers_per_request() {
-    // Every request in an opening burst walks the ladder independently,
-    // because the per-host memory is only written once something succeeds and
-    // nothing has succeeded yet. This test records that rather than asserting
-    // it away, because the obvious fix turned out to cost more than it saved.
+async fn test_a_cold_burst_rediscovers_when_it_has_nothing_else_to_do() {
+    // Concurrent requests straight at `client.send`, all to one host, with no
+    // batch around them. Each walks the ladder alone, because the per-host
+    // memory is only written once something has succeeded and nothing has
+    // yet. Two handshakes apiece where one would have done.
     //
-    // The cost: roughly double the handshakes in the first burst against a
-    // host needing a shift. Measured against a real host, twelve concurrent
-    // requests produced twenty-four attempts. It is bounded by concurrency
-    // and not by scan size, so a thousand-path scan at fifty concurrent pays
-    // about fifty extra handshakes, and only on the first burst; afterwards
-    // the memory answers.
+    // That is the right answer here and the test says so rather than treating
+    // it as a defect. The only way to share the discovery is for these
+    // requests to wait on each other, and a waiter cannot be released until
+    // the leader has classified its response, which needs the body, which
+    // means the wait is a whole request long. Paying for fewer handshakes
+    // with dead time is a bad trade, and it breaks the rule that a slow
+    // request never holds up faster ones.
     //
-    // The attempted fix was a per-host semaphore, so one request discovers
-    // and the rest wait. It worked, 24 attempts down to 13. But a waiter
-    // cannot be released until the leader has classified its response, and
-    // classification needs the body, so the wait is a full request long. That
-    // is a head-of-line stall, which is exactly what request_batch_stream
-    // exists to avoid, and it broke the test that guards that property.
-    // Shortening the wait enough to preserve it removed the entire saving.
-    //
-    // So: leave it, and revisit if scan telemetry ever shows the burst
-    // mattering more than the latency.
+    // A batch does better, because a batch has somewhere else to send the
+    // slot: see `tests/burst_scheduler.rs`. The saving is only available when
+    // there is other work, and this test is the case where there is not.
     const N: usize = 6;
 
     let server = TlsTestServer::start(TlsServerConfig {

From ed2def61345ad296cae917ff48cc0717da415434 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Wed, 30 Sep 2026 14:37:33 -0400
Subject: [PATCH 20/44] Lift the stand-aside ceiling for send_batch, where it
 protects nothing

Load testing the scheduler found it doing nothing at all on any batch
larger than its concurrency limit, which is every batch worth
scheduling.

The ceiling on how many requests may stand aside at once was set to the
concurrency limit for both callers. That is right for the stream
driver, which takes a permit before it spawns: a request that stands
aside hands the permit straight back, the driver spawns another, and
without a bound a batch alternating between two slow hosts would spawn
the lot. It is wrong for send_batch, which spawns every task up front
regardless. There standing aside creates nothing, so the ceiling
protects nothing, and the requests it turns away go and duplicate the
discovery instead.

Measured over a grid of batch shapes, before and after, counting
handshakes above the floor of one per request plus one per host:

  hosts  reqs  conc    ceiling   lifted
      2   100   100         98        0
      2   100    50         63        0
      2   400    50         62        0
      2   400   200        199        0
      4   200   200        196        0
     10   500   100        139        0
     20  2000    50         62        0

Wall time is unchanged in five of those seven. The two that move are
the ones a single wave deep, where the batch is no bigger than its own
concurrency limit: there the discovery cannot overlap with anything
else and the batch pays one extra round trip, 0.22s to 0.32s against a
0.1s server. Multi-wave batches absorb it completely.

Also corrects something I had wrong in the earlier commit message. The
saving is not free. A request standing aside frees its slot, but the
condition only asks whether work for another host exists, not whether
there is enough of it to fill every slot being handed back. When there
is not, the slots idle and the batch pays that round trip. It never
costs time without saving anything, and it never reorders results, but
it is a trade rather than a freebie.

A soak of 10,000 requests alternating both paths: nothing lost, no
hangs, descriptors flat, per-round time flat, memory identical to the
build without the scheduler. The first round now costs one extra
handshake per host rather than a hundred.

One measurement note for anyone repeating this. The first attempt
showed a 3.5x slowdown that was entirely the test harness: the change
bunches connections tightly enough to overflow a listening socket's
default backlog of 128, and the SYN retransmit reads as a stall in the
client. With a real backlog the same case is 0.45s against 0.43s.
---
 CHANGELOG.md |  7 +++++-
 src/batch.rs | 69 ++++++++++++++++++++++++++++++++++++++++++----------
 2 files changed, 62 insertions(+), 14 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 23ea62f..d2a6202 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -54,7 +54,12 @@ the ladder falling back to `compatibility` on the second handshake.
   from twelve handshakes to seven, with nothing idling: standing aside only
   happens when there is undispatched work elsewhere, so when a batch is all
   one host every request goes immediately and discovers it again, which is the
-  cheaper mistake. A slow request still never holds up faster ones behind it
+  cheaper mistake. A slow request still never holds up faster ones behind it.
+  Measured over a grid of batch shapes, the duplicate handshakes go to zero in
+  every one: a scan of 2000 URLs across 20 hosts drops 62 handshakes, 500 URLs
+  across 10 hosts drops 139, and wall time is unchanged. A batch no deeper than
+  its own concurrency limit pays one extra round trip for it, since there the
+  discovery cannot be overlapped with anything
 - `raw_connect`, and requests using `resolve_ip` or `request_target`, widen
   their cipher offer and retry on a handshake the peer could not negotiate,
   the same as an ordinary request. These bypass the pooled client, so they
diff --git a/src/batch.rs b/src/batch.rs
index 38ebb2c..81b839d 100644
--- a/src/batch.rs
+++ b/src/batch.rs
@@ -189,13 +189,24 @@ struct SchedulerState {
     remaining_total: usize,
     remaining_by_host: std::collections::HashMap<String, usize>,
     /// How many requests are standing aside right now, and the ceiling on
-    /// that. A request that has stepped aside holds no permit, so it stops
-    /// counting against concurrency and the driver spawns another in its
-    /// place. Without a ceiling, a batch alternating between two slow hosts
-    /// would have the driver spawn task after task, each stepping aside and
-    /// freeing the permit for the next, until the whole batch was resident.
-    /// The ceiling keeps live tasks inside the envelope the stream already
-    /// had.
+    /// that.
+    ///
+    /// The ceiling is about *spawning*, so the two callers want different
+    /// numbers and passing one for both was a mistake worth recording.
+    ///
+    /// The stream driver takes a permit before it spawns, which is how it
+    /// keeps a million-URL batch from being resident all at once. A request
+    /// that stands aside hands that permit straight back, so the driver
+    /// spawns another, which may also stand aside, and so on: a batch
+    /// alternating between two slow hosts would spawn the lot. There the
+    /// ceiling is the concurrency limit and it is load-bearing.
+    ///
+    /// `send_batch` spawns every task up front regardless. Standing aside
+    /// creates nothing, so a ceiling protects nothing, and setting one only
+    /// throws the benefit away: measured over a grid of batch shapes, the
+    /// duplicate handshakes went to zero exactly when the ceiling was not
+    /// binding and were left completely untouched when it was. There the
+    /// limit is the batch size, which is to say no limit at all.
     aside: usize,
     aside_limit: usize,
 }
@@ -275,7 +286,10 @@ fn host_key(url: &str) -> String {
 }
 
 impl Scheduler {
-    fn new(configs: &[RequestConfig], concurrency: usize) -> Arc<Self> {
+    /// `aside_limit` is how many requests may stand aside at once. See the
+    /// field: the stream driver needs the concurrency limit, `send_batch`
+    /// needs no limit and passes the batch size.
+    fn new(configs: &[RequestConfig], aside_limit: usize) -> Arc<Self> {
         let mut remaining_by_host: std::collections::HashMap<String, usize> =
             std::collections::HashMap::new();
         for c in configs {
@@ -287,7 +301,7 @@ impl Scheduler {
                 remaining_total: configs.len(),
                 remaining_by_host,
                 aside: 0,
-                aside_limit: concurrency.max(1),
+                aside_limit: aside_limit.max(1),
             }),
         })
     }
@@ -401,7 +415,9 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
 ) -> Vec<BatchResult> {
     let semaphore = Arc::new(tokio::sync::Semaphore::new(concurrency));
     let limiter = merge_limiters(shared_limiter, rate_limit);
-    let sched = Scheduler::new(&configs, concurrency);
+    // No ceiling here: the tasks all exist either way, so standing aside
+    // costs nothing that going would not have cost anyway.
+    let sched = Scheduler::new(&configs, configs.len());
     let mut handles = Vec::new();
 
     for config in configs {
@@ -481,6 +497,8 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
 ) -> impl Stream<Item = BatchResult> + Send + 'static {
     let limiter = merge_limiters(shared_limiter, rate_limit);
     let semaphore = Arc::new(tokio::sync::Semaphore::new(concurrency));
+    // Ceiling here, because this driver spawns lazily and a request that
+    // stands aside gives its permit back for the driver to spawn against.
     let sched = Scheduler::new(&configs, concurrency);
 
     stream::once(async move {
@@ -1040,9 +1058,11 @@ mod scheduler_tests {
 
     #[test]
     fn test_the_ceiling_bounds_how_many_can_be_standing_aside() {
-        // Without this, a batch alternating between two slow hosts has the
-        // driver spawn a task, watch it hand the permit straight back, spawn
-        // another, and so on until the whole batch is resident.
+        // The stream driver's limit, which is the concurrency one. Without
+        // it, a batch alternating between two slow hosts has the driver spawn
+        // a task, watch it hand the permit straight back, spawn another, and
+        // so on until the whole batch is resident. `send_batch` wants the
+        // opposite and the test above covers that.
         let mut urls = vec!["https://a.test/"; 10];
         urls.push("https://b.test/");
         let sched = Scheduler::new(&cfgs(&urls), 2);
@@ -1061,6 +1081,29 @@ mod scheduler_tests {
         );
     }
 
+    #[test]
+    fn test_no_ceiling_lets_a_whole_batch_stand_aside() {
+        // What `send_batch` passes, and why it differs from the stream. Its
+        // tasks are all spawned up front, so standing aside creates nothing
+        // and a ceiling protects nothing. Setting one anyway only throws the
+        // saving away: measured across a grid of batch shapes, a ceiling of
+        // the concurrency limit left the duplicate handshakes completely
+        // untouched wherever a batch was bigger than that limit, which is
+        // every batch worth scheduling.
+        let mut urls = vec!["https://a.test/"; 40];
+        urls.push("https://b.test/");
+        let configs = cfgs(&urls);
+        let sched = Scheduler::new(&configs, configs.len());
+        let a = host_key("https://a.test/");
+
+        let _lead = sched.dispatch(&a, true);
+        let aside: Vec<_> = (0..39).map(|_| sched.dispatch(&a, true)).collect();
+        assert!(
+            aside.iter().all(is_aside),
+            "every one of them should wait for the answer the first is fetching"
+        );
+    }
+
     #[test]
     fn test_a_freed_slot_lets_another_stand_aside() {
         let mut urls = vec!["https://a.test/"; 10];

From 3e1629e41f637ee11eee01befcf0060a34faa2ac Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 5 Oct 2026 17:05:25 -0400
Subject: [PATCH 21/44] Stop the antibot scan panicking on a body cut
 mid-character

The classifier reads a 64 KiB window from the top of the body, and sliced
a str straight at that byte. Slicing at a byte that is not a character
boundary panics rather than erring, and max_body_size cuts the body at a
byte count, so a capped read can leave a half-written character sitting
exactly there. It took the whole request down with it.

This only became reachable once dev's max_body_size work met the
classifier, which is why neither branch saw it alone. Walk back to the
nearest boundary before slicing.

Also reformat test_mock.py, which ruff 0.15.10 (what CI pins) wanted.
---
 src/antibot.rs            |  9 ++++++++-
 tests/antibot_classify.rs | 30 ++++++++++++++++++++++++++++++
 tests/python/test_mock.py |  7 +------
 3 files changed, 39 insertions(+), 7 deletions(-)

diff --git a/src/antibot.rs b/src/antibot.rs
index d5e664f..5c859bd 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -149,7 +149,14 @@ impl ResponseFacts<'_> {
     fn body_contains(&self, needle: &str) -> bool {
         // Bodies can be large; a challenge marker is always near the top.
         const WINDOW: usize = 64 * 1024;
-        let end = self.body.len().min(WINDOW);
+        let mut end = self.body.len().min(WINDOW);
+        // The cut can land in the middle of a multi-byte character, and
+        // slicing a str there panics rather than erring. A body truncated by
+        // max_body_size puts arbitrary bytes at the cap, so this is reachable
+        // from the wire. Walk back to the nearest boundary.
+        while end > 0 && !self.body.is_char_boundary(end) {
+            end -= 1;
+        }
         self.body[..end].contains(needle)
     }
 }
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 8d315e0..891f7b3 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -288,3 +288,33 @@ fn test_a_bare_403_is_not_evidence_of_protection() {
     assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
     assert!(!outcome.indicates_protection());
 }
+
+#[test]
+fn test_a_body_cut_mid_character_does_not_panic() {
+    // `max_body_size` cuts the body at a byte count, so the last character can
+    // be left half-written. The classifier scans a 64 KiB window from the top,
+    // and slicing a str at a byte that is not a character boundary panics
+    // rather than erring, which took the whole request down.
+    let h = hdrs(&[("server", "nginx")]);
+
+    // One single-byte character, then two-byte ones, puts every boundary on an
+    // odd offset. The window ends at 65536, which is therefore inside a
+    // character rather than between two.
+    let body = format!("a{}", "\u{5ae}".repeat(40_000));
+    assert!(body.len() > 65_536);
+    assert!(!body.is_char_boundary(65_536));
+
+    let outcome = classify(&facts(200, &h, &body));
+    assert_eq!(outcome, Outcome::Ok);
+}
+
+#[test]
+fn test_a_marker_in_a_multibyte_body_is_still_found() {
+    // Walking back to a boundary must not cost us the match itself.
+    let h = hdrs(&[("server", "cloudflare")]);
+    let body = format!("{}window._cf_chl_opt", "\u{5ae}".repeat(10));
+    assert_eq!(
+        classify(&facts(403, &h, &body)),
+        Outcome::Challenge(Vendor::Cloudflare)
+    );
+}
diff --git a/tests/python/test_mock.py b/tests/python/test_mock.py
index 1b0d010..3348a7d 100644
--- a/tests/python/test_mock.py
+++ b/tests/python/test_mock.py
@@ -405,12 +405,7 @@ async def handler(reader, writer):
         while (line := await reader.readuntil(b"\r\n")) != b"\r\n":
             received.append(line.decode("latin1").rstrip("\r\n"))
         body = "\n".join(received).encode()
-        writer.write(
-            b"HTTP/1.1 200 OK\r\nContent-Length: "
-            + str(len(body)).encode()
-            + b"\r\n\r\n"
-            + body
-        )
+        writer.write(b"HTTP/1.1 200 OK\r\nContent-Length: " + str(len(body)).encode() + b"\r\n\r\n" + body)
         await writer.drain()
         writer.close()
 

From 2ee1bb4a7ccd0366b3c2aa73ca0dd0228991ab01 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Sat, 24 Oct 2026 01:48:20 -0400
Subject: [PATCH 22/44] Remember hosts that refuse every rung of the ladder

---
 CHANGELOG.md                             |   6 ++
 src/client/hyper.rs                      | 101 +++++++++++++++++++++--
 tests/python/test_profiles_and_report.py |  29 ++++++-
 3 files changed, 128 insertions(+), 8 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d2a6202..fa54503 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -25,6 +25,12 @@ the ladder falling back to `compatibility` on the second handshake.
   any refusal. Most 403s are ordinary authorization failures, and retrying each
   one with different TLS settings would double the request count of any scan
   that touches one
+- A host that refuses every rung is remembered too, for ten minutes: during
+  that time a refusal from it is taken as the answer after one request, rather
+  than walking the ladder again. Only success used to be remembered, so a host
+  that refused everything cost two requests for every one a scan sent it, on
+  exactly the hosts most likely to rate-limit or ban. Getting through clears
+  it, and it expires because a refusal is a judgement that changes
 - **`Response.protection`** reports what a bot-management product did:
   `(outcome, vendor)` where outcome is ok, present, challenge, blocked or error
 - **`Response.conclusion`** reduces that to one verdict: reached, blocked,
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 02af531..9f5d007 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1018,6 +1018,11 @@ enum ConnMode {
     Socks5 { proxy_url: String, tls: TlsKey },
 }
 
+/// How long a host that refused every rung keeps getting one request instead of
+/// the whole ladder. Long enough to stop doubling a scan's requests to it, short
+/// enough that a host which relents is noticed. See `ladder_exhausted`.
+const EXHAUSTED_TTL: Duration = Duration::from_secs(600);
+
 pub struct HyperClient {
     // Clients cached by connection mode. A scan using an HTTP proxy may need
     // both a ForwardProxy client (for HTTP targets) and a Tunnel client (for
@@ -1027,6 +1032,10 @@ pub struct HyperClient {
     // the ladder on every request. Written only on success; see
     // `remember_profile`.
     host_profiles: Mutex<HashMap<String, &'static crate::profile::ConnectionProfile>>,
+    // When each host last refused every rung of the ladder. For a while after,
+    // a refusal from it is taken as the answer instead of walking the ladder
+    // again; see `ladder_exhausted`.
+    host_exhausted: Mutex<HashMap<String, Instant>>,
     // Why the last handshake to each host failed.
     //
     // Deliberately on the client rather than on `CachedClient`: a retry with
@@ -1047,6 +1056,7 @@ impl HyperClient {
         HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
             host_profiles: Mutex::new(HashMap::new()),
+            host_exhausted: Mutex::new(HashMap::new()),
             tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
@@ -2406,6 +2416,44 @@ impl HyperClient {
         crate::profile::next_rung(current, failure, tried)
     }
 
+    /// Whether this host recently refused every rung, so a refusal from it now
+    /// should be taken as the answer rather than spending another request on
+    /// the next rung.
+    ///
+    /// Without this, only success was remembered, so a host that refuses
+    /// everything had the whole ladder walked again for every request to it:
+    /// two requests where one tells you the same thing, on exactly the hosts
+    /// most likely to rate-limit or ban. It expires, unlike the success memory,
+    /// because a refusal is a judgement the product makes from things that
+    /// change (rate, reputation, its own rules), and what it refused ten minutes
+    /// ago it may let through now.
+    fn ladder_exhausted(&self, config: &RequestConfig, uri: &http::Uri) -> bool {
+        if config.tls_is_pinned() {
+            return false;
+        }
+        peer_slot_key(uri).is_some_and(|key| self.exhausted_within_ttl(&key))
+    }
+
+    fn exhausted_within_ttl(&self, key: &str) -> bool {
+        self.host_exhausted
+            .lock()
+            .ok()
+            .and_then(|map| map.get(key).copied())
+            .is_some_and(|at| at.elapsed() < EXHAUSTED_TTL)
+    }
+
+    /// Record that a host refused every rung of the ladder.
+    fn remember_exhausted(&self, config: &RequestConfig, uri: &http::Uri) {
+        if config.tls_is_pinned() {
+            return;
+        }
+        if let Some(key) = peer_slot_key(uri)
+            && let Ok(mut map) = self.host_exhausted.lock()
+        {
+            map.insert(key, Instant::now());
+        }
+    }
+
     /// Record that a profile worked against a host.
     ///
     /// Only ever called after an attempt got through. A failure alone is a
@@ -2431,10 +2479,14 @@ impl HyperClient {
         if config.tls_is_pinned() {
             return;
         }
-        if let Some(key) = peer_slot_key(uri)
-            && let Ok(mut map) = self.host_profiles.lock()
-        {
-            map.insert(key, profile);
+        if let Some(key) = peer_slot_key(uri) {
+            if let Ok(mut map) = self.host_profiles.lock() {
+                map.insert(key.clone(), profile);
+            }
+            // whatever it refused before, it lets us through now
+            if let Ok(mut map) = self.host_exhausted.lock() {
+                map.remove(&key);
+            }
         }
     }
 
@@ -2582,7 +2634,14 @@ impl HyperClient {
                             break (resp, cached);
                         }
 
-                        match self.next_rung_for(config, rung, RungFailure::Refused, &tried) {
+                        // A host that recently refused every rung gets one
+                        // request, not the whole ladder again.
+                        let next = if self.ladder_exhausted(config, &uri) {
+                            None
+                        } else {
+                            self.next_rung_for(config, rung, RungFailure::Refused, &tried)
+                        };
+                        match next {
                             Some(next) => {
                                 debug_record(
                                     log,
@@ -2599,7 +2658,15 @@ impl HyperClient {
                             // back as a response rather than an error, so the
                             // caller can read the status, the headers and
                             // whatever the challenge page said.
-                            None => break (resp, cached),
+                            None => {
+                                // Only a ladder that actually ran out counts; a
+                                // single refusal while already exhausted must not
+                                // keep extending it, or the host is never re-tried.
+                                if tried.len() > 1 {
+                                    self.remember_exhausted(config, &uri);
+                                }
+                                break (resp, cached);
+                            }
                         }
                     }
                     Err(mut e) => {
@@ -3155,6 +3222,28 @@ where
 mod tests {
     use super::*;
 
+    #[test]
+    fn test_exhausted_host_expires() {
+        let client = HyperClient::new();
+        assert!(!client.exhausted_within_ttl("example.com:443"));
+
+        client
+            .host_exhausted
+            .lock()
+            .unwrap()
+            .insert("example.com:443".to_string(), Instant::now());
+        assert!(client.exhausted_within_ttl("example.com:443"));
+
+        // a refusal from longer ago than the TTL no longer counts, so the ladder is walked again
+        let long_ago = Instant::now() - EXHAUSTED_TTL - Duration::from_secs(1);
+        client
+            .host_exhausted
+            .lock()
+            .unwrap()
+            .insert("example.com:443".to_string(), long_ago);
+        assert!(!client.exhausted_within_ttl("example.com:443"));
+    }
+
     #[test]
     fn test_backoff_attempt_0() {
         let min = Duration::from_secs(1);
diff --git a/tests/python/test_profiles_and_report.py b/tests/python/test_profiles_and_report.py
index c47bc8b..6f491d6 100644
--- a/tests/python/test_profiles_and_report.py
+++ b/tests/python/test_profiles_and_report.py
@@ -29,7 +29,8 @@ async def _handler(reader, writer):
 
     /              ordinary 200
     /echo-headers  200 whose body is the request headers it received
-    /cf-challenge  403 shaped like a Cloudflare managed challenge
+    /cf-challenge  403 shaped like a Cloudflare managed challenge, unless the
+                   request carries the cookie pass=1
     /akamai-block  403 shaped like an Akamai edge refusal
     /plain-403     403 with no product signature at all
     """
@@ -58,7 +59,9 @@ def respond(status, headers, body):
         if path == "/echo-headers":
             body = "\n".join(received).encode()
             respond(200, [("Content-Type", "text/plain")], body)
-        elif path == "/cf-challenge":
+        elif path == "/cf-challenge" and not any(
+            h.lower().startswith("cookie:") and "pass=1" in h for h in received
+        ):
             # cf-mitigated is the header Cloudflare sets specifically to say it
             # intervened, and is what separates a challenge from a WAF block.
             respond(
@@ -171,6 +174,28 @@ async def test_a_challenge_asks_for_a_browser(server, client):
     assert r.conclusion == ("needs_browser", "cloudflare")
 
 
+async def test_a_host_that_refuses_every_rung_gets_one_request_after(server, client):
+    """Only success used to be remembered, so a host that refused everything had
+    the whole ladder walked again for every request: two requests where one
+    tells you the same thing, on the hosts most likely to rate-limit or ban."""
+    url = f"{server}/cf-challenge"
+
+    r = await client.request(url)
+    assert [a[0] for a in r.attempts] == ["modern", "chrome131"]
+    assert r.conclusion == ("needs_browser", "cloudflare")
+
+    # recently refused on every rung: one request, and the refusal is the answer
+    r = await client.request(url)
+    assert [a[0] for a in r.attempts] == ["modern"]
+    assert r.conclusion == ("needs_browser", "cloudflare")
+
+    # getting through clears it, so the next refusal walks the ladder again
+    r = await client.request(url, headers=[("cookie", "pass=1")])
+    assert r.conclusion == ("reached", "-")
+    r = await client.request(url)
+    assert [a[0] for a in r.attempts] == ["modern", "chrome131"]
+
+
 async def test_a_block_is_not_a_challenge(server, client):
     """The distinction that decides whether to launch a browser. A block might
     yield to a different approach; a challenge will not."""

From 9f32f4699897a997e20f20dfc5dd3a7f86069bdf Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 14:57:11 -0400
Subject: [PATCH 23/44] Don't count Imperva's loader script as a challenge

---
 CHANGELOG.md              |  4 +++
 src/antibot.rs            |  7 +++++-
 tests/antibot_classify.rs | 51 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 61 insertions(+), 1 deletion(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index fa54503..aeee662 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -38,6 +38,10 @@ the ladder falling back to `compatibility` on the second handshake.
   JavaScript challenge, which no HTTP client passes, so the URL should go to a
   real browser rather than be retried
 - **`Response.attempts`** lists what the ladder tried, in order
+- Imperva's `_Incapsula_Resource` loader script no longer counts as a
+  challenge. Imperva appends it to pages it let through, so ordinary pages and
+  the origin's own error pages were judged `needs_browser`. Imperva's incident
+  page still counts
 - **TLS failures now say why.** The reason used to be destroyed in transit:
   hyper-util's error displays as the literal "client error (Connect)" for every
   connect-time failure, so a cipher mismatch, a rejected certificate, a DNS
diff --git a/src/antibot.rs b/src/antibot.rs
index 5c859bd..3e91110 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -215,7 +215,12 @@ pub fn classify(facts: &ResponseFacts) -> Outcome {
     }
 
     // ── Imperva / Incapsula ───────────────────────────────────────
-    if facts.body_contains("_Incapsula_Resource") {
+    // The `&ns=1&cb=` script is the loader Imperva adds to pages it let
+    // through, the origin's own error pages included, so on its own it's not
+    // an interception. Imperva's incident page frames itself as `CWUDNSAI`.
+    if facts.body_contains("_Incapsula_Resource?CWUDNSAI=")
+        || (facts.body_contains("_Incapsula_Resource") && !facts.body_contains("&ns=1&cb="))
+    {
         return Outcome::Challenge(Vendor::Imperva);
     }
 
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 891f7b3..5a2eb8e 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -167,6 +167,57 @@ fn test_akamai_sec_cpt_428_is_its_own_outcome() {
     );
 }
 
+/// The shape of Imperva's incident page, with its identifiers replaced.
+const IMPERVA_INCIDENT: &str = concat!(
+    r#"<html style="height:100%"><head><META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">"#,
+    r#"<script type="text/javascript" src="/_Incapsula_Resource?SWJIYLWA=0123456789abcdef"></script></head>"#,
+    r#"<body style="margin:0px;height:100%"><iframe id="main-iframe" "#,
+    r#"src="/_Incapsula_Resource?CWUDNSAI=24&xinfo=0&incident_id=0-0&edet=12&cinfo=ffffffff&rpinfo=0&mth=GET" "#,
+    r#"frameborder=0 width="100%" height="100%">Request unsuccessful. Incapsula incident ID: 0-0</iframe></body></html>"#
+);
+
+/// The loader Imperva adds to the end of pages it let through.
+const IMPERVA_LOADER: &str = r#"<script type="text/javascript" src="/_Incapsula_Resource?SWJIYLWA=0123456789abcdef&ns=1&cb=123456789" async></script>"#;
+
+#[test]
+fn test_imperva_incident_page_is_a_challenge() {
+    let h = hdrs(&[("x-cdn", "Imperva")]);
+    for status in [200, 403, 503] {
+        assert_eq!(
+            classify(&facts(status, &h, IMPERVA_INCIDENT)),
+            Outcome::Challenge(Vendor::Imperva)
+        );
+    }
+}
+
+#[test]
+fn test_imperva_loader_on_a_page_is_not_a_challenge() {
+    // The origin's own page, a 200 or its own 403, with Imperva's loader
+    // appended.
+    let h = hdrs(&[("x-cdn", "Imperva")]);
+    let page = format!(
+        "<html><title>IIS Windows Server</title><body>welcome{IMPERVA_LOADER}</body></html>"
+    );
+    let refusal = format!("<html><h1>403 Forbidden</h1>{IMPERVA_LOADER}</html>");
+    for (status, body) in [(200, &page), (403, &refusal)] {
+        assert_ne!(
+            classify(&facts(status, &h, body)),
+            Outcome::Challenge(Vendor::Imperva)
+        );
+    }
+}
+
+#[test]
+fn test_imperva_script_without_the_loader_form_is_a_challenge() {
+    // Anything else naming the resource still counts, as before.
+    let h = hdrs(&[]);
+    let body = r#"<html><head><script src="/_Incapsula_Resource?SWJIYLWA=0123456789abcdef"></script></head><body></body></html>"#;
+    assert_eq!(
+        classify(&facts(200, &h, body)),
+        Outcome::Challenge(Vendor::Imperva)
+    );
+}
+
 // ── The false positives that caught the first pass ────────────────
 
 #[test]

From 841f3b0da0a8c2def12cad79cb06712a90f889ae Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 15:22:12 -0400
Subject: [PATCH 24/44] Move the anti-bot rules out of code into
 antibot_rules.json

---
 CHANGELOG.md                       |   5 +
 README.md                          |   2 +-
 antibot_rules.json                 | 126 ++++++++++++
 src/antibot.rs                     | 311 +++++++++++++++--------------
 src/python.rs                      |  22 ++
 src/report.rs                      |  22 +-
 src/response.rs                    |   7 +-
 tests/antibot_classify.rs          |  34 ++--
 tests/antibot_rules.rs             | 115 +++++++++++
 tests/antibot_set_rules.rs         |  31 +++
 tests/python/test_antibot_rules.py |  51 +++++
 11 files changed, 547 insertions(+), 179 deletions(-)
 create mode 100644 antibot_rules.json
 create mode 100644 tests/antibot_rules.rs
 create mode 100644 tests/antibot_set_rules.rs
 create mode 100644 tests/python/test_antibot_rules.py

diff --git a/CHANGELOG.md b/CHANGELOG.md
index aeee662..8e5d803 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -42,6 +42,11 @@ the ladder falling back to `compatibility` on the second handshake.
   challenge. Imperva appends it to pages it let through, so ordinary pages and
   the origin's own error pages were judged `needs_browser`. Imperva's incident
   page still counts
+- **The anti-bot rules are data.** What `protection` recognises now lives in
+  `antibot_rules.json` instead of code, and `blasthttp.set_antibot_rules(path)`
+  (`antibot::set_rules` in Rust) replaces them at runtime, so a rule can change
+  without a rebuild. `Vendor` is a name rather than a fixed list, so a rules
+  file can add a product
 - **TLS failures now say why.** The reason used to be destroyed in transit:
   hyper-util's error displays as the literal "client error (Connect)" for every
   connect-time failure, so a cipher mismatch, a rejected certificate, a DNS
diff --git a/README.md b/README.md
index 9eb29d8..28352b6 100644
--- a/README.md
+++ b/README.md
@@ -26,7 +26,7 @@ cargo add blasthttp
 - **Rust performance** — async I/O, zero-copy where possible, and native concurrency give significant speed improvements over pure Python HTTP clients
 - **SSL cert info on every request** — extracts CN, SANs, emails, issuer, validity dates, and fingerprint during the TLS handshake that's already happening, eliminating the need for a separate connection to gather this information
 - **Connection profiles, and automatic shifting between them**: `modern` by default (11 cipher suites, TLS 1.2+), `compatibility` for servers that speak only RC4, 3DES, SEED, Camellia, anonymous DH or SSLv3, and `chrome` to imitate a browser's TLS, HTTP/2 and headers together. A handshake the peer cannot negotiate widens the offer; a refusal from a recognised protection product changes what the client claims to be. What works is remembered per host
-- **Says why it failed**: a blocked request reports which product blocked it, and a JavaScript challenge reports `needs_browser`, because no HTTP client passes that tier and retrying will not change it
+- **Says why it failed**: a blocked request reports which product blocked it, and a JavaScript challenge reports `needs_browser`, because no HTTP client passes that tier and retrying will not change it. The products and how to recognise them are rules in [`antibot_rules.json`](antibot_rules.json), and `blasthttp.set_antibot_rules(path)` swaps in your own at runtime
 - **No cert validation by default** — offensive-first: connects to self-signed, expired, and misconfigured TLS without extra config
 - **HTTP/2 support** — automatic via ALPN negotiation, falls back to HTTP/1.1
 - **Low-level primitives** — `RawConnection` for byte-level TCP/TLS I/O (bypasses HTTP framing entirely) and `blasthttp.h2` for manual H2 frame construction (includes a permissive HPACK encoder that emits bytes a strict encoder refuses, the building block for H2 smuggling / CRLF-injection tooling)
diff --git a/antibot_rules.json b/antibot_rules.json
new file mode 100644
index 0000000..b3bc56e
--- /dev/null
+++ b/antibot_rules.json
@@ -0,0 +1,126 @@
+{
+  "refusal_statuses": [401, 403, 405, 406, 429, 503],
+  "rules": [
+    {
+      "vendor": "cloudflare",
+      "outcome": "challenge",
+      "when": {"header_contains": {"name": "cf-mitigated", "value": "challenge"}},
+      "why": "The unambiguous one, and the only header Cloudflare sets specifically to say it intervened."
+    },
+    {
+      "vendor": "cloudflare",
+      "outcome": "challenge",
+      "when": {"any": [
+        {"body_contains": "/cdn-cgi/challenge-platform/"},
+        {"body_contains": "window._cf_chl_opt"}
+      ]},
+      "why": "The challenge page's own markers, for when cf-mitigated is missing."
+    },
+    {
+      "vendor": "akamai",
+      "outcome": "blocked",
+      "when": {"any": [
+        {"body_contains": "edgesuite"},
+        {"all": [
+          {"any": [
+            {"header_contains": {"name": "server", "value": "akamaighost"}},
+            {"header_present": "akamai-grn"},
+            {"header_present": "x-akam-sw-version"},
+            {"header_present": "x-akamai-transformed"},
+            {"header_contains": {"name": "server-timing", "value": "ak_p"}}
+          ]},
+          {"body_contains": "Access Denied"}
+        ]}
+      ]},
+      "why": "The edge error page. It HTML-entity-encodes its own punctuation, so the body carries errors&#46;edgesuite&#46;net and a search for errors.edgesuite.net never fires; the bare label edgesuite survives that. server: AkamaiGHost is not reliably present either: www.akamai.com's own 403 sends no server header and identifies itself through akamai-grn, x-akam-sw-version and an ak_p server-timing entry instead, so any one of the edge headers will do. Reference # on its own stays insufficient, because plenty of unrelated error pages carry one."
+    },
+    {
+      "vendor": "akamai",
+      "outcome": "challenge",
+      "when": {"all": [
+        {"status": [428]},
+        {"any": [{"cookie_set": "sec_cpt"}, {"body_contains": "sec-cpt"}]}
+      ]},
+      "why": "sec-cpt is an adaptive challenge that fires after _abck already passed, and it answers 428 rather than 403. Its own outcome, not a block."
+    },
+    {
+      "vendor": "datadome",
+      "outcome": "challenge",
+      "when": {"body_contains": "geo.captcha-delivery.com"}
+    },
+    {
+      "vendor": "perimeterx",
+      "outcome": "challenge",
+      "when": {"any": [{"body_contains": "/px/captcha"}, {"body_contains": "px-captcha"}]}
+    },
+    {
+      "vendor": "imperva",
+      "outcome": "challenge",
+      "when": {"any": [
+        {"body_contains": "_Incapsula_Resource?CWUDNSAI="},
+        {"all": [
+          {"body_contains": "_Incapsula_Resource"},
+          {"not": {"body_contains": "&ns=1&cb="}}
+        ]}
+      ]},
+      "why": "The &ns=1&cb= script is the loader Imperva adds to pages it let through, the origin's own error pages included, so on its own it's not an interception. Imperva's incident page frames itself as CWUDNSAI."
+    },
+    {
+      "vendor": "awswaf",
+      "outcome": "challenge",
+      "when": {"header_present": "x-amzn-waf-action"},
+      "why": "The header carries the action, and there is more than one: challenge for the silent JavaScript kind, captcha for the kind wanting a person. Matching only challenge missed wizzair.com, which answers 405 with captcha, so the ladder read a genuine interception as an ordinary refusal and declined to try anything else. Match the header's presence and let the value be detail."
+    },
+    {
+      "vendor": "kasada",
+      "outcome": "blocked",
+      "when": {"all": [
+        {"any": [{"header_present": "x-kpsdk-ct"}, {"body_contains": "KPSDK"}]},
+        {"status": [429]}
+      ]},
+      "why": "Kasada answers 429 rather than 403 for a refusal."
+    },
+    {
+      "vendor": "kasada",
+      "outcome": "present",
+      "when": {"any": [{"header_present": "x-kpsdk-ct"}, {"body_contains": "KPSDK"}]}
+    },
+    {
+      "vendor": "cloudflare",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"any": [{"cookie_set": "__cf_bm"}, {"cookie_set": "cf_clearance"}]},
+      "why": "From here on, sightings of a product on a response nothing above matched. One that set its cookie while refusing us is still a block."
+    },
+    {
+      "vendor": "akamai",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"any": [{"cookie_set": "_abck"}, {"cookie_set": "ak_bmsc"}, {"cookie_set": "bm_sz"}]}
+    },
+    {
+      "vendor": "datadome",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"cookie_set": "datadome"}
+    },
+    {
+      "vendor": "perimeterx",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"cookie_prefix": "_px"}
+    },
+    {
+      "vendor": "imperva",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"any": [{"cookie_prefix": "visid_incap_"}, {"cookie_prefix": "incap_ses_"}]}
+    },
+    {
+      "vendor": "f5",
+      "outcome": "present",
+      "on_refusal": "blocked",
+      "when": {"header_contains": {"name": "server", "value": "volt-adc"}}
+    }
+  ]
+}
diff --git a/src/antibot.rs b/src/antibot.rs
index 3e91110..8e7e7e6 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -16,38 +16,39 @@
 //! 2. A response that merely *references* a vendor's script is not a challenge
 //!    page. Every page on a protected site carries the loader; only a page
 //!    shaped like an interception is a block.
+//!
+//! The rules themselves are data, in `antibot_rules.json`: the copy built in
+//! is the default, and [`set_rules`] swaps in another at runtime, so a rule
+//! can change without a rebuild.
 
 use std::fmt;
+use std::sync::{Arc, LazyLock, RwLock};
+
+use serde::Deserialize;
+
+/// Which product answered, named by the rule that matched. Deliberately
+/// conservative: guessing a vendor wrong is worse than reporting unknown.
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+pub struct Vendor(String);
+
+impl Vendor {
+    pub fn new(name: impl Into<String>) -> Self {
+        Vendor(name.into())
+    }
 
-/// Which product answered. Not exhaustive, and deliberately conservative:
-/// guessing a vendor wrong is worse than reporting `Unknown`.
-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
-pub enum Vendor {
-    Cloudflare,
-    Akamai,
-    DataDome,
-    PerimeterX,
-    Imperva,
-    F5,
-    Kasada,
-    AwsWaf,
-    Unknown,
+    /// A refusal no rule could put a name to.
+    pub fn unknown() -> Self {
+        Vendor::new("unknown")
+    }
+
+    pub fn is_unknown(&self) -> bool {
+        self.0 == "unknown"
+    }
 }
 
 impl fmt::Display for Vendor {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
-        let s = match self {
-            Vendor::Cloudflare => "cloudflare",
-            Vendor::Akamai => "akamai",
-            Vendor::DataDome => "datadome",
-            Vendor::PerimeterX => "perimeterx",
-            Vendor::Imperva => "imperva",
-            Vendor::F5 => "f5",
-            Vendor::Kasada => "kasada",
-            Vendor::AwsWaf => "awswaf",
-            Vendor::Unknown => "unknown",
-        };
-        f.write_str(s)
+        f.write_str(&self.0)
     }
 }
 
@@ -92,12 +93,12 @@ impl Outcome {
     /// fingerprint. The cost of the alternative is doubling the request count
     /// of every scan that touches a 403, which is most of them.
     pub fn indicates_protection(&self) -> bool {
-        !matches!(self.vendor(), None | Some(Vendor::Unknown))
+        self.vendor().is_some_and(|v| !v.is_unknown())
     }
 
-    pub fn vendor(&self) -> Option<Vendor> {
+    pub fn vendor(&self) -> Option<&Vendor> {
         match self {
-            Outcome::Present(v) | Outcome::Challenge(v) | Outcome::Blocked(v) => Some(*v),
+            Outcome::Present(v) | Outcome::Challenge(v) | Outcome::Blocked(v) => Some(v),
             _ => None,
         }
     }
@@ -161,146 +162,154 @@ impl ResponseFacts<'_> {
     }
 }
 
-/// Classify one response.
+/// The rules a classification runs, as loaded from `antibot_rules.json`.
 ///
-/// Order matters: a challenge outranks a mere sighting of the vendor, and a
-/// block outranks both, because a host that challenges us has still told us
-/// more than one that refuses outright.
-pub fn classify(facts: &ResponseFacts) -> Outcome {
-    // ── Cloudflare ────────────────────────────────────────────────
-    // `cf-mitigated: challenge` is the unambiguous one and the only header
-    // Cloudflare sets specifically to say it intervened.
-    if facts.header_matches("cf-mitigated", "challenge") {
-        return Outcome::Challenge(Vendor::Cloudflare);
-    }
-    if facts.body_contains("/cdn-cgi/challenge-platform/")
-        || facts.body_contains("window._cf_chl_opt")
-    {
-        return Outcome::Challenge(Vendor::Cloudflare);
-    }
+/// Order matters, and the first rule to match decides: a challenge outranks a
+/// mere sighting of the vendor, and a block outranks both, because a host that
+/// challenges us has still told us more than one that refuses outright.
+#[derive(Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
+pub struct Rules {
+    /// Statuses a protection product uses to refuse. 429 belongs because a rate
+    /// limit and a bot block are the same outcome for a scan, whoever sent it.
+    refusal_statuses: Vec<u16>,
+    rules: Vec<Rule>,
+}
 
-    // ── Akamai ────────────────────────────────────────────────────
-    // The edge error page, which is fussier to spot than it looks.
-    //
-    // It HTML-entity-encodes its own punctuation, so the body carries
-    // `errors&#46;edgesuite&#46;net` and a search for `errors.edgesuite.net`
-    // never fires. Matching the bare label `edgesuite` survives that, since
-    // only the dots and slashes get encoded.
-    //
-    // `server: AkamaiGHost` is also not reliably present: www.akamai.com's own
-    // 403 sends no `server` header at all and identifies itself through
-    // `akamai-grn`, `x-akam-sw-version` and an `ak_p` server-timing entry
-    // instead. So the server header is one signal among several rather than a
-    // requirement. `Reference #` on its own stays insufficient, because plenty
-    // of unrelated error pages carry one.
-    if facts.body_contains("edgesuite")
-        || (is_akamai_edge(facts) && facts.body_contains("Access Denied"))
-    {
-        return Outcome::Blocked(Vendor::Akamai);
-    }
-    // sec-cpt is an adaptive challenge that fires after _abck already passed,
-    // and it answers 428 rather than 403. Its own outcome, not a block.
-    if facts.status == 428 && (facts.sets_cookie("sec_cpt") || facts.body_contains("sec-cpt")) {
-        return Outcome::Challenge(Vendor::Akamai);
-    }
+#[derive(Debug, Deserialize)]
+#[serde(deny_unknown_fields)]
+struct Rule {
+    vendor: String,
+    outcome: RuleOutcome,
+    /// What the outcome becomes when the status is a refusal.
+    on_refusal: Option<RuleOutcome>,
+    when: Condition,
+    /// Why the rule is shaped the way it is. Not used, but worth keeping.
+    #[allow(dead_code)]
+    why: Option<String>,
+}
 
-    // ── DataDome ──────────────────────────────────────────────────
-    if facts.body_contains("geo.captcha-delivery.com") {
-        return Outcome::Challenge(Vendor::DataDome);
-    }
+#[derive(Debug, Clone, Copy, Deserialize)]
+#[serde(rename_all = "snake_case")]
+enum RuleOutcome {
+    Present,
+    Challenge,
+    Blocked,
+}
+
+#[derive(Debug, Deserialize)]
+#[serde(rename_all = "snake_case", deny_unknown_fields)]
+enum Condition {
+    All(Vec<Condition>),
+    Any(Vec<Condition>),
+    Not(Box<Condition>),
+    Status(Statuses),
+    HeaderPresent(String),
+    /// Case-insensitive, in both the name and the value.
+    HeaderContains {
+        name: String,
+        value: String,
+    },
+    CookieSet(String),
+    CookiePrefix(String),
+    BodyContains(String),
+}
 
-    // ── PerimeterX / HUMAN ────────────────────────────────────────
-    if facts.body_contains("/px/captcha") || facts.body_contains("px-captcha") {
-        return Outcome::Challenge(Vendor::PerimeterX);
+#[derive(Debug, Deserialize)]
+#[serde(untagged)]
+enum Statuses {
+    Codes(Vec<u16>),
+    Named(NamedStatuses),
+}
+
+#[derive(Debug, Deserialize)]
+#[serde(rename_all = "snake_case")]
+enum NamedStatuses {
+    Refusal,
+}
+
+impl Rules {
+    pub fn from_json(json: &str) -> Result<Rules, serde_json::Error> {
+        let rules: Rules = serde_json::from_str(json)?;
+        if rules.rules.iter().any(|r| r.vendor.is_empty()) {
+            return Err(serde::de::Error::custom("every rule needs a vendor"));
+        }
+        Ok(rules)
     }
 
-    // ── Imperva / Incapsula ───────────────────────────────────────
-    // The `&ns=1&cb=` script is the loader Imperva adds to pages it let
-    // through, the origin's own error pages included, so on its own it's not
-    // an interception. Imperva's incident page frames itself as `CWUDNSAI`.
-    if facts.body_contains("_Incapsula_Resource?CWUDNSAI=")
-        || (facts.body_contains("_Incapsula_Resource") && !facts.body_contains("&ns=1&cb="))
-    {
-        return Outcome::Challenge(Vendor::Imperva);
+    /// Classify one response.
+    pub fn classify(&self, facts: &ResponseFacts) -> Outcome {
+        let refusal = self.is_refusal(facts.status);
+        if let Some(rule) = self.rules.iter().find(|r| self.matches(&r.when, facts)) {
+            let vendor = Vendor::new(&rule.vendor);
+            return match (refusal, rule.on_refusal) {
+                (true, Some(outcome)) => outcome,
+                _ => rule.outcome,
+            }
+            .with(vendor);
+        }
+        // Nothing identifiable
+        if (200..400).contains(&facts.status) {
+            Outcome::Ok
+        } else if refusal {
+            Outcome::Blocked(Vendor::unknown())
+        } else {
+            Outcome::Error
+        }
     }
 
-    // ── AWS WAF ───────────────────────────────────────────────────
-    // The header carries the action, and there is more than one: `challenge`
-    // for the silent JavaScript kind, `captcha` for the kind wanting a person.
-    // Matching only `challenge` missed wizzair.com, which answers 405 with
-    // `captcha`, so the ladder read a genuine interception as an ordinary
-    // refusal and declined to try anything else. Match the header's presence
-    // and let the value be detail.
-    if facts.has_header("x-amzn-waf-action") {
-        return Outcome::Challenge(Vendor::AwsWaf);
+    fn is_refusal(&self, status: u16) -> bool {
+        self.refusal_statuses.contains(&status)
     }
 
-    // ── Kasada ────────────────────────────────────────────────────
-    // Kasada answers 429 rather than 403 for a refusal.
-    if facts.has_header("x-kpsdk-ct") || facts.body_contains("KPSDK") {
-        return if facts.status == 429 {
-            Outcome::Blocked(Vendor::Kasada)
-        } else {
-            Outcome::Present(Vendor::Kasada)
-        };
+    fn matches(&self, condition: &Condition, facts: &ResponseFacts) -> bool {
+        match condition {
+            Condition::All(conditions) => conditions.iter().all(|c| self.matches(c, facts)),
+            Condition::Any(conditions) => conditions.iter().any(|c| self.matches(c, facts)),
+            Condition::Not(condition) => !self.matches(condition, facts),
+            Condition::Status(Statuses::Codes(codes)) => codes.contains(&facts.status),
+            Condition::Status(Statuses::Named(NamedStatuses::Refusal)) => {
+                self.is_refusal(facts.status)
+            }
+            Condition::HeaderPresent(name) => facts.has_header(name),
+            Condition::HeaderContains { name, value } => {
+                facts.header_matches(name, &value.to_ascii_lowercase())
+            }
+            Condition::CookieSet(name) => facts.sets_cookie(name),
+            Condition::CookiePrefix(prefix) => facts.sets_cookie_prefixed(prefix),
+            Condition::BodyContains(needle) => facts.body_contains(needle),
+        }
     }
+}
 
-    // ── Product present, and we got through ───────────────────────
-    // Reached only when nothing above fired, so these are sightings on a
-    // response that actually carried content.
-    let present = if facts.sets_cookie("__cf_bm") || facts.sets_cookie("cf_clearance") {
-        Some(Vendor::Cloudflare)
-    } else if facts.sets_cookie("_abck")
-        || facts.sets_cookie("ak_bmsc")
-        || facts.sets_cookie("bm_sz")
-    {
-        Some(Vendor::Akamai)
-    } else if facts.sets_cookie("datadome") {
-        Some(Vendor::DataDome)
-    } else if facts.sets_cookie_prefixed("_px") {
-        Some(Vendor::PerimeterX)
-    } else if facts.sets_cookie_prefixed("visid_incap_") || facts.sets_cookie_prefixed("incap_ses_")
-    {
-        Some(Vendor::Imperva)
-    } else if facts.header_matches("server", "volt-adc") {
-        Some(Vendor::F5)
-    } else {
-        None
-    };
-
-    if let Some(vendor) = present {
-        // A product that set its cookie while refusing us is still a block.
-        return if is_refusal(facts.status) {
-            Outcome::Blocked(vendor)
-        } else {
-            Outcome::Present(vendor)
-        };
+impl RuleOutcome {
+    fn with(self, vendor: Vendor) -> Outcome {
+        match self {
+            RuleOutcome::Present => Outcome::Present(vendor),
+            RuleOutcome::Challenge => Outcome::Challenge(vendor),
+            RuleOutcome::Blocked => Outcome::Blocked(vendor),
+        }
     }
+}
 
-    // ── Nothing identifiable ──────────────────────────────────────
-    if (200..400).contains(&facts.status) {
-        Outcome::Ok
-    } else if is_refusal(facts.status) {
-        Outcome::Blocked(Vendor::Unknown)
-    } else {
-        Outcome::Error
+impl Default for Rules {
+    /// The rules built in from `antibot_rules.json`.
+    fn default() -> Self {
+        Rules::from_json(include_str!("../antibot_rules.json"))
+            .expect("the built-in antibot_rules.json is valid")
     }
 }
 
-/// Statuses a protection product uses to refuse. 429 is in here because a rate
-/// limit and a bot block are the same outcome for a scan, whoever sent it.
-fn is_refusal(status: u16) -> bool {
-    matches!(status, 401 | 403 | 405 | 406 | 429 | 503)
+static RULES: LazyLock<RwLock<Arc<Rules>>> =
+    LazyLock::new(|| RwLock::new(Arc::new(Rules::default())));
+
+/// Replace the rules every later classification runs, in this process.
+pub fn set_rules(rules: Rules) {
+    *RULES.write().unwrap_or_else(|e| e.into_inner()) = Arc::new(rules);
 }
 
-/// Is this response coming off Akamai's edge at all?
-///
-/// Any one of these is enough. They are the headers the edge attaches to its
-/// own responses regardless of whether it is serving content or refusing.
-fn is_akamai_edge(facts: &ResponseFacts) -> bool {
-    facts.header_matches("server", "akamaighost")
-        || facts.has_header("akamai-grn")
-        || facts.has_header("x-akam-sw-version")
-        || facts.has_header("x-akamai-transformed")
-        || facts.header_matches("server-timing", "ak_p")
+/// Classify one response with the current rules.
+pub fn classify(facts: &ResponseFacts) -> Outcome {
+    let rules = RULES.read().unwrap_or_else(|e| e.into_inner()).clone();
+    rules.classify(facts)
 }
diff --git a/src/python.rs b/src/python.rs
index af39244..00b5466 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -2193,6 +2193,27 @@ fn register_h2_submodule<'py>(parent: &Bound<'py, PyModule>) -> PyResult<()> {
     Ok(())
 }
 
+/// Replace the anti-bot rules every response in this process is classified
+/// with, from a JSON file shaped like the built-in `antibot_rules.json`.
+/// `None` restores the built-in rules. Raises ValueError on a bad file.
+#[pyfunction]
+#[pyo3(signature = (path=None))]
+fn set_antibot_rules(path: Option<std::path::PathBuf>) -> PyResult<()> {
+    let rules = match path {
+        None => crate::antibot::Rules::default(),
+        Some(path) => {
+            let json = std::fs::read_to_string(&path).map_err(|e| {
+                pyo3::exceptions::PyValueError::new_err(format!("{}: {e}", path.display()))
+            })?;
+            crate::antibot::Rules::from_json(&json).map_err(|e| {
+                pyo3::exceptions::PyValueError::new_err(format!("{}: {e}", path.display()))
+            })?
+        }
+    };
+    crate::antibot::set_rules(rules);
+    Ok(())
+}
+
 // ── Module registration ───────────────────────────────────────────
 
 #[pymodule]
@@ -2211,6 +2232,7 @@ fn blasthttp(m: &Bound<'_, PyModule>) -> PyResult<()> {
     m.add_class::<PyHeaders>()?;
     m.add_class::<PyRequest>()?;
     m.add_class::<PyRawConnection>()?;
+    m.add_function(wrap_pyfunction!(set_antibot_rules, m)?)?;
     m.add("HTTPStatusError", m.py().get_type::<HTTPStatusError>())?;
     m.add("TransportError", m.py().get_type::<TransportError>())?;
     register_h2_submodule(m)?;
diff --git a/src/report.rs b/src/report.rs
index bb7ed42..673ea63 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -20,7 +20,7 @@
 
 use serde::{Deserialize, Serialize};
 
-use crate::antibot::{Outcome, Vendor};
+use crate::antibot::Outcome;
 
 /// What one rung of the ladder produced.
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
@@ -62,7 +62,7 @@ impl Attempt {
     pub fn from_outcome(profile: &str, status: u16, outcome: &Outcome) -> Self {
         let vendor = outcome
             .vendor()
-            .filter(|v| !matches!(v, Vendor::Unknown))
+            .filter(|v| !v.is_unknown())
             .map(|v| v.to_string());
         let o = match outcome {
             Outcome::Ok | Outcome::Present(_) => AttemptOutcome::Reached { status },
@@ -111,7 +111,7 @@ impl Conclusion {
     pub fn from_attempts(last: &Outcome, attempts: &[Attempt]) -> Self {
         let vendor = last
             .vendor()
-            .filter(|v| !matches!(v, Vendor::Unknown))
+            .filter(|v| !v.is_unknown())
             .map(|v| v.to_string());
         match last {
             Outcome::Ok | Outcome::Present(_) => Conclusion::Reached,
@@ -132,13 +132,15 @@ impl Conclusion {
 #[cfg(test)]
 mod tests {
     use super::*;
+    use crate::antibot::Vendor;
 
     #[test]
     fn test_a_challenge_asks_for_a_browser_and_a_block_does_not() {
         // The distinction the whole escalation path rests on. A block might
         // yield to different settings; a challenge needs JavaScript and never
         // will, so only one of these justifies starting a browser.
-        let challenge = Conclusion::from_attempts(&Outcome::Challenge(Vendor::Cloudflare), &[]);
+        let challenge =
+            Conclusion::from_attempts(&Outcome::Challenge(Vendor::new("cloudflare")), &[]);
         assert!(challenge.needs_browser());
         assert_eq!(
             challenge,
@@ -147,7 +149,7 @@ mod tests {
             }
         );
 
-        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::Akamai), &[]);
+        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::new("akamai")), &[]);
         assert!(!blocked.needs_browser());
     }
 
@@ -155,13 +157,13 @@ mod tests {
     fn test_an_unnamed_vendor_is_left_unnamed() {
         // Reporting "unknown" as though it were a product would invite a
         // consumer to branch on it. An absent vendor is the honest answer.
-        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::Unknown), &[]);
+        let blocked = Conclusion::from_attempts(&Outcome::Blocked(Vendor::unknown()), &[]);
         assert_eq!(blocked, Conclusion::Blocked { vendor: None });
     }
 
     #[test]
     fn test_reaching_it_is_not_an_escalation() {
-        for outcome in [Outcome::Ok, Outcome::Present(Vendor::Akamai)] {
+        for outcome in [Outcome::Ok, Outcome::Present(Vendor::new("akamai"))] {
             let c = Conclusion::from_attempts(&outcome, &[]);
             assert_eq!(c, Conclusion::Reached);
             assert!(!c.needs_browser());
@@ -171,7 +173,11 @@ mod tests {
     #[test]
     fn test_attempts_record_the_profile_that_produced_them() {
         // The log is only useful if it says which rung did what.
-        let a = Attempt::from_outcome("modern", 403, &Outcome::Challenge(Vendor::PerimeterX));
+        let a = Attempt::from_outcome(
+            "modern",
+            403,
+            &Outcome::Challenge(Vendor::new("perimeterx")),
+        );
         assert_eq!(a.profile, "modern");
         assert_eq!(
             a.outcome,
diff --git a/src/response.rs b/src/response.rs
index 5f25b5c..c4b7820 100644
--- a/src/response.rs
+++ b/src/response.rs
@@ -534,7 +534,10 @@ mod protection_tests {
             ],
             "<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD></HTML>",
         );
-        assert_eq!(blocked.protection(), Outcome::Blocked(Vendor::Akamai));
+        assert_eq!(
+            blocked.protection(),
+            Outcome::Blocked(Vendor::new("akamai"))
+        );
     }
 
     #[test]
@@ -548,7 +551,7 @@ mod protection_tests {
         );
         assert_eq!(
             challenge.protection(),
-            Outcome::Challenge(Vendor::Cloudflare)
+            Outcome::Challenge(Vendor::new("cloudflare"))
         );
 
         let ordinary = response_with(200, &[("server", "nginx")], "<html>hello</html>");
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 5a2eb8e..6c3c030 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -66,7 +66,7 @@ fn test_akamai_access_denied_is_a_block() {
     ]);
     assert_eq!(
         classify(&facts(403, &h, AKAMAI_DENIED)),
-        Outcome::Blocked(Vendor::Akamai)
+        Outcome::Blocked(Vendor::new("akamai"))
     );
 }
 
@@ -80,7 +80,7 @@ fn test_akamai_block_detected_without_a_server_header() {
             &h,
             "<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD></HTML>"
         )),
-        Outcome::Blocked(Vendor::Akamai)
+        Outcome::Blocked(Vendor::new("akamai"))
     );
 }
 
@@ -90,7 +90,7 @@ fn test_entity_encoded_edgesuite_still_matches() {
     let h = hdrs(&[("content-type", "text/html")]);
     assert_eq!(
         classify(&facts(403, &h, AKAMAI_DENIED)),
-        Outcome::Blocked(Vendor::Akamai)
+        Outcome::Blocked(Vendor::new("akamai"))
     );
 }
 
@@ -106,7 +106,7 @@ fn test_akamai_session_cookie_on_success_is_present() {
         ("set-cookie", "bm_sz=8C2...; Path=/"),
     ]);
     let outcome = classify(&facts(200, &h, "<html><body>real content</body></html>"));
-    assert_eq!(outcome, Outcome::Present(Vendor::Akamai));
+    assert_eq!(outcome, Outcome::Present(Vendor::new("akamai")));
     assert!(outcome.got_through());
 }
 
@@ -122,7 +122,7 @@ fn test_cloudflare_managed_challenge() {
     ]);
     assert_eq!(
         classify(&facts(403, &h, "<html>Just a moment...</html>")),
-        Outcome::Challenge(Vendor::Cloudflare)
+        Outcome::Challenge(Vendor::new("cloudflare"))
     );
 }
 
@@ -137,7 +137,7 @@ fn test_cloudflare_waf_block_is_not_a_challenge() {
         "expected a block, got {:?}",
         outcome
     );
-    assert_ne!(outcome, Outcome::Challenge(Vendor::Cloudflare));
+    assert_ne!(outcome, Outcome::Challenge(Vendor::new("cloudflare")));
 }
 
 #[test]
@@ -149,7 +149,7 @@ fn test_cloudflare_bm_cookie_on_success_is_present() {
     ]);
     assert_eq!(
         classify(&facts(200, &h, "<html>content</html>")),
-        Outcome::Present(Vendor::Cloudflare)
+        Outcome::Present(Vendor::new("cloudflare"))
     );
 }
 
@@ -163,7 +163,7 @@ fn test_akamai_sec_cpt_428_is_its_own_outcome() {
     ]);
     assert_eq!(
         classify(&facts(428, &h, r#"{"chlg_duration":30,"difficulty":5}"#)),
-        Outcome::Challenge(Vendor::Akamai)
+        Outcome::Challenge(Vendor::new("akamai"))
     );
 }
 
@@ -185,7 +185,7 @@ fn test_imperva_incident_page_is_a_challenge() {
     for status in [200, 403, 503] {
         assert_eq!(
             classify(&facts(status, &h, IMPERVA_INCIDENT)),
-            Outcome::Challenge(Vendor::Imperva)
+            Outcome::Challenge(Vendor::new("imperva"))
         );
     }
 }
@@ -202,7 +202,7 @@ fn test_imperva_loader_on_a_page_is_not_a_challenge() {
     for (status, body) in [(200, &page), (403, &refusal)] {
         assert_ne!(
             classify(&facts(status, &h, body)),
-            Outcome::Challenge(Vendor::Imperva)
+            Outcome::Challenge(Vendor::new("imperva"))
         );
     }
 }
@@ -214,7 +214,7 @@ fn test_imperva_script_without_the_loader_form_is_a_challenge() {
     let body = r#"<html><head><script src="/_Incapsula_Resource?SWJIYLWA=0123456789abcdef"></script></head><body></body></html>"#;
     assert_eq!(
         classify(&facts(200, &h, body)),
-        Outcome::Challenge(Vendor::Imperva)
+        Outcome::Challenge(Vendor::new("imperva"))
     );
 }
 
@@ -266,7 +266,7 @@ fn test_reference_number_alone_is_not_akamai() {
         &h,
         "<html><h1>Access Denied</h1><p>Reference #12345</p></html>",
     ));
-    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::unknown()));
 }
 
 // ── Ordinary responses ────────────────────────────────────────────
@@ -293,7 +293,7 @@ fn test_unidentified_refusal_reports_unknown_vendor() {
     // to guess a vendor.
     let h = hdrs(&[("server", "nginx")]);
     let outcome = classify(&facts(403, &h, "forbidden"));
-    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::unknown()));
     assert!(!outcome.got_through());
 }
 
@@ -305,7 +305,7 @@ fn test_product_cookie_with_refusal_still_counts_as_blocked() {
         ("set-cookie", "__cf_bm=xyz; Path=/"),
     ]);
     let outcome = classify(&facts(403, &h, "<html>denied</html>"));
-    assert_eq!(outcome, Outcome::Blocked(Vendor::Cloudflare));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::new("cloudflare")));
     assert!(!outcome.got_through());
 }
 
@@ -321,7 +321,7 @@ fn test_aws_waf_captcha_is_a_challenge() {
         ("x-cache", "Error from cloudfront"),
     ]);
     let outcome = classify(&facts(405, &h, ""));
-    assert_eq!(outcome, Outcome::Challenge(Vendor::AwsWaf));
+    assert_eq!(outcome, Outcome::Challenge(Vendor::new("awswaf")));
     assert!(
         outcome.indicates_protection(),
         "the ladder only moves on positive evidence of a product"
@@ -336,7 +336,7 @@ fn test_a_bare_403_is_not_evidence_of_protection() {
     // that touches one.
     let h = hdrs(&[("server", "nginx")]);
     let outcome = classify(&facts(403, &h, "forbidden"));
-    assert_eq!(outcome, Outcome::Blocked(Vendor::Unknown));
+    assert_eq!(outcome, Outcome::Blocked(Vendor::unknown()));
     assert!(!outcome.indicates_protection());
 }
 
@@ -366,6 +366,6 @@ fn test_a_marker_in_a_multibyte_body_is_still_found() {
     let body = format!("{}window._cf_chl_opt", "\u{5ae}".repeat(10));
     assert_eq!(
         classify(&facts(403, &h, &body)),
-        Outcome::Challenge(Vendor::Cloudflare)
+        Outcome::Challenge(Vendor::new("cloudflare"))
     );
 }
diff --git a/tests/antibot_rules.rs b/tests/antibot_rules.rs
new file mode 100644
index 0000000..ae14509
--- /dev/null
+++ b/tests/antibot_rules.rs
@@ -0,0 +1,115 @@
+// Loading anti-bot rules from JSON: what a rules file can say, and how a bad
+// one fails.
+
+use blasthttp::antibot::{Outcome, ResponseFacts, Rules, Vendor};
+
+fn facts<'a>(status: u16, headers: &'a [(String, String)], body: &'a str) -> ResponseFacts<'a> {
+    ResponseFacts {
+        status,
+        headers,
+        body,
+    }
+}
+
+fn header(name: &str, value: &str) -> Vec<(String, String)> {
+    vec![(name.to_string(), value.to_string())]
+}
+
+const CUSTOM: &str = r#"{
+    "refusal_statuses": [403],
+    "rules": [
+        {
+            "vendor": "examplewaf",
+            "outcome": "challenge",
+            "when": {"all": [{"status": "refusal"}, {"body_contains": "example-challenge"}]}
+        },
+        {
+            "vendor": "examplewaf",
+            "outcome": "present",
+            "on_refusal": "blocked",
+            "when": {"cookie_prefix": "exwaf_"}
+        },
+        {
+            "vendor": "otherwaf",
+            "outcome": "blocked",
+            "when": {"all": [
+                {"header_contains": {"name": "X-Edge", "value": "OtherWAF"}},
+                {"not": {"status": [200]}}
+            ]}
+        }
+    ]
+}"#;
+
+#[test]
+fn test_a_rules_file_can_name_a_new_vendor() {
+    let rules = Rules::from_json(CUSTOM).unwrap();
+    let none = vec![];
+    assert_eq!(
+        rules.classify(&facts(403, &none, "<html>example-challenge</html>")),
+        Outcome::Challenge(Vendor::new("examplewaf"))
+    );
+    // the same body on a status the file doesn't call a refusal
+    assert_eq!(
+        rules.classify(&facts(503, &none, "<html>example-challenge</html>")),
+        Outcome::Error
+    );
+}
+
+#[test]
+fn test_on_refusal_changes_the_outcome_on_a_refusal_only() {
+    let rules = Rules::from_json(CUSTOM).unwrap();
+    let cookie = header("set-cookie", "exwaf_session=abc; Path=/");
+    assert_eq!(
+        rules.classify(&facts(200, &cookie, "welcome")),
+        Outcome::Present(Vendor::new("examplewaf"))
+    );
+    assert_eq!(
+        rules.classify(&facts(403, &cookie, "go away")),
+        Outcome::Blocked(Vendor::new("examplewaf"))
+    );
+}
+
+#[test]
+fn test_header_contains_ignores_case() {
+    let rules = Rules::from_json(CUSTOM).unwrap();
+    let edge = header("x-edge", "otherwaf-7");
+    assert_eq!(
+        rules.classify(&facts(404, &edge, "")),
+        Outcome::Blocked(Vendor::new("otherwaf"))
+    );
+    assert_eq!(rules.classify(&facts(200, &edge, "")), Outcome::Ok);
+}
+
+#[test]
+fn test_a_rules_file_replaces_the_built_in_rules() {
+    let rules = Rules::from_json(CUSTOM).unwrap();
+    let mitigated = header("cf-mitigated", "challenge");
+    assert_eq!(
+        Rules::default().classify(&facts(403, &mitigated, "")),
+        Outcome::Challenge(Vendor::new("cloudflare"))
+    );
+    assert_eq!(
+        rules.classify(&facts(403, &mitigated, "")),
+        Outcome::Blocked(Vendor::unknown())
+    );
+}
+
+#[test]
+fn test_a_bad_rules_file_is_refused() {
+    for bad in [
+        // not JSON
+        "{",
+        // a condition that doesn't exist
+        r#"{"refusal_statuses": [], "rules": [{"vendor": "x", "outcome": "challenge", "when": {"body_matches": "x"}}]}"#,
+        // an outcome that doesn't exist
+        r#"{"refusal_statuses": [], "rules": [{"vendor": "x", "outcome": "maybe", "when": {"body_contains": "x"}}]}"#,
+        // a misspelled field
+        r#"{"refusal_statuses": [], "rules": [{"vendor": "x", "outcome": "challenge", "wen": {"body_contains": "x"}}]}"#,
+        // no vendor
+        r#"{"refusal_statuses": [], "rules": [{"vendor": "", "outcome": "challenge", "when": {"body_contains": "x"}}]}"#,
+        // no rules at all
+        r#"{"refusal_statuses": []}"#,
+    ] {
+        assert!(Rules::from_json(bad).is_err(), "accepted: {bad}");
+    }
+}
diff --git a/tests/antibot_set_rules.rs b/tests/antibot_set_rules.rs
new file mode 100644
index 0000000..9f6d2b5
--- /dev/null
+++ b/tests/antibot_set_rules.rs
@@ -0,0 +1,31 @@
+// `set_rules` swaps the rules every later classification in the process
+// runs. Its own test binary, since the rules are process-wide.
+
+use blasthttp::antibot::{Outcome, ResponseFacts, Rules, Vendor, classify, set_rules};
+
+#[test]
+fn test_set_rules_replaces_the_rules_classify_runs() {
+    let none = vec![];
+    let page = ResponseFacts {
+        status: 200,
+        headers: &none,
+        body: "<html>example-challenge</html>",
+    };
+    assert_eq!(classify(&page), Outcome::Ok);
+
+    set_rules(
+        Rules::from_json(
+            r#"{"refusal_statuses": [403], "rules": [
+                {"vendor": "examplewaf", "outcome": "challenge", "when": {"body_contains": "example-challenge"}}
+            ]}"#,
+        )
+        .unwrap(),
+    );
+    assert_eq!(
+        classify(&page),
+        Outcome::Challenge(Vendor::new("examplewaf"))
+    );
+
+    set_rules(Rules::default());
+    assert_eq!(classify(&page), Outcome::Ok);
+}
diff --git a/tests/python/test_antibot_rules.py b/tests/python/test_antibot_rules.py
new file mode 100644
index 0000000..5d046a8
--- /dev/null
+++ b/tests/python/test_antibot_rules.py
@@ -0,0 +1,51 @@
+"""Replacing the anti-bot rules from Python, with a JSON file shaped like the built-in one."""
+
+import json
+
+import pytest
+
+import blasthttp
+
+CHALLENGE_PAGE = b"<html>example-challenge</html>"
+
+
+def protection(body):
+    return blasthttp.Response("https://example.test/", 200, headers=[], body=body).protection
+
+
+@pytest.fixture
+def restore_rules():
+    yield
+    blasthttp.set_antibot_rules()
+
+
+def test_a_rules_file_replaces_the_rules(tmp_path, restore_rules):
+    rules = tmp_path / "rules.json"
+    rules.write_text(
+        json.dumps(
+            {
+                "refusal_statuses": [403],
+                "rules": [
+                    {"vendor": "examplewaf", "outcome": "challenge", "when": {"body_contains": "example-challenge"}}
+                ],
+            }
+        )
+    )
+    assert protection(CHALLENGE_PAGE) == ("ok", "-")
+    blasthttp.set_antibot_rules(str(rules))
+    assert protection(CHALLENGE_PAGE) == ("challenge", "examplewaf")
+    blasthttp.set_antibot_rules()
+    assert protection(CHALLENGE_PAGE) == ("ok", "-")
+
+
+def test_a_bad_rules_file_raises_and_keeps_the_rules(tmp_path, restore_rules):
+    rules = tmp_path / "rules.json"
+    rules.write_text('{"refusal_statuses": [], "rules": [{"vendor": "x", "outcome": "maybe", "when": {}}]}')
+    with pytest.raises(ValueError, match="rules.json"):
+        blasthttp.set_antibot_rules(str(rules))
+    with pytest.raises(ValueError, match="missing.json"):
+        blasthttp.set_antibot_rules(str(tmp_path / "missing.json"))
+    assert protection(b'<html><script src="/_Incapsula_Resource?SWJIYLWA=abc"></script></html>') == (
+        "challenge",
+        "imperva",
+    )

From c496315fddedb5a1aa3d306fa298f61e469567cb Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 15:41:42 -0400
Subject: [PATCH 25/44] Format test_profiles_and_report.py

---
 tests/python/test_profiles_and_report.py | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/tests/python/test_profiles_and_report.py b/tests/python/test_profiles_and_report.py
index 6f491d6..a36fd9a 100644
--- a/tests/python/test_profiles_and_report.py
+++ b/tests/python/test_profiles_and_report.py
@@ -59,9 +59,7 @@ def respond(status, headers, body):
         if path == "/echo-headers":
             body = "\n".join(received).encode()
             respond(200, [("Content-Type", "text/plain")], body)
-        elif path == "/cf-challenge" and not any(
-            h.lower().startswith("cookie:") and "pass=1" in h for h in received
-        ):
+        elif path == "/cf-challenge" and not any(h.lower().startswith("cookie:") and "pass=1" in h for h in received):
             # cf-mitigated is the header Cloudflare sets specifically to say it
             # intervened, and is what separates a challenge from a WAF block.
             respond(

From 020335bf79bea54a25c4b817cab857bc92a49c1d Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 16:47:14 -0400
Subject: [PATCH 26/44] Give pre-TLS connect failures their cause, and classify
 connect timeouts as timeouts

On the pooled path, a failure before TLS was reported from hyper-util's
top-level error, which reads "client error (Connect)" whatever went wrong.
The cause is further down the source() chain, so build the message from the
whole chain, and report a timeout anywhere in it as a timeout instead of a
retryable connection error, matching the direct path.

Also drop the substring check for "ssl", "tls" or "certificate" on
dispatch_direct's send_request errors. TLS is finished by then, so it never
matched.
---
 CHANGELOG.md            |   5 ++
 src/client/hyper.rs     | 106 +++++++++++++++++++++++++++++++++++-----
 tests/connect_errors.rs |  53 ++++++++++++++++++++
 3 files changed, 152 insertions(+), 12 deletions(-)
 create mode 100644 tests/connect_errors.rs

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8e5d803..14f3cdd 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -57,6 +57,11 @@ the ladder falling back to `compatibility` on the second handshake.
 - Fixes a consequence of that: TLS failures were being retried, because they
   were misclassified as connection errors, contradicting the rule that says
   they should not be
+- Connect failures before TLS say why too. A refused connection, a DNS failure
+  or a proxy that refuses the tunnel now includes the cause, e.g. "tcp connect
+  error: Connection refused (os error 111)", not only "client error
+  (Connect)", and a connect
+  timeout is reported as a timeout instead of a retryable connection error
 - Asking for a pre-TLS1.2 version without naming a profile now selects
   `compatibility`, rather than failing with "no ciphers available" because the
   default offers nothing that exists in SSLv3
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 9f5d007..f86ef91 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1233,8 +1233,8 @@ async fn dispatch_request(
 /// `test_tls_error_is_not_retryable` says they should not be.
 ///
 /// The real reason was recorded by the connector on its way out, so look it up
-/// by host. Absent an entry, the failure happened before TLS and connection is
-/// the honest answer.
+/// by host. Absent an entry, the failure happened before TLS, and the cause is
+/// further down the error's `source()` chain.
 fn classify_dispatch_error(
     e: &hyper_util::client::legacy::Error,
     uri: &http::Uri,
@@ -1252,7 +1252,38 @@ fn classify_dispatch_error(
             format!("TLS handshake failed: {}", describe_tls_failure(&failure)),
             failure,
         ),
-        None => ClientError::connection(format!("request failed: {}", e)),
+        None => error_from_chain("request failed", e),
+    }
+}
+
+/// Build a `ClientError` from every error in `e`'s `source()` chain.
+///
+/// hyper and hyper-util keep the cause below the top-level error. hyper-util's
+/// top level reads "client error (Connect)" whatever went wrong, the
+/// connector's error under it says "tcp connect error", and only the
+/// `io::Error` under that says "Connection refused". So the message is the
+/// whole chain, and a timeout anywhere in it makes this a timeout rather than
+/// a connection error, which is what the direct path reports for the same
+/// failure.
+fn error_from_chain(context: &str, e: &(dyn std::error::Error + 'static)) -> ClientError {
+    let mut parts = Vec::new();
+    let mut timed_out = false;
+    let mut current = Some(e);
+    while let Some(err) = current {
+        if let Some(io) = err.downcast_ref::<std::io::Error>()
+            && io.kind() == std::io::ErrorKind::TimedOut
+        {
+            timed_out = true;
+        }
+        parts.push(err.to_string());
+        current = err.source();
+    }
+
+    let message = format!("{}: {}", context, parts.join(": "));
+    if timed_out {
+        ClientError::timeout(message)
+    } else {
+        ClientError::connection(message)
     }
 }
 
@@ -1730,15 +1761,13 @@ async fn dispatch_direct(
     }
     debug_record(log, v, 1, "   Sending request via dispatch_direct...");
 
-    let hyper_response = sender.send_request(request).await.map_err(|e| {
-        let msg = format!("dispatch_direct request failed: {}", e);
-        let err_str = e.to_string().to_lowercase();
-        if err_str.contains("ssl") || err_str.contains("tls") || err_str.contains("certificate") {
-            ClientError::tls(msg)
-        } else {
-            ClientError::connection(msg)
-        }
-    })?;
+    // TLS is already done by this point, and a failure there was reported
+    // with its reason by `connect_stream`, so whatever fails here is the
+    // connection.
+    let hyper_response = sender
+        .send_request(request)
+        .await
+        .map_err(|e| error_from_chain("dispatch_direct request failed", &e))?;
 
     let resp = parse_response(hyper_response, config, log).await?;
 
@@ -3222,6 +3251,59 @@ where
 mod tests {
     use super::*;
 
+    /// Stands in for hyper-util's errors: says nothing itself, and keeps the
+    /// cause in `source()`.
+    #[derive(Debug)]
+    struct Wrapper(&'static str, std::io::Error);
+
+    impl std::fmt::Display for Wrapper {
+        fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+            f.write_str(self.0)
+        }
+    }
+
+    impl std::error::Error for Wrapper {
+        fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+            Some(&self.1)
+        }
+    }
+
+    #[test]
+    fn test_error_from_chain_includes_the_cause() {
+        let e = Wrapper(
+            "tcp connect error",
+            std::io::Error::from(std::io::ErrorKind::ConnectionRefused),
+        );
+        let err = error_from_chain("request failed", &e);
+        assert_eq!(err.kind, super::super::ErrorKind::Connection);
+        assert!(
+            err.message
+                .starts_with("request failed: tcp connect error: "),
+            "{}",
+            err.message
+        );
+        assert!(
+            err.message.to_lowercase().contains("refused"),
+            "{}",
+            err.message
+        );
+    }
+
+    #[test]
+    fn test_error_from_chain_finds_a_timeout_below_the_top() {
+        let e = Wrapper(
+            "client error (Connect)",
+            std::io::Error::new(std::io::ErrorKind::TimedOut, "proxy tunnel timed out"),
+        );
+        let err = error_from_chain("request failed", &e);
+        assert_eq!(err.kind, super::super::ErrorKind::Timeout);
+        assert!(
+            err.message.contains("proxy tunnel timed out"),
+            "{}",
+            err.message
+        );
+    }
+
     #[test]
     fn test_exhausted_host_expires() {
         let client = HyperClient::new();
diff --git a/tests/connect_errors.rs b/tests/connect_errors.rs
new file mode 100644
index 0000000..a45dd17
--- /dev/null
+++ b/tests/connect_errors.rs
@@ -0,0 +1,53 @@
+//! A connect failure should say why it failed, on both the pooled path and the
+//! direct one. hyper-util's own error reads "client error (Connect)" whatever
+//! the cause, so these check that the cause underneath makes it out.
+
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::client::{ErrorKind, HttpClient};
+use blasthttp::config::RequestConfig;
+use tokio::net::TcpListener;
+
+/// A local port with nothing listening on it.
+async fn closed_port() -> u16 {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    listener.local_addr().unwrap().port()
+}
+
+fn config_for(url: String) -> RequestConfig {
+    let mut config = RequestConfig::new(url);
+    config.timeout_seconds = Some(5);
+    config.retries = Some(0);
+    config
+}
+
+#[tokio::test]
+async fn test_refused_connection_says_so_on_the_pooled_path() {
+    let port = closed_port().await;
+    let config = config_for(format!("http://127.0.0.1:{}/", port));
+
+    let err = HyperClient::new().send(&config).await.unwrap_err();
+
+    assert_eq!(err.kind, ErrorKind::Connection);
+    assert!(
+        err.message.to_lowercase().contains("refused"),
+        "message should give the cause: {}",
+        err.message
+    );
+}
+
+#[tokio::test]
+async fn test_refused_connection_says_so_on_the_direct_path() {
+    let port = closed_port().await;
+    let mut config = config_for(format!("http://localhost:{}/", port));
+    // resolve_ip sends the request down the direct path
+    config.resolve_ip = Some("127.0.0.1".to_string());
+
+    let err = HyperClient::new().send(&config).await.unwrap_err();
+
+    assert_eq!(err.kind, ErrorKind::Connection);
+    assert!(
+        err.message.to_lowercase().contains("refused"),
+        "message should give the cause: {}",
+        err.message
+    );
+}

From b9e1434f76857cf3d1c1a68f4abde780bdbfb0bf Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 16:57:59 -0400
Subject: [PATCH 27/44] Carry TransportError on batch results, and raise it
 from the mock

A failed batch request reported only its message, so a caller had to parse
it to tell a timeout from a refused connection from a TLS failure.
BatchResult now has transport_error: the TransportError request() would
have raised. error stays the message string.

The mock raised RuntimeError where the real client raises TransportError,
so code that catches TransportError crashed under the mock on any URL a
test didn't mock. It now raises TransportError for an unmocked URL and for
too many redirects, and a mocked batch failure carries it on
transport_error.

Fixes #139
Fixes #140
---
 CHANGELOG.md                             | 11 ++++-
 src/mock.rs                              | 36 ++++++++++++---
 src/python.rs                            | 55 ++++++++++++++++++----
 tests/python/test_mock.py                | 58 ++++++++++++++++++++++++
 tests/python/test_profiles_and_report.py | 22 +++++++++
 5 files changed, 164 insertions(+), 18 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 14f3cdd..9a68b68 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -60,8 +60,15 @@ the ladder falling back to `compatibility` on the second handshake.
 - Connect failures before TLS say why too. A refused connection, a DNS failure
   or a proxy that refuses the tunnel now includes the cause, e.g. "tcp connect
   error: Connection refused (os error 111)", not only "client error
-  (Connect)", and a connect
-  timeout is reported as a timeout instead of a retryable connection error
+  (Connect)", and a connect timeout is reported as a timeout instead of a
+  retryable connection error
+- Batch failures carry the same detail. `BatchResult.transport_error` is the
+  `TransportError` that `request()` would have raised, with `kind`,
+  `retryable`, `conclusion` and the rest. `error` is still the message string
+- `blasthttp.mock` raises `TransportError` where the real client would: for a
+  URL with no mock response (`kind="connection"`) and for too many redirects
+  (`kind="too_many_redirects"`). A mocked batch request for an unmocked URL
+  gets a `transport_error` too
 - Asking for a pre-TLS1.2 version without naming a profile now selects
   `compatibility`, rather than failing with "no ciphers available" because the
   default offers nothing that exists in SSLv3
diff --git a/src/mock.rs b/src/mock.rs
index bd270ea..6e5ff36 100644
--- a/src/mock.rs
+++ b/src/mock.rs
@@ -15,7 +15,8 @@
 //   (auth tuple → header, cookies dict → header, raise_error, etc.)
 //   is the consumer's responsibility, not ours.
 
-use crate::python::PyResponse;
+use crate::client::ClientError;
+use crate::python::{PyResponse, TransportError, client_error_to_py};
 use crate::response::Response;
 use pyo3::exceptions::{PyRuntimeError, PyStopAsyncIteration, PyTypeError};
 use pyo3::prelude::*;
@@ -439,10 +440,12 @@ fn plan_dispatch(
     }
 
     let Some(handler) = pick_handler(py, state, url, method, &req_headers_dict, body)? else {
-        return Err(PyRuntimeError::new_err(format!(
+        // A TransportError like the real client's for an unreachable host,
+        // so `except TransportError` handles both the same way.
+        return Err(client_error_to_py(ClientError::connection(format!(
             "No mock response registered for {} {}",
             method, url
-        )));
+        ))));
     };
 
     match handler.kind {
@@ -801,9 +804,11 @@ impl PyBlasthttpMock {
                     Some(loc) => {
                         hops += 1;
                         if hops > max_hops {
-                            return Err(PyRuntimeError::new_err(format!(
-                                "exceeded {} redirects starting from mock dispatch",
-                                max_hops
+                            return Err(client_error_to_py(ClientError::too_many_redirects(
+                                format!(
+                                    "exceeded {} redirects starting from mock dispatch",
+                                    max_hops
+                                ),
                             )));
                         }
                         current_url = loc;
@@ -1064,7 +1069,7 @@ impl PyMockBatchIterator {
                     return Python::attach(|py| -> PyResult<Py<PyAny>> {
                         match resp_or_err {
                             Ok(resp) => batch_result_obj(py, &entry.url, Some(resp), None),
-                            Err(e) => batch_result_obj(py, &entry.url, None, Some(e.to_string())),
+                            Err(e) => batch_error_obj(py, &entry.url, e),
                         }
                     });
                 }
@@ -1156,6 +1161,23 @@ fn batch_result_obj(
     Ok(cls.call((url,), Some(&kwargs))?.unbind())
 }
 
+/// Build a failed `blasthttp.BatchResult` from a mock dispatch error.
+///
+/// A `TransportError` goes on `transport_error`, with its bare message as
+/// `error` the way a real batch reports it. Anything else, like an exception
+/// from a user's callback, keeps the old `"Type: message"` string.
+fn batch_error_obj(py: Python<'_>, url: &str, e: PyErr) -> PyResult<Py<PyAny>> {
+    let value = e.value(py);
+    if !value.is_instance_of::<TransportError>() {
+        return batch_result_obj(py, url, None, Some(e.to_string()));
+    }
+    let cls = py.import("blasthttp")?.getattr("BatchResult")?;
+    let kwargs = PyDict::new(py);
+    kwargs.set_item("error", value.str()?)?;
+    kwargs.set_item("transport_error", value)?;
+    Ok(cls.call((url,), Some(&kwargs))?.unbind())
+}
+
 // ── Submodule registration ────────────────────────────────────────
 
 pub fn register_mock_submodule<'py>(parent: &Bound<'py, PyModule>) -> PyResult<()> {
diff --git a/src/python.rs b/src/python.rs
index 00b5466..3fe1af6 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -419,7 +419,7 @@ fn conclusion_to_py(c: &crate::report::Conclusion) -> (String, String) {
     (name.to_string(), vendor.unwrap_or_else(|| "-".to_string()))
 }
 
-fn client_error_to_py(e: ClientError) -> PyErr {
+pub(crate) fn client_error_to_py(e: ClientError) -> PyErr {
     let kind = match &e.kind {
         ErrorKind::Connection => "connection",
         ErrorKind::Timeout => "timeout",
@@ -857,6 +857,13 @@ struct PyBatchResult {
     url: String,
     response: Option<Response>,
     error: Option<String>,
+    /// The failure behind `error`, kept so `transport_error` can say more
+    /// than the message. Turned into a Python object only when asked for, so
+    /// a batch doesn't take the GIL for every failed request.
+    client_error: Option<ClientError>,
+    /// A `TransportError` handed to the constructor, which wins over
+    /// `client_error`.
+    transport_error: Option<Py<PyAny>>,
 }
 
 #[pymethods]
@@ -865,15 +872,30 @@ impl PyBatchResult {
     /// fixture mocks. Provide either `response` (a Response instance)
     /// for success, or `error` (a string) for failure. Both default to
     /// None; passing neither yields a degenerate "no response, no error"
-    /// result that `success` reports as False.
+    /// result that `success` reports as False. `transport_error`, if
+    /// given, must be a `TransportError`.
     #[new]
-    #[pyo3(signature = (url, response=None, error=None))]
-    fn new(url: String, response: Option<PyRef<'_, PyResponse>>, error: Option<String>) -> Self {
-        PyBatchResult {
+    #[pyo3(signature = (url, response=None, error=None, transport_error=None))]
+    fn new(
+        url: String,
+        response: Option<PyRef<'_, PyResponse>>,
+        error: Option<String>,
+        transport_error: Option<Bound<'_, PyAny>>,
+    ) -> PyResult<Self> {
+        if let Some(ref t) = transport_error
+            && !t.is_instance_of::<TransportError>()
+        {
+            return Err(pyo3::exceptions::PyTypeError::new_err(
+                "transport_error must be a TransportError",
+            ));
+        }
+        Ok(PyBatchResult {
             url,
             response: response.map(|r| r.inner.clone()),
             error,
-        }
+            client_error: None,
+            transport_error: transport_error.map(Bound::unbind),
+        })
     }
 
     #[getter]
@@ -897,6 +919,19 @@ impl PyBatchResult {
         self.error.clone()
     }
 
+    /// The `TransportError` that `request()` would have raised for this
+    /// failure, with `kind`, `tls_failure`, `retryable`, `conclusion` and
+    /// `attempts`. None if the request succeeded.
+    #[getter]
+    fn transport_error(&self, py: Python<'_>) -> Option<Py<PyAny>> {
+        if let Some(ref t) = self.transport_error {
+            return Some(t.clone_ref(py));
+        }
+        self.client_error
+            .clone()
+            .map(|e| client_error_to_py(e).into_value(py).into_any())
+    }
+
     /// True if the request succeeded
     #[getter]
     fn success(&self) -> bool {
@@ -913,14 +948,16 @@ impl PyBatchResult {
 }
 
 fn to_py_batch_result(r: BatchResult) -> PyBatchResult {
-    let (response, error) = match r.result {
-        Ok(resp) => (Some(resp), None),
-        Err(e) => (None, Some(e.message)),
+    let (response, error, client_error) = match r.result {
+        Ok(resp) => (Some(resp), None, None),
+        Err(e) => (None, Some(e.message.clone()), Some(e)),
     };
     PyBatchResult {
         url: r.url,
         response,
         error,
+        client_error,
+        transport_error: None,
     }
 }
 
diff --git a/tests/python/test_mock.py b/tests/python/test_mock.py
index 3348a7d..452094f 100644
--- a/tests/python/test_mock.py
+++ b/tests/python/test_mock.py
@@ -333,6 +333,64 @@ async def test_batch_unmatched_url_yields_error_result():
     assert "No mock response" in results[0].error
 
 
+# ── errors match the real client ─────────────────────────────────
+
+
+@pytest.mark.asyncio
+async def test_unmatched_url_raises_transport_error():
+    """The real client raises TransportError for a host it can't reach, so the
+    mock does too. Code written as `except TransportError` that branches on
+    `kind` would otherwise crash under the mock on any URL a test didn't mock."""
+    mock = BlasthttpMock()
+    with pytest.raises(blasthttp.TransportError) as exc:
+        await mock.request("http://nothing/")
+    assert "No mock response" in str(exc.value)
+    assert exc.value.kind == "connection"
+    assert exc.value.conclusion == "unreachable"
+
+
+@pytest.mark.asyncio
+async def test_unmatched_url_is_still_a_runtime_error():
+    mock = BlasthttpMock()
+    with pytest.raises(RuntimeError):
+        await mock.request("http://nothing/")
+
+
+@pytest.mark.asyncio
+async def test_redirect_loop_raises_transport_error():
+    mock = BlasthttpMock()
+    mock.add_response(url="http://loop/", status_code=302, headers={"Location": "http://loop/"})
+    with pytest.raises(blasthttp.TransportError) as exc:
+        await mock.request("http://loop/", follow_redirects=True, max_redirects=2)
+    assert exc.value.kind == "too_many_redirects"
+    assert exc.value.retryable is False
+
+
+@pytest.mark.asyncio
+async def test_batch_unmatched_url_carries_transport_error():
+    mock = BlasthttpMock()
+    results = await mock.request_batch([blasthttp.BatchConfig("http://nothing/")])
+    t = results[0].transport_error
+    assert isinstance(t, blasthttp.TransportError)
+    assert t.kind == "connection"
+    assert results[0].error == str(t)
+
+
+@pytest.mark.asyncio
+async def test_batch_callback_error_has_no_transport_error():
+    """An exception from a test's own callback is not a transport failure, so
+    it keeps the plain error string and nothing more."""
+    mock = BlasthttpMock()
+
+    def boom(request):
+        raise ValueError("callback broke")
+
+    mock.add_callback(boom, url="http://cb/")
+    results = await mock.request_batch([blasthttp.BatchConfig("http://cb/")])
+    assert "callback broke" in results[0].error
+    assert results[0].transport_error is None
+
+
 # ── should_intercept + pass-through ──────────────────────────────
 
 
diff --git a/tests/python/test_profiles_and_report.py b/tests/python/test_profiles_and_report.py
index a36fd9a..220a684 100644
--- a/tests/python/test_profiles_and_report.py
+++ b/tests/python/test_profiles_and_report.py
@@ -238,6 +238,28 @@ async def test_transport_error_is_still_a_runtime_error(client):
         await client.request("http://127.0.0.1:1/", timeout=5)
 
 
+async def test_a_failed_batch_request_carries_the_transport_error(client):
+    """A batch used to report only the message, so a caller had to parse it to
+    tell a timeout from a refused connection from a TLS failure."""
+    results = await client.request_batch([blasthttp.BatchConfig("http://127.0.0.1:1/", timeout=5)])
+    r = results[0]
+    assert isinstance(r.transport_error, blasthttp.TransportError)
+    assert r.transport_error.kind == "connection"
+    assert r.transport_error.retryable is True
+    assert r.transport_error.conclusion == "unreachable"
+    assert r.error == str(r.transport_error)
+
+
+async def test_a_successful_batch_request_has_no_transport_error(server, client):
+    results = await client.request_batch([blasthttp.BatchConfig(f"{server}/")])
+    assert results[0].transport_error is None
+
+
+def test_batch_result_takes_only_a_transport_error():
+    with pytest.raises(TypeError):
+        blasthttp.BatchResult("http://x/", error="e", transport_error=ValueError("e"))
+
+
 async def test_batch_results_carry_the_report(server, client):
     cfgs = [
         blasthttp.BatchConfig(f"{server}/"),

From f7d661b4e7423c37e9284b6203004208abd4b433 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 2 Nov 2026 18:21:52 -0500
Subject: [PATCH 28/44] Keep anonymous suites out once verification is on

An aNULL suite sends no certificate, and OpenSSL skips verify-peer and the
hostname check when none arrives, so a caller who set verify_certs could get
a handshake that reported success while checking nothing. Append !aNULL to
whichever cipher list is in effect when verification is on, including an
explicit cipher_string. With verification off nothing changes.
---
 CHANGELOG.md            |   1 +
 README.md               |   4 +-
 src/client/hyper.rs     |  62 ++++++++++++++++--
 src/profile.rs          |   4 +-
 tests/legacy_default.rs | 139 ++++++++++++++++++++++++++++++++++++++++
 5 files changed, 201 insertions(+), 9 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d9321c0..d739396 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -110,6 +110,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - SSLv3 works. `min_tls_version` / `max_tls_version` / `--min-tls` / `--max-tls` accept `3.0` (also `ssl3`, `sslv3`), and an SSLv3-only server is reachable without asking for it
 - The OpenSSL build cache is keyed on the feature flags and version as well as the target, so changing what the build asks for triggers a rebuild. The first run after upgrading rebuilds once
 - Export ciphers are not supported. OpenSSL removed them outright in 1.1.0, so no build flag brings them back
+- With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
 
 ## 0.10.1
 
diff --git a/README.md b/README.md
index 0928e47..06fbae1 100644
--- a/README.md
+++ b/README.md
@@ -117,7 +117,7 @@ Output is JSON (one object per response), including status, headers, redirect ch
 | `-x, --proxy` | HTTP/SOCKS proxy URL | |
 | `--no-proxy` | Host that bypasses the proxy: `host`, `*.suffix`, IP, CIDR, or `*` (repeatable) | |
 | `--profile` | Connection profile: `modern`, `compatibility`, `chrome` | `modern` |
-| `--ciphers` | OpenSSL cipher string | from the profile |
+| `--ciphers` | OpenSSL cipher string (`:!aNULL` is appended under `--verify`) | from the profile |
 | `--min-tls` | Minimum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
 | `--max-tls` | Maximum TLS version (`3.0` for SSLv3, or 1.0–1.3) | |
 | `-v, --verbose` | Verbose output (-vv includes body) | |
@@ -245,7 +245,7 @@ All parameters except `url` are optional:
 | `proxy` | `str` | HTTP/SOCKS proxy URL |
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
 | `profile` | `str` | Connection profile: `modern` (default), `compatibility`, `chrome` |
-| `cipher_string` | `str` | OpenSSL cipher string. Overrides the profile's, and pins it so the ladder will not move |
+| `cipher_string` | `str` | OpenSSL cipher string. Overrides the profile's, and pins it so the ladder will not move. `:!aNULL` is appended whenever `verify_certs` is on, since an anonymous suite would skip the check |
 | `min_tls_version` | `str` | Minimum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `max_tls_version` | `str` | Maximum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `retries` | `int` | Number of retries on failure |
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index f86ef91..f825c84 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -8,6 +8,7 @@ use http_body_util::BodyExt;
 use hyper_util::client::legacy::Client;
 use hyper_util::client::legacy::connect::HttpConnector;
 use hyper_util::rt::TokioExecutor;
+use std::borrow::Cow;
 use std::collections::HashMap;
 use std::future::Future;
 use std::io::Read;
@@ -192,6 +193,21 @@ pub(crate) fn bisect_disabled(layer: &str) -> bool {
 /// Precedence is: an explicit setting beats the profile, and the profile beats
 /// the default. A caller who names a cipher string means it, and several
 /// existing tests depend on that still being true.
+/// The cipher list to offer: `base` untouched when verification is off, with
+/// `!aNULL` appended when it is on.
+///
+/// An anonymous suite sends no certificate, and OpenSSL skips verify-peer (and
+/// the hostname check with it) when none arrives, so offering one to a caller
+/// who set `verify_certs` would report a verified handshake that checked
+/// nothing. This applies to a caller's own `cipher_string` too.
+fn verified_cipher_list(base: &str, verify: bool) -> Cow<'_, str> {
+    if verify {
+        Cow::Owned(format!("{base}:!aNULL"))
+    } else {
+        Cow::Borrowed(base)
+    }
+}
+
 fn apply_tls_settings(
     builder: &mut openssl::ssl::SslConnectorBuilder,
     config: &RequestConfig,
@@ -275,12 +291,15 @@ fn apply_tls_settings(
     // reach the wire whatever the security level says, so a server speaking
     // only one of them is unreachable, which is the whole reason this project
     // builds its own OpenSSL.
-    let ciphers = config
-        .cipher_string
-        .as_deref()
-        .unwrap_or(profile.tls.cipher_list);
+    let ciphers = verified_cipher_list(
+        config
+            .cipher_string
+            .as_deref()
+            .unwrap_or(profile.tls.cipher_list),
+        config.should_verify_certs(),
+    );
     builder
-        .set_cipher_list(ciphers)
+        .set_cipher_list(&ciphers)
         .map_err(|e| ClientError::tls(format!("invalid cipher string '{}': {}", ciphers, e)))?;
 
     let min_version = config
@@ -4083,4 +4102,37 @@ mod tests {
         assert!(TunnelProxy::from_url("ftp://proxy:21").is_err());
         assert!(TunnelProxy::from_url("http://:8080").is_err());
     }
+
+    // ── verified_cipher_list ──────────────────────────────────────
+
+    #[test]
+    fn test_unverified_list_is_untouched() {
+        assert_eq!(verified_cipher_list("ALL", false), "ALL");
+        assert_eq!(verified_cipher_list("RC4-SHA", false), "RC4-SHA");
+        assert_eq!(
+            verified_cipher_list("ADH-AES128-SHA", false),
+            "ADH-AES128-SHA"
+        );
+    }
+
+    #[test]
+    fn test_unverified_list_is_borrowed() {
+        assert!(matches!(
+            verified_cipher_list("ALL", false),
+            Cow::Borrowed("ALL")
+        ));
+    }
+
+    #[test]
+    fn test_verifying_drops_anonymous_suites() {
+        assert_eq!(verified_cipher_list("ALL", true), "ALL:!aNULL");
+    }
+
+    #[test]
+    fn test_verifying_drops_them_from_a_caller_list_too() {
+        assert_eq!(
+            verified_cipher_list("ADH-AES128-SHA", true),
+            "ADH-AES128-SHA:!aNULL"
+        );
+    }
 }
diff --git a/src/profile.rs b/src/profile.rs
index 5542141..8254071 100644
--- a/src/profile.rs
+++ b/src/profile.rs
@@ -227,8 +227,8 @@ pub const COMPATIBILITY: ConnectionProfile = ConnectionProfile {
         ciphersuites: None,
         // Every suite the build provides except the eNULL ones. Includes the
         // aNULL suites, which skip authentication: worth reaching for a
-        // scanner, and this client already does not verify certificates by
-        // default, so they give up nothing that was being enforced.
+        // scanner while verification is off, and dropped whenever it is on
+        // (see `verified_cipher_list`).
         cipher_list: "ALL",
         sigalgs: None,
         groups: None,
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index 738b073..7a25b58 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -486,3 +486,142 @@ async fn test_a_pinned_request_does_not_teach_the_host_memory() {
 
     server.shutdown().await;
 }
+
+// ── Anonymous suites must not survive turning verification on ─────
+//
+// The other half of `test_anon_dh_reachable_by_default`: with verification off
+// anonymous suites stay reachable, and with it on they are never offered, since
+// a suite with no certificate would make the check pass while checking nothing.
+
+/// An anonymous-DH-only server. These suites only exist at TLS 1.2 and below.
+async fn anon_dh_server() -> TlsTestServer {
+    TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("ADH-AES128-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await
+}
+
+/// Ask for verification against an anonymous-DH server and assert we do not
+/// reach it. `cipher_string` is whatever the caller would have set.
+async fn assert_unreachable_with_verification(cipher_string: Option<&str>) {
+    let server = anon_dh_server().await;
+
+    let mut config = default_config(&server.url());
+    config.verify_certs = Some(true);
+    config.cipher_string = cipher_string.map(str::to_string);
+
+    let result = HyperClient::new().send(&config).await;
+    assert!(
+        result.is_err(),
+        "verify_certs was on, so a server with no certificate must be out of \
+         reach, but the request succeeded with status {:?}",
+        result.as_ref().ok().map(|r| r.status)
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_anon_dh_unreachable_when_verifying() {
+    // No pin, so this walks the ladder: `compatibility` must not bring the
+    // anonymous suites back either.
+    assert_unreachable_with_verification(None).await;
+}
+
+#[tokio::test]
+async fn test_anon_dh_unreachable_when_verifying_with_explicit_ciphers() {
+    // `ALL` admits the anonymous suites. The caller also asked for
+    // verification, and that is the half we honor.
+    assert_unreachable_with_verification(Some("ALL")).await;
+}
+
+#[tokio::test]
+async fn test_anon_dh_unreachable_when_verifying_on_direct_path() {
+    // `resolve_ip` builds its TLS context through `connect_stream`, not the
+    // pooled connector, so cover that path too.
+    let server = anon_dh_server().await;
+
+    let mut config = default_config(&server.url());
+    config.verify_certs = Some(true);
+    config.resolve_ip = Some("127.0.0.1".to_string());
+
+    let result = HyperClient::new().send(&config).await;
+    assert!(
+        result.is_err(),
+        "the direct path must exclude anonymous suites too, but got status {:?}",
+        result.as_ref().ok().map(|r| r.status)
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_anon_dh_reachable_on_direct_path_without_verification() {
+    // With verification off the same path still reaches it.
+    let server = anon_dh_server().await;
+
+    let mut config = default_config(&server.url());
+    config.resolve_ip = Some("127.0.0.1".to_string());
+
+    let result = HyperClient::new().send(&config).await;
+    assert!(
+        result.is_ok(),
+        "verification is off, so the direct path should still reach an \
+         anonymous-DH server: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
+/// The cipher suites `compatibility` offers, with verification on or off.
+async fn compatibility_hello_ciphers(verify: bool) -> Vec<u16> {
+    let server = TlsTestServer::start(TlsServerConfig::default()).await;
+
+    let mut config = default_config(&server.url());
+    config.profile = Some("compatibility".to_string());
+    config.verify_certs = Some(verify);
+    // The self-signed cert fails verification; only the hello matters here.
+    let _ = HyperClient::new().send(&config).await;
+
+    let hello = server
+        .hello
+        .lock()
+        .expect("hello slot poisoned")
+        .clone()
+        .expect("server recorded no ClientHello");
+    server.shutdown().await;
+    hello.ciphers
+}
+
+#[tokio::test]
+async fn test_verification_removes_only_anonymous_suites() {
+    // TLS_DH_anon_WITH_AES_128_CBC_SHA (ADH-AES128-SHA).
+    const ADH_AES128_SHA: u16 = 0x0034;
+
+    let unverified = compatibility_hello_ciphers(false).await;
+    let verified = compatibility_hello_ciphers(true).await;
+
+    assert!(
+        unverified.contains(&ADH_AES128_SHA),
+        "with verification off, compatibility must still offer anonymous suites"
+    );
+    assert!(
+        !verified.contains(&ADH_AES128_SHA),
+        "with verification on, anonymous suites must not be offered"
+    );
+    // Removing them must cost nothing else: what remains is the unverified
+    // offer, in the same order, minus the anonymous suites.
+    let mut remaining = unverified.iter().filter(|c| verified.contains(c));
+    assert!(
+        verified.iter().all(|c| remaining.next() == Some(c)),
+        "verification should only remove suites, not add or reorder them"
+    );
+    assert!(
+        verified.len() < unverified.len(),
+        "expected fewer suites with verification on"
+    );
+}

From eb6959aad45639bd706fd7b9185138a5a9a57013 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Mon, 26 Oct 2026 15:43:15 -0400
Subject: [PATCH 29/44] Only trust Cloudflare challenge body markers on a
 refusal

---
 antibot_rules.json        | 11 +++++++----
 tests/antibot_classify.rs | 40 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+), 4 deletions(-)

diff --git a/antibot_rules.json b/antibot_rules.json
index b3bc56e..ad8dace 100644
--- a/antibot_rules.json
+++ b/antibot_rules.json
@@ -10,11 +10,14 @@
     {
       "vendor": "cloudflare",
       "outcome": "challenge",
-      "when": {"any": [
-        {"body_contains": "/cdn-cgi/challenge-platform/"},
-        {"body_contains": "window._cf_chl_opt"}
+      "when": {"all": [
+        {"status": "refusal"},
+        {"any": [
+          {"body_contains": "/cdn-cgi/challenge-platform/"},
+          {"body_contains": "window._cf_chl_opt"}
+        ]}
       ]},
-      "why": "The challenge page's own markers, for when cf-mitigated is missing."
+      "why": "The challenge page's own markers, for when cf-mitigated is missing. Only on a refusal: on a 200 they're the vendor's script loading normally, as in Turnstile's api.js and pages carrying Cloudflare's injected JavaScript detections."
     },
     {
       "vendor": "akamai",
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 6c3c030..f0858a5 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -126,6 +126,46 @@ fn test_cloudflare_managed_challenge() {
     );
 }
 
+#[test]
+fn test_cloudflare_script_is_not_a_challenge() {
+    // Turnstile's api.js names the challenge platform, but it's a script that
+    // loaded fine, not an interception.
+    let h = hdrs(&[
+        ("server", "cloudflare"),
+        ("content-type", "application/javascript; charset=UTF-8"),
+    ]);
+    let body = r#"var u="/cdn-cgi/challenge-platform/";function render(){}"#;
+    assert_ne!(
+        classify(&facts(200, &h, body)),
+        Outcome::Challenge(Vendor::new("cloudflare"))
+    );
+}
+
+#[test]
+fn test_cloudflare_javascript_detections_page_is_not_a_challenge() {
+    // Cloudflare injects this script tag into ordinary pages on sites with
+    // JavaScript detections on.
+    let h = hdrs(&[("server", "cloudflare"), ("content-type", "text/html")]);
+    let body = r#"<html><body>welcome<script src="/cdn-cgi/challenge-platform/scripts/jsd/main.js"></script></body></html>"#;
+    assert_ne!(
+        classify(&facts(200, &h, body)),
+        Outcome::Challenge(Vendor::new("cloudflare"))
+    );
+}
+
+#[test]
+fn test_cloudflare_challenge_body_on_a_refusal_is_a_challenge() {
+    // Without cf-mitigated, the markers still identify the challenge page.
+    let h = hdrs(&[("server", "cloudflare")]);
+    let body = "<html><script>window._cf_chl_opt={};</script></html>";
+    for status in [403, 503] {
+        assert_eq!(
+            classify(&facts(status, &h, body)),
+            Outcome::Challenge(Vendor::new("cloudflare"))
+        );
+    }
+}
+
 #[test]
 fn test_cloudflare_waf_block_is_not_a_challenge() {
     // A plain WAF refusal carries no cf-mitigated header. Calling it a

From 0c313338fd7fca0a272e51604b17e0b574b9ee66 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 3 Nov 2026 15:10:19 -0500
Subject: [PATCH 30/44] Decode zstd

chrome131 advertises zstd in accept-encoding, as Chrome does, and Cloudflare
answers with it, so bodies from those hosts came back undecoded. Decode every
frame, skip skippable ones, and refuse windows over RFC 9659's 8 MiB.
---
 CHANGELOG.md        |   1 +
 Cargo.lock          |  16 ++++++
 Cargo.toml          |   1 +
 src/client/hyper.rs | 121 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 139 insertions(+)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d739396..b28bf32 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -110,6 +110,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - SSLv3 works. `min_tls_version` / `max_tls_version` / `--min-tls` / `--max-tls` accept `3.0` (also `ssl3`, `sslv3`), and an SSLv3-only server is reachable without asking for it
 - The OpenSSL build cache is keyed on the feature flags and version as well as the target, so changing what the build asks for triggers a rebuild. The first run after upgrading rebuilds once
 - Export ciphers are not supported. OpenSSL removed them outright in 1.1.0, so no build flag brings them back
+- `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
 
 ## 0.10.1
diff --git a/Cargo.lock b/Cargo.lock
index 5c59da7..22de470 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -105,6 +105,7 @@ dependencies = [
  "psl",
  "pyo3",
  "pyo3-async-runtimes",
+ "ruzstd",
  "serde",
  "serde_json",
  "thiserror",
@@ -706,6 +707,15 @@ dependencies = [
  "proc-macro2",
 ]
 
+[[package]]
+name = "ruzstd"
+version = "0.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "69e4ce4a039f42bf25920b95667e4d789e98749ab11bbb8d2c2936f788421bb5"
+dependencies = [
+ "twox-hash",
+]
+
 [[package]]
 name = "serde"
 version = "1.0.229"
@@ -940,6 +950,12 @@ version = "0.2.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b"
 
+[[package]]
+name = "twox-hash"
+version = "2.1.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "86a801b3cea342a06d468c8710662aa29e5e05e4f5c0d62f00bbb7f2ad7941c2"
+
 [[package]]
 name = "unicode-ident"
 version = "1.0.24"
diff --git a/Cargo.toml b/Cargo.toml
index d278f64..9434ece 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -45,6 +45,7 @@ http = "1"
 # Decompression
 flate2 = "1"
 brotli = "9"
+ruzstd = "0.9.1"
 
 # Public Suffix List — decides where a cookie's `Domain` stops being allowed
 # to widen. Without it `Domain=com` or `Domain=github.io` is accepted and the
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index f825c84..cb1250d 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -3092,6 +3092,7 @@ enum Codec {
     Gzip,
     Deflate,
     Brotli,
+    Zstd,
     Identity,
 }
 
@@ -3103,6 +3104,7 @@ impl Codec {
             "gzip" | "x-gzip" => Some(Codec::Gzip),
             "deflate" => Some(Codec::Deflate),
             "br" => Some(Codec::Brotli),
+            "zstd" => Some(Codec::Zstd),
             "identity" => Some(Codec::Identity),
             _ => None,
         }
@@ -3113,6 +3115,7 @@ impl Codec {
             Codec::Gzip => "gzip",
             Codec::Deflate => "deflate",
             Codec::Brotli => "brotli",
+            Codec::Zstd => "zstd",
             Codec::Identity => "identity",
         }
     }
@@ -3125,6 +3128,7 @@ fn decode_one(codec: Codec, data: &[u8], max_size: usize) -> Inflated {
         Codec::Gzip => read_bounded(flate2::read::GzDecoder::new(data), max_size, codec),
         Codec::Deflate => decode_deflate(data, max_size),
         Codec::Brotli => read_bounded(brotli::Decompressor::new(data, 4096), max_size, codec),
+        Codec::Zstd => decode_zstd(data, max_size),
         Codec::Identity => Inflated::Whole(data.to_vec()),
     }
 }
@@ -3175,6 +3179,56 @@ fn decode_deflate(data: &[u8], max_size: usize) -> Inflated {
     }
 }
 
+/// RFC 9659's window limit for `Content-Encoding: zstd`. A frame asking for
+/// more is refused rather than allowed to make us allocate it.
+const ZSTD_MAX_WINDOW: u64 = 8 * 1024 * 1024;
+
+/// Undo `Content-Encoding: zstd`, which may hold several frames back to back,
+/// including skippable ones. The decoder reads one frame at a time, so this
+/// walks them, stopping at `max_size` bytes of output.
+fn decode_zstd(data: &[u8], max_size: usize) -> Inflated {
+    use ruzstd::decoding::StreamingDecoder;
+    use ruzstd::decoding::errors::{FrameDecoderError, ReadFrameHeaderError};
+
+    let fail = |out: Vec<u8>, e: String| {
+        if out.is_empty() {
+            Inflated::Failed(format!("zstd decompression failed: {}", e))
+        } else {
+            Inflated::Partial(out, format!("zstd stream did not decode cleanly: {}", e))
+        }
+    };
+
+    let mut src = data;
+    let mut out = Vec::new();
+    loop {
+        let decoder = match StreamingDecoder::new_with_max_window_size(&mut src, ZSTD_MAX_WINDOW) {
+            Ok(d) => d,
+            Err(FrameDecoderError::ReadFrameHeaderError(ReadFrameHeaderError::SkipFrame {
+                length,
+                ..
+            })) => {
+                let Some(rest) = src.get(length as usize..) else {
+                    return fail(out, "skippable frame runs past the end".to_string());
+                };
+                src = rest;
+                if src.is_empty() {
+                    break;
+                }
+                continue;
+            }
+            Err(e) => return fail(out, e.to_string()),
+        };
+        let room = (max_size - out.len()) as u64;
+        if let Err(e) = decoder.take(room).read_to_end(&mut out) {
+            return fail(out, e.to_string());
+        }
+        if out.len() >= max_size || src.is_empty() {
+            break;
+        }
+    }
+    Inflated::Whole(out)
+}
+
 /// Whether `data` starts with something that reads as a zlib header (RFC 1950
 /// 2.2): compression method 8 in the low nibble of CMF, and a CMF/FLG pair
 /// that is a multiple of 31. True for every zlib stream, and true for a raw
@@ -3540,11 +3594,16 @@ mod tests {
         out
     }
 
+    fn zstd_bytes(data: &[u8]) -> Vec<u8> {
+        ruzstd::encoding::compress_to_vec(data, ruzstd::encoding::CompressionLevel::Fastest)
+    }
+
     fn each_codec() -> Vec<(&'static str, Vec<u8>)> {
         vec![
             ("gzip", gzip_bytes(SAMPLE)),
             ("deflate", deflate_bytes(SAMPLE)),
             ("br", brotli_bytes(SAMPLE)),
+            ("zstd", zstd_bytes(SAMPLE)),
         ]
     }
 
@@ -3911,6 +3970,68 @@ mod tests {
         assert_eq!(decoded("gzip,gzip", &twice, NO_LIMIT), SAMPLE);
     }
 
+    // ── zstd ──────────────────────────────────────────────────────
+
+    #[test]
+    fn test_zstd_reads_every_frame() {
+        // A body may be several frames back to back; all of them are content.
+        let mut two = zstd_bytes(b"first ");
+        two.extend(zstd_bytes(b"second"));
+        assert_eq!(decoded("zstd", &two, NO_LIMIT), b"first second");
+    }
+
+    #[test]
+    fn test_zstd_skips_skippable_frames() {
+        // Magic 0x184D2A50, a little-endian length, then that many bytes.
+        let mut body = vec![0x50, 0x2A, 0x4D, 0x18, 3, 0, 0, 0, b'x', b'y', b'z'];
+        body.extend(zstd_bytes(SAMPLE));
+        assert_eq!(decoded("zstd", &body, NO_LIMIT), SAMPLE);
+    }
+
+    #[test]
+    fn test_zstd_output_is_bounded() {
+        let zeros = vec![0u8; 200_000];
+        assert_eq!(decoded("zstd", &zstd_bytes(&zeros), 4096).len(), 4096);
+        // The cap spans frames too.
+        let mut two = zstd_bytes(&zeros);
+        two.extend(zstd_bytes(&zeros));
+        assert_eq!(decoded("zstd", &two, 250_000).len(), 250_000);
+    }
+
+    #[test]
+    fn test_zstd_refuses_an_oversized_window() {
+        // A frame header asking for a 16 MiB window, over RFC 9659's 8 MiB.
+        // Refused before anything is allocated, and the body comes back raw.
+        let body = [0x28, 0xB5, 0x2F, 0xFD, 0x00, 14 << 3];
+        let (raw, reason) = undecoded("zstd", &body, NO_LIMIT);
+        assert_eq!(raw, body);
+        assert!(
+            reason.contains("zstd"),
+            "reason should name the codec: {}",
+            reason
+        );
+    }
+
+    #[test]
+    fn test_zstd_flags_a_corrupt_stream() {
+        let big = SAMPLE.repeat(400);
+        let mut z = zstd_bytes(&big);
+        let mid = z.len() / 2;
+        z[mid] ^= 0xff;
+        let out = decompress("zstd", &z, NO_LIMIT);
+        assert!(
+            out.decode_error.is_some(),
+            "a corrupted zstd body must be flagged"
+        );
+        assert_ne!(out.body, big);
+    }
+
+    #[test]
+    fn test_zstd_stacks_with_other_codings() {
+        let stacked = zstd_bytes(&gzip_bytes(SAMPLE));
+        assert_eq!(decoded("gzip, zstd", &stacked, NO_LIMIT), SAMPLE);
+    }
+
     #[test]
     fn test_decompress_ignores_identity_entries_in_a_list() {
         let gz = gzip_bytes(SAMPLE);

From 2f9290419ef86fc4c989e6e058c845d3e7c8abfc Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 3 Nov 2026 15:11:21 -0500
Subject: [PATCH 31/44] Let a bare cipher_string reach pre-1.2 servers

With no profile named, a cipher_string resolved to modern and inherited its
TLS 1.2 floor, and pinning kept the ladder from widening it, so
cipher_string="RC4-MD5" couldn't reach a TLS 1.0 server. Select compatibility
instead, as a pre-1.2 version request already does.
---
 CHANGELOG.md            |  1 +
 README.md               |  2 +-
 src/config.rs           | 32 +++++++++++++++++++++---
 tests/legacy_default.rs | 55 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 86 insertions(+), 4 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index b28bf32..4152b57 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -110,6 +110,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - SSLv3 works. `min_tls_version` / `max_tls_version` / `--min-tls` / `--max-tls` accept `3.0` (also `ssl3`, `sslv3`), and an SSLv3-only server is reachable without asking for it
 - The OpenSSL build cache is keyed on the feature flags and version as well as the target, so changing what the build asks for triggers a rebuild. The first run after upgrading rebuilds once
 - Export ciphers are not supported. OpenSSL removed them outright in 1.1.0, so no build flag brings them back
+- A `cipher_string` with no `profile` named selects `compatibility`, the same as asking for a pre-1.2 TLS version, so no TLS 1.2 floor applies and `cipher_string="RC4-MD5"` reaches a TLS 1.0 server. Naming a profile as well keeps that profile's floor
 - `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
 
diff --git a/README.md b/README.md
index 06fbae1..b6dd58c 100644
--- a/README.md
+++ b/README.md
@@ -245,7 +245,7 @@ All parameters except `url` are optional:
 | `proxy` | `str` | HTTP/SOCKS proxy URL |
 | `no_proxy` | `list[str]` | Hosts that bypass the proxy |
 | `profile` | `str` | Connection profile: `modern` (default), `compatibility`, `chrome` |
-| `cipher_string` | `str` | OpenSSL cipher string. Overrides the profile's, and pins it so the ladder will not move. `:!aNULL` is appended whenever `verify_certs` is on, since an anonymous suite would skip the check |
+| `cipher_string` | `str` | OpenSSL cipher string. Overrides the profile's, and pins it so the ladder will not move. With no `profile` named it selects `compatibility`, so no TLS version floor applies. `:!aNULL` is appended whenever `verify_certs` is on, since an anonymous suite would skip the check |
 | `min_tls_version` | `str` | Minimum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `max_tls_version` | `str` | Maximum TLS version (`"3.0"` for SSLv3, or `"1.0"`–`"1.3"`) |
 | `retries` | `int` | Number of retries on failure |
diff --git a/src/config.rs b/src/config.rs
index 6275890..ed846fd 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -115,9 +115,14 @@ impl RequestConfig {
         // version alone produces "no ciphers available": a request that names
         // a protocol, and is told that protocol is impossible.
         //
-        // The ladder cannot rescue this, because naming a version pins the
-        // configuration and pinning is what stops the ladder moving.
-        if self.wants_legacy_protocol() {
+        // Naming a cipher string is the same request from the other side: the
+        // caller wants exactly those ciphers, and the default's TLS 1.2 floor
+        // would make RC4 or 3DES against a TLS 1.0 server impossible. Under
+        // `compatibility` the string replaces its list and no floor applies.
+        //
+        // The ladder cannot rescue either, because both pin the configuration
+        // and pinning is what stops the ladder moving.
+        if self.wants_legacy_protocol() || self.cipher_string.is_some() {
             return crate::profile::by_name("compatibility")
                 .unwrap_or_else(|_| crate::profile::default_profile());
         }
@@ -366,6 +371,27 @@ fn ip_in_cidr(ip: IpAddr, net: IpAddr, prefix_len: u8) -> bool {
 mod tests {
     use super::*;
 
+    fn profile_of(c: &RequestConfig) -> &'static str {
+        c.resolved_profile().name
+    }
+
+    #[test]
+    fn a_cipher_string_alone_selects_compatibility() {
+        let mut c = RequestConfig::new("https://example.com".to_string());
+        assert_eq!(profile_of(&c), "modern");
+        c.cipher_string = Some("RC4-MD5".to_string());
+        assert_eq!(profile_of(&c), "compatibility");
+    }
+
+    #[test]
+    fn a_named_profile_wins_over_a_cipher_string() {
+        // Asking for both means the profile's version floor applies too.
+        let mut c = RequestConfig::new("https://example.com".to_string());
+        c.cipher_string = Some("RC4-MD5".to_string());
+        c.profile = Some("modern".to_string());
+        assert_eq!(profile_of(&c), "modern");
+    }
+
     fn pats(v: &[&str]) -> Vec<String> {
         v.iter().map(|s| s.to_string()).collect()
     }
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index 7a25b58..fc994a1 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -625,3 +625,58 @@ async fn test_verification_removes_only_anonymous_suites() {
         "expected fewer suites with verification on"
     );
 }
+
+// ── A cipher string alone carries no version floor ────────────────
+//
+// Naming ciphers is asking for exactly those, and it pins the configuration,
+// so the ladder will not widen it. If the default's TLS 1.2 floor came along,
+// RC4 against a TLS 1.0 server would be impossible to ask for.
+
+/// A server that speaks only TLS 1.0 with RC4-MD5.
+async fn rc4_md5_tls10_server() -> TlsTestServer {
+    TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("RC4-MD5".to_string()),
+        min_tls_version: Some(SslVersion::TLS1),
+        max_tls_version: Some(SslVersion::TLS1),
+        ..Default::default()
+    })
+    .await
+}
+
+#[tokio::test]
+async fn test_cipher_string_alone_reaches_a_tls10_server() {
+    let server = rc4_md5_tls10_server().await;
+
+    let mut config = default_config(&server.url());
+    config.cipher_string = Some("RC4-MD5".to_string());
+
+    let result = HyperClient::new().send(&config).await;
+    assert!(
+        result.is_ok(),
+        "cipher_string with no version set should reach a TLS 1.0 RC4-MD5 \
+         server: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_cipher_string_alone_reaches_a_tls10_server_on_direct_path() {
+    let server = rc4_md5_tls10_server().await;
+
+    let mut config = default_config(&server.url());
+    config.cipher_string = Some("RC4-MD5".to_string());
+    config.resolve_ip = Some("127.0.0.1".to_string());
+
+    let result = HyperClient::new().send(&config).await;
+    assert!(
+        result.is_ok(),
+        "the direct path should honour a bare cipher_string the same way: {:?}",
+        result.as_ref().err()
+    );
+    assert_eq!(result.unwrap().status, 200);
+
+    server.shutdown().await;
+}

From 1d57909af996c955bb8c56ba87fd1ce5a6ba3924 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Tue, 3 Nov 2026 15:13:41 -0500
Subject: [PATCH 32/44] Carry a TLS failure on its own error instead of a
 per-host map

The connector recorded handshake failures by host:port and nothing removed
them, so after one failure every later error to that host, including a
refused connection, was reported as that TLS failure and never retried.
hyper-util keeps the connector's error as source(), so attach the failure
there and find it by walking the chain.
---
 CHANGELOG.md            |   1 +
 src/client/hyper.rs     | 134 +++++++++++++++++-----------------------
 tests/connect_errors.rs |  36 +++++++++++
 3 files changed, 94 insertions(+), 77 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4152b57..4d41d3c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -110,6 +110,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - SSLv3 works. `min_tls_version` / `max_tls_version` / `--min-tls` / `--max-tls` accept `3.0` (also `ssl3`, `sslv3`), and an SSLv3-only server is reachable without asking for it
 - The OpenSSL build cache is keyed on the feature flags and version as well as the target, so changing what the build asks for triggers a rebuild. The first run after upgrading rebuilds once
 - Export ciphers are not supported. OpenSSL removed them outright in 1.1.0, so no build flag brings them back
+- A TLS failure is reported only for the request that hit it. A later error to the same host, such as a refused connection, is classified on its own and retried as usual
 - A `cipher_string` with no `profile` named selects `compatibility`, the same as asking for a pre-1.2 TLS version, so no TLS 1.2 floor applies and `cipher_string="RC4-MD5"` reaches a TLS 1.0 server. Naming a profile as well keeps that profile's floor
 - `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index cb1250d..00544b5 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -359,20 +359,29 @@ type CertSlot = Arc<Mutex<Option<CertInfo>>>;
 /// will lag, but the value will always be an IP that *was* used).
 type PeerSlot = Arc<Mutex<HashMap<String, IpAddr>>>;
 
-/// Where the connector leaves the reason a TLS handshake failed, keyed the
-/// same way as `PeerSlot`.
+/// A failed TLS handshake, as the connector hands it to hyper-util.
 ///
-/// This exists because the reason is destroyed on the way out. The connector
-/// hands hyper-util a `Box<dyn Error>`, hyper-util wraps it in an error whose
-/// `Display` is the literal string "client error (Connect)", and nothing ever
-/// calls `.source()` to get back down to it. So by the time a failure reaches
-/// the caller, a cipher mismatch, a rejected certificate, a DNS failure and a
-/// refused connection are indistinguishable.
-///
-/// Keyed per host rather than held in a single slot because a cached client
-/// serves many hosts concurrently, and `CertSlot` already demonstrates what
-/// last-writer-wins does to a shared slot under load.
-type TlsFailureSlot = Arc<Mutex<HashMap<String, TlsFailure>>>;
+/// hyper-util wraps it in an error whose `Display` is just "client error
+/// (Connect)", but keeps it as that error's `source()`, so
+/// `classify_dispatch_error` finds it again by walking the chain. The reason
+/// travels with the error of the request it belongs to.
+#[derive(Debug)]
+struct TlsHandshakeError {
+    failure: TlsFailure,
+    inner: Box<dyn std::error::Error + Send + Sync>,
+}
+
+impl std::fmt::Display for TlsHandshakeError {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        self.inner.fmt(f)
+    }
+}
+
+impl std::error::Error for TlsHandshakeError {
+    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+        Some(&*self.inner)
+    }
+}
 
 /// Build the lookup key for `PeerSlot` from a URI's host and port.
 /// HTTPS defaults to 443, everything else to 80 — matches what
@@ -492,8 +501,6 @@ struct OpenSslConnector {
     // opened, read after each redirect hop so the right IP gets stamped
     // on the `RedirectHop` (or final `Response`).
     peer_slot: PeerSlot,
-    // Why the last handshake to a given host failed. See `TlsFailureSlot`.
-    tls_failure_slot: TlsFailureSlot,
     connect_timeout: Duration,
     // Set when requests go through a CONNECT or SOCKS5 proxy. The connector
     // opens the tunnel itself, so that TLS is with the target, inside it.
@@ -656,7 +663,6 @@ impl OpenSslConnector {
         config: &RequestConfig,
         cert_slot: CertSlot,
         peer_slot: PeerSlot,
-        tls_failure_slot: TlsFailureSlot,
     ) -> Result<Self, ClientError> {
         // Ensure legacy ciphers (RC4, DES, etc.) are available
         ensure_legacy_provider();
@@ -703,7 +709,6 @@ impl OpenSslConnector {
             ssl,
             cert_slot,
             peer_slot,
-            tls_failure_slot,
             connect_timeout,
             proxy: None,
         })
@@ -832,7 +837,6 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
         let ssl_connector = self.ssl.clone();
         let profile_active = self.profile_active;
         let cert_slot = self.cert_slot.clone();
-        let tls_failure_slot = self.tls_failure_slot.clone();
         let peer_slot = self.peer_slot.clone();
         let connect_timeout = self.connect_timeout;
 
@@ -926,32 +930,27 @@ impl tower_service::Service<http::Uri> for OpenSslConnector {
             let mut stream = tokio_openssl::SslStream::new(ssl_conf, tcp_stream)
                 .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> { Box::new(e) })?;
 
-            // Record why, then box the error away as hyper-util requires.
-            // This is the only point where the reason still exists.
-            let record = |failure: TlsFailure| {
-                if let Some(ref key) = slot_key
-                    && let Ok(mut slot) = tls_failure_slot.lock()
-                {
-                    slot.insert(key.clone(), failure);
-                }
-            };
-
+            // Attach why, so it survives being boxed away as hyper-util requires.
             tokio::time::timeout(connect_timeout, Pin::new(&mut stream).connect())
                 .await
                 .map_err(|_| -> Box<dyn std::error::Error + Send + Sync> {
-                    record(TlsFailure::Timeout);
-                    Box::new(std::io::Error::new(
-                        std::io::ErrorKind::TimedOut,
-                        format!(
-                            "TLS handshake with {} timed out after {}s",
-                            host,
-                            connect_timeout.as_secs()
-                        ),
-                    ))
+                    Box::new(TlsHandshakeError {
+                        failure: TlsFailure::Timeout,
+                        inner: Box::new(std::io::Error::new(
+                            std::io::ErrorKind::TimedOut,
+                            format!(
+                                "TLS handshake with {} timed out after {}s",
+                                host,
+                                connect_timeout.as_secs()
+                            ),
+                        )),
+                    })
                 })?
                 .map_err(|e| -> Box<dyn std::error::Error + Send + Sync> {
-                    record(TlsFailure::from_openssl(&e));
-                    Box::new(e)
+                    Box::new(TlsHandshakeError {
+                        failure: TlsFailure::from_openssl(&e),
+                        inner: Box::new(e),
+                    })
                 })?;
 
             // Extract cert info after successful handshake
@@ -1055,13 +1054,6 @@ pub struct HyperClient {
     // a refusal from it is taken as the answer instead of walking the ladder
     // again; see `ladder_exhausted`.
     host_exhausted: Mutex<HashMap<String, Instant>>,
-    // Why the last handshake to each host failed.
-    //
-    // Deliberately on the client rather than on `CachedClient`: a retry with
-    // different TLS settings builds a *different* cached client, and it still
-    // needs to read why the previous attempt failed. Per-cached-client would
-    // partition this map by exactly the thing it exists to inform.
-    tls_failures: TlsFailureSlot,
 }
 
 impl Default for HyperClient {
@@ -1076,7 +1068,6 @@ impl HyperClient {
             cached: Mutex::new(std::collections::HashMap::new()),
             host_profiles: Mutex::new(HashMap::new()),
             host_exhausted: Mutex::new(HashMap::new()),
-            tls_failures: Arc::new(Mutex::new(HashMap::new())),
         }
     }
 
@@ -1143,12 +1134,7 @@ impl HyperClient {
 
         let cert_slot: CertSlot = Arc::new(Mutex::new(None));
         let peer_slot: PeerSlot = Arc::new(Mutex::new(HashMap::new()));
-        let connector = OpenSslConnector::new(
-            config,
-            cert_slot.clone(),
-            peer_slot.clone(),
-            self.tls_failures.clone(),
-        )?;
+        let connector = OpenSslConnector::new(config, cert_slot.clone(), peer_slot.clone())?;
         let mut builder = Client::builder(TokioExecutor::new());
 
         // Shape the HTTP/2 SETTINGS and connection window to the profile.
@@ -1212,7 +1198,6 @@ async fn dispatch_request(
     config: &RequestConfig,
     log: &DebugLog,
     redirect_cookies: Option<&str>,
-    tls_failures: &TlsFailureSlot,
 ) -> Result<SingleResponse, ClientError> {
     // The pooled high-level client populates Host / :authority from the URI
     // itself, so we don't add a Host header here. Adding it would cause
@@ -1238,7 +1223,7 @@ async fn dispatch_request(
     let hyper_response = client
         .request(request)
         .await
-        .map_err(|e| classify_dispatch_error(&e, uri, tls_failures))?;
+        .map_err(|e| classify_dispatch_error(&e))?;
 
     parse_response(hyper_response, config, log).await
 }
@@ -1251,28 +1236,24 @@ async fn dispatch_request(
 /// connection error. That also meant TLS failures were being retried, which
 /// `test_tls_error_is_not_retryable` says they should not be.
 ///
-/// The real reason was recorded by the connector on its way out, so look it up
-/// by host. Absent an entry, the failure happened before TLS, and the cause is
-/// further down the error's `source()` chain.
-fn classify_dispatch_error(
-    e: &hyper_util::client::legacy::Error,
-    uri: &http::Uri,
-    tls_failures: &TlsFailureSlot,
-) -> ClientError {
-    let recorded = peer_slot_key(uri).and_then(|key| {
-        tls_failures
-            .lock()
-            .ok()
-            .and_then(|slot| slot.get(&key).cloned())
-    });
-
-    match recorded {
-        Some(failure) => ClientError::tls_detailed(
-            format!("TLS handshake failed: {}", describe_tls_failure(&failure)),
-            failure,
-        ),
-        None => error_from_chain("request failed", e),
+/// The connector attaches a failed handshake's reason as a `TlsHandshakeError`
+/// somewhere down the `source()` chain. Without one, the failure happened
+/// before TLS, and `error_from_chain` reports the cause.
+fn classify_dispatch_error(e: &hyper_util::client::legacy::Error) -> ClientError {
+    let mut current: Option<&(dyn std::error::Error + 'static)> = Some(e);
+    while let Some(err) = current {
+        if let Some(tls) = err.downcast_ref::<TlsHandshakeError>() {
+            return ClientError::tls_detailed(
+                format!(
+                    "TLS handshake failed: {}",
+                    describe_tls_failure(&tls.failure)
+                ),
+                tls.failure.clone(),
+            );
+        }
+        current = err.source();
     }
+    error_from_chain("request failed", e)
 }
 
 /// Build a `ClientError` from every error in `e`'s `source()` chain.
@@ -2414,7 +2395,6 @@ impl HyperClient {
                 config,
                 log,
                 hop_cookies,
-                &self.tls_failures,
             )
             .await?
         };
diff --git a/tests/connect_errors.rs b/tests/connect_errors.rs
index a45dd17..2d600c4 100644
--- a/tests/connect_errors.rs
+++ b/tests/connect_errors.rs
@@ -2,9 +2,13 @@
 //! direct one. hyper-util's own error reads "client error (Connect)" whatever
 //! the cause, so these check that the cause underneath makes it out.
 
+mod support;
+
 use blasthttp::client::hyper::HyperClient;
 use blasthttp::client::{ErrorKind, HttpClient};
 use blasthttp::config::RequestConfig;
+use openssl::ssl::SslVersion;
+use support::tls_server::{TlsServerConfig, TlsTestServer};
 use tokio::net::TcpListener;
 
 /// A local port with nothing listening on it.
@@ -51,3 +55,35 @@ async fn test_refused_connection_says_so_on_the_direct_path() {
         err.message
     );
 }
+
+#[tokio::test]
+async fn test_an_earlier_tls_failure_does_not_relabel_a_later_error() {
+    // A TLS failure belongs to the request that hit it. Once the server is
+    // gone, the next request to the same host:port is refused, and that has
+    // to read as a connection error (retryable), not as the old TLS failure.
+    let server = TlsTestServer::start(TlsServerConfig {
+        max_tls_version: Some(SslVersion::TLS1),
+        ..Default::default()
+    })
+    .await;
+    let mut config = config_for(server.url());
+    config.verify_certs = Some(false);
+    // Pinned at 1.2, so the handshake fails and nothing widens past it.
+    config.min_tls_version = Some("1.2".to_string());
+
+    let client = HyperClient::new();
+    let first = client.send(&config).await.unwrap_err();
+    assert_eq!(first.kind, ErrorKind::Tls, "setup: {}", first.message);
+    assert!(first.tls_failure.is_some());
+
+    server.shutdown().await;
+
+    let second = client.send(&config).await.unwrap_err();
+    assert_eq!(
+        second.kind,
+        ErrorKind::Connection,
+        "a refused connection was reported as: {}",
+        second.message
+    );
+    assert!(second.tls_failure.is_none());
+}

From 6361453570b2d5adbaf0efc9f567b9b1a72c9682 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Thu, 5 Nov 2026 14:47:43 -0500
Subject: [PATCH 33/44] Keep a refusal when the escalation after it fails

---
 CHANGELOG.md        |  1 +
 src/client/hyper.rs | 15 ++++++++---
 tests/ladder.rs     | 62 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 74 insertions(+), 4 deletions(-)
 create mode 100644 tests/ladder.rs

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4d41d3c..7eb868b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -114,6 +114,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A `cipher_string` with no `profile` named selects `compatibility`, the same as asking for a pre-1.2 TLS version, so no TLS 1.2 floor applies and `cipher_string="RC4-MD5"` reaches a TLS 1.0 server. Naming a profile as well keeps that profile's floor
 - `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
+- A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error
 
 ## 0.10.1
 
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 00544b5..99681a0 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -2626,6 +2626,9 @@ impl HyperClient {
             let mut rung = self.starting_profile(config, &uri);
             let mut tried: Vec<&'static str> = Vec::new();
             let mut attempts: Vec<crate::report::Attempt> = Vec::new();
+            // The last refusal received, so an escalation that then fails
+            // outright cannot turn a real response into an error.
+            let mut refusal = None;
             let (resp, cached) = loop {
                 tried.push(rung.name);
                 let rung_config = config.with_profile(rung.name);
@@ -2680,6 +2683,7 @@ impl HyperClient {
                                         resp.status, rung.name, next.name
                                     ),
                                 );
+                                refusal = Some((resp, cached));
                                 rung = next;
                             }
                             // Out of rungs: the refusal is the answer. Hand it
@@ -2727,11 +2731,14 @@ impl HyperClient {
                                 );
                                 rung = next;
                             }
-                            // Nothing else to try, or the failure says nothing
-                            // about our offer. Report the error we actually
-                            // got, carrying what was tried so the caller can
-                            // see the ladder ran and where it stopped.
+                            // Nothing else to try. An earlier rung's refusal is
+                            // still the answer; otherwise report the error,
+                            // carrying what was tried.
                             None => {
+                                if let Some(kept) = refusal.take() {
+                                    self.remember_exhausted(config, &uri);
+                                    break kept;
+                                }
                                 e.attempts = attempts;
                                 return Err(e);
                             }
diff --git a/tests/ladder.rs b/tests/ladder.rs
new file mode 100644
index 0000000..6681c0a
--- /dev/null
+++ b/tests/ladder.rs
@@ -0,0 +1,62 @@
+//! The profile ladder, seen from the response a caller gets back.
+
+use blasthttp::client::HttpClient;
+use blasthttp::client::hyper::HyperClient;
+use blasthttp::config::RequestConfig;
+use blasthttp::report::{AttemptOutcome, Conclusion};
+use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::net::TcpListener;
+
+/// Answers one connection with a Cloudflare challenge, then stops listening,
+/// so every later connection to the port is refused.
+async fn challenge_once() -> u16 {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    tokio::spawn(async move {
+        let (mut sock, _) = listener.accept().await.unwrap();
+        drop(listener);
+        let mut buf = [0u8; 4096];
+        let _ = sock.read(&mut buf).await;
+        let body = "<html>Just a moment...</html>";
+        let reply = format!(
+            "HTTP/1.1 403 Forbidden\r\ncf-mitigated: challenge\r\nserver: cloudflare\r\n\
+             content-length: {}\r\nconnection: close\r\n\r\n{}",
+            body.len(),
+            body
+        );
+        let _ = sock.write_all(reply.as_bytes()).await;
+        let _ = sock.shutdown().await;
+    });
+    port
+}
+
+#[tokio::test]
+async fn test_a_failed_escalation_keeps_the_refusal_it_had() {
+    // A refusal moves the ladder on; if the next rung never gets a response,
+    // the caller still gets the refusal, not the later error.
+    let port = challenge_once().await;
+    let mut config = RequestConfig::new(format!("http://127.0.0.1:{port}/"));
+    config.timeout_seconds = Some(5);
+    config.retries = Some(0);
+
+    let resp = HyperClient::new()
+        .send(&config)
+        .await
+        .unwrap_or_else(|e| panic!("the 403 was lost to: {}", e.message));
+
+    assert_eq!(resp.status, 403);
+    assert!(matches!(resp.conclusion(), Conclusion::NeedsBrowser { .. }));
+    assert!(
+        resp.attempts.len() >= 2,
+        "the escalation should be recorded: {:?}",
+        resp.attempts
+    );
+    assert!(matches!(
+        resp.attempts[0].outcome,
+        AttemptOutcome::Challenged { status: 403, .. }
+    ));
+    assert!(matches!(
+        resp.attempts.last().unwrap().outcome,
+        AttemptOutcome::HandshakeFailed { .. }
+    ));
+}

From 16d6d301add3b33069fe2426607c6ef2a3f3413d Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Thu, 5 Nov 2026 15:01:02 -0500
Subject: [PATCH 34/44] Only count Akamai block markers on a refusal

---
 CHANGELOG.md              |  1 +
 antibot_rules.json        | 27 +++++++++++++++------------
 tests/antibot_classify.rs | 18 ++++++++++++++++++
 3 files changed, 34 insertions(+), 12 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7eb868b..6c19567 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -115,6 +115,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
 - A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error
+- Akamai's block rule only fires on a refusal status. edgesuite.net also serves customer assets, so a 200 page loading one was classified as blocked and re-sent
 
 ## 0.10.1
 
diff --git a/antibot_rules.json b/antibot_rules.json
index ad8dace..a8aec7d 100644
--- a/antibot_rules.json
+++ b/antibot_rules.json
@@ -22,20 +22,23 @@
     {
       "vendor": "akamai",
       "outcome": "blocked",
-      "when": {"any": [
-        {"body_contains": "edgesuite"},
-        {"all": [
-          {"any": [
-            {"header_contains": {"name": "server", "value": "akamaighost"}},
-            {"header_present": "akamai-grn"},
-            {"header_present": "x-akam-sw-version"},
-            {"header_present": "x-akamai-transformed"},
-            {"header_contains": {"name": "server-timing", "value": "ak_p"}}
-          ]},
-          {"body_contains": "Access Denied"}
+      "when": {"all": [
+        {"status": "refusal"},
+        {"any": [
+          {"body_contains": "edgesuite"},
+          {"all": [
+            {"any": [
+              {"header_contains": {"name": "server", "value": "akamaighost"}},
+              {"header_present": "akamai-grn"},
+              {"header_present": "x-akam-sw-version"},
+              {"header_present": "x-akamai-transformed"},
+              {"header_contains": {"name": "server-timing", "value": "ak_p"}}
+            ]},
+            {"body_contains": "Access Denied"}
+          ]}
         ]}
       ]},
-      "why": "The edge error page. It HTML-entity-encodes its own punctuation, so the body carries errors&#46;edgesuite&#46;net and a search for errors.edgesuite.net never fires; the bare label edgesuite survives that. server: AkamaiGHost is not reliably present either: www.akamai.com's own 403 sends no server header and identifies itself through akamai-grn, x-akam-sw-version and an ak_p server-timing entry instead, so any one of the edge headers will do. Reference # on its own stays insufficient, because plenty of unrelated error pages carry one."
+      "why": "The edge error page. It HTML-entity-encodes its own punctuation, so the body carries errors&#46;edgesuite&#46;net and a search for errors.edgesuite.net never fires; the bare label edgesuite survives that. server: AkamaiGHost is not reliably present either: www.akamai.com's own 403 sends no server header and identifies itself through akamai-grn, x-akam-sw-version and an ak_p server-timing entry instead, so any one of the edge headers will do. Reference # on its own stays insufficient, because plenty of unrelated error pages carry one. Only on a refusal: edgesuite.net also serves customer assets, so a page that loaded fine can name it."
     },
     {
       "vendor": "akamai",
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index f0858a5..4680c59 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -94,6 +94,24 @@ fn test_entity_encoded_edgesuite_still_matches() {
     );
 }
 
+#[test]
+fn test_akamai_markers_on_a_page_that_loaded_are_not_a_block() {
+    // edgesuite.net also hosts customer assets, and a page can say "Access
+    // Denied" without being refused. Neither is a block on a 200.
+    let assets = r#"<html><img src="https://a1.g.akamai.net.edgesuite.net/logo.png"></html>"#;
+    let article = "<html><h1>Fixing Access Denied errors</h1></html>";
+    for (h, body) in [
+        (hdrs(&[("content-type", "text/html")]), assets),
+        (
+            hdrs(&[("akamai-grn", "0.6d551702.1790644996.315f62a4")]),
+            article,
+        ),
+    ] {
+        let outcome = classify(&facts(200, &h, body));
+        assert!(outcome.got_through(), "{body} classified as {outcome:?}");
+    }
+}
+
 #[test]
 fn test_akamai_session_cookie_on_success_is_present() {
     // The other half of the same comparison: through, and handed a session.

From 98f7767538754912771c4e01b727413c9eccbd59 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Thu, 5 Nov 2026 15:09:15 -0500
Subject: [PATCH 35/44] Classify a 404 or 5xx as reached, not unreachable

---
 CHANGELOG.md              |  3 ++-
 src/antibot.rs            |  8 +++---
 tests/antibot_classify.rs | 12 ++++++++-
 tests/antibot_rules.rs    |  2 +-
 tests/ladder.rs           | 51 +++++++++++++++++++++++++++------------
 5 files changed, 53 insertions(+), 23 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6c19567..d865d5a 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -31,7 +31,8 @@ the ladder falling back to `compatibility` on the second handshake.
   exactly the hosts most likely to rate-limit or ban. Getting through clears
   it, and it expires because a refusal is a judgement that changes
 - **`Response.protection`** reports what a bot-management product did:
-  `(outcome, vendor)` where outcome is ok, present, challenge, blocked or error
+  `(outcome, vendor)` where outcome is ok, present, challenge or blocked. A
+  404 or a 500 is ok: the server answered and no product intervened
 - **`Response.conclusion`** reduces that to one verdict: reached, blocked,
   needs_browser, needs_legacy_tls or unreachable. `needs_browser` means a
   JavaScript challenge, which no HTTP client passes, so the URL should go to a
diff --git a/src/antibot.rs b/src/antibot.rs
index 8e7e7e6..4c8dd70 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -248,13 +248,11 @@ impl Rules {
             }
             .with(vendor);
         }
-        // Nothing identifiable
-        if (200..400).contains(&facts.status) {
-            Outcome::Ok
-        } else if refusal {
+        // Nothing identifiable. A 404 or a 500 is still the server answering.
+        if refusal {
             Outcome::Blocked(Vendor::unknown())
         } else {
-            Outcome::Error
+            Outcome::Ok
         }
     }
 
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 4680c59..279a8d8 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -342,7 +342,17 @@ fn test_ordinary_404_is_not_a_block() {
     // A missing page is a real answer from the origin, and counting it as a
     // block would make an ordinary scan look like it was being refused.
     let h = hdrs(&[("server", "nginx")]);
-    assert_eq!(classify(&facts(404, &h, "not found")), Outcome::Error);
+    assert_eq!(classify(&facts(404, &h, "not found")), Outcome::Ok);
+}
+
+#[test]
+fn test_server_errors_are_answers() {
+    // A 500 or a 502 is the server answering, not a failure to reach it.
+    let h = hdrs(&[("server", "nginx")]);
+    for status in [400, 410, 500, 502, 504] {
+        let outcome = classify(&facts(status, &h, "error"));
+        assert_eq!(outcome, Outcome::Ok, "status {status}");
+    }
 }
 
 #[test]
diff --git a/tests/antibot_rules.rs b/tests/antibot_rules.rs
index ae14509..33185b5 100644
--- a/tests/antibot_rules.rs
+++ b/tests/antibot_rules.rs
@@ -51,7 +51,7 @@ fn test_a_rules_file_can_name_a_new_vendor() {
     // the same body on a status the file doesn't call a refusal
     assert_eq!(
         rules.classify(&facts(503, &none, "<html>example-challenge</html>")),
-        Outcome::Error
+        Outcome::Ok
     );
 }
 
diff --git a/tests/ladder.rs b/tests/ladder.rs
index 6681c0a..a237c00 100644
--- a/tests/ladder.rs
+++ b/tests/ladder.rs
@@ -7,40 +7,46 @@ use blasthttp::report::{AttemptOutcome, Conclusion};
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
 use tokio::net::TcpListener;
 
-/// Answers one connection with a Cloudflare challenge, then stops listening,
-/// so every later connection to the port is refused.
-async fn challenge_once() -> u16 {
+/// Answers one connection with `status`, `headers` and `body`, then stops
+/// listening, so every later connection to the port is refused.
+async fn serve_once(status: &str, headers: &str, body: &'static str) -> u16 {
     let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
     let port = listener.local_addr().unwrap().port();
+    let reply = format!(
+        "HTTP/1.1 {status}\r\n{headers}content-length: {}\r\nconnection: close\r\n\r\n{body}",
+        body.len()
+    );
     tokio::spawn(async move {
         let (mut sock, _) = listener.accept().await.unwrap();
         drop(listener);
         let mut buf = [0u8; 4096];
         let _ = sock.read(&mut buf).await;
-        let body = "<html>Just a moment...</html>";
-        let reply = format!(
-            "HTTP/1.1 403 Forbidden\r\ncf-mitigated: challenge\r\nserver: cloudflare\r\n\
-             content-length: {}\r\nconnection: close\r\n\r\n{}",
-            body.len(),
-            body
-        );
         let _ = sock.write_all(reply.as_bytes()).await;
         let _ = sock.shutdown().await;
     });
     port
 }
 
+fn config(port: u16) -> RequestConfig {
+    let mut config = RequestConfig::new(format!("http://127.0.0.1:{port}/"));
+    config.timeout_seconds = Some(5);
+    config.retries = Some(0);
+    config
+}
+
 #[tokio::test]
 async fn test_a_failed_escalation_keeps_the_refusal_it_had() {
     // A refusal moves the ladder on; if the next rung never gets a response,
     // the caller still gets the refusal, not the later error.
-    let port = challenge_once().await;
-    let mut config = RequestConfig::new(format!("http://127.0.0.1:{port}/"));
-    config.timeout_seconds = Some(5);
-    config.retries = Some(0);
+    let port = serve_once(
+        "403 Forbidden",
+        "cf-mitigated: challenge\r\nserver: cloudflare\r\n",
+        "<html>Just a moment...</html>",
+    )
+    .await;
 
     let resp = HyperClient::new()
-        .send(&config)
+        .send(&config(port))
         .await
         .unwrap_or_else(|e| panic!("the 403 was lost to: {}", e.message));
 
@@ -60,3 +66,18 @@ async fn test_a_failed_escalation_keeps_the_refusal_it_had() {
         AttemptOutcome::HandshakeFailed { .. }
     ));
 }
+
+#[tokio::test]
+async fn test_a_not_found_page_is_reached() {
+    // A 404 is the server answering. It must not read as an unreachable host.
+    let port = serve_once("404 Not Found", "server: nginx\r\n", "not found").await;
+
+    let resp = HyperClient::new().send(&config(port)).await.unwrap();
+
+    assert_eq!(resp.status, 404);
+    assert_eq!(resp.conclusion(), Conclusion::Reached);
+    assert_eq!(
+        resp.attempts[0].outcome,
+        AttemptOutcome::Reached { status: 404 }
+    );
+}

From 5d4d8acbb4dfb8f969df5163302fc92560c21ddf Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Thu, 5 Nov 2026 15:22:04 -0500
Subject: [PATCH 36/44] Stop treating a bot product's cookie or script on a
 refusal as its block

---
 CHANGELOG.md              |  1 +
 antibot_rules.json        | 21 ++++++++------
 src/antibot.rs            | 12 +++++---
 tests/antibot_classify.rs | 58 +++++++++++++++++++++++++++++++--------
 4 files changed, 67 insertions(+), 25 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d865d5a..411b80e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -117,6 +117,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
 - A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error
 - Akamai's block rule only fires on a refusal status. edgesuite.net also serves customer assets, so a 200 page loading one was classified as blocked and re-sent
+- A product's cookie or injected script on a refusal no longer names it as what refused. Bot-management products set their cookies and Cloudflare injects its JavaScript detections script on every response, the origin's own 401s and 403s included, so an ordinary auth failure behind a CDN walked the ladder and could report `needs_browser`. Cloudflare's own error page (a firewall-rule block such as 1020) is recognised as a Cloudflare block instead
 
 ## 0.10.1
 
diff --git a/antibot_rules.json b/antibot_rules.json
index a8aec7d..2939f2f 100644
--- a/antibot_rules.json
+++ b/antibot_rules.json
@@ -13,11 +13,20 @@
       "when": {"all": [
         {"status": "refusal"},
         {"any": [
-          {"body_contains": "/cdn-cgi/challenge-platform/"},
+          {"body_contains": "/cdn-cgi/challenge-platform/h/"},
           {"body_contains": "window._cf_chl_opt"}
         ]}
       ]},
-      "why": "The challenge page's own markers, for when cf-mitigated is missing. Only on a refusal: on a 200 they're the vendor's script loading normally, as in Turnstile's api.js and pages carrying Cloudflare's injected JavaScript detections."
+      "why": "The challenge page's own markers, for when cf-mitigated is missing. Only on a refusal: on a 200 they're the vendor's script loading normally, as in Turnstile's api.js. The /h/ path is the challenge orchestrator; the bare /cdn-cgi/challenge-platform/ prefix also matches the JavaScript detections script Cloudflare injects into every HTML page, origin error pages included."
+    },
+    {
+      "vendor": "cloudflare",
+      "outcome": "blocked",
+      "when": {"all": [
+        {"status": "refusal"},
+        {"body_contains": "cf-error-details"}
+      ]},
+      "why": "Cloudflare's own error page, as served for a firewall rule block (1020). It carries no cf-mitigated header."
     },
     {
       "vendor": "akamai",
@@ -94,38 +103,32 @@
     {
       "vendor": "cloudflare",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"any": [{"cookie_set": "__cf_bm"}, {"cookie_set": "cf_clearance"}]},
-      "why": "From here on, sightings of a product on a response nothing above matched. One that set its cookie while refusing us is still a block."
+      "why": "From here on, sightings of a product on a response nothing above matched. Only a sighting, even on a refusal: these products set their cookies on every response, the origin's own 401s and 403s included, so a cookie is no evidence the product is what refused."
     },
     {
       "vendor": "akamai",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"any": [{"cookie_set": "_abck"}, {"cookie_set": "ak_bmsc"}, {"cookie_set": "bm_sz"}]}
     },
     {
       "vendor": "datadome",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"cookie_set": "datadome"}
     },
     {
       "vendor": "perimeterx",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"cookie_prefix": "_px"}
     },
     {
       "vendor": "imperva",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"any": [{"cookie_prefix": "visid_incap_"}, {"cookie_prefix": "incap_ses_"}]}
     },
     {
       "vendor": "f5",
       "outcome": "present",
-      "on_refusal": "blocked",
       "when": {"header_contains": {"name": "server", "value": "volt-adc"}}
     }
   ]
diff --git a/src/antibot.rs b/src/antibot.rs
index 4c8dd70..f3fcceb 100644
--- a/src/antibot.rs
+++ b/src/antibot.rs
@@ -240,13 +240,17 @@ impl Rules {
     /// Classify one response.
     pub fn classify(&self, facts: &ResponseFacts) -> Outcome {
         let refusal = self.is_refusal(facts.status);
-        if let Some(rule) = self.rules.iter().find(|r| self.matches(&r.when, facts)) {
-            let vendor = Vendor::new(&rule.vendor);
-            return match (refusal, rule.on_refusal) {
+        for rule in self.rules.iter().filter(|r| self.matches(&r.when, facts)) {
+            let outcome = match (refusal, rule.on_refusal) {
                 (true, Some(outcome)) => outcome,
                 _ => rule.outcome,
+            };
+            // A sighting means the product let us through, which a refusal
+            // isn't, and it is no evidence the product is what refused.
+            if refusal && matches!(outcome, RuleOutcome::Present) {
+                continue;
             }
-            .with(vendor);
+            return outcome.with(Vendor::new(&rule.vendor));
         }
         // Nothing identifiable. A 404 or a 500 is still the server answering.
         if refusal {
diff --git a/tests/antibot_classify.rs b/tests/antibot_classify.rs
index 279a8d8..69f97c4 100644
--- a/tests/antibot_classify.rs
+++ b/tests/antibot_classify.rs
@@ -198,6 +198,52 @@ fn test_cloudflare_waf_block_is_not_a_challenge() {
     assert_ne!(outcome, Outcome::Challenge(Vendor::new("cloudflare")));
 }
 
+#[test]
+fn test_cloudflare_challenge_orchestrator_on_a_refusal_is_a_challenge() {
+    let h = hdrs(&[("server", "cloudflare")]);
+    let body = r#"<html><script src="/cdn-cgi/challenge-platform/h/b/orchestrate/chl_page/v1?ray=0"></script></html>"#;
+    assert_eq!(
+        classify(&facts(403, &h, body)),
+        Outcome::Challenge(Vendor::new("cloudflare"))
+    );
+}
+
+#[test]
+fn test_cloudflare_error_page_is_a_cloudflare_block() {
+    let h = hdrs(&[("server", "cloudflare")]);
+    let body = r#"<html><div id="cf-error-details" class="cf-error-details-wrapper"><h1>Error 1020</h1></div></html>"#;
+    assert_eq!(
+        classify(&facts(403, &h, body)),
+        Outcome::Blocked(Vendor::new("cloudflare"))
+    );
+}
+
+#[test]
+fn test_origin_refusal_behind_a_bot_product_is_not_vendor_evidence() {
+    // Bot products set their cookies and inject their scripts on every
+    // response, the origin's own refusals included. Those alone must not name
+    // the product as what refused, or every ordinary 401/403 walks the ladder.
+    let jsd = r#"<html><h1>401 Unauthorized</h1><script src="/cdn-cgi/challenge-platform/scripts/jsd/main.js"></script></html>"#;
+    for (cookie, body) in [
+        ("__cf_bm=xyz; Path=/", jsd),
+        ("_abck=A7F1~0~YAAQ; Path=/", "<html>forbidden</html>"),
+        ("datadome=abc; Path=/", "<html>forbidden</html>"),
+        ("_px3=abc; Path=/", "<html>forbidden</html>"),
+        ("incap_ses_123=abc; Path=/", "<html>forbidden</html>"),
+    ] {
+        let h = hdrs(&[("set-cookie", cookie)]);
+        for status in [401, 403, 405] {
+            let outcome = classify(&facts(status, &h, body));
+            assert_eq!(
+                outcome,
+                Outcome::Blocked(Vendor::unknown()),
+                "{cookie} at {status}"
+            );
+            assert!(!outcome.indicates_protection());
+        }
+    }
+}
+
 #[test]
 fn test_cloudflare_bm_cookie_on_success_is_present() {
     // www.cloudflare.com sets __cf_bm on an ordinary 200.
@@ -365,18 +411,6 @@ fn test_unidentified_refusal_reports_unknown_vendor() {
     assert!(!outcome.got_through());
 }
 
-#[test]
-fn test_product_cookie_with_refusal_still_counts_as_blocked() {
-    // Setting a session cookie while refusing is still a refusal.
-    let h = hdrs(&[
-        ("server", "cloudflare"),
-        ("set-cookie", "__cf_bm=xyz; Path=/"),
-    ]);
-    let outcome = classify(&facts(403, &h, "<html>denied</html>"));
-    assert_eq!(outcome, Outcome::Blocked(Vendor::new("cloudflare")));
-    assert!(!outcome.got_through());
-}
-
 #[test]
 fn test_aws_waf_captcha_is_a_challenge() {
     // wizzair.com answers 405 with `captcha` rather than `challenge`. A rule

From 174b926f88b6d3c6751654cd74e30ea5bf2aed86 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 14:31:57 -0500
Subject: [PATCH 37/44] Keep a refusal when the deadline runs out during an
 escalation

---
 CHANGELOG.md        |  2 +-
 src/client/hyper.rs | 33 ++++++++++++--------
 tests/ladder.rs     | 76 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 96 insertions(+), 15 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 411b80e..9f6e790 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -115,7 +115,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A `cipher_string` with no `profile` named selects `compatibility`, the same as asking for a pre-1.2 TLS version, so no TLS 1.2 floor applies and `cipher_string="RC4-MD5"` reaches a TLS 1.0 server. Naming a profile as well keeps that profile's floor
 - `Content-Encoding: zstd` is decoded, including bodies of several frames. `chrome131` advertises it, as Chrome does, and Cloudflare answers with it. A frame asking for a window over 8 MiB (RFC 9659's limit) is refused and the body returned undecoded, so a response can't make the client allocate an arbitrary window
 - With `verify_certs` on, anonymous (aNULL) suites are never offered, including from an explicit `cipher_string`. They send no certificate, so verification would report success while checking nothing. With verification off, nothing is excluded
-- A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error
+- A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error. The same holds when the timeout runs out during that next attempt: `timeout` still bounds the whole request, redirects and ladder included, but a refusal already received is returned instead of a timeout error
 - Akamai's block rule only fires on a refusal status. edgesuite.net also serves customer assets, so a 200 page loading one was classified as blocked and re-sent
 - A product's cookie or injected script on a refusal no longer names it as what refused. Bot-management products set their cookies and Cloudflare injects its JavaScript detections script on every response, the origin's own 401s and 403s included, so an ordinary auth failure behind a CDN walked the ladder and could report `needs_browser`. Cloudflare's own error page (a firewall-rule block such as 1020) is recognised as a Cloudflare block instead
 
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index 99681a0..be5a879 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -2302,8 +2302,10 @@ impl HttpClient for HyperClient {
                 tokio::time::sleep(backoff).await;
             }
 
-            match tokio::time::timeout(timeout_duration, self.send_inner(config, &log)).await {
-                Ok(Ok(response)) => {
+            // One deadline for the whole request, redirects and ladder included.
+            let deadline = tokio::time::Instant::now() + timeout_duration;
+            match self.send_inner(config, &log, deadline).await {
+                Ok(response) => {
                     let status = response.status;
                     if super::ErrorKind::Status(status).is_retryable() && attempt < max_retries {
                         debug_record(
@@ -2320,7 +2322,7 @@ impl HttpClient for HyperClient {
                     }
                     return Ok(response);
                 }
-                Ok(Err(e)) => {
+                Err(e) => {
                     if e.kind.is_retryable() && attempt < max_retries {
                         debug_record(
                             &log,
@@ -2333,12 +2335,6 @@ impl HttpClient for HyperClient {
                     }
                     return Err(e);
                 }
-                Err(_) => {
-                    return Err(ClientError::timeout(format!(
-                        "request timed out after {}s",
-                        config.timeout()
-                    )));
-                }
             }
         }
 
@@ -2522,9 +2518,12 @@ impl HyperClient {
         &self,
         config: &RequestConfig,
         log: &DebugLog,
+        deadline: tokio::time::Instant,
     ) -> Result<Response, ClientError> {
         let v = config.verbosity;
         let start = Instant::now();
+        let timed_out =
+            || ClientError::timeout(format!("request timed out after {}s", config.timeout()));
 
         let sanitized_url = sanitize_uri(&config.url);
         let mut uri: http::Uri = sanitized_url.parse().map(normalize_empty_path).map_err(
@@ -2558,7 +2557,10 @@ impl HyperClient {
         // Bypasses the cached connection pool — opens a fresh TCP connection.
         if config.resolve_ip.is_some() || config.request_target.is_some() {
             let redirect_chain: Vec<RedirectHop> = Vec::new();
-            let (resp, cert_info, peer_ip, attempts) = dispatch_direct(&uri, config, log).await?;
+            let (resp, cert_info, peer_ip, attempts) =
+                tokio::time::timeout_at(deadline, dispatch_direct(&uri, config, log))
+                    .await
+                    .map_err(|_| timed_out())??;
             let hop_ms = start.elapsed().as_millis();
             debug_record(log, v, 1, &format!("<- {} ({}ms)", resp.status, hop_ms));
 
@@ -2633,9 +2635,10 @@ impl HyperClient {
                 tried.push(rung.name);
                 let rung_config = config.with_profile(rung.name);
 
-                match self
-                    .attempt_hop(&rung_config, &uri, log, hop_cookies.as_deref())
+                let hop = self.attempt_hop(&rung_config, &uri, log, hop_cookies.as_deref());
+                match tokio::time::timeout_at(deadline, hop)
                     .await
+                    .unwrap_or_else(|_| Err(timed_out()))
                 {
                     Ok((resp, cached)) => {
                         let outcome = crate::antibot::classify(&crate::antibot::ResponseFacts {
@@ -2736,7 +2739,11 @@ impl HyperClient {
                             // carrying what was tried.
                             None => {
                                 if let Some(kept) = refusal.take() {
-                                    self.remember_exhausted(config, &uri);
+                                    // Running out of time says nothing about
+                                    // the host, so it doesn't count as exhausted.
+                                    if e.kind != super::ErrorKind::Timeout {
+                                        self.remember_exhausted(config, &uri);
+                                    }
                                     break kept;
                                 }
                                 e.attempts = attempts;
diff --git a/tests/ladder.rs b/tests/ladder.rs
index a237c00..94533c7 100644
--- a/tests/ladder.rs
+++ b/tests/ladder.rs
@@ -1,7 +1,7 @@
 //! The profile ladder, seen from the response a caller gets back.
 
-use blasthttp::client::HttpClient;
 use blasthttp::client::hyper::HyperClient;
+use blasthttp::client::{ErrorKind, HttpClient};
 use blasthttp::config::RequestConfig;
 use blasthttp::report::{AttemptOutcome, Conclusion};
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
@@ -81,3 +81,77 @@ async fn test_a_not_found_page_is_reached() {
         AttemptOutcome::Reached { status: 404 }
     );
 }
+
+/// Answers the first connection with a Cloudflare challenge after `delay`,
+/// then accepts every later connection and never answers it.
+async fn slow_challenge_then_silence(delay: std::time::Duration) -> u16 {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    tokio::spawn(async move {
+        let mut first = true;
+        let mut held = Vec::new();
+        loop {
+            let (mut sock, _) = listener.accept().await.unwrap();
+            if !first {
+                held.push(sock);
+                continue;
+            }
+            first = false;
+            tokio::spawn(async move {
+                let mut buf = [0u8; 4096];
+                let _ = sock.read(&mut buf).await;
+                tokio::time::sleep(delay).await;
+                let body = "<html>Just a moment...</html>";
+                let reply = format!(
+                    "HTTP/1.1 403 Forbidden\r\ncf-mitigated: challenge\r\n\
+                     content-length: {}\r\nconnection: close\r\n\r\n{body}",
+                    body.len()
+                );
+                let _ = sock.write_all(reply.as_bytes()).await;
+                let _ = sock.shutdown().await;
+            });
+        }
+    });
+    port
+}
+
+#[tokio::test]
+async fn test_an_escalation_that_runs_out_of_time_keeps_the_refusal() {
+    // The refusal arrives most of the way through the timeout and the next
+    // rung never answers. The caller gets the refusal, not a timeout.
+    let port = slow_challenge_then_silence(std::time::Duration::from_millis(1500)).await;
+    let mut config = config(port);
+    config.timeout_seconds = Some(2);
+
+    let resp = HyperClient::new()
+        .send(&config)
+        .await
+        .unwrap_or_else(|e| panic!("the 403 was lost to: {}", e.message));
+
+    assert_eq!(resp.status, 403);
+    match &resp.attempts.last().unwrap().outcome {
+        AttemptOutcome::HandshakeFailed { reason } => {
+            assert!(reason.contains("timed out"), "{reason}")
+        }
+        other => panic!("the last rung should have timed out: {other:?}"),
+    }
+}
+
+#[tokio::test]
+async fn test_a_server_that_never_answers_still_times_out() {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    tokio::spawn(async move {
+        let mut held = Vec::new();
+        loop {
+            held.push(listener.accept().await.unwrap().0);
+        }
+    });
+    let mut config = config(port);
+    config.timeout_seconds = Some(1);
+
+    let err = HyperClient::new().send(&config).await.unwrap_err();
+
+    assert_eq!(err.kind, ErrorKind::Timeout);
+    assert_eq!(err.message, "request timed out after 1s");
+}

From e8d0169e457bc8be735377291e8a042653ca2659 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 14:47:27 -0500
Subject: [PATCH 38/44] Conclude unreachable, not needs_legacy_tls, once
 compatibility has failed

---
 CHANGELOG.md            |  1 +
 src/client/mod.rs       | 16 +++++++++----
 tests/legacy_default.rs | 52 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 65 insertions(+), 4 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9f6e790..06e919c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -118,6 +118,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A refusal that sends the ladder to another profile is kept. If that next attempt gets no response at all (a failed handshake, a reset, a refused connection), the caller gets the refusal, with the failed attempt listed in `attempts`, rather than the error. The same holds when the timeout runs out during that next attempt: `timeout` still bounds the whole request, redirects and ladder included, but a refusal already received is returned instead of a timeout error
 - Akamai's block rule only fires on a refusal status. edgesuite.net also serves customer assets, so a 200 page loading one was classified as blocked and re-sent
 - A product's cookie or injected script on a refusal no longer names it as what refused. Bot-management products set their cookies and Cloudflare injects its JavaScript detections script on every response, the origin's own 401s and 403s included, so an ordinary auth failure behind a CDN walked the ladder and could report `needs_browser`. Cloudflare's own error page (a firewall-rule block such as 1020) is recognised as a Cloudflare block instead
+- A handshake failure concludes `needs_legacy_tls` only when `compatibility` wasn't tried, such as when a pin stopped the ladder. Once the widest offer has failed too, as with a server wanting a client certificate, the conclusion is `unreachable`
 
 ## 0.10.1
 
diff --git a/src/client/mod.rs b/src/client/mod.rs
index 5a6e683..ba90f25 100644
--- a/src/client/mod.rs
+++ b/src/client/mod.rs
@@ -265,12 +265,20 @@ impl ClientError {
     ///
     /// A peer that could not negotiate with us is reported separately from one
     /// we never reached: the first is alive and wants cryptography we declined
-    /// or cannot do, which is actionable, and the second is not.
+    /// or cannot do, which is actionable, and the second is not. Once
+    /// `compatibility`, the widest offer, has failed too, legacy TLS is not
+    /// what is missing.
     pub fn conclusion(&self) -> crate::report::Conclusion {
+        let widest_tried = self
+            .attempts
+            .iter()
+            .any(|a| a.profile == crate::profile::COMPATIBILITY.name);
         match &self.tls_failure {
-            Some(f) if f.suggests_wider_offer() => crate::report::Conclusion::NeedsLegacyTls {
-                reason: self.message.clone(),
-            },
+            Some(f) if f.suggests_wider_offer() && !widest_tried => {
+                crate::report::Conclusion::NeedsLegacyTls {
+                    reason: self.message.clone(),
+                }
+            }
             _ => crate::report::Conclusion::Unreachable {
                 reason: self.message.clone(),
             },
diff --git a/tests/legacy_default.rs b/tests/legacy_default.rs
index fc994a1..e5d59f9 100644
--- a/tests/legacy_default.rs
+++ b/tests/legacy_default.rs
@@ -17,6 +17,7 @@ mod support;
 use blasthttp::client::HttpClient;
 use blasthttp::client::hyper::HyperClient;
 use blasthttp::config::RequestConfig;
+use blasthttp::report::Conclusion;
 use openssl::ssl::SslVersion;
 use support::tls_server::{TlsServerConfig, TlsTestServer};
 
@@ -680,3 +681,54 @@ async fn test_cipher_string_alone_reaches_a_tls10_server_on_direct_path() {
 
     server.shutdown().await;
 }
+
+// ── What a failure concludes ──────────────────────────────────────
+
+#[tokio::test]
+async fn test_a_failure_after_the_widest_offer_does_not_ask_for_legacy_tls() {
+    // No profile offers a NULL cipher, so modern and compatibility both fail.
+    // Legacy TLS was offered and refused; asking for it again helps nothing.
+    let server = TlsTestServer::start(TlsServerConfig {
+        cipher_list: Some("NULL-SHA".to_string()),
+        max_tls_version: Some(SslVersion::TLS1_2),
+        ..Default::default()
+    })
+    .await;
+
+    let err = HyperClient::new()
+        .send(&default_config(&server.url()))
+        .await
+        .unwrap_err();
+
+    let profiles: Vec<&str> = err.attempts.iter().map(|a| a.profile.as_str()).collect();
+    assert_eq!(profiles, ["modern", "compatibility"]);
+    assert!(
+        matches!(err.conclusion(), Conclusion::Unreachable { .. }),
+        "got {:?}",
+        err.conclusion()
+    );
+
+    server.shutdown().await;
+}
+
+#[tokio::test]
+async fn test_a_pin_that_stops_the_ladder_still_asks_for_legacy_tls() {
+    // The pin is why nothing wider was tried, so unpinning is the answer.
+    let server = TlsTestServer::start(TlsServerConfig {
+        max_tls_version: Some(SslVersion::TLS1),
+        ..Default::default()
+    })
+    .await;
+    let mut config = default_config(&server.url());
+    config.min_tls_version = Some("1.2".to_string());
+
+    let err = HyperClient::new().send(&config).await.unwrap_err();
+
+    assert!(
+        matches!(err.conclusion(), Conclusion::NeedsLegacyTls { .. }),
+        "got {:?}",
+        err.conclusion()
+    );
+
+    server.shutdown().await;
+}

From 9df34fd3d340a93f64c5af740dcacc6ed757ef43 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 15:03:12 -0500
Subject: [PATCH 39/44] Document blocked with no vendor as an ordinary refusal

---
 CHANGELOG.md  |  3 ++-
 src/python.rs | 14 ++++++++++----
 src/report.rs |  6 ++++--
 3 files changed, 16 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 06e919c..d5743f6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -36,7 +36,8 @@ the ladder falling back to `compatibility` on the second handshake.
 - **`Response.conclusion`** reduces that to one verdict: reached, blocked,
   needs_browser, needs_legacy_tls or unreachable. `needs_browser` means a
   JavaScript challenge, which no HTTP client passes, so the URL should go to a
-  real browser rather than be retried
+  real browser rather than be retried. A `blocked` with vendor `-` is an
+  ordinary refusal from the origin, not a product
 - **`Response.attempts`** lists what the ladder tried, in order
 - Imperva's `_Incapsula_Resource` loader script no longer counts as a
   challenge. Imperva appends it to pages it let through, so ordinary pages and
diff --git a/src/python.rs b/src/python.rs
index c707b08..bfe44da 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -736,7 +736,7 @@ impl PyResponse {
     ///
     /// Returns a tuple of `(outcome, vendor)`, both lowercase strings:
     ///
-    ///   outcome  ok | present | challenge | blocked | error
+    ///   outcome  ok | present | challenge | blocked
     ///   vendor   cloudflare | akamai | datadome | perimeterx | imperva |
     ///            f5 | kasada | unknown | "-" when nothing was detected
     ///
@@ -751,15 +751,21 @@ impl PyResponse {
     /// What this request amounts to, as a single verdict a caller can branch
     /// on. Returns `(conclusion, vendor)`:
     ///
-    ///   reached          got the page
-    ///   blocked          a product refused us
+    ///   reached          got the page, or the server's own answer such as
+    ///                    a 404 or a 500
+    ///   blocked          refused. vendor names the product when one was
+    ///                    recognised; "-" is an ordinary refusal (auth,
+    ///                    forbidden path, method not allowed) that a
+    ///                    different connection shape won't change
     ///   needs_browser    a JavaScript challenge; hand this URL to a real
     ///                    browser, because no HTTP client passes this tier
     ///   needs_legacy_tls the peer wants cryptography we declined or cannot do
     ///   unreachable      never got a response
     ///
     /// `needs_browser` is the one worth acting on. It means retrying with
-    /// different settings cannot help, which is the opposite of `blocked`.
+    /// different settings cannot help. A `blocked` with a named vendor is
+    /// the only refusal the ladder tries other settings against, so by the
+    /// time it is reported they have been tried.
     #[getter]
     fn conclusion(&self) -> (String, String) {
         conclusion_to_py(&self.inner.conclusion())
diff --git a/src/report.rs b/src/report.rs
index 673ea63..4ea3b6c 100644
--- a/src/report.rs
+++ b/src/report.rs
@@ -76,9 +76,11 @@ impl Attempt {
 /// What the whole request amounts to, once every rung has been tried.
 #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
 pub enum Conclusion {
-    /// Got the page.
+    /// Got the page, or the server's own answer such as a 404 or a 500.
     Reached,
-    /// A product refused us and nothing we can change is likely to help.
+    /// Refused. `vendor` names the product when one was recognised, and the
+    /// ladder has already tried what it could; `None` is an ordinary refusal
+    /// from the origin (auth, forbidden path) that no connection shape changes.
     Blocked {
         #[serde(skip_serializing_if = "Option::is_none")]
         vendor: Option<String>,

From 5d8b1251aa0c11287f070e49bd38014e1b48b9d6 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 15:27:08 -0500
Subject: [PATCH 40/44] Retry a connection reset during the TLS handshake

---
 CHANGELOG.md            |  1 +
 src/client/mod.rs       | 22 +++++++++++-
 tests/connect_errors.rs | 76 ++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 97 insertions(+), 2 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index d5743f6..fc3a55c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -120,6 +120,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - Akamai's block rule only fires on a refusal status. edgesuite.net also serves customer assets, so a 200 page loading one was classified as blocked and re-sent
 - A product's cookie or injected script on a refusal no longer names it as what refused. Bot-management products set their cookies and Cloudflare injects its JavaScript detections script on every response, the origin's own 401s and 403s included, so an ordinary auth failure behind a CDN walked the ladder and could report `needs_browser`. Cloudflare's own error page (a firewall-rule block such as 1020) is recognised as a Cloudflare block instead
 - A handshake failure concludes `needs_legacy_tls` only when `compatibility` wasn't tried, such as when a pin stopped the ladder. Once the widest offer has failed too, as with a server wanting a client certificate, the conclusion is `unreachable`
+- A connection reset during the TLS handshake is a `connection` error and retried, as before 1.0, rather than a non-retryable `tls` one. `tls_failure` still reads `reset`
 
 ## 0.10.1
 
diff --git a/src/client/mod.rs b/src/client/mod.rs
index ba90f25..b09e93e 100644
--- a/src/client/mod.rs
+++ b/src/client/mod.rs
@@ -252,10 +252,18 @@ impl ClientError {
     /// A TLS failure that knows why. Prefer this over `tls` wherever an
     /// `openssl::ssl::Error` is in hand, since the reason is what the profile
     /// ladder branches on and it cannot be recovered from the message text.
+    ///
+    /// A reset is a connection failure and a timeout is a timeout, whichever
+    /// stage they hit; `tls_failure` still says it was during the handshake.
     pub fn tls_detailed(message: String, failure: TlsFailure) -> Self {
+        let kind = match failure {
+            TlsFailure::Reset => ErrorKind::Connection,
+            TlsFailure::Timeout => ErrorKind::Timeout,
+            _ => ErrorKind::Tls,
+        };
         ClientError {
             message,
-            kind: ErrorKind::Tls,
+            kind,
             tls_failure: Some(failure),
             attempts: Vec::new(),
         }
@@ -578,4 +586,16 @@ mod tls_failure_tests {
         assert_eq!(detailed.kind, ErrorKind::Tls);
         assert_eq!(detailed.tls_failure, Some(TlsFailure::NoSharedCipher));
     }
+
+    #[test]
+    fn test_a_handshake_reset_or_timeout_keeps_its_own_kind() {
+        let reset = ClientError::tls_detailed("x".into(), TlsFailure::Reset);
+        assert_eq!(reset.kind, ErrorKind::Connection);
+        assert!(reset.kind.is_retryable());
+        assert_eq!(reset.tls_failure, Some(TlsFailure::Reset));
+
+        let timeout = ClientError::tls_detailed("x".into(), TlsFailure::Timeout);
+        assert_eq!(timeout.kind, ErrorKind::Timeout);
+        assert_eq!(timeout.tls_failure, Some(TlsFailure::Timeout));
+    }
 }
diff --git a/tests/connect_errors.rs b/tests/connect_errors.rs
index 2d600c4..0759d68 100644
--- a/tests/connect_errors.rs
+++ b/tests/connect_errors.rs
@@ -5,7 +5,7 @@
 mod support;
 
 use blasthttp::client::hyper::HyperClient;
-use blasthttp::client::{ErrorKind, HttpClient};
+use blasthttp::client::{ErrorKind, HttpClient, TlsFailure};
 use blasthttp::config::RequestConfig;
 use openssl::ssl::SslVersion;
 use support::tls_server::{TlsServerConfig, TlsTestServer};
@@ -87,3 +87,77 @@ async fn test_an_earlier_tls_failure_does_not_relabel_a_later_error() {
     );
     assert!(second.tls_failure.is_none());
 }
+
+/// Accepts connections, reads the ClientHello and hangs up. Counts how many
+/// connections it saw.
+async fn hangs_up_on_hello() -> (u16, std::sync::Arc<std::sync::atomic::AtomicUsize>) {
+    use tokio::io::AsyncReadExt;
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    let seen = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0));
+    let counter = seen.clone();
+    tokio::spawn(async move {
+        loop {
+            let (mut sock, _) = listener.accept().await.unwrap();
+            counter.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
+            let mut buf = [0u8; 4096];
+            let _ = sock.read(&mut buf).await;
+            drop(sock);
+        }
+    });
+    (port, seen)
+}
+
+#[tokio::test]
+async fn test_a_hangup_during_the_handshake_is_a_retryable_connection_error() {
+    let (port, seen) = hangs_up_on_hello().await;
+    let mut config = config_for(format!("https://127.0.0.1:{}/", port));
+    config.retries = Some(1);
+    config.retry_wait_min_ms = Some(10);
+    config.retry_wait_max_ms = Some(10);
+
+    let err = HyperClient::new().send(&config).await.unwrap_err();
+
+    assert_eq!(err.kind, ErrorKind::Connection, "{}", err.message);
+    assert_eq!(err.tls_failure, Some(TlsFailure::Reset));
+    assert_eq!(seen.load(std::sync::atomic::Ordering::SeqCst), 2);
+}
+
+/// Accepts connections and never answers them.
+async fn silent_server() -> u16 {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    tokio::spawn(async move {
+        let mut held = Vec::new();
+        loop {
+            held.push(listener.accept().await.unwrap().0);
+        }
+    });
+    port
+}
+
+#[tokio::test]
+async fn test_a_handshake_timeout_reads_the_same_on_both_paths() {
+    // A peer that accepts and never answers the ClientHello is a timeout on
+    // the pooled path and the direct one alike.
+    let port = silent_server().await;
+    let mut seen = Vec::new();
+    for direct in [false, true] {
+        let mut config = config_for(format!("https://127.0.0.1:{}/", port));
+        config.timeout_seconds = Some(1);
+        if direct {
+            config.resolve_ip = Some("127.0.0.1".to_string());
+        }
+
+        let err = HyperClient::new().send(&config).await.unwrap_err();
+
+        assert_eq!(
+            err.kind,
+            ErrorKind::Timeout,
+            "direct={direct}: {}",
+            err.message
+        );
+        seen.push(err.tls_failure);
+    }
+    assert_eq!(seen[0], seen[1]);
+}

From b8cc955e06eec5916382368648a74d4048e732dc Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 15:32:24 -0500
Subject: [PATCH 41/44] Take the batch rate limit at the send, not the spawn

---
 CHANGELOG.md |  1 +
 src/batch.rs | 72 ++++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 62 insertions(+), 11 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index fc3a55c..e9af851 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -121,6 +121,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A product's cookie or injected script on a refusal no longer names it as what refused. Bot-management products set their cookies and Cloudflare injects its JavaScript detections script on every response, the origin's own 401s and 403s included, so an ordinary auth failure behind a CDN walked the ladder and could report `needs_browser`. Cloudflare's own error page (a firewall-rule block such as 1020) is recognised as a Cloudflare block instead
 - A handshake failure concludes `needs_legacy_tls` only when `compatibility` wasn't tried, such as when a pin stopped the ladder. Once the widest offer has failed too, as with a server wanting a client certificate, the conclusion is `unreachable`
 - A connection reset during the TLS handshake is a `connection` error and retried, as before 1.0, rather than a non-retryable `tls` one. `tls_failure` still reads `reset`
+- A batch's rate limit is taken when a request is sent rather than when it's spawned, so requests that stood aside for a host's first request go at the configured rate when it finishes instead of all at once
 
 ## 0.10.1
 
diff --git a/src/batch.rs b/src/batch.rs
index 81b839d..a0a984f 100644
--- a/src/batch.rs
+++ b/src/batch.rs
@@ -421,15 +421,11 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
     let mut handles = Vec::new();
 
     for config in configs {
-        // Rate limit: pace dispatch before spawning the task
-        if let Some(ref limiter) = limiter {
-            limiter.acquire().await;
-        }
-
         let client = client.clone();
         let permit = semaphore.clone();
         let url = config.url.clone();
         let sched = sched.clone();
+        let limiter = limiter.clone();
 
         let handle = tokio::spawn(async move {
             // Stand aside before taking a permit rather than after, so the
@@ -439,6 +435,11 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
             let (_lead, _) = await_turn(&sched, &host, &config, None).await;
 
             let _permit = permit.acquire().await.unwrap();
+            // Paced at the send, not the spawn, so a request that stood aside
+            // pays when it actually goes instead of bursting with the rest.
+            if let Some(limiter) = limiter {
+                limiter.acquire().await;
+            }
             let result = client.send(&config).await;
             BatchResult { url, result }
         });
@@ -483,9 +484,8 @@ pub async fn send_batch<C: HttpClient + Send + Sync + 'static>(
 /// `__anext__` (a `future_into_py` block running on the tokio runtime).
 ///
 /// Concurrency is gated *before* spawn by a semaphore acquire on the driver,
-/// so at most `concurrency` requests are in-flight at any time. Rate-limit
-/// acquire happens before the semaphore so dispatch pacing matches
-/// `send_batch`. In-flight tasks are NOT cancelled if the consumer drops
+/// so at most `concurrency` requests are in-flight at any time. The rate
+/// limit is taken just before each send, as in `send_batch`. In-flight tasks are NOT cancelled if the consumer drops
 /// the stream — they run to completion and their sends fail silently. This
 /// also matches `send_batch`.
 pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
@@ -506,9 +506,6 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
 
         tokio::spawn(async move {
             for config in configs {
-                if let Some(ref l) = limiter {
-                    l.acquire().await;
-                }
                 let permit = match semaphore.clone().acquire_owned().await {
                     Ok(p) => p,
                     Err(_) => break,
@@ -517,6 +514,7 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
                 let tx = tx.clone();
                 let sched = sched.clone();
                 let semaphore = semaphore.clone();
+                let limiter = limiter.clone();
                 tokio::spawn(async move {
                     let host = host_key(&config.url);
                     // The driver holds the permit here, unlike `send_batch`,
@@ -532,6 +530,10 @@ pub fn send_batch_stream<C: HttpClient + Send + Sync + 'static>(
                     };
 
                     let _permit = permit;
+                    // Paced at the send, as in `send_batch`.
+                    if let Some(limiter) = limiter {
+                        limiter.acquire().await;
+                    }
                     let url = config.url.clone();
                     let result = client.send(&config).await;
                     let _ = tx.unbounded_send(BatchResult { url, result });
@@ -739,6 +741,54 @@ mod tests {
         );
     }
 
+    /// Records when each send happened.
+    struct Recording {
+        inner: MockClient,
+        sends: std::sync::Mutex<Vec<Instant>>,
+    }
+
+    impl HttpClient for Recording {
+        async fn send(&self, config: &RequestConfig) -> Result<Response, ClientError> {
+            self.sends.lock().unwrap().push(Instant::now());
+            self.inner.send(config).await
+        }
+    }
+
+    #[tokio::test]
+    async fn test_requests_that_stood_aside_still_keep_the_rate() {
+        // Followers of a slow host stand aside while its first request runs.
+        // When it finishes they must go at the configured rate, not at once.
+        let client = Arc::new(Recording {
+            inner: MockClient::new(200, "ok".to_string()).with_delay(Duration::from_secs(2)),
+            sends: std::sync::Mutex::new(Vec::new()),
+        });
+        let configs: Vec<RequestConfig> = (0..8)
+            .map(|i| format!("https://a.com/{i}"))
+            .chain((0..8).map(|i| format!("https://b.com/{i}")))
+            .map(RequestConfig::new)
+            .collect();
+
+        send_batch(client.clone(), configs, 50, Some(5.0), None).await;
+
+        let sends = client.sends.lock().unwrap().clone();
+        assert_eq!(sends.len(), 16);
+        let busiest = sends
+            .iter()
+            .map(|t| {
+                sends
+                    .iter()
+                    .filter(|u| **u >= *t && **u < *t + Duration::from_secs(1))
+                    .count()
+            })
+            .max()
+            .unwrap();
+        // 5 a second, plus one for the bucket's first token.
+        assert!(
+            busiest <= 6,
+            "{busiest} sends inside one second at a rate of 5"
+        );
+    }
+
     #[tokio::test]
     async fn test_rate_limit_with_concurrency() {
         // Rate limit AND concurrency together

From 5ce501bcc52028db35067f71280a5024bae3134d Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 15:57:14 -0500
Subject: [PATCH 42/44] Claim one browser when the ladder switches to chrome131

---
 CHANGELOG.md        |  1 +
 README.md           |  2 +-
 src/client/hyper.rs | 31 +++++++++++++--
 src/config.rs       |  7 ++++
 src/python.rs       |  3 ++
 tests/ladder.rs     | 97 +++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 137 insertions(+), 4 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index e9af851..23e638e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -122,6 +122,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A handshake failure concludes `needs_legacy_tls` only when `compatibility` wasn't tried, such as when a pin stopped the ladder. Once the widest offer has failed too, as with a server wanting a client certificate, the conclusion is `unreachable`
 - A connection reset during the TLS handshake is a `connection` error and retried, as before 1.0, rather than a non-retryable `tls` one. `tls_failure` still reads `reset`
 - A batch's rate limit is taken when a request is sent rather than when it's spawned, so requests that stood aside for a host's first request go at the configured rate when it finishes instead of all at once
+- When the ladder switches to `chrome131` after a denial, Chrome's User-Agent and client hints replace the caller's, so the request doesn't claim Chrome 131 in its handshake and hints and another browser in its User-Agent. First contact keeps the caller's User-Agent, every other header and cookie the caller added stays, and naming the profile yourself keeps your headers as before
 
 ## 0.10.1
 
diff --git a/README.md b/README.md
index b6dd58c..6af36f9 100644
--- a/README.md
+++ b/README.md
@@ -25,7 +25,7 @@ cargo add blasthttp
 - **Batch connection reuse** — connections are pooled and reused within a batch, dramatically reducing overhead when scanning many URLs on the same hosts
 - **Rust performance** — async I/O, zero-copy where possible, and native concurrency give significant speed improvements over pure Python HTTP clients
 - **SSL cert info on every request** — extracts CN, SANs, emails, issuer, validity dates, and fingerprint during the TLS handshake that's already happening, eliminating the need for a separate connection to gather this information
-- **Connection profiles, and automatic shifting between them**: `modern` by default (11 cipher suites, TLS 1.2+), `compatibility` for servers that speak only RC4, 3DES, SEED, Camellia, anonymous DH or SSLv3, and `chrome` to imitate a browser's TLS, HTTP/2 and headers together. A handshake the peer cannot negotiate widens the offer; a refusal from a recognised protection product changes what the client claims to be. What works is remembered per host
+- **Connection profiles, and automatic shifting between them**: `modern` by default (11 cipher suites, TLS 1.2+), `compatibility` for servers that speak only RC4, 3DES, SEED, Camellia, anonymous DH or SSLv3, and `chrome` to imitate a browser's TLS, HTTP/2 and headers together. A handshake the peer cannot negotiate widens the offer; a refusal from a recognised protection product changes what the client claims to be. When it switches to `chrome` that way, Chrome's User-Agent and client hints replace any you set, so the request claims one browser; your other headers and cookies stay. What works is remembered per host
 - **Says why it failed**: a blocked request reports which product blocked it, and a JavaScript challenge reports `needs_browser`, because no HTTP client passes that tier and retrying will not change it. The products and how to recognise them are rules in [`antibot_rules.json`](antibot_rules.json), and `blasthttp.set_antibot_rules(path)` swaps in your own at runtime
 - **No cert validation by default** — offensive-first: connects to self-signed, expired, and misconfigured TLS without extra config
 - **HTTP/2 support** — automatic via ALPN negotiation, falls back to HTTP/1.1
diff --git a/src/client/hyper.rs b/src/client/hyper.rs
index be5a879..55b1ed6 100644
--- a/src/client/hyper.rs
+++ b/src/client/hyper.rs
@@ -1885,6 +1885,14 @@ async fn dispatch_forward_proxy(
     parse_response(hyper_response, config, log).await
 }
 
+/// Headers that say which browser a request claims to be.
+const IDENTITY_HEADERS: [&str; 4] = [
+    "user-agent",
+    "sec-ch-ua",
+    "sec-ch-ua-mobile",
+    "sec-ch-ua-platform",
+];
+
 fn build_request(
     uri: &http::Uri,
     config: &RequestConfig,
@@ -1954,8 +1962,9 @@ fn build_request(
     // answers an honest curl with 200 and the same client wearing only a
     // Chrome User-Agent with 403.
     //
-    // A caller's own header still wins over the profile's, since naming one
-    // explicitly means it.
+    // A caller's own header wins over the profile's, since naming one
+    // explicitly means it, except the browser identity once the ladder has
+    // switched profile (below).
     // A profile with no headers of its own means the client's own minimal
     // defaults, which is what `compatibility` and `modern` both want.
     let header_profile = if bisect_disabled("headers") {
@@ -1963,9 +1972,22 @@ fn build_request(
     } else {
         config.resolved_profile()
     };
+    // When the ladder chose the profile, its browser identity replaces the
+    // caller's; every other header the caller set still wins.
+    let profile_owns = |name: &str| {
+        config.profile_from_ladder
+            && IDENTITY_HEADERS
+                .iter()
+                .any(|h| h.eq_ignore_ascii_case(name))
+            && header_profile
+                .headers
+                .iter()
+                .any(|(p, _)| p.eq_ignore_ascii_case(name))
+    };
     if !header_profile.headers.is_empty() {
         for (name, value) in header_profile.headers {
-            let already_set = custom.iter().any(|(k, _)| k.eq_ignore_ascii_case(name));
+            let already_set =
+                !profile_owns(name) && custom.iter().any(|(k, _)| k.eq_ignore_ascii_case(name));
             if !already_set {
                 builder = builder.header(*name, *value);
             }
@@ -2014,6 +2036,9 @@ fn build_request(
                 builder = builder.header(name.as_str(), merged);
                 continue;
             }
+            if profile_owns(name) {
+                continue;
+            }
             builder = builder.header(name.as_str(), value.as_str());
         }
     }
diff --git a/src/config.rs b/src/config.rs
index ed846fd..6ddf987 100644
--- a/src/config.rs
+++ b/src/config.rs
@@ -64,6 +64,11 @@ pub struct RequestConfig {
     pub alpn_protocols: Option<Vec<String>>,
     #[serde(default)]
     pub verbosity: u8,
+    /// Set when the ladder, not the caller, chose `profile`. The profile's
+    /// browser identity headers then replace the caller's, since a switch
+    /// made because of a denial has to claim one browser consistently.
+    #[serde(skip)]
+    pub profile_from_ladder: bool,
 }
 
 impl RequestConfig {
@@ -93,6 +98,7 @@ impl RequestConfig {
             resolve_ip: None,
             alpn_protocols: None,
             verbosity: 0,
+            profile_from_ladder: false,
         }
     }
 
@@ -159,6 +165,7 @@ impl RequestConfig {
     /// connector from another.
     pub fn with_profile(&self, name: &str) -> RequestConfig {
         let mut c = self.clone();
+        c.profile_from_ladder = !self.tls_is_pinned();
         c.profile = Some(name.to_string());
         c
     }
diff --git a/src/python.rs b/src/python.rs
index bfe44da..6a5b01c 100644
--- a/src/python.rs
+++ b/src/python.rs
@@ -1184,6 +1184,7 @@ impl BlastHTTP {
             resolve_ip,
             alpn_protocols,
             verbosity: 0,
+            profile_from_ladder: false,
         };
 
         let limiter = self.rate_limiter.clone();
@@ -1387,6 +1388,7 @@ impl BlastHTTP {
             resolve_ip: None,
             alpn_protocols: None,
             verbosity: 0,
+            profile_from_ladder: false,
         };
 
         let limiter = self.rate_limiter.clone();
@@ -1775,6 +1777,7 @@ impl PyBatchConfig {
             resolve_ip: self.resolve_ip,
             alpn_protocols: self.alpn_protocols,
             verbosity: 0,
+            profile_from_ladder: false,
         })
     }
 }
diff --git a/tests/ladder.rs b/tests/ladder.rs
index 94533c7..261c06c 100644
--- a/tests/ladder.rs
+++ b/tests/ladder.rs
@@ -155,3 +155,100 @@ async fn test_a_server_that_never_answers_still_times_out() {
     assert_eq!(err.kind, ErrorKind::Timeout);
     assert_eq!(err.message, "request timed out after 1s");
 }
+
+type Seen = std::sync::Arc<std::sync::Mutex<Vec<Vec<(String, String)>>>>;
+
+/// Refuses the first request with a Cloudflare challenge and answers every
+/// later one with a 200, recording each request's headers.
+async fn challenge_first_then_ok() -> (u16, Seen) {
+    let listener = TcpListener::bind("127.0.0.1:0").await.unwrap();
+    let port = listener.local_addr().unwrap().port();
+    let seen: Seen = Default::default();
+    let record = seen.clone();
+    tokio::spawn(async move {
+        loop {
+            let (mut sock, _) = listener.accept().await.unwrap();
+            let mut buf = vec![0u8; 16384];
+            let n = sock.read(&mut buf).await.unwrap_or(0);
+            let head = String::from_utf8_lossy(&buf[..n]).to_string();
+            let headers: Vec<(String, String)> = head
+                .lines()
+                .skip(1)
+                .filter_map(|l| l.split_once(':'))
+                .map(|(k, v)| (k.trim().to_ascii_lowercase(), v.trim().to_string()))
+                .collect();
+            let first = {
+                let mut s = record.lock().unwrap();
+                s.push(headers);
+                s.len() == 1
+            };
+            let reply = if first {
+                "HTTP/1.1 403 Forbidden\r\ncf-mitigated: challenge\r\ncontent-length: 0\r\nconnection: close\r\n\r\n"
+            } else {
+                "HTTP/1.1 200 OK\r\ncontent-length: 2\r\nconnection: close\r\n\r\nok"
+            };
+            let _ = sock.write_all(reply.as_bytes()).await;
+            let _ = sock.shutdown().await;
+        }
+    });
+    (port, seen)
+}
+
+fn values<'a>(headers: &'a [(String, String)], name: &str) -> Vec<&'a str> {
+    headers
+        .iter()
+        .filter(|(k, _)| k == name)
+        .map(|(_, v)| v.as_str())
+        .collect()
+}
+
+fn caller_headers() -> Vec<(String, String)> {
+    [
+        ("User-Agent", "custom/1.0"),
+        ("Accept", "application/json"),
+        ("X-Added", "1"),
+        ("Cookie", "session=abc"),
+    ]
+    .into_iter()
+    .map(|(k, v)| (k.to_string(), v.to_string()))
+    .collect()
+}
+
+#[tokio::test]
+async fn test_a_switch_after_a_denial_claims_one_browser() {
+    // First contact keeps the caller's User-Agent. Once a denial moves the
+    // ladder to chrome131, its browser identity replaces the caller's, and
+    // everything else the caller added stays.
+    let (port, seen) = challenge_first_then_ok().await;
+    let mut config = config(port);
+    config.headers = Some(caller_headers());
+
+    let resp = HyperClient::new().send(&config).await.unwrap();
+    assert_eq!(resp.status, 200);
+
+    let seen = seen.lock().unwrap();
+    assert_eq!(seen.len(), 2);
+    assert_eq!(values(&seen[0], "user-agent"), ["custom/1.0"]);
+
+    let switched = &seen[1];
+    let ua = values(switched, "user-agent");
+    assert_eq!(ua.len(), 1, "one User-Agent, got {ua:?}");
+    assert!(ua[0].contains("Chrome/131"), "{}", ua[0]);
+    assert!(values(switched, "sec-ch-ua")[0].contains("131"));
+    assert_eq!(values(switched, "accept"), ["application/json"]);
+    assert_eq!(values(switched, "x-added"), ["1"]);
+    assert_eq!(values(switched, "cookie"), ["session=abc"]);
+}
+
+#[tokio::test]
+async fn test_a_profile_the_caller_named_keeps_their_user_agent() {
+    let (port, seen) = challenge_first_then_ok().await;
+    let mut config = config(port);
+    config.headers = Some(caller_headers());
+    config.profile = Some("chrome131".to_string());
+
+    HyperClient::new().send(&config).await.unwrap();
+
+    let seen = seen.lock().unwrap();
+    assert_eq!(values(&seen[0], "user-agent"), ["custom/1.0"]);
+}

From 368cde062f0ceea9776ff5abfac54d6f6b350d47 Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Fri, 6 Nov 2026 16:00:35 -0500
Subject: [PATCH 43/44] Keep pinned batch requests out of host discovery

---
 CHANGELOG.md |  1 +
 src/batch.rs | 91 ++++++++++++++++++++++++++++++++++++++++++++++------
 2 files changed, 83 insertions(+), 9 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 23e638e..d5d301b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -122,6 +122,7 @@ the ladder falling back to `compatibility` on the second handshake.
 - A handshake failure concludes `needs_legacy_tls` only when `compatibility` wasn't tried, such as when a pin stopped the ladder. Once the widest offer has failed too, as with a server wanting a client certificate, the conclusion is `unreachable`
 - A connection reset during the TLS handshake is a `connection` error and retried, as before 1.0, rather than a non-retryable `tls` one. `tls_failure` still reads `reset`
 - A batch's rate limit is taken when a request is sent rather than when it's spawned, so requests that stood aside for a host's first request go at the configured rate when it finishes instead of all at once
+- A batch request with its TLS pinned (`profile`, `cipher_string` or a TLS version) skips host discovery. It never reads or writes the per-host memory, so it no longer stands aside for a host's first request, and never makes unpinned requests wait on it
 - When the ladder switches to `chrome131` after a denial, Chrome's User-Agent and client hints replace the caller's, so the request doesn't claim Chrome 131 in its handshake and hints and another browser in its User-Agent. First contact keeps the caller's User-Agent, every other header and cookie the caller added stays, and naming the profile yourself keeps your headers as before
 
 ## 0.10.1
diff --git a/src/batch.rs b/src/batch.rs
index a0a984f..244b371 100644
--- a/src/batch.rs
+++ b/src/batch.rs
@@ -290,15 +290,19 @@ impl Scheduler {
     /// field: the stream driver needs the concurrency limit, `send_batch`
     /// needs no limit and passes the batch size.
     fn new(configs: &[RequestConfig], aside_limit: usize) -> Arc<Self> {
+        // Pinned requests never consult the per-host memory, so they take no
+        // part in discovering it and `await_turn` lets them straight through.
         let mut remaining_by_host: std::collections::HashMap<String, usize> =
             std::collections::HashMap::new();
-        for c in configs {
+        let mut remaining_total = 0;
+        for c in configs.iter().filter(|c| !c.tls_is_pinned()) {
             *remaining_by_host.entry(host_key(&c.url)).or_insert(0) += 1;
+            remaining_total += 1;
         }
         Arc::new(Scheduler {
             state: std::sync::Mutex::new(SchedulerState {
                 gates: std::collections::HashMap::new(),
-                remaining_total: configs.len(),
+                remaining_total,
                 remaining_by_host,
                 aside: 0,
                 aside_limit: aside_limit.max(1),
@@ -388,6 +392,9 @@ async fn await_turn(
     Option<LeaderGuard>,
     Option<tokio::sync::OwnedSemaphorePermit>,
 ) {
+    if config.tls_is_pinned() {
+        return (None, permit);
+    }
     match sched.dispatch(host, true) {
         Dispatch::Lead(g) => (Some(g), permit),
         Dispatch::Go => (None, permit),
@@ -744,12 +751,34 @@ mod tests {
     /// Records when each send happened.
     struct Recording {
         inner: MockClient,
-        sends: std::sync::Mutex<Vec<Instant>>,
+        sends: std::sync::Mutex<Vec<(String, Instant)>>,
+    }
+
+    impl Recording {
+        fn new(delay: Duration) -> Self {
+            Recording {
+                inner: MockClient::new(200, "ok".to_string()).with_delay(delay),
+                sends: std::sync::Mutex::new(Vec::new()),
+            }
+        }
+
+        fn sent_at(&self, url: &str) -> Instant {
+            self.sends
+                .lock()
+                .unwrap()
+                .iter()
+                .find(|(u, _)| u == url)
+                .unwrap()
+                .1
+        }
     }
 
     impl HttpClient for Recording {
         async fn send(&self, config: &RequestConfig) -> Result<Response, ClientError> {
-            self.sends.lock().unwrap().push(Instant::now());
+            self.sends
+                .lock()
+                .unwrap()
+                .push((config.url.clone(), Instant::now()));
             self.inner.send(config).await
         }
     }
@@ -758,10 +787,7 @@ mod tests {
     async fn test_requests_that_stood_aside_still_keep_the_rate() {
         // Followers of a slow host stand aside while its first request runs.
         // When it finishes they must go at the configured rate, not at once.
-        let client = Arc::new(Recording {
-            inner: MockClient::new(200, "ok".to_string()).with_delay(Duration::from_secs(2)),
-            sends: std::sync::Mutex::new(Vec::new()),
-        });
+        let client = Arc::new(Recording::new(Duration::from_secs(2)));
         let configs: Vec<RequestConfig> = (0..8)
             .map(|i| format!("https://a.com/{i}"))
             .chain((0..8).map(|i| format!("https://b.com/{i}")))
@@ -770,7 +796,7 @@ mod tests {
 
         send_batch(client.clone(), configs, 50, Some(5.0), None).await;
 
-        let sends = client.sends.lock().unwrap().clone();
+        let sends: Vec<Instant> = client.sends.lock().unwrap().iter().map(|s| s.1).collect();
         assert_eq!(sends.len(), 16);
         let busiest = sends
             .iter()
@@ -789,6 +815,53 @@ mod tests {
         );
     }
 
+    fn pinned(url: &str) -> RequestConfig {
+        let mut c = RequestConfig::new(url.to_string());
+        c.profile = Some("chrome131".to_string());
+        c
+    }
+
+    #[tokio::test]
+    async fn test_pinned_requests_do_not_stand_aside() {
+        // Pinned requests learn nothing from a host's first request, so
+        // waiting for it is pure latency.
+        let client = Arc::new(Recording::new(Duration::from_millis(500)));
+        let configs: Vec<RequestConfig> = ["a", "a", "a", "b", "b", "b"]
+            .iter()
+            .enumerate()
+            .map(|(i, h)| pinned(&format!("https://{h}.com/{i}")))
+            .collect();
+
+        send_batch(client.clone(), configs, 50, None, None).await;
+
+        let sends: Vec<Instant> = client.sends.lock().unwrap().iter().map(|s| s.1).collect();
+        let spread = *sends.iter().max().unwrap() - *sends.iter().min().unwrap();
+        assert!(
+            spread < Duration::from_millis(200),
+            "sends spread over {spread:?}"
+        );
+    }
+
+    #[tokio::test]
+    async fn test_a_pinned_request_does_not_lead_a_host() {
+        // A pinned first request never records what the host wants, so the
+        // unpinned requests behind it must not wait on it.
+        let client = Arc::new(Recording::new(Duration::from_millis(500)));
+        let configs = vec![
+            pinned("https://a.com/pinned"),
+            RequestConfig::new("https://a.com/open".to_string()),
+            RequestConfig::new("https://b.com/other".to_string()),
+        ];
+
+        send_batch(client.clone(), configs, 50, None, None).await;
+
+        let waited = client.sent_at("https://a.com/open") - client.sent_at("https://a.com/pinned");
+        assert!(
+            waited < Duration::from_millis(200),
+            "waited {waited:?} on a pinned request"
+        );
+    }
+
     #[tokio::test]
     async fn test_rate_limit_with_concurrency() {
         // Rate limit AND concurrency together

From 1a6a1bd740e3c82035a98b52a5579e3407a78daa Mon Sep 17 00:00:00 2001
From: liquidsec <paul.mueller08@gmail.com>
Date: Wed, 11 Nov 2026 11:51:50 -0500
Subject: [PATCH 44/44] Temporarily build OpenSSL in the PEP 517 backend

---
 pyproject.toml                   |  4 +++-
 scripts/openssl_build_backend.py | 25 +++++++++++++++++++++++++
 2 files changed, 28 insertions(+), 1 deletion(-)
 create mode 100644 scripts/openssl_build_backend.py

diff --git a/pyproject.toml b/pyproject.toml
index 3b67538..b1ded7c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,8 @@
 [build-system]
 requires = ["maturin>=1.0,<2.0"]
-build-backend = "maturin"
+# TEMPORARY: restore `build-backend = "maturin"` and drop backend-path before merging into dev.
+build-backend = "openssl_build_backend"
+backend-path = ["scripts"]
 
 [project]
 name = "blasthttp"
diff --git a/scripts/openssl_build_backend.py b/scripts/openssl_build_backend.py
new file mode 100644
index 0000000..ece864f
--- /dev/null
+++ b/scripts/openssl_build_backend.py
@@ -0,0 +1,25 @@
+# TEMPORARY: remove before merging into dev, and restore `build-backend = "maturin"` in pyproject.toml.
+# Builds the vendored OpenSSL before maturin runs, so unreleased 1.0 installs from a git source with just `uv sync`.
+import subprocess
+import sys
+from pathlib import Path
+
+import maturin
+from maturin import *  # re-exports maturin's other PEP 517 hooks
+
+BUILD_OPENSSL = Path(__file__).resolve().parent / "build-openssl.sh"
+
+
+def _build_openssl():
+    # the script returns immediately once OpenSSL is built
+    subprocess.run(["bash", str(BUILD_OPENSSL)], check=True, stdout=sys.stderr)
+
+
+def build_wheel(wheel_directory, config_settings=None, metadata_directory=None):
+    _build_openssl()
+    return maturin.build_wheel(wheel_directory, config_settings, metadata_directory)
+
+
+def build_editable(wheel_directory, config_settings=None, metadata_directory=None):
+    _build_openssl()
+    return maturin.build_editable(wheel_directory, config_settings, metadata_directory)