pjhartlieb · Nov 10, 2026 · Nov 6, 2026 · Nov 6, 2026 · Nov 6, 2026 · Nov 6, 2026
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 [![bbot_banner](https://github.com/user-attachments/assets/f02804ce-9478-4f1e-ac4d-9cf5620a3214)](https://github.com/blacklanternsecurity/bbot)
 
-[![Python Version](https://img.shields.io/badge/python-3.10+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-AGPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA)
+[![Python Version](https://img.shields.io/badge/python-3.11+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-AGPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA)
 
 ### **BEE·bot** is a multipurpose scanner inspired by [Spiderfoot](https://github.com/smicallef/spiderfoot), built to automate your **Recon**, **Bug Bounties**, and **ASM**!
 
@@ -20,6 +20,8 @@ pipx install --pip-args '\--pre' bbot
 
 _For more installation methods, including [Docker](https://hub.docker.com/r/blacklanternsecurity/bbot), see [Getting Started](https://www.blacklanternsecurity.com/bbot/Stable/)_
 
+> **Upgrading from 3.0?** See the [3.0 → 3.1 Migration Guide](https://www.blacklanternsecurity.com/bbot/Stable/migration/3.1_breaking_changes/) ([source](docs/migration/3.1_breaking_changes.md)) for changes that can affect existing presets, configs, custom modules, and output.
+
 > **Upgrading from 2.x?** BBOT 3.0 contains breaking changes to the CLI, presets, modules, events, and Python API. See the [2.x → 3.0 Migration Guide](https://www.blacklanternsecurity.com/bbot/Stable/migration/3.0_breaking_changes/) ([source](docs/migration/3.0_breaking_changes.md)) before upgrading.
 
 > **Speed tip:** BBOT's DNS resolver ([blastdns](https://github.com/blacklanternsecurity/blastdns)) spins up multiple threads per resolver in `/etc/resolv.conf`. Adding more unfiltered resolvers dramatically speeds up scans. See the [sample resolv.conf](docs/data/resolv-sample.conf) and [Tips and Tricks](https://www.blacklanternsecurity.com/bbot/Stable/scanning/tips_and_tricks/#speed-up-scans-with-more-dns-resolvers) for details.
@@ -56,7 +58,6 @@ output_modules:
 
 config:
   dns:
-    threads: 25
     brute_threads: 1000
   # put your API keys here
   # modules:

diff --git a/bbot/defaults.yml b/bbot/defaults.yml
@@ -71,7 +71,7 @@ dns:
   # state bounded by the resolver count rather than growing with query volume
   brute_persistent_socket: true
   # nameservers to use for DNS brute-forcing
-  # default is updated weekly and contains ~10K high-quality public servers
+  # default is updated weekly and contains ~5K high-quality public servers
   brute_nameservers: https://raw.githubusercontent.com/blacklanternsecurity/public-dns-servers/master/nameservers.txt
   # How many of those to draw at random, once per scan (0 = all of them)
   # The dead ones in the draw are dropped at startup, and every brute-force run in the scan shares the rest

diff --git a/bbot/modules/lightfuzz/lightfuzz.py b/bbot/modules/lightfuzz/lightfuzz.py
@@ -42,8 +42,8 @@ class Config(BaseModuleConfig):
             "try_bypasses",
             description=(
                 "How to handle WAF-tagged targets. 'always' skips them entirely; 'never' fuzzes them "
-                "raw; 'try_bypasses' (default) asks nowafpls whether body padding gets through, "
-                "and fuzzes with padding when it does."
+                "regardless, still padding request bodies where padding gets through; 'try_bypasses' "
+                "(default) asks nowafpls whether body padding gets through, and fuzzes with padding when it does."
             ),
         )
         emit_baseline_responses: bool = Field(
@@ -468,7 +468,7 @@ async def filter_event(self, event):
                     return False, "WAF is bypassable via body padding, but this event has no POST-style probe"
                 # STATUS_NO_INTERFERENCE: the POST body wasn't gated, so POST-style probes run unpadded.
                 # It says nothing about other channels, so handle_event still skips GET/COOKIE/HEADER.
-            # avoid_wafs == "never": fall through and fuzz raw
+            # avoid_wafs == "never": fuzz regardless; prepare_request still pads bodies when bypassable
 
         # Skip WEB_PARAMETERs on static-asset URLs (pdf, doc, xml, etc.) — fuzzing them is pointless
         if event.type == "WEB_PARAMETER":

diff --git a/bbot/presets/subdomain-enum.yml b/bbot/presets/subdomain-enum.yml
@@ -10,7 +10,6 @@ output_modules:
 
 config:
   dns:
-    threads: 25
     brute_threads: 1000
   # put your API keys here
   # modules:

diff --git a/bbot/presets/web/lightfuzz-heavy.yml b/bbot/presets/web/lightfuzz-heavy.yml
@@ -1,4 +1,4 @@
-description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. Still skips confirmed WAFs."
+description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it."
 
 include:
   - lightfuzz

diff --git a/bbot/presets/web/lightfuzz.yml b/bbot/presets/web/lightfuzz.yml
@@ -1,4 +1,4 @@
-description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. Skips confirmed WAFs."
+description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it."
 
 include:
   - lightfuzz-light

diff --git a/bbot/scanner/preset/args.py b/bbot/scanner/preset/args.py
@@ -463,7 +463,11 @@ def create_parser(self, *args, **kwargs):
         )
         misc.add_argument("--custom-yara-rules", "-cy", help="Add custom yara rules to excavate")
 
-        misc.add_argument("--user-agent", "-ua", help="Set the user-agent for all HTTP requests")
+        misc.add_argument(
+            "--user-agent",
+            "-ua",
+            help="Set the user-agent for HTTP requests (most API calls send BBOT/<version>)",
+        )
         misc.add_argument(
             "--user-agent-suffix", "-uas", help="Suffix to append to the user-agent", metavar="SUFFIX", default=None
         )

diff --git a/docs/migration/3.1_breaking_changes.md b/docs/migration/3.1_breaking_changes.md
@@ -6,6 +6,12 @@ If a change is missing here, please open an issue.
 
 ---
 
+## Python
+
+BBOT 3.1 requires Python 3.11 or newer, and Python 3.10 is no longer supported. On 3.10, `pip install --upgrade bbot` doesn't fail; it stays on the latest 3.0 release, so upgrade Python first. Python 3.15 is now supported.
+
+---
+
 ## Config and Presets
 
 ### Custom module folders (`module_dirs`)
@@ -77,6 +83,22 @@ By default, open ports other than 80 and 443 are dropped on hosts tagged `cdn` o
 
 ## Modules
 
+### `lightfuzz` probes WAF-protected hosts instead of skipping them
+
+In 3.0, lightfuzz skipped every host tagged `waf`. In 3.1, `modules.lightfuzz.avoid_wafs` defaults to `try_bypasses`: lightfuzz sends each WAF-tagged host a test payload in a POST body, first unpadded and then padded (the nowafpls technique). It fuzzes the host only if that probe gets past the WAF: either the padding gets the payload through, or the WAF didn't block it even unpadded. Even then, only request-body parameters are fuzzed. The `lightfuzz`, `lightfuzz-heavy`, `lightfuzz-max`, and `lightfuzz-xss` presets also enable the `nowafpls` module, which runs the same probe on every WAF-tagged host and emits a `FINDING` when padding gets through. Expect more traffic to WAF-fronted targets after upgrading. See [WAF Handling](../modules/lightfuzz.md#waf-handling).
+
+| 3.0 | 3.1 |
+|-----|-----|
+| the top-level `avoid_wafs` (default `true`) controlled lightfuzz | no longer read |
+| `modules.lightfuzz.avoid_wafs` was ignored | controls lightfuzz: `always` (skip), `never` (fuzz whatever the probe finds, still padding request bodies where padding gets through), or `try_bypasses` (default); `true` / `false` still work as `always` / `never` |
+| `lightfuzz-max` still skipped WAF-tagged hosts | `lightfuzz-max` sets `never` and fuzzes them whatever the probe finds, still padding POST and JSON bodies where padding gets through |
+
+`lightfuzz-light` still skips WAF-tagged hosts. To keep the 3.0 behavior with the other presets, set `avoid_wafs` to `always` and exclude nowafpls:
+
+```bash
+bbot -t evilcorp.com -p lightfuzz -c modules.lightfuzz.avoid_wafs=always -em nowafpls
+```
+
 ### `dnsdumpster` requires an API key
 
 dnsdumpster no longer has a keyless mode. Set `modules.dnsdumpster.api_key`, or the module is disabled at setup.
@@ -89,6 +111,10 @@ dnsdumpster no longer has a keyless mode. Set `modules.dnsdumpster.api_key`, or
 
 With `urls: true`, wayback now probes each host once and drops URLs on hosts that don't respond. Expect fewer, but live, URLs from wayback.
 
+### API requests identify as BBOT
+
+Modules that query third-party APIs through BBOT's API request helper now send `User-Agent: BBOT/<version>` instead of the scan's user agent, so `web.user_agent` and `web.user_agent_suffix` no longer apply to those requests. Not every module uses the helper yet, so some third-party requests still send `web.user_agent`; don't rely on `BBOT/<version>` for allowlisting. Requests to your targets still use `web.user_agent`.
+
 ---
 
 ## Event Output

diff --git a/docs/modules/custom_yara_rules.md b/docs/modules/custom_yara_rules.md
@@ -163,5 +163,5 @@ bbot -m http --custom-yara-rules=substack.yara -t http://www.blacklanternsecurit
 We get the following result. Note that the finding now contains the actual title tag that was identified with the regex.
 
 ```
-[FINDING] {"description": "Custom Yara Rule [ContainsTitle] with description: [Contains an HTML title] Matched via identifier [title_value] and extracted [<title>Black Lantern Security</title>]", "host": "www.blacklanternsecurity.com", "url": "https://www.blacklanternsecurity.com/"}	excavate	(cdn-github, cdn-ip)
+[FINDING] {"description": "Custom Yara Rule [ContainsTitle] with description: [Contains an HTML title] Matched via identifier [title_value] and extracted [<title>Black Lantern Security</title>]", "host": "www.blacklanternsecurity.com", "url": "https://www.blacklanternsecurity.com/"}	excavate	(cdn, github)
 ```
diff --git a/docs/modules/lightfuzz.md b/docs/modules/lightfuzz.md
@@ -202,6 +202,16 @@ Injects URLs pointing to unique Interactsh subdomains as parameter values. Three
 
 If the server fetches the injected URL, the resulting DNS or HTTP interaction is detected out-of-band via Interactsh. HTTP interactions are CONFIRMED confidence (the server made a full HTTP request), while DNS-only interactions are MEDIUM confidence (DNS resolution occurred, but the server may not have completed the request). Requires Interactsh to be enabled (it is by default).
 
+## WAF Handling
+
+`avoid_wafs` controls what lightfuzz does with targets tagged `waf`:
+
+| Value | Behavior |
+|---|---|
+| `always` | Skip WAF-protected targets entirely |
+| `try_bypasses` (default) | Ask the `nowafpls` helper whether the WAF blocks payloads in the request body, and whether padding the body gets past it. POST body and JSON parameters are fuzzed with padding if it does, unpadded if the WAF isn't blocking them, and skipped otherwise. Cookie and header parameters are skipped, since they have no body to pad. GET parameters are skipped too, unless `try_get_as_post` is on (as in `lightfuzz-heavy` and `lightfuzz-xss`), in which case they are retested as POST bodies and handled like POST parameters above. |
+| `never` | Fuzz every parameter on WAF-protected targets, whatever the probe finds. POST body and JSON parameters are still padded where `nowafpls` finds padding gets past the WAF. |
+
 ## Presets
 
 Lightfuzz comes with pre-defined presets. Unless you really know BBOT inside and out, we recommend using one of them. This is because Lightfuzz needs to change several important BBOT settings to work correctly, including:
@@ -220,7 +230,7 @@ The most minimal option. Best for running alongside larger scans with minimal ov
 | **Submodules** | `path`, `sqli`, `xss` only |
 | **Companion modules** | None |
 | **POST fuzzing** | Disabled |
-| **WAF avoidance** | On |
+| **WAF avoidance** | On (`avoid_wafs: always`): WAF-protected targets are skipped |
 
 ### `-p lightfuzz`
 
@@ -231,7 +241,7 @@ The default starting point. If you're not sure which to use, start here.
 | **Submodules** | All 9 (`cmdi`, `crypto`, `path`, `serial`, `sqli`, `ssti`, `xss`, `esi`, `ssrf`) |
 | **Companion modules** | `badsecrets`, `hunt`, `reflected_parameters` |
 | **POST fuzzing** | Disabled, but `try_post_as_get` is on (POST params retested as GET) |
-| **WAF avoidance** | On |
+| **WAF avoidance** | Try bypasses (`avoid_wafs: try_bypasses`): see [WAF Handling](#waf-handling) |
 
 ### `-p lightfuzz-heavy`
 
@@ -250,7 +260,7 @@ Everything in `lightfuzz-heavy`, plus:
 
 | Added setting | Value |
 |---|---|
-| **WAF avoidance** | Off — WAF-protected targets are fuzzed |
+| **WAF avoidance** | Off (`avoid_wafs: never`): every parameter on WAF-protected targets is fuzzed, and request bodies are still padded where padding gets past the WAF |
 | **Query string collapsing** | Off — each unique parameter-value pair is fuzzed individually instead of deduplicating by parameter name |
 | **Force common headers** | On — headers like `X-Forwarded-For` are fuzzed even if not observed on the target |
 | **Speculate params** | On — potential parameters are extracted from JSON/XML response bodies |
@@ -265,8 +275,9 @@ A focused preset for XSS hunting only. Demonstrates how to build a single-submod
 |---|---|
 | **Submodules** | `xss` only |
 | **Companion modules** | `paramminer_getparams`, `reflected_parameters` |
-| **POST fuzzing** | Disabled |
+| **POST fuzzing** | Enabled, and `try_get_as_post` is on (GET params retested as POST) |
 | **Query string collapsing** | Off |
+| **WAF avoidance** | Try bypasses (`avoid_wafs: try_bypasses`) |
 
 ### Spider Preset
 

diff --git a/docs/modules/nuclei.md b/docs/modules/nuclei.md
@@ -122,11 +122,11 @@ These options stack on top of whichever **mode** you've chosen, and on top of ea
 
 **include_tags** force-runs tags that Nuclei excludes by default. Nuclei ships an ignore list (`dos`, `local`, `fuzz`, `bruteforce`, `txt-service`) that applies before any option here, and this is the only way to reach those templates. Be careful with `fuzz` in particular: those templates carry very large payload lists and account for the overwhelming majority of Nuclei's potential request volume.
 
-**profile** selects a Nuclei template profile, which is a curated bundle of filter settings. You can give it the id of one of the profiles that ships with nuclei-templates (`recommended`, `pentest`, `cves`, `kev`, `misconfigurations`, `wordpress`, and more -- run `nuclei -tpl` for the full list), or a path to a profile file of your own. The `recommended` profile is a good default: alongside a severity floor it carries a maintained list of templates known to produce false positives, and that list updates along with the templates.
+**profile** selects a Nuclei template profile, which is a curated bundle of filter settings. You can give it the id of one of the profiles that ships with nuclei-templates (`recommended`, `pentest`, `cves`, `kev`, `misconfigurations`, `wordpress`, and more -- run `nuclei -tpl` for the full list), or a path to a profile file of your own. The `recommended` profile is a good default: alongside a severity floor it carries a maintained list of templates known to produce false positives, and that list updates along with the templates. The `nuclei-recommended` preset (`-p nuclei-recommended`) runs nuclei with this profile.
 
 **template_condition** is passed to Nuclei's `-template-condition` and filters on the template's metadata. Useful fields include `severity`, `max_request`, `epss_score`, `epss_percentile`, `cvss_score`, `cve_id`, `cwe_id`, `cpe`, `vendor`, `product` and `tags`. For example, `epss_score >= 0.1 || contains(tags, 'kev')` limits the scan to vulnerabilities with real-world exploitation evidence. Be aware that Nuclei treats an unrecognized field name as a non-match rather than an error, so BBOT counts the matching templates at startup and refuses to scan if the expression matches nothing.
 
-**new_templates** limits the scan to templates added in the most recent nuclei-templates release, which is typically 100-130 templates. **new_templates_versions** does the same for specific releases, e.g. `v10.4.8,v10.4.9`. These are handy for re-scanning a target you've already covered, to see only what's new.
+**new_templates** limits the scan to templates added in the most recent nuclei-templates release, which is typically 100-130 templates. **new_templates_versions** does the same for specific releases, e.g. `v10.4.8,v10.4.9`. These are handy for re-scanning a target you've already covered, to see only what's new. The `nuclei-new` preset (`-p nuclei-new`) does this with the latest release.
 
 ### Example Commands
 

diff --git a/docs/modules/wayback.md b/docs/modules/wayback.md
@@ -34,6 +34,7 @@ Before emission, URLs go through several cleanup steps:
 - **URL collapsing** - Groups of similar URLs (e.g. pagination, search results) are deduplicated based on the `garbage_threshold` setting
 - **HTTP/HTTPS deduplication** - When both `http://` and `https://` variants exist, only the HTTPS version is kept
 - **Blacklist filtering** - URLs containing known CDN/WAF paths (e.g. `_Incapsula_Resource`, `/cdn-cgi/`) are filtered out
+- **Dead-host filtering** - Each host is probed once over HTTP, and URLs on hosts that don't answer are dropped. Any response counts as alive; only connection failures and timeouts don't. With `archive` enabled, those URLs are still used for archive retrieval.
 
 ### Parameter Extraction (`parameters: True`)