diff --git a/README.md b/README.md index 6f3d1b5c34..81a6a5d834 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ [![bbot_banner](https://github.com/user-attachments/assets/f02804ce-9478-4f1e-ac4d-9cf5620a3214)](https://github.com/blacklanternsecurity/bbot) -[![Python Version](https://img.shields.io/badge/python-3.10+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-AGPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA) +[![Python Version](https://img.shields.io/badge/python-3.11+-FF8400)](https://www.python.org) [![License](https://img.shields.io/badge/license-AGPLv3-FF8400.svg)](https://github.com/blacklanternsecurity/bbot/blob/dev/LICENSE) [![PyPi Downloads](https://static.pepy.tech/personalized-badge/bbot?right_color=orange&left_color=grey)](https://pepy.tech/project/bbot) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![Tests](https://github.com/blacklanternsecurity/bbot/actions/workflows/tests.yml/badge.svg?branch=stable)](https://github.com/blacklanternsecurity/bbot/actions?query=workflow%3A"tests") [![Codecov](https://codecov.io/gh/blacklanternsecurity/bbot/branch/dev/graph/badge.svg?token=IR5AZBDM5K)](https://codecov.io/gh/blacklanternsecurity/bbot) [![Discord](https://img.shields.io/discord/859164869970362439)](https://discord.com/invite/PZqkgxu5SA) ### **BEE·bot** is a multipurpose scanner inspired by [Spiderfoot](https://github.com/smicallef/spiderfoot), built to automate your **Recon**, **Bug Bounties**, and **ASM**! @@ -20,6 +20,8 @@ pipx install --pip-args '\--pre' bbot _For more installation methods, including [Docker](https://hub.docker.com/r/blacklanternsecurity/bbot), see [Getting Started](https://www.blacklanternsecurity.com/bbot/Stable/)_ +> **Upgrading from 3.0?** See the [3.0 → 3.1 Migration Guide](https://www.blacklanternsecurity.com/bbot/Stable/migration/3.1_breaking_changes/) ([source](docs/migration/3.1_breaking_changes.md)) for changes that can affect existing presets, configs, custom modules, and output. + > **Upgrading from 2.x?** BBOT 3.0 contains breaking changes to the CLI, presets, modules, events, and Python API. See the [2.x → 3.0 Migration Guide](https://www.blacklanternsecurity.com/bbot/Stable/migration/3.0_breaking_changes/) ([source](docs/migration/3.0_breaking_changes.md)) before upgrading. > **Speed tip:** BBOT's DNS resolver ([blastdns](https://github.com/blacklanternsecurity/blastdns)) spins up multiple threads per resolver in `/etc/resolv.conf`. Adding more unfiltered resolvers dramatically speeds up scans. See the [sample resolv.conf](docs/data/resolv-sample.conf) and [Tips and Tricks](https://www.blacklanternsecurity.com/bbot/Stable/scanning/tips_and_tricks/#speed-up-scans-with-more-dns-resolvers) for details. @@ -56,7 +58,6 @@ output_modules: config: dns: - threads: 25 brute_threads: 1000 # put your API keys here # modules: diff --git a/bbot/defaults.yml b/bbot/defaults.yml index 714295efc6..f7bc88a72d 100644 --- a/bbot/defaults.yml +++ b/bbot/defaults.yml @@ -71,7 +71,7 @@ dns: # state bounded by the resolver count rather than growing with query volume brute_persistent_socket: true # nameservers to use for DNS brute-forcing - # default is updated weekly and contains ~10K high-quality public servers + # default is updated weekly and contains ~5K high-quality public servers brute_nameservers: https://raw.githubusercontent.com/blacklanternsecurity/public-dns-servers/master/nameservers.txt # How many of those to draw at random, once per scan (0 = all of them) # The dead ones in the draw are dropped at startup, and every brute-force run in the scan shares the rest diff --git a/bbot/modules/lightfuzz/lightfuzz.py b/bbot/modules/lightfuzz/lightfuzz.py index 71d2749604..78f3a5385c 100644 --- a/bbot/modules/lightfuzz/lightfuzz.py +++ b/bbot/modules/lightfuzz/lightfuzz.py @@ -42,8 +42,8 @@ class Config(BaseModuleConfig): "try_bypasses", description=( "How to handle WAF-tagged targets. 'always' skips them entirely; 'never' fuzzes them " - "raw; 'try_bypasses' (default) asks nowafpls whether body padding gets through, " - "and fuzzes with padding when it does." + "regardless, still padding request bodies where padding gets through; 'try_bypasses' " + "(default) asks nowafpls whether body padding gets through, and fuzzes with padding when it does." ), ) emit_baseline_responses: bool = Field( @@ -468,7 +468,7 @@ async def filter_event(self, event): return False, "WAF is bypassable via body padding, but this event has no POST-style probe" # STATUS_NO_INTERFERENCE: the POST body wasn't gated, so POST-style probes run unpadded. # It says nothing about other channels, so handle_event still skips GET/COOKIE/HEADER. - # avoid_wafs == "never": fall through and fuzz raw + # avoid_wafs == "never": fuzz regardless; prepare_request still pads bodies when bypassable # Skip WEB_PARAMETERs on static-asset URLs (pdf, doc, xml, etc.) — fuzzing them is pointless if event.type == "WEB_PARAMETER": diff --git a/bbot/presets/subdomain-enum.yml b/bbot/presets/subdomain-enum.yml index d4230c3fc6..994d8d4160 100644 --- a/bbot/presets/subdomain-enum.yml +++ b/bbot/presets/subdomain-enum.yml @@ -10,7 +10,6 @@ output_modules: config: dns: - threads: 25 brute_threads: 1000 # put your API keys here # modules: diff --git a/bbot/presets/web/lightfuzz-heavy.yml b/bbot/presets/web/lightfuzz-heavy.yml index c67474af1e..3bc6dac1dd 100644 --- a/bbot/presets/web/lightfuzz-heavy.yml +++ b/bbot/presets/web/lightfuzz-heavy.yml @@ -1,4 +1,4 @@ -description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. Still skips confirmed WAFs." +description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it." include: - lightfuzz diff --git a/bbot/presets/web/lightfuzz.yml b/bbot/presets/web/lightfuzz.yml index 3ae97c8c75..d3d8259e67 100644 --- a/bbot/presets/web/lightfuzz.yml +++ b/bbot/presets/web/lightfuzz.yml @@ -1,4 +1,4 @@ -description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. Skips confirmed WAFs." +description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it." include: - lightfuzz-light diff --git a/bbot/scanner/preset/args.py b/bbot/scanner/preset/args.py index 15a2c72839..f22c565c09 100644 --- a/bbot/scanner/preset/args.py +++ b/bbot/scanner/preset/args.py @@ -463,7 +463,11 @@ def create_parser(self, *args, **kwargs): ) misc.add_argument("--custom-yara-rules", "-cy", help="Add custom yara rules to excavate") - misc.add_argument("--user-agent", "-ua", help="Set the user-agent for all HTTP requests") + misc.add_argument( + "--user-agent", + "-ua", + help="Set the user-agent for HTTP requests (most API calls send BBOT/)", + ) misc.add_argument( "--user-agent-suffix", "-uas", help="Suffix to append to the user-agent", metavar="SUFFIX", default=None ) diff --git a/docs/migration/3.1_breaking_changes.md b/docs/migration/3.1_breaking_changes.md index 8cc15594a4..31bfab81d7 100644 --- a/docs/migration/3.1_breaking_changes.md +++ b/docs/migration/3.1_breaking_changes.md @@ -6,6 +6,12 @@ If a change is missing here, please open an issue. --- +## Python + +BBOT 3.1 requires Python 3.11 or newer, and Python 3.10 is no longer supported. On 3.10, `pip install --upgrade bbot` doesn't fail; it stays on the latest 3.0 release, so upgrade Python first. Python 3.15 is now supported. + +--- + ## Config and Presets ### Custom module folders (`module_dirs`) @@ -77,6 +83,22 @@ By default, open ports other than 80 and 443 are dropped on hosts tagged `cdn` o ## Modules +### `lightfuzz` probes WAF-protected hosts instead of skipping them + +In 3.0, lightfuzz skipped every host tagged `waf`. In 3.1, `modules.lightfuzz.avoid_wafs` defaults to `try_bypasses`: lightfuzz sends each WAF-tagged host a test payload in a POST body, first unpadded and then padded (the nowafpls technique). It fuzzes the host only if that probe gets past the WAF: either the padding gets the payload through, or the WAF didn't block it even unpadded. Even then, only request-body parameters are fuzzed. The `lightfuzz`, `lightfuzz-heavy`, `lightfuzz-max`, and `lightfuzz-xss` presets also enable the `nowafpls` module, which runs the same probe on every WAF-tagged host and emits a `FINDING` when padding gets through. Expect more traffic to WAF-fronted targets after upgrading. See [WAF Handling](../modules/lightfuzz.md#waf-handling). + +| 3.0 | 3.1 | +|-----|-----| +| the top-level `avoid_wafs` (default `true`) controlled lightfuzz | no longer read | +| `modules.lightfuzz.avoid_wafs` was ignored | controls lightfuzz: `always` (skip), `never` (fuzz whatever the probe finds, still padding request bodies where padding gets through), or `try_bypasses` (default); `true` / `false` still work as `always` / `never` | +| `lightfuzz-max` still skipped WAF-tagged hosts | `lightfuzz-max` sets `never` and fuzzes them whatever the probe finds, still padding POST and JSON bodies where padding gets through | + +`lightfuzz-light` still skips WAF-tagged hosts. To keep the 3.0 behavior with the other presets, set `avoid_wafs` to `always` and exclude nowafpls: + +```bash +bbot -t evilcorp.com -p lightfuzz -c modules.lightfuzz.avoid_wafs=always -em nowafpls +``` + ### `dnsdumpster` requires an API key dnsdumpster no longer has a keyless mode. Set `modules.dnsdumpster.api_key`, or the module is disabled at setup. @@ -89,6 +111,10 @@ dnsdumpster no longer has a keyless mode. Set `modules.dnsdumpster.api_key`, or With `urls: true`, wayback now probes each host once and drops URLs on hosts that don't respond. Expect fewer, but live, URLs from wayback. +### API requests identify as BBOT + +Modules that query third-party APIs through BBOT's API request helper now send `User-Agent: BBOT/` instead of the scan's user agent, so `web.user_agent` and `web.user_agent_suffix` no longer apply to those requests. Not every module uses the helper yet, so some third-party requests still send `web.user_agent`; don't rely on `BBOT/` for allowlisting. Requests to your targets still use `web.user_agent`. + --- ## Event Output diff --git a/docs/modules/custom_yara_rules.md b/docs/modules/custom_yara_rules.md index cf6b540fb6..dc0f825d53 100644 --- a/docs/modules/custom_yara_rules.md +++ b/docs/modules/custom_yara_rules.md @@ -163,5 +163,5 @@ bbot -m http --custom-yara-rules=substack.yara -t http://www.blacklanternsecurit We get the following result. Note that the finding now contains the actual title tag that was identified with the regex. ``` -[FINDING] {"description": "Custom Yara Rule [ContainsTitle] with description: [Contains an HTML title] Matched via identifier [title_value] and extracted [Black Lantern Security]", "host": "www.blacklanternsecurity.com", "url": "https://www.blacklanternsecurity.com/"} excavate (cdn-github, cdn-ip) +[FINDING] {"description": "Custom Yara Rule [ContainsTitle] with description: [Contains an HTML title] Matched via identifier [title_value] and extracted [Black Lantern Security]", "host": "www.blacklanternsecurity.com", "url": "https://www.blacklanternsecurity.com/"} excavate (cdn, github) ``` diff --git a/docs/modules/lightfuzz.md b/docs/modules/lightfuzz.md index 194da52273..5cb3f692a1 100644 --- a/docs/modules/lightfuzz.md +++ b/docs/modules/lightfuzz.md @@ -202,6 +202,16 @@ Injects URLs pointing to unique Interactsh subdomains as parameter values. Three If the server fetches the injected URL, the resulting DNS or HTTP interaction is detected out-of-band via Interactsh. HTTP interactions are CONFIRMED confidence (the server made a full HTTP request), while DNS-only interactions are MEDIUM confidence (DNS resolution occurred, but the server may not have completed the request). Requires Interactsh to be enabled (it is by default). +## WAF Handling + +`avoid_wafs` controls what lightfuzz does with targets tagged `waf`: + +| Value | Behavior | +|---|---| +| `always` | Skip WAF-protected targets entirely | +| `try_bypasses` (default) | Ask the `nowafpls` helper whether the WAF blocks payloads in the request body, and whether padding the body gets past it. POST body and JSON parameters are fuzzed with padding if it does, unpadded if the WAF isn't blocking them, and skipped otherwise. Cookie and header parameters are skipped, since they have no body to pad. GET parameters are skipped too, unless `try_get_as_post` is on (as in `lightfuzz-heavy` and `lightfuzz-xss`), in which case they are retested as POST bodies and handled like POST parameters above. | +| `never` | Fuzz every parameter on WAF-protected targets, whatever the probe finds. POST body and JSON parameters are still padded where `nowafpls` finds padding gets past the WAF. | + ## Presets Lightfuzz comes with pre-defined presets. Unless you really know BBOT inside and out, we recommend using one of them. This is because Lightfuzz needs to change several important BBOT settings to work correctly, including: @@ -220,7 +230,7 @@ The most minimal option. Best for running alongside larger scans with minimal ov | **Submodules** | `path`, `sqli`, `xss` only | | **Companion modules** | None | | **POST fuzzing** | Disabled | -| **WAF avoidance** | On | +| **WAF avoidance** | On (`avoid_wafs: always`): WAF-protected targets are skipped | ### `-p lightfuzz` @@ -231,7 +241,7 @@ The default starting point. If you're not sure which to use, start here. | **Submodules** | All 9 (`cmdi`, `crypto`, `path`, `serial`, `sqli`, `ssti`, `xss`, `esi`, `ssrf`) | | **Companion modules** | `badsecrets`, `hunt`, `reflected_parameters` | | **POST fuzzing** | Disabled, but `try_post_as_get` is on (POST params retested as GET) | -| **WAF avoidance** | On | +| **WAF avoidance** | Try bypasses (`avoid_wafs: try_bypasses`): see [WAF Handling](#waf-handling) | ### `-p lightfuzz-heavy` @@ -250,7 +260,7 @@ Everything in `lightfuzz-heavy`, plus: | Added setting | Value | |---|---| -| **WAF avoidance** | Off — WAF-protected targets are fuzzed | +| **WAF avoidance** | Off (`avoid_wafs: never`): every parameter on WAF-protected targets is fuzzed, and request bodies are still padded where padding gets past the WAF | | **Query string collapsing** | Off — each unique parameter-value pair is fuzzed individually instead of deduplicating by parameter name | | **Force common headers** | On — headers like `X-Forwarded-For` are fuzzed even if not observed on the target | | **Speculate params** | On — potential parameters are extracted from JSON/XML response bodies | @@ -265,8 +275,9 @@ A focused preset for XSS hunting only. Demonstrates how to build a single-submod |---|---| | **Submodules** | `xss` only | | **Companion modules** | `paramminer_getparams`, `reflected_parameters` | -| **POST fuzzing** | Disabled | +| **POST fuzzing** | Enabled, and `try_get_as_post` is on (GET params retested as POST) | | **Query string collapsing** | Off | +| **WAF avoidance** | Try bypasses (`avoid_wafs: try_bypasses`) | ### Spider Preset diff --git a/docs/modules/nuclei.md b/docs/modules/nuclei.md index 6894da01ea..eca74cbf21 100644 --- a/docs/modules/nuclei.md +++ b/docs/modules/nuclei.md @@ -122,11 +122,11 @@ These options stack on top of whichever **mode** you've chosen, and on top of ea **include_tags** force-runs tags that Nuclei excludes by default. Nuclei ships an ignore list (`dos`, `local`, `fuzz`, `bruteforce`, `txt-service`) that applies before any option here, and this is the only way to reach those templates. Be careful with `fuzz` in particular: those templates carry very large payload lists and account for the overwhelming majority of Nuclei's potential request volume. -**profile** selects a Nuclei template profile, which is a curated bundle of filter settings. You can give it the id of one of the profiles that ships with nuclei-templates (`recommended`, `pentest`, `cves`, `kev`, `misconfigurations`, `wordpress`, and more -- run `nuclei -tpl` for the full list), or a path to a profile file of your own. The `recommended` profile is a good default: alongside a severity floor it carries a maintained list of templates known to produce false positives, and that list updates along with the templates. +**profile** selects a Nuclei template profile, which is a curated bundle of filter settings. You can give it the id of one of the profiles that ships with nuclei-templates (`recommended`, `pentest`, `cves`, `kev`, `misconfigurations`, `wordpress`, and more -- run `nuclei -tpl` for the full list), or a path to a profile file of your own. The `recommended` profile is a good default: alongside a severity floor it carries a maintained list of templates known to produce false positives, and that list updates along with the templates. The `nuclei-recommended` preset (`-p nuclei-recommended`) runs nuclei with this profile. **template_condition** is passed to Nuclei's `-template-condition` and filters on the template's metadata. Useful fields include `severity`, `max_request`, `epss_score`, `epss_percentile`, `cvss_score`, `cve_id`, `cwe_id`, `cpe`, `vendor`, `product` and `tags`. For example, `epss_score >= 0.1 || contains(tags, 'kev')` limits the scan to vulnerabilities with real-world exploitation evidence. Be aware that Nuclei treats an unrecognized field name as a non-match rather than an error, so BBOT counts the matching templates at startup and refuses to scan if the expression matches nothing. -**new_templates** limits the scan to templates added in the most recent nuclei-templates release, which is typically 100-130 templates. **new_templates_versions** does the same for specific releases, e.g. `v10.4.8,v10.4.9`. These are handy for re-scanning a target you've already covered, to see only what's new. +**new_templates** limits the scan to templates added in the most recent nuclei-templates release, which is typically 100-130 templates. **new_templates_versions** does the same for specific releases, e.g. `v10.4.8,v10.4.9`. These are handy for re-scanning a target you've already covered, to see only what's new. The `nuclei-new` preset (`-p nuclei-new`) does this with the latest release. ### Example Commands diff --git a/docs/modules/wayback.md b/docs/modules/wayback.md index e4bb10c774..180122b86a 100644 --- a/docs/modules/wayback.md +++ b/docs/modules/wayback.md @@ -34,6 +34,7 @@ Before emission, URLs go through several cleanup steps: - **URL collapsing** - Groups of similar URLs (e.g. pagination, search results) are deduplicated based on the `garbage_threshold` setting - **HTTP/HTTPS deduplication** - When both `http://` and `https://` variants exist, only the HTTPS version is kept - **Blacklist filtering** - URLs containing known CDN/WAF paths (e.g. `_Incapsula_Resource`, `/cdn-cgi/`) are filtered out +- **Dead-host filtering** - Each host is probed once over HTTP, and URLs on hosts that don't answer are dropped. Any response counts as alive; only connection failures and timeouts don't. With `archive` enabled, those URLs are still used for archive retrieval. ### Parameter Extraction (`parameters: True`) diff --git a/docs/release_history.md b/docs/release_history.md index aa0aa7af06..f301a52095 100644 --- a/docs/release_history.md +++ b/docs/release_history.md @@ -1,3 +1,112 @@ +### 3.1.0 - Nov 10, 2026 +- [https://github.com/blacklanternsecurity/bbot/pull/3584](https://github.com/blacklanternsecurity/bbot/pull/3584) + +!!! warning "Check your setup before upgrading" + + 3.1 includes changes that can affect existing presets, configs, custom modules, and anything that consumes scan output. Read the [3.0 to 3.1 migration guide](./migration/3.1_breaking_changes.md) before upgrading. The changes most likely to bite: + + - Python 3.11 or newer is required. On 3.10, `pip install --upgrade bbot` stays on 3.0. + - The `portfilter` module is gone. CDN/WAF port filtering is built in and on by default; remove `portfilter` from presets, and pass `--allow-cdn-ports` to turn the filter off. + - `module_dirs` only works as a top-level preset key, and BBOT asks before loading each custom module folder. Runs without a terminal need `-y`, and `-S` no longer implies it. + - Preset `conditions` are read-only: they can warn or abort, but can't modify the preset. + - dnsdumpster requires an API key, and waf_bypass only tests scan targets by default. + - lightfuzz now probes WAF-tagged hosts with nowafpls instead of skipping them, and fuzzes one only if the probe gets past the WAF. For the old behavior, set `modules.lightfuzz.avoid_wafs: always` and pass `-em nowafpls`. + +#### Highlights + +- **DNS brute-forcing moves to BlastDNS:** massdns is gone, and brute-forcing runs through blastdns 2.0 with adaptive rate control. With higher default DNS concurrency, a typical `subdomain-enum` scan finishes in about 40% less time than on 3.0.2, with the same subdomains found ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457), [#3606](https://github.com/blacklanternsecurity/bbot/pull/3606), [#3612](https://github.com/blacklanternsecurity/bbot/pull/3612)) +- **New `--custom-nameservers` flag** for choosing the resolvers BBOT uses for normal resolution ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)) +- **baddns 2.5.0**, with a new `DELEGATION` submodule that is on by default ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)) +- **CDN/WAF port filtering in core, on by default** ([#3534](https://github.com/blacklanternsecurity/bbot/pull/3534)) +- **Smarter API rate limiting:** `Retry-After` is honored, API keys rotate on a 429, and rate limits no longer count as failures ([#3461](https://github.com/blacklanternsecurity/bbot/pull/3461)) +- **Hardened custom module and preset loading** ([#3593](https://github.com/blacklanternsecurity/bbot/pull/3593)) +- **New modules:** favicon, js_unpacker, nowafpls, otilabs, scanmalware +- **Python 3.15 supported, Python 3.10 dropped** ([#3474](https://github.com/blacklanternsecurity/bbot/pull/3474)) + +#### Core & Packaging + +- Supported Python is now 3.11 - 3.15 ([#3474](https://github.com/blacklanternsecurity/bbot/pull/3474)). +- CDN/WAF port filtering moved from the `portfilter` module into core and is on by default. Open ports other than 80 and 443 are dropped on hosts tagged `cdn` or `waf`; `--allow-cdn-ports` turns it off ([#3534](https://github.com/blacklanternsecurity/bbot/pull/3534)). +- `api_request` rate limiting overhauled, plus config deprecation support via `Field(deprecated=True)`. Calls made through `api_request` now send a `BBOT/` User-Agent instead of the scan's user agent ([#3461](https://github.com/blacklanternsecurity/bbot/pull/3461)). +- CPE-derived technologies are named by product, with the CPE kept in a new `cpe` field ([#3535](https://github.com/blacklanternsecurity/bbot/pull/3535)). +- In-scope events are no longer tagged `affiliate` ([#3524](https://github.com/blacklanternsecurity/bbot/pull/3524)). +- FINDING text output shows the host when it isn't otherwise present. +- FINDING has an optional `port` for findings that aren't on a URL, such as a TLS or TCP service. It's part of the event's identity and shown as `host:port` in text output, and legba and medusa findings now keep their port ([#3629](https://github.com/blacklanternsecurity/bbot/pull/3629)). +- `HTTP_RESPONSE` events carry `url_extension` ([#3406](https://github.com/blacklanternsecurity/bbot/pull/3406)). +- `ClosestHostEvent` keeps its inherited port, and the port is restored from JSON ([#3462](https://github.com/blacklanternsecurity/bbot/pull/3462)). +- `download()` respects `cache_hrs` in its freshness check ([#3563](https://github.com/blacklanternsecurity/bbot/pull/3563)). +- Fix hostless sort sentinel type mismatch in `Target.add()` ([#3391](https://github.com/blacklanternsecurity/bbot/pull/3391)). +- Version output includes the git commit hash for git checkouts ([#3530](https://github.com/blacklanternsecurity/bbot/pull/3530)). +- `helpers.response_to_json()` removed in favor of `response_to_event_dict()` ([#3473](https://github.com/blacklanternsecurity/bbot/pull/3473)). + +#### DNS + +- massdns removed. DNS brute-forcing runs through blastdns 2.0, with its own retry budget, a short per-query timeout, and one persistent socket per resolver ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)). +- New `dns.brute_max_resolvers` (default 600, `0` for the whole list). Each scan draws that many brute-force resolvers at random, drops the dead ones at startup, and shares the rest across every brute-force run in the scan ([#3612](https://github.com/blacklanternsecurity/bbot/pull/3612)). +- Ordinary DNS concurrency raised and made explicit: `dns.threads` 10 to 50, and a new `dns.max_concurrency` (256) ([#3606](https://github.com/blacklanternsecurity/bbot/pull/3606)). +- New `--custom-nameservers` flag (`dns.nameservers`). An invalid value aborts the scan during prep instead of producing a "completed" scan with zero results ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)). +- Resolver diagnostics are reported when brute-force delivery drops ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)). +- Already-resolved events skip ahead in dnsresolve's queue ([#3533](https://github.com/blacklanternsecurity/bbot/pull/3533)). +- Fix nested wildcard detection in `is_wildcard` ([#3536](https://github.com/blacklanternsecurity/bbot/pull/3536)). + +#### BadDNS + +- baddns 2.5.0 ([#3457](https://github.com/blacklanternsecurity/bbot/pull/3457)). +- New `DELEGATION` submodule, enabled by default and in the `baddns` / `baddns-heavy` presets. Checks `_acme-challenge`, `_dmarc` and common DKIM selectors for dangling delegations. +- `NS` gained a lame-delegation finding. +- On-demand TLS platform detection, S3/GCS bucket references in HTML (`references` submodule, in `baddns-heavy`), and a fix for S3 bucket takeover detection. +- Signature refresh: 37 stale signatures removed, 78 updated, and 5 added, plus 14 that suppress NS false positives on registrar and parking nameservers. +- Word matching runs every signature in a single YARA pass per response. + +#### Modules + +**New:** + +- `favicon` - technology detection from favicon hashes ([#3532](https://github.com/blacklanternsecurity/bbot/pull/3532)). +- `js_unpacker` - unpacks source maps, webpack, obfuscator.io, Next.js and Dean Edwards packer JavaScript, and feeds the results back into the scan; in the `js-audit` preset ([#3335](https://github.com/blacklanternsecurity/bbot/pull/3335)). +- `nowafpls` - WAF bypass detection via body padding. lightfuzz, ajaxpro and generic_ssrf pad their payloads on hosts where the bypass is confirmed ([#3336](https://github.com/blacklanternsecurity/bbot/pull/3336)). +- `otilabs` - subdomain enumeration via the OTI Labs Domain Intelligence API ([#3496](https://github.com/blacklanternsecurity/bbot/pull/3496)). +- `scanmalware` - subdomains from ScanMalware's public scan archive. Keyless and opt-in (`-m scanmalware`) ([#3528](https://github.com/blacklanternsecurity/bbot/pull/3528)). + +**Removed:** portfilter (replaced by core CDN/WAF port filtering). + +**Notable improvements:** + +- telerik overhaul: native RAU and DialogHandler crypto, and exploitable RAU detected without uploading a file ([#3333](https://github.com/blacklanternsecurity/bbot/pull/3333)). +- nuclei: nine new template-selection options, new `nuclei-recommended` and `nuclei-new` presets, budget mode fixed (a budget of 10 now means about 15 requests per host, down from 134), and budget mode's template index cached ([#3479](https://github.com/blacklanternsecurity/bbot/pull/3479)). Results are parented to the URL, or the host and port, that nuclei actually scanned, so findings on sibling subdomains or on several ports of one host are no longer merged or mislabeled ([#3629](https://github.com/blacklanternsecurity/bbot/pull/3629)). +- excavate: long redirect detection ([#3481](https://github.com/blacklanternsecurity/bbot/pull/3481)), relative URLs resolved against `` ([#3522](https://github.com/blacklanternsecurity/bbot/pull/3522)), and hostnames no longer extracted from partway through a hyphenated label ([#3590](https://github.com/blacklanternsecurity/bbot/pull/3590)). +- lightfuzz: positive confirmation required for code-change findings ([#3357](https://github.com/blacklanternsecurity/bbot/pull/3357)), tighter serial error resolution ([#3427](https://github.com/blacklanternsecurity/bbot/pull/3427)), unbounded queue growth from parameter re-discovery fixed ([#3405](https://github.com/blacklanternsecurity/bbot/pull/3405)), baseline URLs counted against spider distance ([#3523](https://github.com/blacklanternsecurity/bbot/pull/3523)), raw Cookie header read in the sqli cookie test ([#3498](https://github.com/blacklanternsecurity/bbot/pull/3498)), and WAF gating comments corrected ([#3608](https://github.com/blacklanternsecurity/bbot/pull/3608)). +- dnsdumpster converted to the authenticated API ([#3458](https://github.com/blacklanternsecurity/bbot/pull/3458)). +- wayback verifies hosts are alive before emitting URLs ([#3359](https://github.com/blacklanternsecurity/bbot/pull/3359)). +- waf_bypass only attempts bypasses against scan targets by default ([#3460](https://github.com/blacklanternsecurity/bbot/pull/3460)). +- crt requests an explicit `/` path, which crt.sh requires ([#3428](https://github.com/blacklanternsecurity/bbot/pull/3428)). +- SQL output modules request sqlalchemy's asyncio extra ([#3505](https://github.com/blacklanternsecurity/bbot/pull/3505)). + +#### Performance + +- DeepDiff selectively replaced ([#3340](https://github.com/blacklanternsecurity/bbot/pull/3340)), then removed entirely ([#3604](https://github.com/blacklanternsecurity/bbot/pull/3604)). +- Faster `compute_simhash`, with corrected executor placements ([#3331](https://github.com/blacklanternsecurity/bbot/pull/3331)). + +#### Security + +- Custom module loading hardened: BBOT asks before loading each custom module folder, refuses world-writable folders and files, and refuses custom modules that reuse an existing module name. `config.module_dirs` is deprecated and ignored ([#3593](https://github.com/blacklanternsecurity/bbot/pull/3593)). +- Preset `conditions` are read-only, and module `version` and the global `home` option are validated before they reach an install command ([#3593](https://github.com/blacklanternsecurity/bbot/pull/3593)). +- Symlinks in unarchive output are rejected after extraction ([#3560](https://github.com/blacklanternsecurity/bbot/pull/3560)). +- gitdumper: origin check and depth limit on directory listing ([#3466](https://github.com/blacklanternsecurity/bbot/pull/3466)). +- `dnspython` removed. Nothing imported it, and its old pin held installs below 2.9.0, the fix for GHSA-45c3-73f7-pv4m ([#3610](https://github.com/blacklanternsecurity/bbot/pull/3610)). +- Security dependency updates ([#3557](https://github.com/blacklanternsecurity/bbot/pull/3557)). + +#### Dependencies + +- blastdns 2.0.4 ([#3612](https://github.com/blacklanternsecurity/bbot/pull/3612), [#3631](https://github.com/blacklanternsecurity/bbot/pull/3631)), blasthttp 0.10.1, baddns 2.5.0, badsecrets 1.3.0 ([#3526](https://github.com/blacklanternsecurity/bbot/pull/3526)), cloudcheck 11.2.0 ([#3559](https://github.com/blacklanternsecurity/bbot/pull/3559)), radixtarget 4.3.1 ([#3558](https://github.com/blacklanternsecurity/bbot/pull/3558), [#3617](https://github.com/blacklanternsecurity/bbot/pull/3617)), pycryptodome 4.3.0 ([#3542](https://github.com/blacklanternsecurity/bbot/pull/3542), [#3597](https://github.com/blacklanternsecurity/bbot/pull/3597), [#3620](https://github.com/blacklanternsecurity/bbot/pull/3620)). +- `deepdiff`, `cachebox` and `orderly-set` removed ([#3604](https://github.com/blacklanternsecurity/bbot/pull/3604)). +- `pyjwt`, `lxml` and `socksio` removed, and `xmltojson` replaced by `xmltodict` ([#3627](https://github.com/blacklanternsecurity/bbot/pull/3627)). +- trufflehog 3.97.0 to 3.99.8 ([#3605](https://github.com/blacklanternsecurity/bbot/pull/3605)). + +#### Documentation + +- New [3.0 to 3.1 migration guide](./migration/3.1_breaking_changes.md) ([#3594](https://github.com/blacklanternsecurity/bbot/pull/3594)), and docs updated for 3.1 ([#3613](https://github.com/blacklanternsecurity/bbot/pull/3613)). + ### 3.0.2 - Aug 24, 2026 - [https://github.com/blacklanternsecurity/bbot/pull/3332](https://github.com/blacklanternsecurity/bbot/pull/3332) diff --git a/docs/scanning/advanced.md b/docs/scanning/advanced.md index 838230f527..3496cd3612 100644 --- a/docs/scanning/advanced.md +++ b/docs/scanning/advanced.md @@ -148,7 +148,7 @@ Misc: --custom-yara-rules, -cy CUSTOM_YARA_RULES Add custom yara rules to excavate --user-agent, -ua USER_AGENT - Set the user-agent for all HTTP requests + Set the user-agent for HTTP requests (most API calls send BBOT/) --user-agent-suffix, -uas SUFFIX Suffix to append to the user-agent diff --git a/docs/scanning/configuration.md b/docs/scanning/configuration.md index 7bca62e73b..ceab9f6e82 100644 --- a/docs/scanning/configuration.md +++ b/docs/scanning/configuration.md @@ -2,7 +2,7 @@ Normally, [Presets](presets.md) are used to configure a scan. However, there may be cases where you want to change BBOT's global defaults so a certain option is always set, even if it's not specified in a preset. -BBOT has a YAML config at `~/.config/bbot/bbot.yml`. This is the first config that BBOT loads, so it's a good place to put default settings like `http_proxy`, `max_threads`, or `http_user_agent`. **API keys** and other sensitive values go in a companion file at `~/.config/bbot/secrets.yml`. +BBOT has a YAML config at `~/.config/bbot/bbot.yml`. This is the first config that BBOT loads, so it's a good place to put default settings like `web.http_proxy`, `web.user_agent`, or `dns.threads`. **API keys** and other sensitive values go in a companion file at `~/.config/bbot/secrets.yml`. For a list of all possible config options, see: @@ -131,7 +131,7 @@ dns: # state bounded by the resolver count rather than growing with query volume brute_persistent_socket: true # nameservers to use for DNS brute-forcing - # default is updated weekly and contains ~10K high-quality public servers + # default is updated weekly and contains ~5K high-quality public servers brute_nameservers: https://raw.githubusercontent.com/blacklanternsecurity/public-dns-servers/master/nameservers.txt # How many of those to draw at random, once per scan (0 = all of them) # The dead ones in the draw are dropped at startup, and every brute-force run in the scan shares the rest @@ -539,7 +539,7 @@ In addition to the stated options for each module, the following universal optio | modules.legba.telnet_wordlist | Union[str, list[str]] | Wordlist for TELNET combined username:password, newline separated. Accepts a URL or local file path, or a list of URLs/paths to merge multiple wordlists (duplicates removed). | https://raw.githubusercontent.com/danielmiessler/SecLists/refs/heads/master/Passwords/Default-Credentials/telnet-betterdefaultpasslist.txt | | modules.legba.version | VersionStr | legba version | 1.1.1 | | modules.legba.vnc_wordlist | Union[str, list[str]] | Wordlist for VNC passwords, newline separated. Accepts a URL or local file path, or a list of URLs/paths to merge multiple wordlists (duplicates removed). | https://raw.githubusercontent.com/danielmiessler/SecLists/refs/heads/master/Passwords/Default-Credentials/vnc-betterdefaultpasslist.txt | -| modules.lightfuzz.avoid_wafs | Literal['always', 'never', 'try_bypasses'] | How to handle WAF-tagged targets. 'always' skips them entirely; 'never' fuzzes them raw; 'try_bypasses' (default) asks nowafpls whether body padding gets through, and fuzzes with padding when it does. | try_bypasses | +| modules.lightfuzz.avoid_wafs | Literal['always', 'never', 'try_bypasses'] | How to handle WAF-tagged targets. 'always' skips them entirely; 'never' fuzzes them regardless, still padding request bodies where padding gets through; 'try_bypasses' (default) asks nowafpls whether body padding gets through, and fuzzes with padding when it does. | try_bypasses | | modules.lightfuzz.disable_post | bool | Disable processing of POST parameters, avoiding form submissions. | False | | modules.lightfuzz.emit_baseline_responses | bool | Emit canonical baseline responses as HTTP_RESPONSE events so excavate can mine them for new params/URLs. | True | | modules.lightfuzz.enabled_submodules | list[str] | A list of submodules to enable. Empty list enabled all modules. | ['sqli', 'cmdi', 'xss', 'path', 'ssti', 'crypto', 'serial', 'esi', 'ssrf'] | diff --git a/docs/scanning/events.md b/docs/scanning/events.md index 45cdeae3cd..e8f9e35883 100644 --- a/docs/scanning/events.md +++ b/docs/scanning/events.md @@ -85,7 +85,8 @@ These attributes allow us to construct a visual graph of events (e.g. in [Neo4j] "tags": [ "a-record", "aaaa-record", - "cdn-akamai", + "akamai", + "cdn", "cname-record", "in-scope", "subdomain" diff --git a/docs/scanning/output.md b/docs/scanning/output.md index b8d6489495..6c03f20fed 100644 --- a/docs/scanning/output.md +++ b/docs/scanning/output.md @@ -73,10 +73,10 @@ The `csv` output module produces a CSV like this: | Event type | Event data | IP Address | Source Module | Scope Distance | Event Tags | | ---------- | ----------------------- | ---------- | ------------- | -------------- | -------------------------------------------------------------------------------------------------------- | -| DNS_NAME | evilcorp.com | 1.2.3.4 | TARGET | 0 | a-record,cdn-github,distance-0,domain,in-scope,mx-record,ns-record,resolved,soa-record,target,txt-record | -| DNS_NAME | www.evilcorp.com | 2.3.4.5 | certspotter | 0 | a-record,aaaa-record,cdn-github,cname-record,distance-0,in-scope,resolved,subdomain | -| URL | http://www.evilcorp.com | 2.3.4.5 | http | 0 | a-record,aaaa-record,cdn-github,cname-record,distance-0,in-scope,resolved,subdomain | -| DNS_NAME | admin.evilcorp.com | 5.6.7.8 | otx | 0 | a-record,aaaa-record,cloud-azure,cname-record,distance-0,in-scope,resolved,subdomain | +| DNS_NAME | evilcorp.com | 1.2.3.4 | TARGET | 0 | a-record,cdn,distance-0,domain,github,in-scope,mx-record,ns-record,resolved,soa-record,target,txt-record | +| DNS_NAME | www.evilcorp.com | 2.3.4.5 | certspotter | 0 | a-record,aaaa-record,cdn,cname-record,distance-0,github,in-scope,resolved,subdomain | +| URL | http://www.evilcorp.com | 2.3.4.5 | http | 0 | a-record,aaaa-record,cdn,cname-record,distance-0,github,in-scope,resolved,subdomain | +| DNS_NAME | admin.evilcorp.com | 5.6.7.8 | otx | 0 | a-record,aaaa-record,azure,cloud,cname-record,distance-0,in-scope,resolved,subdomain | ### JSON @@ -260,9 +260,9 @@ The `asset_inventory` module produces a CSV like this: | Host | Provider | IP(s) | Status | Open Ports | | ------------------ | ----------- | ------- | ------ | ---------- | -| evilcorp.com | cdn-github | 1.2.3.4 | Active | 80,443 | -| www.evilcorp.com | cdn-github | 2.3.4.5 | Active | 22,80,443 | -| admin.evilcorp.com | cloud-azure | 5.6.7.8 | N/A | | +| evilcorp.com | github | 1.2.3.4 | Active | 80,443 | +| www.evilcorp.com | github | 2.3.4.5 | Active | 22,80,443 | +| admin.evilcorp.com | azure | 5.6.7.8 | N/A | | ### SQLite diff --git a/docs/scanning/presets.md b/docs/scanning/presets.md index 460ec77448..4e1769ce97 100644 --- a/docs/scanning/presets.md +++ b/docs/scanning/presets.md @@ -213,7 +213,7 @@ When BBOT is used as a Python library, there is no prompt; it logs a warning for ### Conditions -Sometimes, you might need to add custom logic to a preset. BBOT supports this via `conditions`. The `conditions` attribute allows you to specify a list of custom conditions that will be evaluated before the scan starts. This is useful for performing last-minute sanity checks, or changing the behavior of the scan based on custom criteria. +Sometimes, you might need to add custom logic to a preset. BBOT supports this via `conditions`. The `conditions` attribute allows you to specify a list of custom conditions that will be evaluated before the scan starts. This is useful for last-minute sanity checks, like warning about or aborting on a risky combination of settings. ```yaml title="my_preset.yml" description: Abort if nuclei templates aren't specified diff --git a/docs/scanning/presets_list.md b/docs/scanning/presets_list.md index a796185395..f7cb53f471 100644 --- a/docs/scanning/presets_list.md +++ b/docs/scanning/presets_list.md @@ -272,11 +272,11 @@ Modules: [0]("") ## **lightfuzz** -Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. Skips confirmed WAFs. +Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it. ??? note "`lightfuzz.yml`" ```yaml title="~/.bbot/presets/web/lightfuzz.yml" - description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. Skips confirmed WAFs." + description: "Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it." include: - lightfuzz-light @@ -301,11 +301,11 @@ Modules: [0]("") ## **lightfuzz-heavy** -Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. Still skips confirmed WAFs. +Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it. ??? note "`lightfuzz-heavy.yml`" ```yaml title="~/.bbot/presets/web/lightfuzz-heavy.yml" - description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. Still skips confirmed WAFs." + description: "Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it." include: - lightfuzz @@ -763,7 +763,6 @@ Enumerate subdomains via APIs, brute-force config: dns: - threads: 25 brute_threads: 1000 # put your API keys here # modules: @@ -1083,8 +1082,8 @@ Here is a the same data, but in a table: | iis-shortnames | web | Recursively enumerate IIS shortnames | 0 | | | js-audit | web | Hunt for leaked credentials and vulnerable libraries in client-side JavaScript | 7 | badsecrets, http, js_unpacker, retirejs, robots, trufflehog, wayback | | kitchen-sink | | Everything everywhere all at once | 7 | baddns, baddns_direct, baddns_zone, http, hunt, reflected_parameters, webbrute | -| lightfuzz | web | Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. Skips confirmed WAFs. | 6 | badsecrets, http, hunt, lightfuzz, nowafpls, reflected_parameters | -| lightfuzz-heavy | web | Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. Still skips confirmed WAFs. | 8 | badsecrets, http, hunt, lightfuzz, nowafpls, reflected_parameters, robots, wayback | +| lightfuzz | web | Default fuzzing: all 9 submodules (cmdi, crypto, path, serial, sqli, ssti, xss, esi, ssrf) plus companion modules (badsecrets, hunt, reflected_parameters). POST fuzzing disabled but try_post_as_get enabled, so POST params are retested as GET. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it. | 6 | badsecrets, http, hunt, lightfuzz, nowafpls, reflected_parameters | +| lightfuzz-heavy | web | Aggressive fuzzing: everything in lightfuzz, plus paramminer brute-force parameter discovery (headers, GET params, cookies), POST request fuzzing enabled, try_get_as_post enabled (GET params retested as POST), and robots.txt parsing. On WAF-tagged hosts, only request-body parameters are fuzzed, and only if the WAF doesn't block them or nowafpls body padding gets them past it. | 8 | badsecrets, http, hunt, lightfuzz, nowafpls, reflected_parameters, robots, wayback | | lightfuzz-light | web | Minimal fuzzing: only path traversal, SQLi, and XSS submodules. No POST requests. No companion modules. Safest option for running alongside larger scans with minimal overhead. | 2 | http, lightfuzz | | lightfuzz-max | web | Maximum fuzzing: everything in lightfuzz-heavy, plus the heavy paramminer variant (1-3 letter brute-force on GET params, case mutation on case-sensitive backends, recycle_words on all paramminer modules), WAF targets are no longer skipped, each unique parameter-value pair is fuzzed individually (no collapsing), common headers like X-Forwarded-For are fuzzed even if not observed, and potential parameters are speculated from JSON/XML response bodies. Significantly increases scan time. | 8 | badsecrets, http, hunt, lightfuzz, nowafpls, reflected_parameters, robots, wayback | | lightfuzz-xss | web | XSS-only: enables only the xss submodule with paramminer_getparams and reflected_parameters. POST fuzzing enabled and try_get_as_post retests every discovered GET parameter as POST, so the nowafpls body-padding bypass gets a chance on WAF-tagged targets (a padded-POST XSS is triggerable from an attacker-controlled form). | 5 | http, lightfuzz, nowafpls, paramminer_getparams, reflected_parameters | diff --git a/docs/scanning/tips_and_tricks.md b/docs/scanning/tips_and_tricks.md index 747c5e5203..4c506c0679 100644 --- a/docs/scanning/tips_and_tricks.md +++ b/docs/scanning/tips_and_tricks.md @@ -59,11 +59,17 @@ For OSINT, it's critical that every resolver is **unfiltered**. Specialized reso --8<-- "docs/data/resolv-sample.conf" ``` -Copy this to `/etc/resolv.conf` (or append the `nameserver` lines to your existing config). With all 11 resolvers at the default 10 threads each, blastdns will run 110 workers in parallel instead of the typical 10-30 you get from a default OS config. +Copy this to `/etc/resolv.conf` (or append the `nameserver` lines to your existing config). At the default 50 queries per resolver, five or six resolvers already reach the 256 total, so if you add more, raise `dns.max_concurrency` to match. !!! tip If your system uses `systemd-resolved` or `resolvconf`, you may need to configure the upstream forwarders there instead of editing `/etc/resolv.conf` directly. +You can also skip `/etc/resolv.conf` and give BBOT its resolvers directly. `--custom-nameservers` (or `dns.nameservers` in your config) replaces the system resolvers for that scan: + +```bash +bbot -t evilcorp.com -f subdomain-enum --custom-nameservers 8.8.8.8 1.1.1.1 9.9.9.10 64.6.64.6 74.82.42.42 +``` + ### Web Spider The web spider is great for finding juicy data like subdomains, email addresses, and javascript secrets buried in webpages. However since it can lengthen the duration of a scan, it's disabled by default. To enable it, use one of the built-in spider presets: diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index 8d6546ab70..c30f76181d 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -4,7 +4,7 @@ - `Fatal error from pip prevented installation.` - `ERROR: No matching distribution found for bbot` -If you get errors like the above, it's probably because your Python version is too old. BBOT requires Python 3.10+. +If you get errors like the above, it's probably because your Python version is too old. BBOT requires Python 3.11+. ```bash # install a newer version of python