[liquidsec]

Release 3.1.0

These notes cover everything since 3.0.2. The full migration guide is in docs/migration/3.1_breaking_changes.md (#3594).

Breaking changes

• Python 3.10 dropped, 3.15 added (Drop Python 3.10, add Python 3.15 #3474). requires-python is now >=3.11,<3.16; CI tests 3.11 through 3.15. 3.10 reaches EOL this month.
• portfilter module removed (Filter CDN/WAF ports by default and remove the portfilter module #3534). CDN/WAF port filtering now happens in core and is on by default; opt out with --allow-cdn-ports.
• Custom module loading hardened (Harden preset and custom module loading #3593). config.module_dirs is deprecated and ignored; module_dirs is honored only as a top-level preset key. BBOT asks before loading each custom module folder (-y answers yes, -S no longer implies -y), refuses world-writable folders and files, and refuses custom modules that reuse an existing module name. Preset conditions are read-only, and version / home are validated.
• lightfuzz probes WAF-tagged hosts instead of skipping them (Add nowafpls Cloudflare WAF bypass detector #3336). modules.lightfuzz.avoid_wafs defaults to try_bypasses, which fuzzes a WAF-tagged host only if a nowafpls probe gets past the WAF, and the lightfuzz presets enable the nowafpls module. The top-level avoid_wafs is no longer read. For the 3.0 behavior, set modules.lightfuzz.avoid_wafs: always and pass -em nowafpls.
• dnsdumpster requires an API key (Convert dnsdumpster to the authenticated API #3458), waf_bypass targets scan targets only by default (waf_bypass: only attempt bypasses against scan targets by default #3460), CPE-derived technology names (Name CPE-derived technologies by product and retain the CPE #3535) and no affiliate tag on in-scope events (Don't tag in-scope events as affiliate #3524) can affect existing presets and output consumers.

Highlights

DNS (#3457)

• massdns removed. DNS brute-forcing now runs through blastdns 2.0, and the massdns dependency is gone entirely.
• Brute-force gets its own retry budget, a short per-query timeout, and persistent sockets (one per resolver rather than one per in-flight query).
• New --custom-nameservers flag (dns.nameservers), replacing the system resolvers for normal resolution. An invalid value aborts the scan during prep instead of producing a "completed" scan with zero results.
• Resolver diagnostics reported when brute-force delivery drops.
• New dns.brute_max_resolvers (default 600, 0 for the whole list) (Draw a random sample of brute-force resolvers once per scan #3612). Each scan draws that many brute-force resolvers at random, once. Dead ones are dropped at startup, and every brute-force run in the scan shares the rest, so pacing learned in one run carries into the next. Also raises the blastdns floor to 2.0.3, which paces attempts when per-attempt loss shows a shared bottleneck.
• Ordinary DNS concurrency raised and made explicit (Raise DNS concurrency defaults and size dnsresolve from them #3606): dns.threads 10 to 50, new dns.max_concurrency (256), and the dnsresolve worker count derived from the client's total slots. On a large wildcard subdomain-enum scan this halved wall time compared with 3.0.2, with the same subdomains found. Lower dns.threads for a fragile resolver. The subdomain-enum preset no longer sets dns.threads: 25, so it gets the new default too (3.1 documentation update #3613).
• Already-resolved events skip ahead in dnsresolve's queue (Let already-resolved events skip ahead in dnsresolve's queue #3533).
• Fix nested wildcard detection in is_wildcard (Fix nested wildcard detection in is_wildcard #3536).

baddns 2.5.0 (#3457)

• New DELEGATION submodule, enabled by default and in the baddns / baddns-heavy presets. Checks _acme-challenge, _dmarc and 15 common DKIM selectors for dangling delegations, with wildcard and generic-finding filtering to keep volume down.
• NS gained a lame-delegation finding (ceiling raised to HIGH/CONFIRMED).
• baddns's YARA-backed WordMatcher is passed through, matching all signatures in a single pass.
• blastdns capped at <2.1 to match baddns's own ceiling.

New modules

• favicon: technology detection from favicon hashes (Add favicon module for technology detection #3532).
• js_unpacker: detects and unpacks obfuscated JavaScript (Add js_unpacker module for detecting and unpacking obfuscated JS #3335).
• nowafpls: WAF bypass detection via body padding (Add nowafpls Cloudflare WAF bypass detector #3336). lightfuzz, ajaxpro and generic_ssrf pad their payloads on hosts where the bypass is confirmed.
• otilabs: subdomain enumeration (Add otilabs subdomain module #3496).
• scanmalware: subdomains from ScanMalware's public scan archive (Add scanmalware subdomain module #3528). Keyless and opt-in (-m scanmalware); not in subdomain-enum.

Modules

• telerik overhaul with native RAU and DialogHandler crypto (telerik: overhaul, native RAU + DialogHandler crypto #3333).
• nuclei: expose newer template-selection options (Expose nuclei's newer template-selection options #3479). Results are parented to the URL, or the host and port, that nuclei actually scanned, so findings on sibling subdomains or on several ports of one host are no longer merged or mislabeled (Fix nuclei result correlation and add an optional port to FINDING #3629).
• dnsdumpster converted to the authenticated API (Convert dnsdumpster to the authenticated API #3458).
• wayback verifies hosts are alive before emitting URLs (Verify hosts are alive before emitting wayback URLs #3359).
• waf_bypass only attempts bypasses against scan targets by default (waf_bypass: only attempt bypasses against scan targets by default #3460).
• gitdumper: origin check and depth limit on directory listing (Add origin check and depth limit to gitdumper directory listing #3466).
• crt: request an explicit / path, which crt.sh requires (crt: request an explicit / path (blasthttp sends an empty path, which crt.sh rejects with 400) #3428).
• SQL output modules request sqlalchemy's asyncio extra (fix: request sqlalchemy's asyncio extra in the SQL output modules #3505).
• lightfuzz: require positive confirmation for code-change findings (Require positive confirmation for lightfuzz code-change findings #3357), tighten serial error resolution (Tighten lightfuzz serial Error Resolution #3427), fix unbounded queue growth from parameter re-discovery (Fix unbounded lightfuzz queue growth from parameter re-discovery loop #3405), count baseline URLs against spider distance (Count lightfuzz baseline URLs against spider distance #3523), read the raw Cookie header in the sqli cookie test (Fix lightfuzz sqli cookie test to read the raw Cookie header #3498), correct the WAF-gating comments and test the GET-side skip (Clarify lightfuzz's WAF gating comments and pin the GET-side skip #3608).

excavate

• Long redirection response detection (Add long redirection response detection to excavate #3481).
• Relative URLs resolved against <base href> (Resolve excavated relative URLs against <base href> #3522).
• Hostnames are no longer extracted from partway through a hyphenated label (Stop extracting hostnames from partway through a hyphenated label #3590).

Core

• Overhauled api_request rate limiting, plus config deprecation support via Field(deprecated=True) (Overhaul api_request rate limiting and add config deprecation support #3461). API calls made through api_request now send a BBOT/<version> User-Agent instead of the scan's browser User-Agent.
• CPE-derived technologies are named by product, with the CPE retained (Name CPE-derived technologies by product and retain the CPE #3535).
• In-scope events are no longer tagged affiliate (Don't tag in-scope events as affiliate #3524).
• FINDING text output shows the host when it isn't otherwise present.
• FINDING has an optional port for findings that aren't on a URL, such as a TLS or TCP service. It's part of the event's identity and shown as host:port in text output, and legba and medusa findings now keep their port (Fix nuclei result correlation and add an optional port to FINDING #3629).
• HTTP_RESPONSE events carry url_extension (Set url_extension on HTTP_RESPONSE events #3406).
• ClosestHostEvent keeps its inherited port, and the port is restored from JSON (Keep inherited port on ClosestHostEvent and restore port from JSON #3462).
• download() respects cache_hrs in its freshness check (Respect cache_hrs in download() freshness check #3563).
• Reject symlinks in unarchive output after extraction (Reject symlinks in unarchive output after extraction #3560).
• Fix hostless sort sentinel type mismatch in Target.add() (fix TypeError when a hostless target is sorted against a /32 #3391).
• Version output includes the git commit hash for git checkouts (Include git commit hash in version output for git checkouts #3530).

Performance

• Selectively replace DeepDiff (Selectively Replace DeepDiff Due to Performance Issues #3340), then remove the dependency entirely (Remove deepdiff dependency #3604).
• Faster compute_simhash, with corrected executor placements (Speed up compute_simhash and correct executor placements #3331).
• response_to_json removed in favor of response_to_event_dict (Delete duplicate response_to_json, unify on response_to_event_dict #3473).

Dependencies

• blastdns 2.0.4 (Draw a random sample of brute-force resolvers once per scan #3612, Bump blastdns to 2.0.4 #3631), blasthttp 0.10.1, baddns 2.5.0, badsecrets 1.3.0 (Bump badsecrets to 1.3.0 #3526), cloudcheck 11.2.0 (Bump cloudcheck to 11.2.0 #3559), radixtarget 4.3.1 (Bump radixtarget to 4.3.0 #3558, Bump radixtarget from 4.3.0 to 4.3.1 #3617), pycryptodome 4.3.0 (Bump pycryptodome from 3.24.0 to 4.0.0 #3542, Bump pycryptodome from 4.1.0 to 4.2.0 #3597, Bump pycryptodome from 4.2.0 to 4.3.0 #3620).
• deepdiff removed, along with cachebox and orderly-set (Remove deepdiff dependency #3604). ansible-core resolves to 2.22 on Python 3.13+, which is what makes 3.15 installable.
• dnspython removed (deps: drop dnspython; blastdns handles all DNS #3610). Nothing imported it, and its old pin held every install below 2.9.0, the fix for GHSA-45c3-73f7-pv4m. The baddns modules still install it, uncapped, through baddns.
• pyjwt, lxml and socksio removed, and xmltojson replaced by xmltodict (Remove unnecessary dependencies #3627).
• trufflehog 3.97.0 to 3.99.8 (Update trufflehog to 3.99.8 #3605).
• Security dependency updates (Security dependency updates #3557).

Maintenance

• CI: test-matrix status check with every job named (Add a test-matrix status check, name every job #3356), faster test suite via teardown, polling and dep-install fixes (perf(ci): cut test suite wall time via teardown, poll, and dep-install fixes #3404), benchmark base-branch comparison restored and pytest-benchmark 5.3.1 for pytest 9.2 (Restore base-branch comparison in benchmark CI #3611), CLA workflow permissions (fix(ci): grant contents: read to the CLA workflow (dev) #3570), version updater workflow name (Clarify Version Updater workflow name #3415), pinned runner image (ci: pin the runner image to ubuntu-24.04 #3456, ci: pin the test-matrix runner and the CLA workflow ref #3483), weekly grouped dependabot updates with shared auto-merge for minor and patch (ci(dependabot): weekly grouped updates and shared auto-merge for minor and patch #3487).
• Docs: 3.1 breaking changes guide (Add 3.1 breaking changes doc #3594), docs updated for 3.1 (3.1 documentation update #3613), AGENTS.md as an index of the org standards (docs: make AGENTS.md an index of the org standards #3486), mkdocstrings 1.0.8 (Bump mkdocstrings to 1.0.8 #3585), module setup troubleshooting notes (Add module setup troubleshooting notes #3228), AGENTS.md options section and module examples refreshed (AGENTS.md: fix the options section and refresh stale module examples #3420), 3.0.0-3.0.2 release notes (Add 3.0.0-3.0.2 release notes to release history #3408).
• Project metadata lists Black Lantern Security as the author (Update project authorship to Black Lantern Security #3628).
• Merged stable into dev to resolve release conflicts (Merge stable into dev to resolve release conflicts #3592).
• Dependabot bumps across the uv and github-actions groups, and automated docs regeneration.

Refs #3555

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[liquidsec]

recheck

[jonaslejon]

I have read the CLA Document and I hereby sign the CLA

[jonaslejon]

Signed above, but the CLA bot can't save it. Run 42930418939 fails with Could not update the JSON file: Repository rule violations found. Changes must be made through a pull request. It looks like the ruleset on blacklanternsecurity/CLA blocks bls-cla-bot from pushing to main; the last saved signature is from 10-06. Adding the bot as a bypass actor should fix it.