Skip to content

fix(leads): analyze a clean hostname in the domain-intel endpoint - #102

Merged
debpalash merged 1 commit into
debpalash:mainfrom
rudycelekli:fix/domain-intel-normalize-input-20261014
Oct 23, 2026
Merged

debpalash merged 1 commit into
debpalash:mainfrom
rudycelekli:fix/domain-intel-normalize-input-20261014

Conversation

@rudycelekli

@rudycelekli rudycelekli commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

POST /api/leads/domain-intel passed the caller's text to RDAP, Cloudflare DoH and crt.sh almost as typed. Four concrete problems, each reproduced against the route with analyze_domain stubbed to record its argument:

  1. A non-string domain (42, ["a.com"]) raised AttributeError on .strip() and returned HTTP 500.
  2. domain.replace("www.", "") removes a www. anywhere in the name: awww.example.com was analyzed as aexample.com.
  3. The name was not lowercased, but get_crtsh_subdomains lowercases certificate names before endswith("." + domain), so Example.com always reported zero subdomains.
  4. A path, port, query or &exclude=none was interpolated into the rdap.org/domain/{domain} and crt.sh/?q=%25.{domain}&exclude=... URLs unvalidated.

Security-sensitive note (per CONTRIBUTING): the endpoint is editor-only and the fixed third-party hosts do not change; this validates input going into outbound request URLs and does not touch the SSRF guard.

Change

clean_domain() in domain_intel.py takes the hostname of a domain or URL (https://WWW.Example.com:8443/pricing?x=1 becomes example.com), lowercases it, drops a trailing dot and a leading www. only, converts IDNs to punycode, and accepts only a valid multi-label hostname. The route uses it and answers 400 for anything else; analyze_domain and the providers are unchanged.

Evidence

New tests/test_domain_intel_route.py (29 cases). On current main the route tests fail:

assert ['aexample.com'] == ['awww.example.com']
AttributeError: 'int' object has no attribute 'strip'     (apps/api/routers/leads.py:1402)
AttributeError: 'list' object has no attribute 'strip'
assert 200 == 400   for {"domain": "example.com&exclude=none"} and {"domain": "not a domain"}

Verification

  • After: uv run pytest tests/test_domain_intel_route.py: 29 passed.
  • Not verified: live RDAP, DoH and crt.sh calls (no network use in the tests); full backend suite, PostgreSQL suite and frontend not run.
  • Behavior change to be aware of: a bare IPv4 address still passes (it is a valid dotted name); single-label names such as localhost now get 400.

Commit is signed off per the DCO.

POST /api/leads/domain-intel passed the caller's text to RDAP, Cloudflare DoH
and crt.sh almost as typed:

- a non-string domain raised AttributeError (HTTP 500);
- replace("www.", "") removed a "www." anywhere in the name, so awww.example.com
  was analyzed as aexample.com;
- the name was not lowercased, but crt.sh results are, so Example.com always
  reported zero subdomains;
- a path, port, query, space or "&exclude=..." went straight into the RDAP and
  crt.sh request URLs.

Add clean_domain(): take the hostname of a domain or URL, lowercase it, drop a
trailing dot and a leading www. only, convert IDNs to punycode and accept only
a valid hostname. The route returns 400 for anything else.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
@debpalash
debpalash merged commit ff77c11 into debpalash:main Oct 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants