Skip to content

feat: sweep the full dns picture to find who to report a domain to - #88

Merged
jaspermayone merged 1 commit into
jaspermayone/54-digitalocean-xarf-reportsfrom
jaspermayone/54-full-dns-picture
Sep 7, 2026
Merged

jaspermayone merged 1 commit into
jaspermayone/54-digitalocean-xarf-reportsfrom
jaspermayone/54-full-dns-picture

Conversation

@jaspermayone

@jaspermayone jaspermayone commented Sep 7, 2026

Copy link
Copy Markdown
Member

Stacked on #84. Follow-up to the DigitalOcean work: answers "where do I report this" from the whole zone rather than from two record types.

The problem

#84 resolves A and AAAA records, which says which addresses serve a domain. It does not say who can take the page down. A phishing site on platform hosting is reached by a CNAME, and its address is a shared anycast one belonging to a CDN rather than to the platform serving the page. IP matching then finds the wrong party, or nobody.

The sweep

Report::DnsSweepService resolves A, AAAA, CNAME, NS, MX, TXT, SOA and CAA, plus a reverse lookup on each address, and stores the lot in dns_records.

Left out on purpose: DNSKEY and DS have no typed class in Resolv and say nothing about who to report to. SRV is only meaningful under a _service._proto label, not on the domain itself.

Matching, strongest signal first

graph TD
    A[phishing domain] --> B["1. CNAME + reverse lookup<br/>names the platform serving the page"]
    B -->|no match| C["2. A + AAAA against published CIDR ranges<br/>names the network that owns the address"]
    C -->|no match| D["3. NS<br/>names only the DNS operator"]
    B -->|match| E[hosting contact]
    C -->|match| E
    D -->|match| E
    F["MX → mail_hosts<br/>never decides who hosts the page"] -.->|surfaced for a human| G[case report]
Loading

The ordering is not cosmetic. Sweeping github.com live shows it delegates to Route 53 but is served by GitHub:

NS   ns-1707.awsdns-21.co.uk, dns1.p08.nsone.net, ...
PTR  140.82.114.3 -> lb-140-82-114-3-iad.github.com

Matching nameservers first would have reported GitHub's content to Amazon. There is a test pinning that case.

MX is excluded from matching entirely — it names who carries the mail, not who serves the page. mail_hosts surfaces it so a human can route the mail side by hand.

hostname_patterns

nameserver_patterns becomes hostname_patterns: the same globs identify a provider in a CNAME target or a reverse lookup just as well as in an NS record. A bare pattern also covers its subdomains, so digitalocean.com matches droplet.digitalocean.com without every entry needing a glob.

The old column is left in place, unread, rather than renamed. strong_migrations blocks the rename and is right to: the running release still reads the old name during a rolling deploy. This migration adds the new column and copies the values across; a later migration can drop nameserver_patterns once this release is out.

Thirteen existing contacts get patterns for the hostnames that identify them (vercel-dns.com, cloudfront.net, *awsdns-*, domaincontrol.com, wixdns.net and so on). No new contacts were invented — a wrong abuse address sends reports nowhere, so that list stays curated by hand.

The case report

A case that needs manual review now carries the CNAME, the reverse lookups, the mail hosts, TXT (SPF names the sending provider), CAA and SOA, so whoever picks it up has the evidence to route it.

Verification

465 tests pass; RuboCop, Brakeman, bundler-audit and herb are clean. The sweep was run against live domains to confirm the output shape and to find the nameserver ordering bug above.

https://claude.ai/code/session_01UJuHu239wz4x62pLNt651V

@coderabbitai

coderabbitai Bot commented Sep 7, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 22 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: f9aaa410-e944-453e-864a-18abf9c7a9e7

📥 Commits

Reviewing files that changed from the base of the PR and between c655caa and 4fa84eb.

📒 Files selected for processing (13)
  • app/controllers/admin/report/abuse_contacts_controller.rb
  • app/models/report/abuse_contact.rb
  • app/models/report/domain_lookup.rb
  • app/services/report/abuse_contact_importer.rb
  • app/services/report/dns_sweep_service.rb
  • app/services/report/domain_lookup_service.rb
  • app/views/reports/case_report.html.erb
  • db/migrate/20260907120003_add_dns_records_to_report_domain_lookups.rb
  • db/migrate/20260907120004_add_hostname_patterns_to_report_abuse_contacts.rb
  • db/schema.rb
  • db/seeds/abuse_contacts.rb
  • test/models/report/domain_lookup_test.rb
  • test/services/report/dns_sweep_service_test.rb

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@jaspermayone jaspermayone added the migration Includes database migrations. label Sep 7, 2026
@jaspermayone jaspermayone changed the title feat: sweep the full DNS picture to find who to report a domain to feat: sweep the full dns picture to find who to report a domain to Sep 7, 2026
Only A and AAAA records were resolved, which answers which addresses serve
a domain but not who can take the page down. A phishing site on platform
hosting is reached by a CNAME, and its address is a shared anycast one
belonging to a CDN rather than to the platform serving the page.

Report::DnsSweepService resolves A, AAAA, CNAME, NS, MX, TXT, SOA and CAA,
plus a reverse lookup on each address, and stores the lot in dns_records.
DNSKEY and DS are left out: Resolv has no typed class for them and they say
nothing about who to report to. SRV is left out because it is only
meaningful under a _service._proto label, not on the domain itself.

Hosting is now matched strongest signal first. CNAME targets and reverse
lookups name the platform or machine serving the page, so they go first;
the addresses come next; nameservers come last, because they frequently
name a different company. github.com delegates to Route 53 but is served by
GitHub, so matching nameservers first would have reported it to Amazon. MX
is deliberately excluded from matching and surfaced as mail_hosts instead,
for a human to route the mail side by hand.

nameserver_patterns becomes hostname_patterns, since the same globs identify
a provider in a CNAME target or reverse lookup just as well as in an NS
record. The old column is left in place, unread, rather than renamed:
renaming breaks a rolling deploy, and strong_migrations is right to say so.
A later migration can drop it once this release is out.

The case report renders the whole picture, so a case that needs manual
review carries the evidence to route it.

Claude-Session: https://claude.ai/code/session_01UJuHu239wz4x62pLNt651V
@jaspermayone
jaspermayone force-pushed the jaspermayone/54-full-dns-picture branch from 2bf362a to 4fa84eb Compare September 7, 2026 18:14
@jaspermayone
jaspermayone merged commit cb92cc1 into main Sep 7, 2026
4 checks passed
@jaspermayone
jaspermayone deleted the jaspermayone/54-full-dns-picture branch September 7, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

migration Includes database migrations.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant