Repository navigation
Conversation
ValidEmail2.disposable_emails loads the ~180,000 disposable domains into a Set: one String object per domain plus its hash entry, about 35 MB of memory once the first email is checked. The file is sorted (see pull_mailchecker_emails.rb), so lookups now go through ValidEmail2.disposable_domains, a SortedDomainList keeping the file as a single String (2.6 MB) searched by bisection. disposable_emails still returns the same Set, built on demand. Once it is loaded, disposable_domains returns it, so code adding domains to it or removing them keeps working.
Spone
force-pushed
the
sorted-disposable-domains
branch
from
October 12, 2026 14:37
2c6cef4 to
9faaf2c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ValidEmail2.disposable_emailsloads the ~180,000 disposable domains into aSet. Each domain becomes its ownStringplus a hash entry, so the first email checked grows the process by about 35 MB.Change
The domains file is sorted and has no duplicates:
pull_mailchecker_emails.rbwrites it with.uniq.sort. So it can be searched without building aSet:ValidEmail2::SortedDomainListkeeps the file as a single binaryString(about 2.6 MB) and answersinclude?by bisection over its lines. Each step reads one line around the middle offset, so there's no index to build.ValidEmail2.disposable_domainsis whatAddress#disposable_domain?and#disposable_mx_server?now use:SortedDomainList;disposable_emailshas been loaded, that sameSet.Code that adds domains to the
Setor removes them (as one of the specs does) keeps working unchanged.disposable_emailsstill returns the same mutableSet, built on demand. Nothing changes for code calling it, except that it now pays the 14 MB only when it does.Measured
Against
config/disposable_email_domains.txt(178,700 domains). Memory is the process's RSS growth after loading and one lookup, each case in a fresh process:Set)SortedDomainList)A lookup is slower than a
Set's but stays well under the existingperform_under(0.0001)benchmark (100 µs), and a few microseconds per validated email is negligible next to the memory saved in every process.Specs
spec/sorted_domain_list_spec.rb:spec/valid_email2_spec.rb:Set;Setis loaded, domains added to it count.spec/benchmark_spec.rb: a lookup benchmark without theSet.bundle exec rake: 145 examples, 0 failures. RuboCop reports the same 2 offenses as onmain(Metrics/ClassLengthonAddress,Metrics/BlockLengthon the main spec).