Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

Open Source Proxy Benchmarking Tool

Open source proxy benchmarking tool that anyone can run to get accurate, transparent performance results. It is a Scrapy spider that crawls Similarweb Top 50 websites with up to 1,000 URLs per website (maximum 50,000 URLs) and reports the results per proxy provider.

QuickStart

  1. Install Scrapy and scrapy-zyte-api
python3 -m pip install --upgrade pip
python3 -m pip install scrapy
python3 -m pip install scrapy-zyte-api
  1. Configure the proxybench/settings.py file with the credentials you have and comment out the rest
PROXY_PROVIDERS = {
    # https://brightdata.com/products/web-unlocker
    'brightdata_web_unlocker': 'http://lum-customer-<id>-zone-unblocker:<password>@zproxy.lum-superproxy.io:22225',
    # https://oxylabs.io/products/scraper-api/web
    'oxylabs_web_scraper_api': 'http://<user>:<password>@realtime.oxylabs.io:60000',
    # https://www.zyte.com/smart-browser-api-anti-fingerprinting/
    'zyte_smart_browser': '<zyte_data_api_key>',
    # https://www.zyte.com/smart-proxy-manager/
    'zyte_smart_proxy_manager': 'http://<smart_proxy_manager_api_key>:@proxy.zyte.com:8011',
}
  1. Run
scrapy crawl proxybench
  1. Check the results in the log.txt file
 'proxybench/brightdata_web_unlocker/successful': 33753,
 'proxybench/brightdata_web_unlocker/total': 38357,
 'proxybench/oxylabs_web_scraper_api/successful': 32471,
 'proxybench/oxylabs_web_scraper_api/total': 38234,
 'proxybench/zyte_smart_browser/successful': 38397,
 'proxybench/zyte_smart_browser/total': 40462,
 'proxybench/zyte_smart_proxy_manager/successful': 36841,
 'proxybench/zyte_smart_proxy_manager/total': 40239,

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages