Skip to content

bralbral/network_info

BranchesTags
 
 

Repository files navigation

Network Info Parser

This script parses the ARIN/APNIC/LACNIC/AfriNIC/RIPE databases into a local PostgreSQL database. After the parsing is finished you can get the infos for any IPv4 or IPv6 by querying the database.

This project was used in analysing some data dumps and cross referencing the IPs with the networks. It can also be used to easily search for netranges assigned to a company in interest.

I recommend using the docker setup because it removes the hassle of installing everything manually.

Hint: The Database can grow fast so be sure to have enough space. On docker my postgres database uses 4.066GB of space.

Requirements

  • Python3 >= 3.3
  • postgresql
  • python3-netaddr
  • python3-psycopg
  • python3-sqlalchemy

Docker

You can simply pull the image from Docker Hub and connect it to a local database via

docker pull firefart/network_info
docker run --rm firefart/network_info -c postgresql://user:pass@db:5432/network_info

Or you can connect the docker container to another database container.

docker run --name network_info_db -e POSTGRES_DB=network_info -e POSTGRES_USER=network_info -e POSTGRES_PASSWORD=network_info -d postgres:9-alpine
docker run --rm --link network_info_db:postgres firefart/network_info -c postgresql://user:pass@db:5432/network_info

If you have checked out the GIT repo you can run the script via docker compose. I included some binstubs so you don't have to deal with all the docker commands.

If you run

./bin/network_info

the image will be built, a postgres database is connected, the files are downloaded and the parsing begins. The database stays up after the run (you can see it via docker ps) so you can connect it to your script.

For a one shot query you can run

./bin/query IPv4

or

./bin/query IPv6

Or for a psql prompt

./bin/psql

Manual Installation

Installation of needed packages (Example on Ubuntu 16.04):

apt install postgresql python3 python3-netaddr python3-psycopg2 python3-sqlalchemy

or -

apt install postgresql python3 python-pip
pip install -r requirements.txt

Create PostgreSQL database (Use "network_info" as password):

sudo -u postgres createuser --pwprompt --createdb network_info
sudo -u postgres createdb --owner=network_info network_info

Prior to starting this script you need to download the database dumps by executing:

./download_dumps.sh

After importing you can lookup an IP address like:

SELECT block.inetnum, block.netname, block.country, block.description, block.maintained_by, block.created, block.last_modified, block.source FROM block WHERE block.inetnum >> '2001:db8::1' ORDER BY block.inetnum DESC;
SELECT block.inetnum, block.netname, block.country, block.description, block.maintained_by, block.created, block.last_modified, block.source FROM block WHERE block.inetnum >> '8.8.8.8' ORDER BY block.inetnum DESC;

or -

./query_db.sh 192.0.2.1

Sample run (docker compose)

$ ./bin/network_info
Creating network "ripe_default" with the default driver
Creating volume "ripe_pg_data" with local driver
Creating ripe_db_1
Downloading afrinic.db.gz...
Connecting to ftp.afrinic.net (196.216.2.24:21)
afrinic.db.gz        100% |****************************************************************************************************************************|  5419k  0:00:00 ETA
Downloading apnic.db.inetnum.gz...
Connecting to ftp.apnic.net (202.12.29.205:21)
apnic.db.inetnum.gz  100% |****************************************************************************************************************************| 37065k  0:00:00 ETA
Downloading apnic.db.inet6num.gz...
Connecting to ftp.apnic.net (202.12.29.205:21)
apnic.db.inet6num.gz 100% |****************************************************************************************************************************|  1113k  0:00:00 ETA
Downloading arin.db...
Connecting to ftp.arin.net (199.71.0.151:21)
arin.db              100% |****************************************************************************************************************************| 12314k  0:00:00 ETA
Downloading delegated-lacnic-extended-latest...
Connecting to ftp.lacnic.net (200.3.14.11:21)
delegated-lacnic-ext 100% |****************************************************************************************************************************|  2161k  0:00:00 ETA
Downloading ripe.db.inetnum.gz...
Connecting to ftp.ripe.net (193.0.6.140:21)
ripe.db.inetnum.gz   100% |****************************************************************************************************************************|   228M  0:00:00 ETA
Downloading ripe.db.inet6num.gz...
Connecting to ftp.ripe.net (193.0.6.140:21)
ripe.db.inet6num.gz  100% |****************************************************************************************************************************| 24589k  0:00:00 ETA
2020-06-23 20:21:11,836 - create_db - INFO     - MainProcess - afrinic.db.gz - parsing database file: ./databases/afrinic.db.gz
2020-06-23 20:21:20,688 - create_db - INFO     - MainProcess - afrinic.db.gz - Got 171337 blocks
2020-06-23 20:21:20,688 - create_db - INFO     - MainProcess - afrinic.db.gz - database parsing finished: 8.85 seconds
2020-06-23 20:21:20,688 - create_db - INFO     - MainProcess - afrinic.db.gz - parsing blocks
2020-06-23 20:21:40,904 - create_db - INFO     - MainProcess - afrinic.db.gz - block parsing finished: 20.22 seconds
2020-06-23 20:21:40,906 - create_db - INFO     - MainProcess - apnic.db.inet6num.gz - parsing database file: ./databases/apnic.db.inet6num.gz
2020-06-23 20:21:43,364 - create_db - INFO     - MainProcess - apnic.db.inet6num.gz - Got 73190 blocks
2020-06-23 20:21:43,376 - create_db - INFO     - MainProcess - apnic.db.inet6num.gz - database parsing finished: 2.47 seconds
2020-06-23 20:21:43,376 - create_db - INFO     - MainProcess - apnic.db.inet6num.gz - parsing blocks
2020-06-23 20:21:52,059 - create_db - INFO     - MainProcess - apnic.db.inet6num.gz - block parsing finished: 8.68 seconds
2020-06-23 20:21:52,061 - create_db - INFO     - MainProcess - apnic.db.inetnum.gz - parsing database file: ./databases/apnic.db.inetnum.gz
2020-06-23 20:22:21,802 - create_db - INFO     - MainProcess - apnic.db.inetnum.gz - Got 1079011 blocks
2020-06-23 20:22:21,809 - create_db - INFO     - MainProcess - apnic.db.inetnum.gz - database parsing finished: 29.75 seconds
2020-06-23 20:22:21,809 - create_db - INFO     - MainProcess - apnic.db.inetnum.gz - parsing blocks
2020-06-23 20:24:43,940 - create_db - INFO     - MainProcess - apnic.db.inetnum.gz - block parsing finished: 142.13 seconds
2020-06-23 20:24:43,942 - create_db - INFO     - MainProcess - arin.db - parsing database file: ./databases/arin.db
2020-06-23 20:24:43,946 - create_db - INFO     - MainProcess - arin.db - Got 0 blocks
2020-06-23 20:24:44,008 - create_db - INFO     - MainProcess - arin.db - database parsing finished: 0.07 seconds
2020-06-23 20:24:44,008 - create_db - INFO     - MainProcess - arin.db - parsing blocks
2020-06-23 20:24:44,030 - create_db - INFO     - MainProcess - arin.db - block parsing finished: 0.02 seconds
2020-06-23 20:24:44,032 - create_db - INFO     - MainProcess - delegated-lacnic-extended-latest - parsing database file: ./databases/delegated-lacnic-extended-latest
2020-06-23 20:24:44,034 - create_db - WARNING  - MainProcess - delegated-lacnic-extended-latest - line does not start with lacnic: b'2.3|lacnic|20200621|73188|19870101|20200619|-0300'
2020-06-23 20:24:44,035 - create_db - WARNING  - MainProcess - delegated-lacnic-extended-latest - Invalid line: b'lacnic|*|ipv4|*|18136|summary'
2020-06-23 20:24:44,035 - create_db - WARNING  - MainProcess - delegated-lacnic-extended-latest - Invalid line: b'lacnic|*|ipv6|*|43276|summary'
2020-06-23 20:24:44,035 - create_db - WARNING  - MainProcess - delegated-lacnic-extended-latest - Invalid line: b'lacnic|*|asn|*|11776|summary'
2020-06-23 20:24:44,489 - create_db - INFO     - MainProcess - delegated-lacnic-extended-latest - Got 61412 blocks
2020-06-23 20:24:44,489 - create_db - INFO     - MainProcess - delegated-lacnic-extended-latest - database parsing finished: 0.46 seconds
2020-06-23 20:24:44,489 - create_db - INFO     - MainProcess - delegated-lacnic-extended-latest - parsing blocks
2020-06-23 20:24:51,207 - create_db - INFO     - MainProcess - delegated-lacnic-extended-latest - block parsing finished: 6.72 seconds
2020-06-23 20:24:51,209 - create_db - INFO     - MainProcess - ripe.db.inetnum.gz - parsing database file: ./databases/ripe.db.inetnum.gz
2020-06-23 20:27:46,474 - create_db - INFO     - MainProcess - ripe.db.inetnum.gz - Got 4191047 blocks
2020-06-23 20:27:46,476 - create_db - INFO     - MainProcess - ripe.db.inetnum.gz - database parsing finished: 175.27 seconds
2020-06-23 20:27:46,476 - create_db - INFO     - MainProcess - ripe.db.inetnum.gz - parsing blocks
2020-06-23 20:37:17,835 - create_db - INFO     - MainProcess - ripe.db.inetnum.gz - block parsing finished: 571.36 seconds
2020-06-23 20:37:17,837 - create_db - INFO     - MainProcess - ripe.db.inet6num.gz - parsing database file: ./databases/ripe.db.inet6num.gz
2020-06-23 20:37:57,369 - create_db - INFO     - MainProcess - ripe.db.inet6num.gz - Got 1081478 blocks
2020-06-23 20:37:57,706 - create_db - INFO     - MainProcess - ripe.db.inet6num.gz - database parsing finished: 39.87 seconds
2020-06-23 20:37:57,706 - create_db - INFO     - MainProcess - ripe.db.inet6num.gz - parsing blocks
2020-06-23 20:40:21,534 - create_db - INFO     - MainProcess - ripe.db.inet6num.gz - block parsing finished: 143.83 seconds
2020-06-23 20:40:21,534 - create_db - INFO     - MainProcess - empty - script finished: 1149.83 seconds

$ ./bin/query 8.8.8.8
SELECT block.inetnum, block.netname, block.country, block.description, block.maintained_by, block.created, block.last_modified, block.source FROM block WHERE block.inetnum >> '8.8.8.8' ORDER BY block.inetnum DESC;
-[ RECORD 1 ]-+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
inetnum       | 8.0.0.0/8
netname       | IANA-NETBLOCK-8
country       | AU
description   | This network range is not allocated to APNIC. If your whois search has returned this message, then you have searched the APNIC whois database for an address that is allocated by another Regional Internet Registry (RIR). Please search the other RIRs at whois.arin.net or whois.ripe.net for more information about that range.
maintained_by | MAINT-APNIC-AP
created       |
last_modified | 2008-09-04 06:51:28
source        | apnic
-[ RECORD 2 ]-+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
inetnum       | 8.0.0.0/6
netname       | NON-RIPE-NCC-MANAGED-ADDRESS-BLOCK
country       | EU # Country is really world wide
description   | IPv4 address block not managed by the RIPE NCC
maintained_by | RIPE-NCC-HM-MNT
created       | 2019-01-07 10:49:33
last_modified | 2019-01-07 10:49:33
source        | ripe
-[ RECORD 3 ]-+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
inetnum       | 0.0.0.0/0
netname       | IANA-BLK
country       | EU # Country is really world wide
description   | The whole IPv4 address space
maintained_by | AFRINIC-HM-MNT
created       |
last_modified |
source        | afrinic
-[ RECORD 4 ]-+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
inetnum       | 0.0.0.0/0
netname       | IANA-BLK
country       | EU # Country field is actually all countries in the world and not just EU countries
description   | The whole IPv4 address space
maintained_by | RIPE-NCC-HM-MNT
created       | 2002-06-25 14:19:09
last_modified | 2018-11-23 10:30:34
source        | ripe
-[ RECORD 5 ]-+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
inetnum       | 0.0.0.0/0
netname       | IANA-BLOCK
country       | AU
description   | General placeholder reference for all IPv4 addresses
maintained_by | MAINT-APNIC-AP
created       |
last_modified | 2008-09-04 06:51:49
source        | apnic

Export block table

If you need to export data from PG to another source (Clickhouse, Elasticsearch, etc.) you can use:

./bin/export_to_gzip

A compressed CSV file will be created with the current date in the filename.

The script supports various options:

./bin/export_to_gzip [OPTIONS]

Options:
  -d, --delimiter DELIMITER   Field delimiter (default: ',')
  -t, --table TABLE           Table name to export (default: block)
  -o, --output PREFIX         Output file prefix (default: block_dump)
  -h, --host HOST             Database host (default: db)
  -u, --user USER             Database user (default: network_info)
  -n, --database NAME         Database name (default: network_info)
  -p, --password PASSWORD     Database password (default: network_info)
  --help                       Show this help message

Examples:

# Default export (comma delimiter)
./bin/export_to_gzip

# Export with semicolon delimiter
./bin/export_to_gzip -d ';'

# Export as TSV (tab-separated)
./bin/export_to_gzip -d $'\t'

# Export custom table with pipe delimiter
./bin/export_to_gzip -t users -d '|' -o users_dump

Export to MMDB format

You can also export the database to MaxMind DB (MMDB) format for use with GeoIP2-compatible tools:

./bin/export_to_mmdb

An MMDB file will be created with the current date in the filename.

⚠️ HIGH MEMORY USAGE WARNING The MMDB export process loads all data into memory before writing the final file. For large databases (millions of records), this can require several GB of RAM. Ensure you have sufficient memory available before running the export.

The script supports various options:

./bin/export_to_mmdb [OPTIONS]

Options:
  -t, --table TABLE           Table name to export (default: block)
  -o, --output PREFIX         Output file prefix (default: block_dump)
  -h, --host HOST             Database host (default: db)
  -u, --user USER             Database user (default: network_info)
  -n, --database NAME         Database name (default: network_info)
  -p, --password PASSWORD     Database password (default: network_info)
  --help                       Show this help message

Examples:

# Default export
./bin/export_to_mmdb

# Export custom table
./bin/export_to_mmdb -t users -o users_dump

# Export with custom database connection
./bin/export_to_mmdb -h localhost -u myuser -n mydb -p mypass

Using the MMDB file

The MMDB file contains the following network information fields for each IP range:

  • netname - Network name identifier
  • country - Country code (e.g., US, RU, DE)
  • description - Network description
  • maintained_by - Maintainer information
  • created - Creation date (YYYY-MM-DD format)
  • last_modified - Last modification date (YYYY-MM-DD format)
  • source - Source RIR (ARIN, RIPE, APNIC, LACNIC, AfriNIC)
  • status - Network status

Once you have the MMDB file, you can use it with various tools:

Python examples

Simple lookup:

import maxminddb

# Open the MMDB file
with maxminddb.open_database('block_dump_2026-05-01.mmdb') as reader:
    # Lookup an IP address
    result = reader.get('8.8.8.8')
    if result:
        print(f"Network: {result['netname']}")
        print(f"Country: {result['country']}")
        print(f"Source: {result['source']}")

Command line example

# Install mmdblookup tool
sudo apt install mmdb-bin

# Lookup an IP address
mmdblookup --file block_dump_2026-05-01.mmdb --ip 8.8.8.8

About

Parse Network Info Databases (ARIN/APNIC/LACNIC/AfriNIC/RIPE)

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 60.1%
  • Shell 37.9%
  • Dockerfile 2.0%