Go to file
Alexandre Dulaunoy 0450e39bc2
Set theme jekyll-theme-cayman
2022-07-15 12:05:41 +02:00
bin new: [pdns-import-cof] New importer for the Passive DNS backend. 2022-07-15 11:45:41 +02:00
etc new: [db] kvrocks support added instead of redis back-end 2022-07-15 09:49:41 +02:00
LICENSE chg: [doc] Basic README added 2019-02-03 17:30:04 +01:00
README.md chg: [doc] updated 2022-07-15 12:04:42 +02:00
_config.yml Set theme jekyll-theme-cayman 2022-07-15 12:05:41 +02:00
install_server.sh create db directory and correct path 2020-02-12 10:35:39 +01:00
install_server_kvrocks.sh new: [db] kvrocks support added instead of redis back-end 2022-07-15 09:49:41 +02:00
launch_server.sh new: [db] kvrocks support added instead of redis back-end 2022-07-15 09:49:41 +02:00
requirements new: [pdns-import-cof] New importer for the Passive DNS backend. 2022-07-15 11:45:41 +02:00

README.md

analyzer-d4-passivedns

analyzer-d4-passivedns is an analyzer for a D4 network sensor including a complete Passive DNS server. The analyser can process data produced by D4 sensors (in passivedns CSV format (more to come)) or independently from D4 using COF websocket streams.

The package include a Passive DNS server which can be queried later to search for the Passive DNS records.

Features

Overview

Requirements

  • Python 3
  • Redis >5.0 or kvrocks
  • Tornado
  • iptools

Install

Redis

./install_server.sh

All the Python 3 code will be installed in a virtualenv (PDNSENV).

Kvrocks

./install_server_kvrocks.sh

All the Python 3 code will be installed in a virtualenv (PDNSENV).

Running

Start the redis server or kvrocks server

Don't forget to set the DB directory in the redis.conf configuration. By default, the redis for Passive DNS is running on TCP port 6400

./redis/src/redis-server ./etc/redis.conf

or

./kvrocks/src/kvrocks -c ./etc/kvrocks.conf

Start the Passive DNS COF server

. ./PDNSENV/bin/activate
cd ./bin/
python3 ./pdns-cof-server.py

Feeding the Passive DNS server

You have two ways to feed the Passive DNS server. You can combine multiple streams. A sample public COF stream is available from CIRCL with the newly seen IPv6 addresses and DNS records.

(via COF websocket stream) start the importer

python3 pdns-import-cof.py --websocket ws://crh.circl.lu:8888

(via D4) Configure and start the D4 analyzer

cd ./etc
cp analyzer.conf.sample analyzer.conf

Edit the analyzer.conf to match the UUID of the analyzer queue from your D4 server.

[global]
my-uuid = 6072e072-bfaa-4395-9bb1-cdb3b470d715
d4-server = 127.0.0.1:6380
# INFO|DEBUG
logging-level = INFO

then you can start the analyzer which will fetch the data from the analyzer, parse it and populate the Passive DNS database.

. ./PDNSENV/bin/activate/
cd ./bin/
python3 pdns-ingestion.py

Usage

Querying the server

adulau@kolmogorov ~/git/analyzer-d4-passivedns (master)$ curl -s http://127.0.0.1:8400/query/xn--ihuvudetpevap-xfb.se | jq .
{
  "time_first": 1657878272,
  "time_last": 1657878272,
  "count": 1,
  "rrtype": "AAAA",
  "rrname": "xn--ihuvudetpevap-xfb.se",
  "rdata": "2a02:250:0:8::53",
  "origin": "origin not configured"
}
curl -s http://127.0.0.1:8400/query/2a02:250:0:8::53 
{"time_first": 1657878141, "time_last": 1657878141, "count": 1, "rrtype": "AAAA", "rrname": "media.vastporten.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878929, "time_last": 1657878929, "count": 1, "rrtype": "AAAA", "rrname": "www.folkinitiativetarjeplog.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878272, "time_last": 1657878272, "count": 1, "rrtype": "AAAA", "rrname": "xn--ihuvudetpevap-xfb.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878189, "time_last": 1657878189, "count": 1, "rrtype": "AAAA", "rrname": "media.primesteps.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878986, "time_last": 1657878986, "count": 1, "rrtype": "AAAA", "rrname": "media.skellefteaadventurepark.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657874940, "time_last": 1657874940, "count": 1, "rrtype": "AAAA", "rrname": "galleri.torsaspaintball.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657874205, "time_last": 1657874205, "count": 1, "rrtype": "AAAA", "rrname": "www.media1.harlaut.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878165, "time_last": 1657878165, "count": 1, "rrtype": "AAAA", "rrname": "www.sd-nekretnine.rs", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878678, "time_last": 1657878678, "count": 1, "rrtype": "AAAA", "rrname": "www.www2.resultat-balans.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657874288, "time_last": 1657874288, "count": 1, "rrtype": "AAAA", "rrname": "www.assistanshemtjanst.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878943, "time_last": 1657878943, "count": 1, "rrtype": "AAAA", "rrname": "kafekultur.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878141, "time_last": 1657878141, "count": 1, "rrtype": "AAAA", "rrname": "media1.rlab.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878997, "time_last": 1657878997, "count": 1, "rrtype": "AAAA", "rrname": "serbiagreenbuildingexpo.com", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657879064, "time_last": 1657879064, "count": 1, "rrtype": "AAAA", "rrname": "www.framtro.nu", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657874285, "time_last": 1657874285, "count": 1, "rrtype": "AAAA", "rrname": "www.twotheartist.com", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}
{"time_first": 1657878774, "time_last": 1657878774, "count": 1, "rrtype": "AAAA", "rrname": "media.narkesten.se", "rdata": "2a02:250:0:8::53", "origin": "origin not configured"}

License

The software is free software/open source released under the GNU Affero General Public License version 3.