2019-05-10 15:55:59 +02:00
% Full instructions available at:
% https://github.com/elauksap/focus-beamertheme
\documentclass { beamer}
\usetheme [numbering=progressbar] { focus}
\usepackage { tikz}
\usetikzlibrary { positioning}
\usetikzlibrary { shapes,arrows}
\usepackage { transparent}
\usepackage { fancyvrb}
\usepackage { listings}
\usepackage { tabularx}
2019-05-16 16:01:17 +02:00
\usepackage { amsfonts}
\usepackage { csquotes}
2019-05-10 15:55:59 +02:00
\definecolor { main} { RGB} { 47, 161, 219}
\definecolor { background} { RGB} { 240, 247, 255}
\definecolor { textcolor} { RGB} { 85, 87, 83}
\title { D4 Project}
\subtitle { Open and collaborative network monitoring}
\author { Jean-Louis Huynen}
\titlegraphic { \includegraphics [scale=0.20] { ../../logos/d4-logo.pdf} }
\institute { Team CIRCL \\ \url { https://www.d4-project.org/} }
\date { 2019/05/21}
\begin { document}
\begin { frame}
\maketitle
\end { frame}
\begin { frame}
\frametitle { Problem statement}
\begin { itemize}
\item CSIRTs (or private organisations) build their { \bf own honeypot, honeynet or blackhole monitoring network}
\item Designing, managing and operating such infrastructure is a tedious and resource intensive task
\item { \bf Automatic sharing} between monitoring networks from different organisations is missing
\item Sensors and processing are often seen as blackbox or difficult to audit
\end { itemize}
\end { frame}
\begin { frame}
\frametitle { Objective}
\begin { itemize}
\item Based on our experience with
MISP\footnote { \url { https://github.com/MISP/MISP} } where sharing
played an important role, we transpose the model in D4 project
\item Keeping the protocol and code base { \bf simple and minimal}
\item Allowing every organisation to { \bf control and audit their own sensor network}
\item Extending D4 or { \bf encapsulating legacy monitoring protocols} must be as simple as possible
\item Ensuring that the sensor server has { \bf no control on the sensor} (unidirectional streaming)
\item Don't force users to use dedicated sensors and allow { \bf flexibility of sensor support} (software, hardware, virtual)
\end { itemize}
\end { frame}
2019-05-21 08:47:54 +02:00
\begin { frame}
\frametitle { D4 Overview}
\includegraphics [scale=0.38] { ../../diagram/d4-overview.png}
\end { frame}
2019-05-10 15:55:59 +02:00
\begin { frame}
\frametitle { (short) History}
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item D4 Project (co-funded under INEA CEF EU program) started - { \bf 1st November 2018}
\item D4 encapsulation protocol version 1 published - { \bf 1st December 2018}
\item v0.1 release of the D4 core\footnote { \url { https://www.github.com/D4-project/d4-core} } including a server and simple D4 C client - { \bf 21st January 2019}
2019-05-16 16:01:17 +02:00
\item First version of a golang D4
client\footnote { \url { https://www.github.com/D4-project/d4-goclient/} }
2019-05-20 16:22:11 +02:00
running on ARM, MIPS, PPC and x86 - { \bf 14th February 2019}
2019-05-10 15:55:59 +02:00
\end { itemize}
\end { frame}
\begin { frame}
\frametitle { (short) History}
\begin { center}
\begin { tabularx} { \linewidth } %
{ >{ \setlength \hsize { 0.6\hsize } \raggedright } X%
>{ \setlength \hsize { 0.4\hsize } \raggedright } X}
\hline
Release & Date \tabularnewline
\hline
analyzer-d4-passivedns-v0.1 & Apr. 5, 2019 \tabularnewline
analyzer-d4-passivessl-0.1 & Apr. 25, 2019 \tabularnewline
analyzer-d4-pibs-v0.1 & Apr. 8, 2019 \tabularnewline
BGP-Ranking-1.0 & Apr. 25, 2019 \tabularnewline
d4-core-v0.1 & Jan. 25, 2019 \tabularnewline
d4-core-v0.2 & Feb. 14, 2019 \tabularnewline
d4-core-v0.3 & Apr. 8, 2019 \tabularnewline
d4-goclient-v0.1 & Feb. 14, 2019 \tabularnewline
d4-goclient-v0.2 & Apr. 8, 2019 \tabularnewline
d4-server-packer-0.1 & Apr. 25, 2019 \tabularnewline
IPASN-History-1.0 & Apr. 25, 2019 \tabularnewline
sensor-d4-tls-fingerprinting-0.1 & Apr. 25, 2019 \tabularnewline
\hline
\end { tabularx}
\end { center}
2019-05-16 16:01:17 +02:00
see \url { https://github.com/D4-Project}
2019-05-10 15:55:59 +02:00
\end { frame}
\begin { frame}
2019-05-16 16:01:17 +02:00
\frametitle { Roadmap - output}
2019-05-10 15:55:59 +02:00
2019-05-16 16:01:17 +02:00
CIRCL will host a server instance for organisations willing to
contribute to a public dataset without running their own D4 server:
2019-05-10 15:55:59 +02:00
\begin { itemize}
2019-05-21 08:47:54 +02:00
\item [\checkmark ] Blackhole DDoS
2019-05-16 16:01:17 +02:00
\item [\checkmark ] Passive DNS
2019-05-21 08:47:54 +02:00
\item [\checkmark ] Passive SSL
2019-05-16 16:01:17 +02:00
\item BGP mapping
\item egress filtering mapping
2019-05-20 16:22:11 +02:00
\item Radio-Specturm monitoring: 802.11, BLE, etc.
2019-05-16 16:01:17 +02:00
\item ...
2019-05-10 15:55:59 +02:00
\end { itemize}
\end { frame}
\begin { frame}
\frametitle { D4 encapsulation protocol}
\includegraphics [scale=0.38] { ../../diagram/d4-protocol-encapsulation.png}
\end { frame}
\begin { frame}
\frametitle { D4 Header}
\begin { tabular} { |l|l|l|}
\hline
Name & bit size& Description\\
\hline
version & uint 8 & Version of the header \\
type & uint 8 & Data encapsulated type\\
uuid & uint 128 & Sensor UUID\\
timestamp & uint 64 & Encapsulation time\\
hmac & uint 256 & Authentication header (HMAC-SHA-256-128)\\
size & uint 32 & Payload size\\
\hline
\end { tabular}
\end { frame}
\begin { frame}
\frametitle { D4 Header}
\framesubtitle { Types}
\begin { tabular} { |l|l|}
\hline
Type & Description\\
\hline
0 & Reserved\\
1 & pcap (libpcap 2.4)\\
2 & meta header (JSON)\\
3 & generic log line\\
4 & dnscap output\\
5 & pcapng (diagnostic)\\
6 & generic NDJSON or JSON Lines\\
7 & generic YAF (Yet Another Flowmeter)\\
8 & passivedns CSV stream\\
254 & type defined by meta header (type 2)\\
\hline
\end { tabular}
\end { frame}
\begin { frame}
\frametitle { D4 meta header}
\framesubtitle { Meta types}
D4 header includes an easy way to { \bf extend the protocol} (via type 2) without altering the format. Within a D4 session, the initial D4 packet(s) type 2 defines
the custom headers and then the following packets with type 254 is the custom data encapsulated.
\small
\input { meta.tex}
\end { frame}
\begin { frame}
\frametitle { D4 server}
\begin { itemize}
\item D4 core server\footnote { \url { https://github.com/D4-project/d4-core} } is a complete server to handle clients (sensors) including the decapsulation of the D4 protocol, control of sensor registrations, management of decoding protocols and dispatching to adequate decoders/analysers.
\item D4 server is written in Python 3.6 and runs on standard GNU/Linux distribution.
\end { itemize}
\end { frame}
\begin { frame}
\frametitle { D4 server - management interface}
2019-05-20 16:22:11 +02:00
The D4 server provides a { \bf web interface} to manage D4 sensors, sessions and analyzer.
2019-05-10 15:55:59 +02:00
\begin { itemize}
\item Get Sensors status, errors and statistics
\item Get all connected sensors
\item Manage Sensors (stream size limit, secret key, ...)
\item Manage Accepted types
\item UUID/IP blocklist
\item Create Analyzer Queues
\end { itemize}
\end { frame}
\begin { frame}
\frametitle { D4 server - main interface}
2019-05-20 16:22:11 +02:00
\includegraphics [width=\textwidth] { ./d4-5.png}
2019-05-10 15:55:59 +02:00
\end { frame}
\begin { frame}
\frametitle { D4 server - server management}
2019-05-20 16:22:11 +02:00
\includegraphics [width=\textwidth] { ./d4-2.png}
2019-05-10 15:55:59 +02:00
\end { frame}
\begin { frame}
\frametitle { D4 server - server management}
2019-05-20 16:22:11 +02:00
\includegraphics [width=\textwidth] { ./d4-3.png}
2019-05-10 15:55:59 +02:00
\end { frame}
\begin { frame}
\frametitle { D4 server - sensor overview}
2019-05-20 16:22:11 +02:00
\includegraphics [width=\textwidth] { ./d4-1.png}
2019-05-10 15:55:59 +02:00
\end { frame}
\begin { frame}
\frametitle { D4 server - sensor management}
2019-05-20 16:22:11 +02:00
\includegraphics [width=\textwidth] { ./d4-4.png}
2019-05-10 15:55:59 +02:00
\end { frame}
2019-05-16 16:01:17 +02:00
\begin { frame}
\frametitle { }
\begin { center}
2019-05-20 16:22:11 +02:00
{ \bf A distributed Network telescope to observe DDoS attacks}
2019-05-16 16:01:17 +02:00
\end { center}
\vspace { 10pt}
\begin { center}
\includegraphics [width=.7\textwidth] { eventhorizon.png}
\end { center}
\end { frame}
2019-05-10 15:55:59 +02:00
2019-05-16 16:01:17 +02:00
\begin { frame}
\frametitle { Motivation}
DDoS Attacks produce an observable side-effect:
\begin { center}
\scalebox { 0.8} { \input { bsvol.tex} }
\end { center}
\end { frame}
2019-05-10 15:55:59 +02:00
\begin { frame}
2019-05-16 16:01:17 +02:00
\frametitle { What can be derived from backscatter traffic?}
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item External point of view on ongoing Denial of Service attacks:
\begin { itemize}
\item { \bf Confirm} if there is a DDoS attack
\item { \bf Recover} time line of attacked targets
\item { \bf Confirm} which services (DNS, webserver, $ \dots $ )
\item { \bf Observe} Infrastructure changes
\end { itemize}
\item { \bf Assess the state of an infrastructure under denial of service attack}
2019-05-16 16:01:17 +02:00
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item { \bf Detect} failure/addition of intermediate network equipments, firewalls, proxy servers etc
\item { \bf Detect} DDoS mitigation devices
2019-05-16 16:01:17 +02:00
\end { itemize}
2019-05-20 16:22:11 +02:00
\item Create models of DoS/DDoS attacks
2019-05-16 16:01:17 +02:00
\end { itemize}
2019-05-10 15:55:59 +02:00
\end { frame}
2019-05-16 16:01:17 +02:00
\begin { frame}
\frametitle { D4 in this setting}
2019-05-20 16:22:11 +02:00
D4 - for data collection and processing:
2019-05-16 16:01:17 +02:00
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item { \bf provide} various points of observation in non contiguous address space,
\item { \bf aggregate} and { \bf mix} backscatter traffic collected from D4 sensors,
\item { \bf perform} analysis on big amount of data.
2019-05-16 16:01:17 +02:00
\end { itemize}
2019-05-20 16:22:11 +02:00
D4 - from a end-user perspective:
2019-05-16 16:01:17 +02:00
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item { \bf provide} backscatter analysis results,
\item { \bf provide} daily updates,
\item { \bf provide} additional relevant (or pivotal) information (DNS, BGP, etc.),
\item { \bf provide} an API and search capabilities.
2019-05-16 16:01:17 +02:00
\end { itemize}
2019-05-21 09:01:26 +02:00
\end { frame}
\begin { frame}
\frametitle { First release}
\begin { itemize}
\item [\checkmark]
analyzer-d4-pibs\footnote { \url { https://github.com/D4-project/analyzer-d4-pibs} } , an analyzer for a D4 network sensor:
\begin { itemize}
\item { \bf processes} data produced by D4 sensors (pcaps),
2019-05-21 11:36:02 +02:00
\item { \bf displays} potential backscatter traffic on standard output,
2019-05-21 11:57:43 +02:00
\item { \bf focuses} on TCP SYN flood in this first release.
2019-05-21 09:01:26 +02:00
\end { itemize}
\end { itemize}
\end { frame}
2019-05-10 15:55:59 +02:00
2019-05-21 09:01:26 +02:00
\begin { frame}
2019-05-16 16:01:17 +02:00
\begin { center}
2019-05-20 16:22:11 +02:00
{ \bf Passive DNS}
2019-05-16 16:01:17 +02:00
\end { center}
\end { frame}
2019-05-10 15:55:59 +02:00
\begin { frame}
2019-05-16 16:01:17 +02:00
\frametitle { Problem statement}
\begin { itemize}
\item CIRCL (and other CSIRTs) have their own passive DNS\footnote { \url { https://www.circl.lu/services/passive-dns/} } collection mechanisms
\item Current { \bf collection models} are affected with DoH\footnote { DNS over HTTPS} and centralised DNS services
\item DNS answers collection is a tedious process
\item { \bf Sharing Passive DNS stream} between organisation is challenging due to privacy
\end { itemize}
2019-05-10 15:55:59 +02:00
\end { frame}
2019-05-16 16:01:17 +02:00
\begin { frame}
\frametitle { Potential Strategy}
\begin { itemize}
\item Improve { \bf Passive DNS collection diversity} by being closer to the source and limit impact of DoH (e.g. at the OS resolver level)
\item Increasing diversity and { \bf mixing models} before sharing/storing Passive DNS records
\item Simplify process and tools to install for { \bf Passive DNS collection by relying on D4 sensors} instead of custom mechanisms
\item Provide a distributed infrastructure for mixing streams and filtering out the sharing to the validated partners
\end { itemize}
\end { frame}
2019-05-10 15:55:59 +02:00
\begin { frame}
2019-05-16 16:01:17 +02:00
\frametitle { First release}
2019-05-20 16:22:11 +02:00
2019-05-16 16:01:17 +02:00
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item [\checkmark]
analyzer-d4-passivedns\footnote { \url { https://github.com/D4-project/analyzer-d4-passivedns} } , an analyzer for a D4 network sensor:
2019-05-16 16:01:17 +02:00
2019-05-20 16:22:11 +02:00
\begin { itemize}
\item { \bf processes} data produced by D4 sensors (in passivedns CSV format\footnote { \url { https://github.com/gamelinux/passivedns} } ),
\item { \bf ingests} these into a { \bf Passive DNS server} which can be queried later to search for the Passive DNS records,
\item { \bf provides} a lookup server (using on
redis-compatible backend) that is a Passive DNS REST server compliant to the Common Output Format\footnote { \url { https://tools.ietf.org/html/draft-dulaunoy-dnsop-passive-dns-cof-04} } .
\end { itemize}
2019-05-16 16:01:17 +02:00
\end { itemize}
\end { frame}
\begin { frame}
\begin { center}
2019-05-20 16:22:11 +02:00
{ \bf Passive SSL revamping}
2019-05-16 16:01:17 +02:00
\end { center}
\end { frame}
\begin { frame}
\frametitle { A passive SSL fingerprinter}
CSIRT's rationale for collecting TLS handshakes:
\begin { itemize}
2019-05-20 16:22:11 +02:00
\item { \bf pivot} on additional data points,
\item { \bf find} owners of IP addresses,
\item { \bf detect} usage of CIDR blocks,
\item { \bf detect} vulnerable systems,
\item { \bf detect} compromised services,
\item { \bf detect} key material reuse,
\item { \bf detect} weak keys.
2019-05-16 16:01:17 +02:00
\end { itemize}
\end { frame}
\begin { frame}
2019-05-21 08:47:54 +02:00
\frametitle { Objectives - TLS Fingerprinting}
2019-05-20 16:22:11 +02:00
{ \bf Keeping} a log of links between:
2019-05-16 16:01:17 +02:00
\begin { itemize}
\item x509 certificates,
\item ports,
\item IP address,
\item client (ja3),
\item server (ja3s),
\end { itemize}
\begin { displayquote}
``JA3 is a method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence.''\footnote { https://github.com/salesforce/ja3}
\end { displayquote}
\end { frame}
\begin { frame}
2019-05-20 16:22:11 +02:00
\frametitle { Objectives - Mind your Ps and Qs}
{ \bf Collect} and { \bf store} x509 certificates and TLS sessions:
2019-05-16 16:01:17 +02:00
\begin { itemize}
\item Public keys type and size,
2019-05-20 16:22:11 +02:00
\item moduli and exponents,
2019-05-16 16:01:17 +02:00
\item curves parameters.
\end { itemize}
2019-05-21 11:36:02 +02:00
{ \bf Detect} anti patterns in crypto:
2019-05-20 16:22:11 +02:00
\begin { itemize}
2019-05-21 11:36:02 +02:00
\item Shared Public Keys,
2019-05-20 16:22:11 +02:00
\item Moduli that share one prime factor,
\item Moduli that share both prime factor,
\item Small factors,
\item Nonces reuse / common preffix or suffix, etc.
\end { itemize}
2019-05-16 16:01:17 +02:00
\end { frame}
\begin { frame}
\frametitle { First release}
\begin { itemize}
\item [\checkmark] sensor-d4-tls-fingerprinting
\footnote { \url { github.com/D4-project/sensor-d4-tls-fingerprinting} } :
2019-05-20 16:22:11 +02:00
{ \bf Extracts} and { \bf fingerprints} certificates, and { \bf computes} TLSH fuzzy hash.
2019-05-16 16:01:17 +02:00
\item [\checkmark] analyzer-d4-passivessl
\footnote { \url { github.com/D4-project/analyzer-d4-passivessl} } :
2019-05-20 16:22:11 +02:00
{ \bf Stores} Certificates / PK details in a PostgreSQL DB.
2019-05-16 16:01:17 +02:00
\item lookup-d4-passivessl
\footnote { \url { github.com/D4-project/lookup-d4-passivessl} } :
2019-05-20 16:22:11 +02:00
{ \bf Exposes} the DB through a public REST API.
2019-05-16 16:01:17 +02:00
\end { itemize}
2019-05-10 15:55:59 +02:00
\end { frame}
2019-05-20 16:22:11 +02:00
\begin { frame} { Future}
2019-05-16 16:01:17 +02:00
\begin { itemize}
\item { \bf Mixing models for passive collection streams} (for privacy) in next version of D4 core server
2019-05-20 16:22:11 +02:00
\item { \bf Interconnecting private D4 sensor networks} with other D4 sensor networks (sharing to partners filtered stream)
\item { \bf Previewing datasets} collected in D4 sensor network and providing { \bf open data stream} (if contributor agrees to share under specific conditions)
\item { \bf Leverage MISP sharing communities} to augment Threat
Intelligence, and provide accurate metrology.
2019-05-16 16:01:17 +02:00
\end { itemize}
\end { frame}
2019-05-10 15:55:59 +02:00
\begin { frame}
\frametitle { Get in touch if you want to join the project, host a sensor or contribute}
\begin { itemize}
\item Collaboration can include research partnership, sharing of collected streams or improving the software.
\item Contact: info@circl.lu
2019-05-20 16:22:11 +02:00
\item \url { https://github.com/D4-Project}
\item \url { https://twitter.com/d4_ project}
\item \url { https://d4-project.org}
2019-05-10 15:55:59 +02:00
\end { itemize}
\end { frame}
\end { document}