390 lines
14 KiB
TeX
390 lines
14 KiB
TeX
% Full instructions available at:
|
|
% https://github.com/elauksap/focus-beamertheme
|
|
|
|
\documentclass{beamer}
|
|
\usetheme[numbering=progressbar]{focus}
|
|
\usepackage{tikz}
|
|
\usetikzlibrary{positioning}
|
|
\usetikzlibrary{shapes,arrows}
|
|
\usepackage{transparent}
|
|
\usepackage{fancyvrb}
|
|
\usepackage{listings}
|
|
\usepackage{tabularx}
|
|
\usepackage{amsfonts}
|
|
\usepackage{csquotes}
|
|
\definecolor{main}{RGB}{47, 161, 219}
|
|
\definecolor{background}{RGB}{240, 247, 255}
|
|
\definecolor{textcolor}{RGB}{85, 87, 83}
|
|
|
|
\title{D4 Project}
|
|
\subtitle{Open and collaborative network monitoring}
|
|
\author{Jean-Louis Huynen}
|
|
\titlegraphic{\includegraphics[scale=0.20]{../../logos/d4-logo.pdf}}
|
|
\institute{Team CIRCL \\ \url{https://www.d4-project.org/}}
|
|
\date{2019/05/21}
|
|
|
|
\begin{document}
|
|
\begin{frame}
|
|
\maketitle
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Problem statement}
|
|
\begin{itemize}
|
|
\item CSIRTs (or private organisations) build their {\bf own honeypot, honeynet or blackhole monitoring network}
|
|
\item Designing, managing and operating such infrastructure is a tedious and resource intensive task
|
|
\item {\bf Automatic sharing} between monitoring networks from different organisations is missing
|
|
\item Sensors and processing are often seen as blackbox or difficult to audit
|
|
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
|
|
\begin{frame}
|
|
\frametitle{Objective}
|
|
\begin{itemize}
|
|
\item Based on our experience with
|
|
MISP\footnote{\url{https://github.com/MISP/MISP}} where sharing
|
|
played an important role, we transpose the model in D4 project
|
|
\item Keeping the protocol and code base {\bf simple and minimal}
|
|
\item Allowing every organisation to {\bf control and audit their own sensor network}
|
|
\item Extending D4 or {\bf encapsulating legacy monitoring protocols} must be as simple as possible
|
|
\item Ensuring that the sensor server has {\bf no control on the sensor} (unidirectional streaming)
|
|
\item Don't force users to use dedicated sensors and allow {\bf flexibility of sensor support} (software, hardware, virtual)
|
|
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
|
|
\begin{frame}
|
|
\frametitle{(short) History}
|
|
\begin{itemize}
|
|
\item D4 Project (co-funded under INEA CEF EU program) started - 1st November 2018
|
|
\item D4 encapsulation protocol version 1 published - 1st December 2018
|
|
\item v0.1 release of the D4 core\footnote{\url{https://www.github.com/D4-project/d4-core}} including a server and simple D4 C client - 21st January 2019
|
|
\item First version of a golang D4
|
|
client\footnote{\url{https://www.github.com/D4-project/d4-goclient/}}
|
|
running on ARM, MIPS, PPC and x86 - 14th February 2019
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{(short) History}
|
|
\begin{center}
|
|
\begin{tabularx}{\linewidth}%
|
|
{>{\setlength\hsize{0.6\hsize}\raggedright}X%
|
|
>{\setlength\hsize{0.4\hsize}\raggedright}X}
|
|
|
|
\hline
|
|
Release & Date \tabularnewline
|
|
\hline
|
|
analyzer-d4-passivedns-v0.1 & Apr. 5, 2019 \tabularnewline
|
|
analyzer-d4-passivessl-0.1 & Apr. 25, 2019 \tabularnewline
|
|
analyzer-d4-pibs-v0.1 & Apr. 8, 2019 \tabularnewline
|
|
BGP-Ranking-1.0 & Apr. 25, 2019 \tabularnewline
|
|
d4-core-v0.1 & Jan. 25, 2019 \tabularnewline
|
|
d4-core-v0.2 & Feb. 14, 2019 \tabularnewline
|
|
d4-core-v0.3 & Apr. 8, 2019 \tabularnewline
|
|
d4-goclient-v0.1 & Feb. 14, 2019 \tabularnewline
|
|
d4-goclient-v0.2 & Apr. 8, 2019 \tabularnewline
|
|
d4-server-packer-0.1 & Apr. 25, 2019 \tabularnewline
|
|
IPASN-History-1.0 & Apr. 25, 2019 \tabularnewline
|
|
sensor-d4-tls-fingerprinting-0.1 & Apr. 25, 2019 \tabularnewline
|
|
\hline
|
|
|
|
\end{tabularx}
|
|
\end{center}
|
|
|
|
see \url{https://github.com/D4-Project}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 Overview}
|
|
\includegraphics[scale=0.38]{../../diagram/d4-overview.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Roadmap - output}
|
|
|
|
CIRCL will host a server instance for organisations willing to
|
|
contribute to a public dataset without running their own D4 server:
|
|
\begin{itemize}
|
|
\item [\checkmark]Passive SSL
|
|
\item [\checkmark] Passive DNS
|
|
\item [\checkmark]Blackhole DDoS
|
|
\item BGP mapping
|
|
\item egress filtering mapping
|
|
\item Radio monitoring
|
|
\item ...
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 encapsulation protocol}
|
|
\includegraphics[scale=0.38]{../../diagram/d4-protocol-encapsulation.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 Header}
|
|
\begin{tabular}{|l|l|l|}
|
|
\hline
|
|
Name & bit size& Description\\
|
|
\hline
|
|
version & uint 8 & Version of the header \\
|
|
type & uint 8 & Data encapsulated type\\
|
|
uuid & uint 128 & Sensor UUID\\
|
|
timestamp & uint 64 & Encapsulation time\\
|
|
hmac & uint 256 & Authentication header (HMAC-SHA-256-128)\\
|
|
size & uint 32 & Payload size\\
|
|
\hline
|
|
\end{tabular}
|
|
\end{frame}
|
|
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 Header}
|
|
\framesubtitle{Types}
|
|
\begin{tabular}{|l|l|}
|
|
\hline
|
|
Type & Description\\
|
|
\hline
|
|
0 & Reserved\\
|
|
1 & pcap (libpcap 2.4)\\
|
|
2 & meta header (JSON)\\
|
|
3 & generic log line\\
|
|
4 & dnscap output\\
|
|
5 & pcapng (diagnostic)\\
|
|
6 & generic NDJSON or JSON Lines\\
|
|
7 & generic YAF (Yet Another Flowmeter)\\
|
|
8 & passivedns CSV stream\\
|
|
254 & type defined by meta header (type 2)\\
|
|
\hline
|
|
\end{tabular}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 meta header}
|
|
\framesubtitle{Meta types}
|
|
D4 header includes an easy way to {\bf extend the protocol} (via type 2) without altering the format. Within a D4 session, the initial D4 packet(s) type 2 defines
|
|
the custom headers and then the following packets with type 254 is the custom data encapsulated.
|
|
\small
|
|
\input{meta.tex}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server}
|
|
\begin{itemize}
|
|
\item D4 core server\footnote{\url{https://github.com/D4-project/d4-core}} is a complete server to handle clients (sensors) including the decapsulation of the D4 protocol, control of sensor registrations, management of decoding protocols and dispatching to adequate decoders/analysers.
|
|
\item D4 server is written in Python 3.6 and runs on standard GNU/Linux distribution.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - management interface}
|
|
The D4 server provides a web interface to manage D4 sensors, sessions and analyzer.
|
|
\begin{itemize}
|
|
\item Get Sensors status, errors and statistics
|
|
\item Get all connected sensors
|
|
\item Manage Sensors (stream size limit, secret key, ...)
|
|
\item Manage Accepted types
|
|
\item UUID/IP blocklist
|
|
\item Create Analyzer Queues
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - main interface}
|
|
\includegraphics[width=\textwidth]{../../diagram/d4-5.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - server management}
|
|
\includegraphics[width=\textwidth]{../../diagram/d4-2.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - server management}
|
|
\includegraphics[width=\textwidth]{../../diagram/d4-3.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - sensor overview}
|
|
\includegraphics[width=\textwidth]{../../diagram/d4-1.png}
|
|
\end{frame}
|
|
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 server - sensor management}
|
|
\includegraphics[width=\textwidth]{../../diagram/d4-4.png}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{}
|
|
\begin{center}
|
|
A distributed Network telescope to observe DDoS attacks
|
|
\end{center}
|
|
\vspace{10pt}
|
|
\begin{center}
|
|
\includegraphics[width=.7\textwidth]{eventhorizon.png}
|
|
\end{center}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Motivation}
|
|
DDoS Attacks produce an observable side-effect:
|
|
\begin{center}
|
|
\scalebox{0.8}{\input{bsvol.tex}}
|
|
\end{center}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{What can be derived from backscatter traffic?}
|
|
|
|
\begin{itemize}
|
|
\item External point of view on ongoing denial of service attacks
|
|
\item Confirm if there is a DDOS attack
|
|
\item Recover time line of attacked targets
|
|
\item Confirm which services (DNS, webserver, $\dots$)
|
|
\item Infrastructure changes
|
|
\item Assess the state of an infrastructure under denial of service attack
|
|
\begin{itemize}
|
|
\item Detect failure/addition of intermediate network equipments, firewalls, proxy servers etc
|
|
\item Detect DDoS mitigation devices
|
|
\end{itemize}
|
|
\item Create probabilistic models of denial of service attacks
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{D4 in this setting}
|
|
|
|
Aggregating backscatter traffic collected from D4 sensors:
|
|
\begin{itemize}
|
|
\item have various points of observation (non contiguous address space)
|
|
\item perform analysis on bigger amount of data
|
|
\end{itemize}
|
|
|
|
D4 lookup should provide:
|
|
\begin{itemize}
|
|
\item backscatter analysis results,
|
|
\item daily updates,
|
|
\item additional relevant information (DNS, BGP, etc.).
|
|
\end{itemize}
|
|
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\begin{center}
|
|
Passive DNS
|
|
\end{center}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Problem statement}
|
|
\begin{itemize}
|
|
\item CIRCL (and other CSIRTs) have their own passive DNS\footnote{\url{https://www.circl.lu/services/passive-dns/}} collection mechanisms
|
|
\item Current {\bf collection models} are affected with DoH\footnote{DNS over HTTPS} and centralised DNS services
|
|
\item DNS answers collection is a tedious process
|
|
\item {\bf Sharing Passive DNS stream} between organisation is challenging due to privacy
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Potential Strategy}
|
|
\begin{itemize}
|
|
\item Improve {\bf Passive DNS collection diversity} by being closer to the source and limit impact of DoH (e.g. at the OS resolver level)
|
|
\item Increasing diversity and {\bf mixing models} before sharing/storing Passive DNS records
|
|
\item Simplify process and tools to install for {\bf Passive DNS collection by relying on D4 sensors} instead of custom mechanisms
|
|
\item Provide a distributed infrastructure for mixing streams and filtering out the sharing to the validated partners
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{First release}
|
|
\begin{itemize}
|
|
|
|
\item analyzer-d4-passivedns\footnote{\url{https://github.com/D4-project/analyzer-d4-passivedns}} is an analyzer for a D4 network sensor. The analyser can process data produced by D4 sensors (in passivedns CSV format\footnote{\url{https://github.com/gamelinux/passivedns}})
|
|
\item Ingest these into a {\bf Passive DNS server} which can be queried later to search for the Passive DNS records
|
|
\item The lookup server (using on redis-compatible backend) is a Passive DNS REST server compliant to the Common Output Format\footnote{\url{https://tools.ietf.org/html/draft-dulaunoy-dnsop-passive-dns-cof-04}}
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\begin{center}
|
|
Passive SSL revamping
|
|
\end{center}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{A passive SSL fingerprinter}
|
|
CSIRT's rationale for collecting TLS handshakes:
|
|
\begin{itemize}
|
|
\item pivot on additional data points,
|
|
\item find owners of IP addresses,
|
|
\item detect usage of CIDR blocks,
|
|
\item detect vulnerable systems,
|
|
\item detect compromised services,
|
|
\item detect Key material reuse,
|
|
\item detect weak keys.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Objectives}
|
|
History of links between:
|
|
\begin{itemize}
|
|
\item x509 certificates,
|
|
\item ports,
|
|
\item IP address,
|
|
\item client (ja3),
|
|
\item server (ja3s),
|
|
\end{itemize}
|
|
\begin{displayquote}
|
|
``JA3 is a method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence.''\footnote{https://github.com/salesforce/ja3}
|
|
\end{displayquote}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Objectives}
|
|
Mind your Ps and Qs:
|
|
\begin{itemize}
|
|
\item Public keys type and size,
|
|
\item modulos and exponents,
|
|
\item curves parameters.
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{First release}
|
|
\begin{itemize}
|
|
\item[\checkmark] sensor-d4-tls-fingerprinting
|
|
\footnote{\url{github.com/D4-project/sensor-d4-tls-fingerprinting}}:
|
|
Extracts and fingerprints certificates
|
|
\item[\checkmark] analyzer-d4-passivessl
|
|
\footnote{\url{github.com/D4-project/analyzer-d4-passivessl}}:
|
|
Stores Certificates / PK details in a PostgreSQL DB
|
|
\item lookup-d4-passivessl
|
|
\footnote{\url{github.com/D4-project/lookup-d4-passivessl}}:
|
|
Exposes the DB through a public REST API
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}[t]{Future}
|
|
\begin{itemize}
|
|
\item {\bf Mixing models for passive collection streams} (for privacy) in next version of D4 core server
|
|
\item Interconnecting private D4 sensor networks with other D4 sensor networks (sharing to partners filtered stream)
|
|
\item Previewing dataset collected in D4 sensor network and providing {\bf open data stream} (if contributor agrees to share under specific conditions)
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
\begin{frame}
|
|
\frametitle{Get in touch if you want to join the project, host a sensor or contribute}
|
|
\begin{itemize}
|
|
\item Collaboration can include research partnership, sharing of collected streams or improving the software.
|
|
\item Contact: info@circl.lu
|
|
\item \url{https://github.com/D4-Project} - \url{https://twitter.com/d4_project}
|
|
\end{itemize}
|
|
\end{frame}
|
|
|
|
|
|
\end{document} |