MISP
/
misp-training
mirror of https://github.com/MISP/misp-training
2
0
Fork 0
Code Issues Releases Wiki Activity

wip

master
iglocska 2020-01-29 23:25:44 +01:00
parent a87cbafe9c
commit 3894093d4f
No known key found for this signature in database
GPG Key ID: BEA224F1FEF113AC
1 changed files with 76 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

211
b.4-turning-data-into-actionable-intelligence-short/content.tex
View File

@ -5,19 +5,11 @@
\titlepage \titlepage
\end{frame} \end{frame}
\begin{frame}
\frametitle{about CIRCL}
\includegraphics[scale=0.4]{circl.png}
\begin{itemize}
\item The Computer Incident Response Center Luxembourg (CIRCL) is a government-driven initiative designed to provide a systematic response facility to computer security threats and incidents. CIRCL is the CERT for the private sector, communes and non-governmental entities in Luxembourg and is operated by securitymadein.lu g.i.e.
\end{itemize}
\end{frame}
\begin{frame} \begin{frame}
\frametitle{MISP and CIRCL} \frametitle{MISP and CIRCL}
\begin{itemize} \begin{itemize}
\item CIRCL is mandated by the Ministry of Economy and acting as the Luxembourg National CERT for private sector. \item CIRCL is mandated by the Ministry of Economy and acting as the Luxembourg National CERT for private sector.
\item CIRCL leads the development of the Open Source MISP threat intelligence platform which is used by many military or intelligence communities, private companies, financial sector, National CERTs and LEAs globally. \item We lead the development of the Open Source MISP TISP which is used by many military or intelligence communities, private companies, financial sector, National CERTs and LEAs globally.
\item {\bf CIRCL runs multiple large MISP communities performing active daily threat-intelligence sharing}. \item {\bf CIRCL runs multiple large MISP communities performing active daily threat-intelligence sharing}.
\end{itemize} \end{itemize}
\end{frame} \end{frame}
@ -25,24 +17,9 @@
\begin{frame} \begin{frame}
\frametitle{The aim of this presentation} \frametitle{The aim of this presentation}
\begin{itemize} \begin{itemize}
\item To give some insight into what sort of an evolution of our various communities' have gone through as observed over the past 8 years \item Why is contextualisation important?
\item Show the importance of {\bf strong contextualisation}... \item What options do we have in MISP?
\item ...and how that can be leveraged when trying to make our data {\bf actionable} \item How can we leverage this in the end?
\end{itemize}
\end{frame}
\begin{frame}
\frametitle{Development based on practical user feedback}
\begin{itemize}
\item There are many different types of users of an information sharing platform like MISP:
\begin{itemize}
\item {\bf Malware reversers} willing to share indicators of analysis with respective colleagues.
\item {\bf Security analysts} searching, validating and using indicators in operational security.
\item {\bf Intelligence analysts} gathering information about specific adversary groups.
\item {\bf Law-enforcement} relying on indicators to support or bootstrap their DFIR cases.
\item {\bf Risk analysis teams} willing to know about the new threats, likelyhood and occurences.
\item {\bf Fraud analysts} willing to share financial indicators to detect financial frauds.
\end{itemize}
\end{itemize} \end{itemize}
\end{frame} \end{frame}
@ -57,13 +34,6 @@
\end{itemize} \end{itemize}
\end{frame} \end{frame}
\begin{frame}
\frametitle{Initial workflow}
\begin{center}
\includegraphics[width=1.0\linewidth]{workflow_initial2.png}
\end{center}
\end{frame}
\begin{frame} \begin{frame}
\frametitle{Why was it so simplistic?} \frametitle{Why was it so simplistic?}
\begin{itemize} \begin{itemize}
@ -93,62 +63,53 @@
\end{frame} \end{frame}
\begin{frame} \begin{frame}
\frametitle{Our initial solution} \frametitle{Different layers of context}
\begin{itemize} \begin{itemize}
\item Allow users to {\bf tag any information} created in MISP \item Context added by analysts / tools
\item We wanted to be {\bf lax with what we accept} in terms of data, but be {\bf strict on what we fed to our tools}, with strong filter options \item Data that tells a story
\item We had some ideas on how to potentially move forward... \item Encoding analyst knowledge to automatically leverage the above
\end{itemize}
\end{frame}
\section{Context added by analysts / tools}
\begin{frame}
\frametitle{Expressing why data-points matter}
\begin{itemize}
\item An IP address by itself is barely ever interesting
\item We need to tell the recipient / machine why this is relevant
\item All data in MISP has a bare minimum required context
\item We differentiate between indicators and supporting data
\end{itemize} \end{itemize}
\end{frame} \end{frame}
\begin{frame} \begin{frame}
\frametitle{Our initial failures} \frametitle{Broadening the scope of what sort of context we are interested in}
\begin{itemize} \begin{itemize}
\item Try to capture different aspects of contextualisation into {\bf normalised values} (threat level, source reliability, etc) \item {\bf Who} can receive our data? {\bf What} can they do with it?
\begin{itemize} \item {\bf Data accuracy, source reliability}
\item Didn't scale with needs other than our own \item {\bf Why} is this data relevant to us?
\item Incorporating new types of contextualisation would mean {\bf the modification of the software} \item {\bf Who} do we think is behind it, {\bf what tools} were used?
\item Getting communities with {\bf established naming conventions} to use anything but their go-to vocabularies was a pipe-dream \item What sort of {\bf motivations} are we dealing with? Who are the {\bf targets}?
\item Heated arguments over numeric conversions \item How can we {\bf block/detect/remediate} the attack?
\end{itemize} \item What sort of {\bf impact} are we dealing with?
\end{itemize} \end{itemize}
\end{frame} \end{frame}
\begin{frame} \begin{frame}
\frametitle{Human creativity} \frametitle{Tagging and taxonomies}
\begin{itemize}
\item We tried an alternate approach instead: Free tagging
\begin{itemize}
\item Result was spectacularly painful, at least 7 different ways to spell tlp:amber
\item No canonisation for common terms lead to tagging ultimately becoming a highly flawed tool for filtering within a sharing community
\end{itemize}
\end{itemize}
\begin{center}
\includegraphics[scale=0.45]{creativity.png}
\end{center}
\end{frame}
\begin{frame}
\frametitle{How we ended up tackling the issue more successfuly}
\begin{itemize} \begin{itemize}
\item We ended up with a mixed approach, currently implemented by the MISP-taxonomy system \item Simple labels
\begin{itemize} \item Standardising on vocabularies
\item Taxonomies are {\bf vocabularies} of known tags \item Different organisational/community cultures require different nomenclatures
\item Tags would be in a {\bf triple tag format} \item Triple tag system - taxonomies
\begin{itemize} \item JSON libraries that can easily be defined without our intervention
\item[] \texttt{namespace:predicate=''value''} ***********PICPLZ
\end{itemize}
\item Create your own taxonomies, recipients should be able to use data you tag with them without knowing it at the first place
\item Avoid any coding, stick to {\bf JSON}
\end{itemize}
\item Massive success, approaching 100 taxonomies
\item Organisations can solve their own issues without having to rely on us
\end{itemize} \end{itemize}
\includegraphics[scale=0.4]{taxonomy-workflow.png}
\end{frame} \end{frame}
\begin{frame} \begin{frame}
\frametitle{We were still missing something...} \frametitle{Galaxies}
\begin{itemize} \begin{itemize}
\item Taxonomy tags often {\bf non self-explanatory} \item Taxonomy tags often {\bf non self-explanatory}
\item Example: universal understanding of tlp:green vs APT 28 \item Example: universal understanding of tlp:green vs APT 28
@ -168,33 +129,21 @@
\end{frame} \end{frame}
\begin{frame} \begin{frame}
\frametitle{Broadening the scope of what sort of context we are interested in} \frametitle{The emergence of ATT\&CK and similar galaxies}
\begin{itemize} \begin{itemize}
\item {\bf Who} can receive our data? {\bf What} can they do with it? \item Standardising on high-level {\bf TTPs} was a solution to a long list of issues
\item {\bf Data accuracy, source reliability} \item Adoption was rapid, tools producing ATT\&CK data, familiar interface for users
\item {\bf Why} is this data relevant to us? \item A much better take on kill-chain phases in general
\item {\bf Who} do we think is behind it, {\bf what tools} were used? \item Feeds into our {\bf filtering} and {\bf situational awareness} needs extremely well
\item What sort of {\bf motivations} are we dealing with? Who are the {\bf targets}? \item Gave rise to other, ATT\&CK-like systems tackling other concerns
\item How can we {\bf block/detect/remediate} the attack? \begin{itemize}
\item What sort of {\bf impact} are we dealing with? \item {\bf attck4fraud} \footnote{\url{https://www.misp-project.org/galaxy.html\#_attck4fraud}} by Francesco Bigarella from ING
\end{itemize} \item {\bf Election guidelines} \footnote{\url{https://www.misp-project.org/galaxy.html\#_election_guidelines}} by NIS Cooperation Group
\end{itemize}
\end{itemize}
\end{frame} \end{frame}
\begin{frame} \section{Data that tells a story}
\frametitle{Parallel to the contextualisation efforts: False positive handling}
\begin{itemize}
\item Low quality / false positive prone information being shared
\item Lead to {\bf alert-fatigue}
\item Exclude organisation xy out of the community?
\item False positives are often obvious - {\bf can be encoded}
\item {\bf Warninglist system}\footnote{\url{https://github.com/MISP/misp-warninglists}} aims to do that
\item Lists of well-known indicators which are often false-positives like RFC1918 networks, ...
\end{itemize}
\begin{center}
\includegraphics[scale=0.22]{warning-list.png}
\includegraphics[scale=0.45]{warning-list-event.png}
\end{center}
\end{frame}
\begin{frame} \begin{frame}
\frametitle{More complex data-structures for a modern age} \frametitle{More complex data-structures for a modern age}
@ -243,15 +192,35 @@
\end{center} \end{center}
\end{frame} \end{frame}
******TIMELINETHING
\section{Encoding analyst knowledge to automatically leverage the above}
\begin{frame}
\frametitle{False positive handling}
\begin{itemize}
\item Low quality / false positive prone information being shared
\item Lead to {\bf alert-fatigue}
\item Exclude organisation xy out of the community?
\item False positives are often obvious - {\bf can be encoded}
\item {\bf Warninglist system}\footnote{\url{https://github.com/MISP/misp-warninglists}} aims to do that
\item Lists of well-known indicators which are often false-positives like RFC1918 networks, ...
\end{itemize}
\begin{center}
\includegraphics[scale=0.22]{warning-list.png}
\includegraphics[scale=0.45]{warning-list-event.png}
\end{center}
\end{frame}
\begin{frame} \begin{frame}
\frametitle{Making use of all this context} \frametitle{Making use of all this context}
\begin{itemize} \begin{itemize}
\item Most obvious goal: Improve the way we query data \item Providing advanced ways of querying data
\begin{itemize} \begin{itemize}
\item Unified all export APIs \item Unified export APIs
\item Incorporate all contextualisation options into {\bf API filters} \item Incorporating all contextualisation options into {\bf API filters}
\item Allow for an {\bf on-demand} way of {\bf excluding potential false positives} \item Allowing for an {\bf on-demand} way of {\bf excluding potential false positives}
\item Allow users to easily {\bf build their own} export modules feed their various tools \item Allowing users to easily {\bf build their own} export modules feed their various tools
\end{itemize} \end{itemize}
\end{itemize} \end{itemize}
\end{frame} \end{frame}
@ -280,10 +249,10 @@
\begin{frame} \begin{frame}
\frametitle{Synchronisation filters} \frametitle{Synchronisation filters}
\begin{itemize} \begin{itemize}
\item Make decisions on whom to share data with based on context \item Making decisions on whom to share data with based on context
\begin{itemize} \begin{itemize}
\item MISP by default decides based on the information creator's decision who data gets shared with \item MISP by default decides based on the information creator's decision who data gets shared with
\item Community hosts should be able to {\bf act as a safety net} for sharing \item Community hosts can {\bf act as a safety net} for sharing
\begin{itemize} \begin{itemize}
\item {\bf Push filters} - what can I push? \item {\bf Push filters} - what can I push?
\item {\bf Pull filters} - what am I interested in? \item {\bf Pull filters} - what am I interested in?
@ -293,21 +262,6 @@
\end{itemize} \end{itemize}
\end{frame} \end{frame}
\begin{frame}
\frametitle{The emergence of ATT\&CK and similar galaxies}
\begin{itemize}
\item Standardising on high-level {\bf TTPs} was a solution to a long list of issues
\item Adoption was rapid, tools producing ATT\&CK data, familiar interface for users
\item A much better take on kill-chain phases in general
\item Feeds into our {\bf filtering} and {\bf situational awareness} needs extremely well
\item Gave rise to other, ATT\&CK-like systems tackling other concerns
\begin{itemize}
\item {\bf attck4fraud} \footnote{\url{https://www.misp-project.org/galaxy.html\#_attck4fraud}} by Francesco Bigarella from ING
\item {\bf Election guidelines} \footnote{\url{https://www.misp-project.org/galaxy.html\#_election_guidelines}} by NIS Cooperation Group
\end{itemize}
\end{itemize}
\end{frame}
\begin{frame}[fragile] \begin{frame}[fragile]
\frametitle{Example query to generate ATT\&CK heatmaps} \frametitle{Example query to generate ATT\&CK heatmaps}
\texttt{/events/restSearch} \texttt{/events/restSearch}
@ -351,19 +305,6 @@
\end{itemize} \end{itemize}
\end{frame} \end{frame}
\begin{frame}
\frametitle{Scoring Indicators: Our solution}
$$ \texttt{score}(\texttt{\tiny Attribute}) = \texttt{base\_score}(\texttt{\tiny Attribute, Model}) \;\;\bullet\;\; \texttt{decay}(\texttt{\tiny Model, time}) $$
Where,\vspace{0.5cm}
\begin{itemize}
\item \texttt{score} $ \in [0, 100] $
\item \texttt{base\_score} $ \in [0, 100] $
\item \texttt{decay} is a function defined by model's parameters controlling decay speed
\item \texttt{Attribute} Contains \textit{Attribute}'s values and metadata {\scriptsize (\textit{Taxonomies}, \textit{Galaxies}, ...)}
\item \texttt{Model} Contains the \textit{Model}'s configuration
\end{itemize}
\end{frame}
\begin{frame} \begin{frame}
\frametitle{Implementation in MISP: \texttt{Event/view}} \frametitle{Implementation in MISP: \texttt{Event/view}}
\includegraphics[width=1.00\linewidth]{decaying-event.png} \includegraphics[width=1.00\linewidth]{decaying-event.png}