diff --git a/README.md b/README.md index 3c2986b..40be55f 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,30 @@ # analyzer-d4-log -Analyze logs collected though d4 to produce diverse statistics. +This analyzer processes loglines ingested by d4 (as type 3). + +# Architecture +analyzer-d4-log relies on redis to consume grokked loglines. + +To grok the loglines, analyzer-d4-log relies on an external tool: both logstash https://www.elastic.co/logstash +and nifi https://nifi.apache.org/ have been tested for this purpose (using this nifi template https://github.com/D4-project/d4-nifi-templates). +These tools poll directly the d4 server'redis server for loglines and push the results into a specific redis queue that the analyzer consumes. + +![](assets/nifi.png) + +analyzer-d4-log polls this queue periodically to produce counts and statistics of the data. At the moment, only sshd logs are supported but more will come in the future. + +# SSHD log analysis + +## Output generation +Every once in a while, analyzer-d4-log compiles the result into a svg images and csv files. It will also produce a minimalist webpage to navigate the data with a datarangepicker.; +![](assets/analyzer-d4-log.png) + +## MISP export +I addition to this graphical view, the repository contains a MISP_export folder that allows for the publication of a MISP feed of daily events. It compiles the TOP 100 usernames and sources seen in ssh login failure by D4 sensors. + +![](assets/dailyMISPevent.png) + +![](assets/d4_auth_MISPobject.png) + +Since MISP 2.4.128, MISP can conveniently display this data through specialized widgets. + +![](assets/MISP_widgets.png) diff --git a/assets/MISP_widgets.png b/assets/MISP_widgets.png new file mode 100644 index 0000000..8e3e069 Binary files /dev/null and b/assets/MISP_widgets.png differ diff --git a/assets/d4_auth_MISPobject.png b/assets/d4_auth_MISPobject.png new file mode 100644 index 0000000..b3e3d10 Binary files /dev/null and b/assets/d4_auth_MISPobject.png differ diff --git a/assets/dailyMISPevent.png b/assets/dailyMISPevent.png new file mode 100644 index 0000000..e6afe91 Binary files /dev/null and b/assets/dailyMISPevent.png differ diff --git a/assets/nifi.png b/assets/nifi.png new file mode 100644 index 0000000..b7650fb Binary files /dev/null and b/assets/nifi.png differ