You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
Jean-Louis Huynen c7372bc29a
chg: [sshd] make it compatible with analyzer-json by default
8 months ago
MISP_export chg: [misp] MISP export 1 year ago
assets chg [doc] initial README material 1 year ago
conf.sample add: [grok] moving to grokking support - logic refacto 1 year ago
inputreader chg: [sshd] retry on redisreader EOF 1 year ago
logcompiler chg: [sshd] make it compatible with analyzer-json by default 8 months ago
.gitignore chg: [sshd] svg graph generation 1 year ago
LICENSE Initial commit 1 year ago
README.md chg: [doc] typos 1 year ago
go.mod chg: [mod] bump d4-golang-util - fix #13 1 year ago
go.sum chg: [mod] bump d4-golang-util - fix #13 1 year ago
install_server.sh chg: [install] backport 1 year ago
launch_server.sh chg: [install] install and launch scripts + bumping golang-utils 1 year ago
main.go chg: [misp] correct timings 1 year ago
redis.conf add: [conf] redis config 1 year ago

README.md

analyzer-d4-log

This analyzer processes loglines ingested by d4 (as type 3).

Architecture

analyzer-d4-log relies on redis to consume grokked loglines.

To grok the loglines, analyzer-d4-log relies on an external tool: both logstash https://www.elastic.co/logstash and nifi https://nifi.apache.org/ have been tested for this purpose (using this nifi template https://github.com/D4-project/d4-nifi-templates). These tools poll directly d4 server's redis for loglines and push the results into a specific redis queue that the analyzer consumes.

Grokking D4 loglines in nifi

analyzer-d4-log polls this queue periodically to produce counts and statistics of the data. At the moment, only sshd logs are supported but more will come in the future.

SSHD log analysis

Output generation

Every once in a while, analyzer-d4-log compiles the result into a svg images and csv files. It will also produce a minimalist webpage to navigate the data with a datarangepicker.;

MISP export

I addition to this graphical view, the repository contains a MISP_export folder that allows for the publication of a MISP feed of daily events. It compiles the TOP 100 usernames and sources seen in ssh login failure by D4 sensors.

Since MISP 2.4.128, MISP can conveniently display this data through specialized widgets.