D4
/
analyzer-d4-log
mirror of https://github.com/D4-project/analyzer-d4-log
2
0
Fork 0
Code Issues Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
57 Commits
3 Branches
6 Tags
1009 KiB
 
 
 
 
Tag: Branch: Tree: master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Go to file
Jean-Louis Huynen c7372bc29a
chg: [sshd] make it compatible with analyzer-json by default 		3 years ago
MISP_export chg: [misp] MISP export 3 years ago
assets chg [doc] initial README material 3 years ago
conf.sample add: [grok] moving to grokking support - logic refacto 3 years ago
inputreader chg: [sshd] retry on redisreader EOF 3 years ago
logcompiler chg: [sshd] make it compatible with analyzer-json by default 3 years ago
.gitignore chg: [sshd] svg graph generation 3 years ago
LICENSE Initial commit 3 years ago
README.md chg: [doc] typos 3 years ago
go.mod chg: [mod] bump d4-golang-util - fix #13 3 years ago
go.sum chg: [mod] bump d4-golang-util - fix #13 3 years ago
install_server.sh chg: [install] backport 3 years ago
launch_server.sh chg: [install] install and launch scripts + bumping golang-utils 3 years ago
main.go chg: [misp] correct timings 3 years ago
redis.conf add: [conf] redis config 3 years ago

README.md

analyzer-d4-log

This analyzer processes loglines ingested by d4 (as type 3).

Architecture

analyzer-d4-log relies on redis to consume grokked loglines.

To grok the loglines, analyzer-d4-log relies on an external tool: both logstash https://www.elastic.co/logstash and nifi https://nifi.apache.org/ have been tested for this purpose (using this nifi template https://github.com/D4-project/d4-nifi-templates). These tools poll directly d4 server's redis for loglines and push the results into a specific redis queue that the analyzer consumes.

Grokking D4 loglines in nifi

analyzer-d4-log polls this queue periodically to produce counts and statistics of the data. At the moment, only sshd logs are supported but more will come in the future.

SSHD log analysis

Output generation

Every once in a while, analyzer-d4-log compiles the result into a svg images and csv files. It will also produce a minimalist webpage to navigate the data with a datarangepicker.;

MISP export

I addition to this graphical view, the repository contains a MISP_export folder that allows for the publication of a MISP feed of daily events. It compiles the TOP 100 usernames and sources seen in ssh login failure by D4 sensors.

Since MISP 2.4.128, MISP can conveniently display this data through specialized widgets.