A dashboard for a real-time overview of threat intelligence from MISP instances
 
 
 
 
 
Go to file
Sami Mokaddem 07a66403d9
Merge pull request #37 from mokaddem/quick-fix
fix: register tags from attributes
2018-02-22 09:48:18 +01:00
config - Added comment on MaxMind DB 2018-01-06 12:07:14 +01:00
data feature: Added geolocation of phone numbers 2017-11-27 16:32:02 +01:00
doc update: database organisation in doc 2017-12-05 16:28:58 +01:00
helpers Merge remote-tracking branch 'upstream/master' into multipleMerges 2018-01-15 14:19:44 +01:00
screenshots update: Readme + added screenshots 2017-11-21 16:45:20 +01:00
static feature: Added a tooltip in trendings' timeline + open a MISP tab on double click 2018-01-16 09:28:16 +01:00
templates feature: Added a tooltip in trendings' timeline + open a MISP tab on double click 2018-01-16 09:28:16 +01:00
tests addition: added forgotten testall.sh script, which can be used rapidly test MISP-Dashboard 2018-01-15 14:59:53 +01:00
LICENSE add: LICENSE added 2017-10-28 10:33:24 +02:00
README.md Start scripts in background 2018-01-19 23:37:18 +01:00
give_honors_to_org.py Draft support of trophies in ZMQ 2017-11-14 14:05:54 +01:00
install_dependencies.sh fix: a small typo in the wget command 2018-01-15 15:41:37 +01:00
misp-dashboard.wsgi wsgi definition for Apache mod_wsgi 2017-12-15 16:05:25 +01:00
retreive_map_pic.py Started support of MISP ZMQ 2017-10-13 15:03:09 +02:00
server.py feature: Added a tooltip in trendings' timeline + open a MISP tab on double click 2018-01-16 09:28:16 +01:00
start_all.sh - Added debug instructions 2018-01-06 20:51:23 +01:00
util.py fix: Fixed tons of bugs related to migration of handle_contribution to 2017-12-01 15:39:17 +01:00
zmq_dispatcher.py fix: Attribute 'Tag' in misp_json.Attributes does not contain the field 'Tag' anymore 2018-02-22 09:45:33 +01:00
zmq_subscriber.py Append to log instead of replacing 2017-12-05 10:32:12 +01:00

README.md

MISP-Dashboard

An experimental dashboard showing live data and statistics from the ZMQ of one or more MISP instances.

Installation

  • Launch ./install_dependencies.sh from the MISP-Dashboard directory
  • Update the configuration file config.cfg so that it matches your system
    • Fields that you may change:
      • RedisGlobal -> host
      • RedisGlobal -> port
      • RedisGlobal -> zmq_url
      • RedisGlobal -> misp_web_url
      • RedisMap -> pathMaxMindDB

Updating by pulling

  • Re-launch ./install_dependencies.sh to fetch new required dependencies
  • Re-update your configuration file config.cfg

Starting the System

/!\ You do not need to run it as root. Normal privileges are fine.

  • Be sure to have a running redis server
    • e.g. redis-server -p 6250
  • Activate your virtualenv . ./DASHENV/bin/activate
  • Listen to the MISP feed by starting the zmq_subscriber ./zmq_subscriber.py &
  • Start the dispatcher to process received messages ./zmq_dispatcher.py &
  • Start the Flask server ./server.py &
  • Access the interface at http://localhost:8001/

Debug

Debug is fun and gives you more details on what is going on when things fail. Bare in mind running Flask in debug is NOT suitable for production, it will drop you to a Python shell if enabled, to do further digging.

Just before running ./server.py do:

export FLASK_DEBUG=1
export FLASK_APP=server.py
flask run --host=0.0.0.0 --port=8001 # <- Be careful here, this exposes it on ALL ip addresses. Ideally if run locally --host=127.0.0.1

OR, just toggle the debug flag in start_all.sh script.

Happy hacking ;)

Features

Live Dashboard

  • Possibility to subscribe to multiple ZMQ feeds
  • Shows direct contribution made by organisations
  • Shows live resolvable posted locations

Dashboard live

Geolocalisation Dashboard

  • Provides historical geolocalised information to support security teams, CSIRTs or SOC finding threats in their constituency
  • Possibility to get geospatial information from specific regions

Dashbaord geo

Contributors Dashboard

Shows:

  • The monthly rank of all organisation
  • The last organisation that contributed (dynamic updates)
  • The contribution level of all organisation
  • Each category of contribution per organisation
  • The current ranking of the selected organisation (dynamic updates)

Includes:

  • Gamification of the platform:
    • Two different levels of ranking with unique icons
    • Exclusive obtainable badges for source code contributors and donator

Dashboard contributor Dashboard contributor2

Users Dashboard

  • Shows when and how the platform is used:
    • Login punchcard and overtime
    • Contribution vs login

Dashboard users

Trendings Dashboard

  • Provides real time information to support security teams, CSIRTs or SOC showing current threats and activity
    • Shows most active events, categories and tags
    • Shows sightings and discussion overtime

Dashboard users

zmq_subscriber options


A zmq subscriber. It subscribe to a ZMQ then redispatch it to the MISP-dashboard

optional arguments:
  -h, --help            show this help message and exit
  -n ZMQNAME, --name ZMQNAME
                        The ZMQ feed name
  -u ZMQURL, --url ZMQURL
                        The URL to connect to

Deploy in production using mod_wsgi

Install Apache's mod-wsgi for Python3

sudo apt-get install libapache2-mod-wsgi-py3

Caveat: If you already have mod-wsgi installed for Python2, it will be replaced!

The following packages will be REMOVED:
  libapache2-mod-wsgi
The following NEW packages will be installed:
  libapache2-mod-wsgi-py3

Configuration file /etc/apache2/sites-available/misp-dashboard.conf assumes that misp-dashboard is cloned into var/www/misp-dashboard. It runs as user misp in this example. Change the permissions to folder and files accordingly.

<VirtualHost *:8000>
    ServerAdmin admin@misp.local
    ServerName misp.local

    DocumentRoot /var/www/misp-dashboard
    
    WSGIDaemonProcess misp-dashboard \
       user=misp group=misp \
       python-home=/var/www/misp-dashboard/DASHENV \
       processes=1 \
       threads=15 \
       maximum-requests=5000 \
       listen-backlog=100 \
       queue-timeout=45 \
       socket-timeout=60 \
       connect-timeout=15 \
       request-timeout=60 \
       inactivity-timeout=0 \
       deadlock-timeout=60 \
       graceful-timeout=15 \
       eviction-timeout=0 \
       shutdown-timeout=5 \
       send-buffer-size=0 \
       receive-buffer-size=0 \
       header-buffer-size=0 \
       response-buffer-size=0 \
       server-metrics=Off

    WSGIScriptAlias / /var/www/misp-dashboard/misp-dashboard.wsgi

    <Directory /var/www/misp-dashboard>
        WSGIProcessGroup misp-dashboard
        WSGIApplicationGroup %{GLOBAL}
        Require all granted
    </Directory>

    LogLevel info
    ErrorLog /var/log/apache2/misp-dashboard.local_error.log
    CustomLog /var/log/apache2/misp-dashboard.local_access.log combined
    ServerSignature Off
</VirtualHost>

License

Images and logos are handmade for:

  • rankingMISPOrg/
  • rankingMISPMonthly/
  • MISPHonorableIcons/

Note that:

  • Part of MISPHonorableIcons/1.svg comes from octicons.github.com (CC0 - No Rights Reserved)
  • Part of MISPHonorableIcons/2.svg comes from Zeptozephyr (CC0 - No Rights Reserved)
Copyright (C) 2017 CIRCL - Computer Incident Response Center Luxembourg (c/o smile, security made in Lëtzebuerg, Groupement d'Intérêt Economique)
Copyright (c) 2017 Sami Mokaddem


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.