Python library using the MISP Rest API
Go to file
Raphaël Vinot 5cd9e26073
Merge pull request #472 from mback2k/ignore-tests
Update .gitignore to exclude files produced during tests
2019-10-03 14:01:54 -07:00
docs Minor grammar errors 2019-09-26 11:03:03 +02:00
examples Code cleanup 2019-09-26 20:50:53 +02:00
pymisp chg: [types] updated to the latest version 2019-10-02 09:51:24 +02:00
tests chg: [test] remove attribute field which was not foreseen in 2.4 branch 2019-09-16 15:06:05 +02:00
travis chg: Remove legacy tests 2019-07-17 17:13:09 +02:00
.gitchangelog.rc chg: Bump version 2019-03-01 10:46:48 -08:00
.gitignore Update .gitignore to exclude files produced during tests 2019-10-03 19:45:17 +02:00
.gitmodules chg: Add i8n for pdfexport, without all the fonts in the main repo 2019-03-04 16:06:38 -08:00
.readthedocs.yml chg: Build all formats for the documentation 2019-03-01 10:37:47 -08:00
.travis.yml fix: Travis & python2 2019-04-25 13:23:26 +02:00
CHANGELOG.txt chg: Bump Changelog. 2019-08-30 14:20:26 +02:00
LICENSE add license 2014-12-09 14:54:45 +01:00
MANIFEST.in Include documentation and examples in source dist 2017-12-18 11:25:11 +01:00
Pipfile chg: [deps] Bump 2019-07-23 00:02:27 +02:00
Pipfile.lock chg: Bump dependencies 2019-10-01 13:15:10 -07:00
README.md chg: Bump readme 2019-09-10 13:55:58 +02:00
setup.cfg Fix PyPi package. 2015-08-05 17:48:15 +02:00
setup.py fix: [tests] Path to test file 2019-07-22 09:42:15 +02:00

README.md

README

Documentation Status Build Status Coverage Status Python 3.6 PyPi version Number of PyPI downloads

PyMISP - Python Library to access MISP

PyMISP is a Python library to access MISP platforms via their REST API.

PyMISP allows you to fetch events, add or update events/attributes, add or update samples or search for attributes.

Requirements

Install from pip

pip3 install pymisp

Install the latest version from repo

git clone https://github.com/MISP/PyMISP.git && cd PyMISP
git submodule update --init
pip3 install -I .[fileobjects,neo,openioc,virustotal]

Installing it with virtualenv

It is recommended to use virtualenv to not polute your OS python envirenment.

pip3 install virtualenv
git clone https://github.com/MISP/PyMISP.git && cd PyMISP
python3 -m venv ./
source venv/bin/activate
git submodule update --init
pip3 install -I .[fileobjects,neo,openioc,virustotal]

Running the tests

pip3 install -U nose pip setuptools coveralls codecov requests-mock
pip3 install git+https://github.com/kbandla/pydeep.git

git clone https://github.com/viper-framework/viper-test-files.git tests/viper-test-files
nosetests-3.4 --with-coverage --cover-package=pymisp,tests --cover-tests tests/test_*.py

If you have a MISP instance to test against, you can also run the live ones:

Note: You need to update the key in tests/testlive_comprehensive.py to the automation key of your admin account.

nosetests-3.4 --with-coverage --cover-package=pymisp,tests --cover-tests tests/testlive_comprehensive.py

Samples and how to use PyMISP

Various examples and samples scripts are in the examples/ directory.

In the examples directory, you will need to change the keys.py.sample to enter your MISP url and API key.

cd examples
cp keys.py.sample keys.py
vim keys.py

The API key of MISP is available in the Automation section of the MISP web interface.

To test if your URL and API keys are correct, you can test with examples/last.py to fetch the events published in the last x amount of time (supported time indicators: days (d), hours (h) and minutes (m)). last.py

cd examples
python3 last.py -l 10h # 10 hours
python3 last.py -l 5d  #  5 days
python3 last.py -l 45m # 45 minutes

Debugging

You have two options there:

  1. Pass debug=True to PyMISP and it will enable logging.DEBUG to stderr on the whole module

  2. Use the python logging module directly:


import logging
logger = logging.getLogger('pymisp')

# Configure it as you whish, for example, enable DEBUG mode:
logger.setLevel(logging.DEBUG)

Or if you want to write the debug output to a file instead of stderr:

import pymisp
import logging

logger = logging.getLogger('pymisp')
logging.basicConfig(level=logging.DEBUG, filename="debug.log", filemode='w', format=pymisp.FORMAT)

Test cases

  1. The content of mispevent.py is tested on every commit
  2. The tests cases that require a running MISP instance can be run the following way:
# From a pipenv

nosetests-3.4 -s --with-coverage --cover-package=pymisp,tests --cover-tests tests/testlive_comprehensive.py:TestComprehensive.[test_name]

Documentation

PyMISP API documentation is available.

Documentation can be generated with epydoc:

epydoc --url https://github.com/MISP/PyMISP --graph all --name PyMISP --pdf pymisp -o doc

Jupyter notebook

A series of Jupyter notebooks for PyMISP tutorial are available in the repository.

Everything is a Mutable Mapping

... or at least everything that can be imported/exported from/to a json blob

AbstractMISP is the master class, and inherit collections.MutableMapping which means the class can be represented as a python dictionary.

The abstraction assumes every property that should not be seen in the dictionary is prepended with a _, or its name is added to the private list __not_jsonable (accessible through update_not_jsonable and set_not_jsonable.

This master class has helpers that will make it easy to load, and export, to, and from, a json string.

MISPEvent, MISPAttribute, MISPObjectReference, MISPObjectAttribute, and MISPObject are subclasses of AbstractMISP, which mean that they can be handled as python dictionaries.

MISP Objects

Creating a new MISP object generator should be done using a pre-defined template and inherit AbstractMISPObjectGenerator.

Your new MISPObject generator need to generate attributes, and add them as class properties using add_attribute.

When the object is sent to MISP, all the class properties will be exported to the JSON export.