diff --git a/SUMMARY.md b/SUMMARY.md index 646b020..935e903 100755 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -2,6 +2,7 @@ * [Book Convention](book-convention/README.md) * [Quick Start](quick-start/README.md) +* [Requirements](requirements/README.md) * [Get Your Instance](get-your-instance/README.md) * [General Layout](general-layout/README.md) * [General Concepts](general-concepts/README.md) @@ -24,4 +25,5 @@ * [Synchronisation/Sharing](sharing/README.md) * [ZeroMQ - MISP publish-subscribe](misp-zmq/README.md) * [Translations - i18n & l10n](translation/README.md) +* [FAQ](faq/README.md) * [Appendices](appendices/README.md) diff --git a/appendices/README.md b/appendices/README.md index 40db07f..be98c9b 100644 --- a/appendices/README.md +++ b/appendices/README.md @@ -1,3 +1,7 @@ +# Summary + + + # Appendix A: External Authentication #### The external authentication mechanism described @@ -220,3 +224,85 @@ https:///servers/queryACL/findMissingFunctionNames Functions that have not been tied into the new ACL yet show up here. These functions will (until added to the ACL) only be accessible to site admins. +# Appendix C: Official MISP developments + +This section lists the projects that can be found on the main [MISP GitHub](https://github.com/MISP/repositories) page + e know of but not officially support and rely on their respective maintainers to keep up to date to the MISP 2.4 developments. + + +| Project | Description | Status | +| -- | -- | -- | +| [misp-objects](https://github.com/MISP/misp-objects) | Definition, description and relationship types of MISP objects | Core to MISP, frequently updated and tested | + + + +# Appendix D: Third-party development + +This section lists some projects we know of but not officially support and rely on their respective maintainers to keep up to date to the MISP 2.4 developments. + +| Project | Description | Status | +| -- | -- | -- | +| [MISP-STIX-ESM](https://github.com/mohlcyber/MISP-STIX-ESM) | Exports MISP events to STIX and ingest into McAfee ESM | Not tested by MISP core team | +| [Docker MISP](https://github.com/harvard-itsecurity/docker-misp) | Automated Docker MISP container | Not tested by MISP core team | +| [misp42splunk](https://github.com/remg427/misp42splunk) | A Splunk app to use MISP in background and combine with TheHive | Not tested by MISP core team | +| [getmispioc](https://github.com/xme/splunk/tree/master/getmispioc) | getiocmisp is a Splunk custom search command that helps to extract IOCs from a MISP instance. | Not tested by MISP core team | +| [OTX MISP](https://github.com/gcrahay/otx_misp) | Imports Alienvault OTX pulses to a MISP instance | Not tested by MISP core team | +| [BTG](https://github.com/conix-security/BTG) | BTG's purpose is to make fast and efficient search on IOC | Not tested by MISP core team | +| [MISP OSINT Collection](https://github.com/adulau/misp-osint-collection) | Collection of best practices to add OSINT into MISP and/or MISP communities | Not tested by MISP core team | +| [Ansible MISP](https://github.com/StamusNetworks/ansible-misp) | Ansible playbook to install Malware Information Sharing Platform (MISP) | Not tested by MISP core team | +| [IBM XFE module](https://github.com/johestephan/XFE) | Various IBM X-Force Exchange modules | Not tested by MISP core team | +| [MISP dockerized](https://github.com/DCSO/MISP-dockerized-misp-modules) | MISP dockerized is a project designed to provide an easy-to-use and easy-to-install'out of the box' MISP instance that includes everything you need to run MISP with minimal host-side requirements. | Not tested by MISP core team | +| [MISP dockerized modules](https://github.com/DCSO/MISP-dockerized-misp-modules) | MISP-modules for MISP dockerized | Not tested by MISP core team | +| [FireMISP](https://github.com/deralexxx/FireMISP) | FireEye Alert json files to MISP Malware information sharing plattform (Alpha) | Not tested by MISP core team | +| [MISP Chrome Plugin](https://github.com/deralexxx/misp-chrome-plugin) | MISP Chrome plugin for adding and looking up indicators | Not tested by MISP core team | +| [PySight2MISP](https://github.com/deralexxx/PySight2MISP) | PySight2MISP is a project that can be run to be used as glue between iSight intel API and MISP API | Not tested by MISP core team | +| [tie2misp](https://github.com/DCSO/tie2misp) | Import DCSO TIE IOCs as MISP events | Not tested by MISP core team | +| [security onion MISP](https://github.com/weslambert/securityonion-misp) | Grab NIDS rules and Bro Intel generated from a MISP instance and use them in Security Onion | Not tested by MISP core team | +| [virustream](https://github.com/ntddk/virustream) | A script to track malware IOCs with OSINT on Twitter. | Not tested by MISP core team | +| [LAC CSV Import](https://github.com/LAC-Japan/MISP-CSVImport) | Register MISP events based on information described in files such as CSV and TSV. | Not tested by MISP core team | +| [The Hive](https://github.com/TheHive-Project/TheHive) | TheHive: a Scalable, Open Source and Free Security Incident Response Platform | Strong links between core team members, tested and known working | +| [puppet-misp](https://github.com/voxpupuli/puppet-misp) | This module installs and configures MISP - [puppet forge site](https://forge.puppet.com/puppet/misp) | Not tested by MISP core team | +| [ansible MISP](https://github.com/juju4/ansible-MISP) | ansible role to setup MISP | Not tested by MISP core team | +| [OpenDXL ATD MISP](https://github.com/mohlcyber/OpenDXL-ATD-MISP) | Automated threat intelligence collection with McAfee ATD, OpenDXL and MISP | Not tested by MISP core team | +| [IMAP Proxy](https://github.com/CIRCL/IMAP-Proxy) | Modular IMAP proxy (including PyCIRCLeanMail and MISP forward modules) | Not tested by MISP core team | +| [AutoMISP](https://github.com/da667/AutoMISP) | automate your MISP installs - This shell script is designed to automatically install [MISP](https://github.com/MISP/MISP) and the [misp-modules](https://github.com/MISP/misp-modules) extension on either Ubuntu 16.04, or 18.04. | Not tested by MISP core team | +| [Palo Alto Networks report_to_misp](https://github.com/PaloAltoNetworks/report_to_misp) | Parse a report and import the events into MISP | Not tested by MISP core team | +| [Palo Alto Networks minemeld-misp](https://github.com/PaloAltoNetworks/minemeld-misp) | MineMeld nodes for MISP | Not tested by MISP core team | +| [golang-misp](https://github.com/0xrawsec/golang-misp) | Golang Library to interact with your MISP instance | Not tested by MISP core team | +| [go-misp](https://github.com/Zenithar/go-misp) | Golang MISP [API Client](http://zenithar.org/go/misp) | Not tested by MISP core team | +| [MISP MAR](https://github.com/mohlcyber/MISP-MAR) | Integration between MISP platform and McAfee Active Response | Not tested by MISP core team | +| [MISP IoC Validator](https://github.com/tom8941/MISP-IOC-Validator) | Validate IOC from MISP ; Export results and iocs to SIEM and sensors using syslog and CEF format | Not tested by MISP core team | +| [vt2misp](https://github.com/eCrimeLabs/vt2misp) | Script to fetch data from virustotal and add it to a specific event as an object | Not tested by MISP core team | +| [Threat Pinch Lookup](https://github.com/cloudtracer/ThreatPinchLookup) | Documentation and Sharing Repository for ThreatPinch Lookup Chrome & Firefox [Extension](https://chrome.google.com/webstore/detail/threatpinch-lookup/ljdgplocfnmnofbhpkjclbefmjoikgke) | Not tested by MISP core team | +| [dovehawk](https://github.com/tylabs/dovehawk) | Dovehawk is a Bro module that automatically imports MISP indicators and reports Sightings | Not tested by MISP core team | +| [yara-exporter](https://github.com/CERT-Bund/yara-exporter) | Exporting MISP event attributes to yara rules usable with Thor apt scanner | Not tested by MISP core team | +| [volatility-misp](https://github.com/CIRCL/volatility-misp) | Volatility plugin to interface with MISP | Not tested by MISP core team | +| [misp2bro](https://github.com/thnyheim/misp2bro) | Python script that gets IOC from MISP and converts it into BRO intel files. | Not tested by MISP core team | +| [TA-misp](https://github.com/stricaud/TA-misp) | Splunk integration with MISP | Not tested by MISP core team | +| [MISP QRadar](https://github.com/karthikkbala/MISP-QRadar-Integration) | The Project can used to integrate QRadar with MISP Threat Sharing Platform | Not tested by MISP core team | +| [pymisp-suricata_search](https://github.com/raw-data/pymisp-suricata_search) | Multi-threaded suricata search module for MISP | Not tested by MISP core team | +| [MISP-ThreatExchange](https://github.com/EC-DIGIT-CSIRC/MISP-ThreatExchange) | Script to interface MISP with Facebook ThreatExchange | Not tested by MISP core team | +| [aptc](https://github.com/jymcheong/aptc) | [Automated Payload Test Controller](https://jymcheong.github.io/aptc/) | Not tested by MISP core team | +| [aptmap](https://github.com/3c7/aptmap) | A [map](https://aptmap.netlify.com) displaying threat actors from the [misp-galaxy](https://github.com/MISP/misp-galaxy) | Not tested by MISP core team | +| [mispy](https://github.com/nbareil/mispy) | Another MISP module for Python | Not tested by MISP core team | +| [MispSharp](https://github.com/DBHeise/MispSharp) | C# Library for MISP | Not tested by MISP core team | +| [Privacy Aware Sharing of IoCs in MISP](https://github.com/charly077/MISP-privacy-aware-sharing-master-thesis) | [Master Thesis](https://github.com/charly077/MISP-privacy-aware-sharing-master-thesis/blob/master/report/report.pdf) including MISP data. | Master thesis | + + + +# Appendix E: Other Threat Intel Ressources + +A brief list of online ressources that around #ThreatIntel + +* [Curated list of awesome cybersecurity companies and solutions.](https://github.com/Annsec/awesome-cybersecurity/blob/master/README.md) (Updated April 2017) +* [A curated list of awesome malware analysis tools and resources](https://github.com/rshipp/awesome-malware-analysis/blob/master/README.md). Inspired by [awesome-python](https://github.com/vinta/awesome-python) and [awesome-php](https://github.com/ziadoz/awesome-php). +* [An authoritative list of awesome devsecops tools with the help from community experiments and contributions](https://github.com/devsecops/awesome-devsecops/blob/master/README.md).[DEV.SEC.OPS](http://devsecops.org) +* [Advance Python IoC extractor](https://github.com/InQuest/python-iocextract) diff --git a/faq/README.md b/faq/README.md new file mode 100644 index 0000000..976f9f9 --- /dev/null +++ b/faq/README.md @@ -0,0 +1,95 @@ + + +# Frequently Asked Questions + +The following page hosts most frequently asked questions as seen on our [issues](https://github.com/MISP/issues) and [gitter](https://gitter.im/MISP/MISP). + +## Permission issues + +If you have any permission issues, please [set the permissions](https://misp.github.io/MISP/INSTALL.ubuntu1804/#5-set-the-permissions) to something sane first. + +## When to update MISP? + +One question might be how often to update MISP. +You can update MISP as ofte as you like. If you see the follwing: + +![MISP Update](./figures/misp-diag-update.png) + +This means that the main repository has an update available. + +If you want to play it safer or want to integrate it in your Weekly/Bi-Monthly update routine you can track our [Changelog](https://www.misp-project.org/Changelog.txt) a more up to date version is available [here](https://misp.github.io/MISP/Changelog/) + +## Update MISP fails + +If your MISP instance is outdated, meaning ONLY the core, not the modules or dashboard or python modules, you well see the following. + +![MISP outdated](./figures/misp-outdated.png) + +Once you click on update MISP you will be asked confirmation. + +![MISP Update Yes/No](./figures/update-misp-YN.png) + +If you are not on a branch, the UI will tell you this, the update will fail. + +![not on branch](./figures/misp-not-on-branch.png) + +If you cannot write the **.git** files and directory as the user running the web server (and thus PHP), the update will fail. +The following diagnostic check will let you know if you can update or not. + +![.git not writeable](./figures/misp-diag-not-writeable-files-git.png) + +In case you get a file not found on **.git/ORIG_HEAD**, this means that you have never updated your MISP OR you have installed git from an archive file (like .zip/.tar.gz or similar) +Try to click update MISP and see what happens. + +![ORIG_HEAD file not found](./figures/misp-diag-writeable-files-not_found-git.png) + +### What can go wrong if I update MISP? + +In theory nothing. We put great effort into protecting the integrity of the data stored in your MISP instance. +DB upgrades happen upon login or on reload once you have update the repository. +You cannot "break" anything by clicking **Update MISP** worse case it will complain about something and you will certainly find the answer on this page. + +IF not, please open an [issue](https://github.com/MISP/MISP/issues) on GitHub or come to our [gitter](https://gitter.im/MISP/MISP) chat to see if the community can help. + +### error: pathspec 'app/composer.json' did not match any file(s) known to git + +This is **not** an error and can be ignore. Nothing will be impacted by this. + +![pathspec](./figures/misp-pathspec.png) + +### MISP modules "Connection refused" + +![MISP Modules ](./figures/misp-module-system-diag.png) + +If you get have a **Connection refused state** on your modules one of the following might be true. + +- You have no [misp-modules](https://github.com/MISP/misp-modules) not installed +- They are instaled but not running +- Something completly different + +If they are not installed, check out this section of the [INSTALL guide](https://github.com/MISP/misp-modules/#how-to-install-and-start-misp-modules-in-a-python-virtualenv) of [misp-modules](https://github.com/MISP/misp-modules). + +In case they are not running, try this on the console: + +``` +sudo -u www-data /var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s & +``` + +OR if you were foolish enough to not install in a Python virtualenv: + +``` +sudo -u www-data misp-modules -l 127.0.0.1 -s & +``` + +:warning: Running misp-modules like this will certainly kill it once you quit the session. Make sure it is in your **/etc/rc.local** or some ther init script that gets run on boot. + +## Uninstalling MISP + +There is no official procedure to uninstalling a MISP instance. + +If you want to re-use a machine where MISP was installed, wipe the machine and do a fresh install. +Consider the data in your MISP instance as potentially confidential and if you synchronized with other instances, be respectful and wipe it clean. + + diff --git a/faq/figures/misp-diag-not-writeable-files-git.png b/faq/figures/misp-diag-not-writeable-files-git.png new file mode 100644 index 0000000..2cb1fdf Binary files /dev/null and b/faq/figures/misp-diag-not-writeable-files-git.png differ diff --git a/faq/figures/misp-diag-update.png b/faq/figures/misp-diag-update.png new file mode 100644 index 0000000..499c4da Binary files /dev/null and b/faq/figures/misp-diag-update.png differ diff --git a/faq/figures/misp-diag-writeable-files-not_found-git.png b/faq/figures/misp-diag-writeable-files-not_found-git.png new file mode 100644 index 0000000..c1485e4 Binary files /dev/null and b/faq/figures/misp-diag-writeable-files-not_found-git.png differ diff --git a/faq/figures/misp-module-system-diag.png b/faq/figures/misp-module-system-diag.png new file mode 100644 index 0000000..49ad9f2 Binary files /dev/null and b/faq/figures/misp-module-system-diag.png differ diff --git a/faq/figures/misp-not-on-branch.png b/faq/figures/misp-not-on-branch.png new file mode 100644 index 0000000..f52bc46 Binary files /dev/null and b/faq/figures/misp-not-on-branch.png differ diff --git a/faq/figures/misp-outdated.png b/faq/figures/misp-outdated.png new file mode 100644 index 0000000..d0fb2f4 Binary files /dev/null and b/faq/figures/misp-outdated.png differ diff --git a/faq/figures/misp-pathspec.png b/faq/figures/misp-pathspec.png new file mode 100644 index 0000000..918b1e6 Binary files /dev/null and b/faq/figures/misp-pathspec.png differ diff --git a/faq/figures/update-misp-YN.png b/faq/figures/update-misp-YN.png new file mode 100644 index 0000000..142cb09 Binary files /dev/null and b/faq/figures/update-misp-YN.png differ diff --git a/publish.sh b/publish.sh old mode 100644 new mode 100755 index e0355ed..8600c7a --- a/publish.sh +++ b/publish.sh @@ -3,8 +3,8 @@ gitbook build gitbook pdf gitbook epub gitbook mobi -cp book.pdf _book -cp book.epub _book -cp book.mobi _book +mv book.pdf _book +mv book.epub _book +mv book.mobi _book cd _book -rsync -av . circl@cpab.circl.lu:/var/www/nwww.circl.lu/doc/misp/ +rsync -azv . circl@cpab:/var/www/nwww.circl.lu/doc/misp/ && rm -rf _book diff --git a/requirements/README.md b/requirements/README.md new file mode 100644 index 0000000..3389d5c --- /dev/null +++ b/requirements/README.md @@ -0,0 +1,28 @@ +# MISP Instance requirements + + + +## Intro + +There are various ways you can run a MISP instance. + +- Virtualized with docker/ansible/packer etc +- VMware/Virtualbox/Xen etc +- Dedicated hardware +- Road warrior setups +- Air-gapped setups + +Whilst there is never an ultimate answer to what specifications a system needs, we try to give an approximate answer depending on your use case. + +## The biggie + +Having millions of events with millions of attributes (indicators) will eventually result in sub-par performance. +Ideally you have millions of attributes and thousands of events. But this also depends on how you ingest the data. +With millions of attributes a bottleneck could be the correlation engine. +Especially if you have many duplicates in your events. (Use the feed matrix to see if feeds are massively overlapping) + +### Tool assisted sizing + +During a hackathon [misp-sizer](https://www.misp-project.org/MISP-sizer/) was conceived. ([code](https://github.com/MISP/MISP-sizer)) +This can give you a very rough estimate and needs some more [improvements](https://github.com/MISP/MISP-sizer/issues). +