MISP
/
misp-modules
mirror of https://github.com/MISP/misp-modules
2
0
Fork 0
Code Issues Releases Wiki Activity

Deployed 99646ee with MkDocs version: 1.0.4

gh-pages
Alexandre Dulaunoy 2021-04-19 09:39:38 +01:00
parent 29073f4187
commit 566a97a971
3 changed files with 99 additions and 712 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

809
index.html
View File

@ -91,7 +91,7 @@
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" data-md-component="overlay" for="__drawer"></label>
<a href="#misp-modules" tabindex="1" class="md-skip">
<a href="#home" tabindex="1" class="md-skip">
Skip to content
</a>
@ -286,115 +286,6 @@
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" title="How to install and start MISP modules in a Python virtualenv? (recommended)" class="md-nav__link">
How to install and start MISP modules in a Python virtualenv? (recommended)
</a>
</li>
<li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" title="How to install and start MISP modules on RHEL-based distributions ?" class="md-nav__link">
How to install and start MISP modules on RHEL-based distributions ?
</a>
</li>
<li class="md-nav__item">
<a href="#how-to-add-your-own-misp-modules" title="How to add your own MISP modules?" class="md-nav__link">
How to add your own MISP modules?
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#introspection" title="introspection" class="md-nav__link">
introspection
</a>
</li>
<li class="md-nav__item">
<a href="#version" title="version" class="md-nav__link">
version
</a>
</li>
<li class="md-nav__item">
<a href="#additional-configuration-values" title="Additional Configuration Values" class="md-nav__link">
Additional Configuration Values
</a>
</li>
<li class="md-nav__item">
<a href="#handler" title="handler" class="md-nav__link">
handler
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#export-module" title="export module" class="md-nav__link">
export module
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#module-type" title="Module type" class="md-nav__link">
Module type
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#testing-your-modules" title="Testing your modules?" class="md-nav__link">
Testing your modules?
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#enable-your-module-in-the-web-interface" title="Enable your module in the web interface" class="md-nav__link">
Enable your module in the web interface
</a>
</li>
<li class="md-nav__item">
<a href="#set-any-other-required-settings-for-your-module" title="Set any other required settings for your module" class="md-nav__link">
Set any other required settings for your module
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#install-misp-module-on-an-offline-instance" title="Install misp-module on an offline instance." class="md-nav__link">
Install misp-module on an offline instance.
</a>
</li>
<li class="md-nav__item">
@ -405,15 +296,8 @@
</li>
<li class="md-nav__item">
<a href="#tips-for-developers-creating-modules" title="Tips for developers creating modules" class="md-nav__link">
Tips for developers creating modules
</a>
</li>
<li class="md-nav__item">
<a href="#documentation" title="Documentation" class="md-nav__link">
Documentation
<a href="#licenses" title="Licenses" class="md-nav__link">
Licenses
</a>
</li>
@ -601,115 +485,6 @@
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" title="How to install and start MISP modules in a Python virtualenv? (recommended)" class="md-nav__link">
How to install and start MISP modules in a Python virtualenv? (recommended)
</a>
</li>
<li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" title="How to install and start MISP modules on RHEL-based distributions ?" class="md-nav__link">
How to install and start MISP modules on RHEL-based distributions ?
</a>
</li>
<li class="md-nav__item">
<a href="#how-to-add-your-own-misp-modules" title="How to add your own MISP modules?" class="md-nav__link">
How to add your own MISP modules?
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#introspection" title="introspection" class="md-nav__link">
introspection
</a>
</li>
<li class="md-nav__item">
<a href="#version" title="version" class="md-nav__link">
version
</a>
</li>
<li class="md-nav__item">
<a href="#additional-configuration-values" title="Additional Configuration Values" class="md-nav__link">
Additional Configuration Values
</a>
</li>
<li class="md-nav__item">
<a href="#handler" title="handler" class="md-nav__link">
handler
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#export-module" title="export module" class="md-nav__link">
export module
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#module-type" title="Module type" class="md-nav__link">
Module type
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#testing-your-modules" title="Testing your modules?" class="md-nav__link">
Testing your modules?
</a>
<nav class="md-nav">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#enable-your-module-in-the-web-interface" title="Enable your module in the web interface" class="md-nav__link">
Enable your module in the web interface
</a>
</li>
<li class="md-nav__item">
<a href="#set-any-other-required-settings-for-your-module" title="Set any other required settings for your module" class="md-nav__link">
Set any other required settings for your module
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="#install-misp-module-on-an-offline-instance" title="Install misp-module on an offline instance." class="md-nav__link">
Install misp-module on an offline instance.
</a>
</li>
<li class="md-nav__item">
@ -720,15 +495,8 @@
</li>
<li class="md-nav__item">
<a href="#tips-for-developers-creating-modules" title="Tips for developers creating modules" class="md-nav__link">
Tips for developers creating modules
</a>
</li>
<li class="md-nav__item">
<a href="#documentation" title="Documentation" class="md-nav__link">
Documentation
<a href="#licenses" title="Licenses" class="md-nav__link">
Licenses
</a>
</li>
@ -750,494 +518,113 @@
<h1 id="misp-modules">MISP modules<a class="headerlink" href="#misp-modules" title="Permanent link">&para;</a></h1>
<p><a href="https://travis-ci.org/MISP/misp-modules"><img alt="Build Status" src="https://travis-ci.org/MISP/misp-modules.svg?branch=main" /></a>
<a href="https://coveralls.io/github/MISP/misp-modules?branch=main"><img alt="Coverage Status" src="https://coveralls.io/repos/github/MISP/misp-modules/badge.svg?branch=main" /></a>
<a href="https://codecov.io/gh/MISP/misp-modules"><img alt="codecov" src="https://codecov.io/gh/MISP/misp-modules/branch/main/graph/badge.svg" /></a></p>
<p>MISP modules are autonomous modules that can be used to extend <a href="https://github.com/MISP/MISP">MISP</a> for new services such as expansion, import and export.</p>
<h1 id="home">Home<a class="headerlink" href="#home" title="Permanent link">&para;</a></h1>
<p><a href="https://travis-ci.org/MISP/misp-modules"><img alt="Build Status" src="https://travis-ci.org/MISP/misp-modules.svg?branch=master" /></a>
<a href="https://coveralls.io/github/MISP/misp-modules?branch=master"><img alt="Coverage Status" src="https://coveralls.io/repos/github/MISP/misp-modules/badge.svg?branch=master" /></a>
<a href="https://codecov.io/gh/MISP/misp-modules"><img alt="codecov" src="https://codecov.io/gh/MISP/misp-modules/branch/master/graph/badge.svg" /></a>
<a href="https://app.fossa.io/projects/git%2Bgithub.com%2FMISP%2Fmisp-modules?ref=badge_shield"><img alt="FOSSA Status" src="https://app.fossa.io/api/projects/git%2Bgithub.com%MISP%2Fmisp-modules.svg?type=shield" /></a></p>
<p>MISP modules are autonomous modules that can be used for expansion and other services in <a href="https://github.com/MISP/MISP">MISP</a>.</p>
<p>The modules are written in Python 3 following a simple API interface. The objective is to ease the extensions of MISP functionalities
without modifying core components. The API is available via a simple REST API which is independent from MISP installation or configuration.</p>
<p>For more information: <a href="https://www.misp-project.org/misp-training/3.1-misp-modules.pdf">Extending MISP with Python modules</a> slides from <a href="https://github.com/MISP/misp-training">MISP training</a>.</p>
<p>MISP modules support is included in MISP starting from version <code>2.4.28</code>.</p>
<p>For more information: <a href="https://www.circl.lu/assets/files/misp-training/switch2016/2-misp-modules.pdf">Extending MISP with Python modules</a> slides from MISP training.</p>
<h2 id="existing-misp-modules">Existing MISP modules<a class="headerlink" href="#existing-misp-modules" title="Permanent link">&para;</a></h2>
<h3 id="expansion-modules">Expansion modules<a class="headerlink" href="#expansion-modules" title="Permanent link">&para;</a></h3>
<ul>
<li><a href="misp_modules/modules/expansion/apiosintds.py">apiosintDS</a> - a hover and expansion module to query the OSINT.digitalside.it API.</li>
<li><a href="misp_modules/modules/expansion/apivoid.py">API Void</a> - an expansion and hover module to query API Void with a domain attribute.</li>
<li><a href="misp_modules/modules/expansion/assemblyline_submit.py">AssemblyLine submit</a> - an expansion module to submit samples and urls to AssemblyLine.</li>
<li><a href="misp_modules/modules/expansion/assemblyline_query.py">AssemblyLine query</a> - an expansion module to query AssemblyLine and parse the full submission report.</li>
<li><a href="misp_modules/modules/expansion/backscatter_io.py">Backscatter.io</a> - a hover and expansion module to expand an IP address with mass-scanning observations.</li>
<li><a href="misp_modules/modules/expansion/bgpranking.py">BGP Ranking</a> - a hover and expansion module to expand an AS number with the ASN description and its ranking and position in BGP Ranking.</li>
<li><a href="misp_modules/modules/expansion/ransomcoindb.py">RansomcoinDB check</a> - An expansion hover module to query the <a href="https://ransomcoindb.concinnity-risks.com">ransomcoinDB</a>: it contains mapping between BTC addresses and malware hashes. Enrich MISP by querying for BTC -&gt; hash or hash -&gt; BTC addresses.</li>
<li><a href="misp_modules/modules/expansion/btc_scam_check.py">BTC scam check</a> - An expansion hover module to instantly check if a BTC address has been abused.</li>
<li><a href="misp_modules/modules/expansion/btc_steroids.py">BTC transactions</a> - An expansion hover module to get a blockchain balance and the transactions from a BTC address in MISP.</li>
<li><a href="misp_modules/modules/expansion/censys_enrich.py">Censys-enrich</a> - An expansion and module to retrieve information from censys.io about a particular IP or certificate.</li>
<li><a href="misp_modules/modules/expansion/circl_passivedns.py">CIRCL Passive DNS</a> - a hover and expansion module to expand hostname and IP addresses with passive DNS information.</li>
<li><a href="misp_modules/modules/expansion/circl_passivessl.py">CIRCL Passive SSL</a> - a hover and expansion module to expand IP addresses with the X.509 certificate(s) seen.</li>
<li><a href="misp_modules/modules/expansion/countrycode.py">countrycode</a> - a hover module to tell you what country a URL belongs to.</li>
<li><a href="misp_modules/modules/expansion/crowdstrike_falcon.py">CrowdStrike Falcon</a> - an expansion module to expand using CrowdStrike Falcon Intel Indicator API.</li>
<li><a href="misp_modules/modules/expansion/cpe.py">CPE</a> - An expansion module to query the CVE Search API with a cpe code, to get its related vulnerabilities.</li>
<li><a href="misp_modules/modules/expansion/cve.py">CVE</a> - a hover module to give more information about a vulnerability (CVE).</li>
<li><a href="misp_modules/modules/expansion/cve_advanced.py">CVE advanced</a> - An expansion module to query the CIRCL CVE search API for more information about a vulnerability (CVE).</li>
<li><a href="misp_modules/modules/expansion/cuckoo_submit.py">Cuckoo submit</a> - A hover module to submit malware sample, url, attachment, domain to Cuckoo Sandbox.</li>
<li><a href="misp_modules/modules/expansion/cytomic_orion.py">Cytomic Orion</a> - An expansion module to enrich attributes in MISP and share indicators of compromise with Cytomic Orion.</li>
<li><a href="misp_modules/modules/expansion/dbl_spamhaus.py">DBL Spamhaus</a> - a hover module to check Spamhaus DBL for a domain name.</li>
<li><a href="misp_modules/modules/expansion/dns.py">DNS</a> - a simple module to resolve MISP attributes like hostname and domain to expand IP addresses attributes.</li>
<li><a href="misp_modules/modules/expansion/docx_enrich.py">docx-enrich</a> - an enrichment module to get text out of Word document into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/domaintools.py">DomainTools</a> - a hover and expansion module to get information from <a href="http://www.domaintools.com/">DomainTools</a> whois.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/backscatter_io.py">Backscatter.io</a> - a hover and expansion module to expand an IP address with mass-scanning observations.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/bgpranking.py">BGP Ranking</a> - a hover and expansion module to expand an AS number with the ASN description, its history, and position in BGP Ranking.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/btc_scam_check.py">BTC scam check</a> - An expansion hover module to instantly check if a BTC address has been abused.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/btc_steroids.py">BTC transactions</a> - An expansion hover module to get a blockchain balance and the transactions from a BTC address in MISP.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/circl_passivedns.py">CIRCL Passive DNS</a> - a hover and expansion module to expand hostname and IP addresses with passive DNS information.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/circl_passivessl.py">CIRCL Passive SSL</a> - a hover and expansion module to expand IP addresses with the X.509 certificate seen.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/countrycode.py">countrycode</a> - a hover module to tell you what country a URL belongs to.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/crowdstrike_falcon.py">CrowdStrike Falcon</a> - an expansion module to expand using CrowdStrike Falcon Intel Indicator API.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/cve.py">CVE</a> - a hover module to give more information about a vulnerability (CVE).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/cve_advanced.py">CVE advanced</a> - An expansion module to query the CIRCL CVE search API for more information about a vulnerability (CVE).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/cuckoo_submit.py">Cuckoo submit</a> - A hover module to submit malware sample, url, attachment, domain to Cuckoo Sandbox.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/dbl_spamhaus.py">DBL Spamhaus</a> - a hover module to check Spamhaus DBL for a domain name.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/dns.py">DNS</a> - a simple module to resolve MISP attributes like hostname and domain to expand IP addresses attributes.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/docx-enrich.py">docx-enrich</a> - an enrichment module to get text out of Word document into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/domaintools.py">DomainTools</a> - a hover and expansion module to get information from <a href="http://www.domaintools.com/">DomainTools</a> whois.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/eupi.py">EUPI</a> - a hover and expansion module to get information about an URL from the <a href="https://phishing-initiative.eu/?lang=en">Phishing Initiative project</a>.</li>
<li><a href="misp_modules/modules/expansion/eql.py">EQL</a> - an expansion module to generate event query language (EQL) from an attribute. <a href="https://eql.readthedocs.io/en/latest/">Event Query Language</a></li>
<li><a href="misp_modules/modules/expansion/eupi.py">EUPI</a> - a hover and expansion module to get information about an URL from the <a href="https://phishing-initiative.eu/?lang=en">Phishing Initiative project</a>.</li>
<li><a href="misp_modules/modules/expansion/farsight_passivedns.py">Farsight DNSDB Passive DNS</a> - a hover and expansion module to expand hostname and IP addresses with passive DNS information.</li>
<li><a href="misp_modules/modules/expansion/geoip_country.py">GeoIP</a> - a hover and expansion module to get GeoIP information from geolite/maxmind.</li>
<li><a href="misp_modules/modules/expansion/geoip_city.py">GeoIP_City</a> - a hover and expansion module to get GeoIP City information from geolite/maxmind.</li>
<li><a href="misp_modules/modules/expansion/geoip_asn.py">GeoIP_ASN</a> - a hover and expansion module to get GeoIP ASN information from geolite/maxmind.</li>
<li><a href="misp_modules/modules/expansion/greynoise.py">Greynoise</a> - a hover to get information from greynoise.</li>
<li><a href="misp_modules/modules/expansion/hashdd.py">hashdd</a> - a hover module to check file hashes against <a href="http://www.hashdd.com">hashdd.com</a> including NSLR dataset.</li>
<li><a href="misp_modules/modules/expansion/hibp.py">hibp</a> - a hover module to lookup against Have I Been Pwned?</li>
<li><a href="misp_modules/modules/expansion/html_to_markdown.py">html_to_markdown</a> - Simple HTML to markdown converter</li>
<li><a href="misp_modules/modules/expansion/intel471.py">intel471</a> - an expansion module to get info from <a href="https://intel471.com">Intel471</a>.</li>
<li><a href="misp_modules/modules/expansion/ipasn.py">IPASN</a> - a hover and expansion to get the BGP ASN of an IP address.</li>
<li><a href="misp_modules/modules/expansion/iprep.py">iprep</a> - an expansion module to get IP reputation from packetmail.net.</li>
<li><a href="misp_modules/modules/expansion/joesandbox_submit.py">Joe Sandbox submit</a> - Submit files and URLs to Joe Sandbox.</li>
<li><a href="misp_modules/modules/expansion/joesandbox_query.py">Joe Sandbox query</a> - Query Joe Sandbox with the link of an analysis and get the parsed data.</li>
<li><a href="misp_modules/modules/expansion/lastline_submit.py">Lastline submit</a> - Submit files and URLs to Lastline.</li>
<li><a href="misp_modules/modules/expansion/lastline_query.py">Lastline query</a> - Query Lastline with the link to an analysis and parse the report.</li>
<li><a href="misp_modules/modules/expansion/macaddress_io.py">macaddress.io</a> - a hover module to retrieve vendor details and other information regarding a given MAC address or an OUI from <a href="https://macaddress.io">MAC address Vendor Lookup</a>. See <a href="https://macaddress.io/integrations/MISP-module">integration tutorial here</a>.</li>
<li><a href="misp_modules/modules/expansion/macvendors.py">macvendors</a> - a hover module to retrieve mac vendor information.</li>
<li><a href="misp_modules/modules/expansion/malwarebazaar.py">MALWAREbazaar</a> - an expansion module to query MALWAREbazaar with some payload.</li>
<li><a href="misp_modules/modules/expansion/ocr_enrich.py">ocr-enrich</a> - an enrichment module to get OCRized data from images into MISP.</li>
<li><a href="misp_modules/modules/expansion/ods_enrich.py">ods-enrich</a> - an enrichment module to get text out of OpenOffice spreadsheet document into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/odt_enrich.py">odt-enrich</a> - an enrichment module to get text out of OpenOffice document into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/onyphe.py">onyphe</a> - a modules to process queries on Onyphe.</li>
<li><a href="misp_modules/modules/expansion/onyphe_full.py">onyphe_full</a> - a modules to process full queries on Onyphe.</li>
<li><a href="misp_modules/modules/expansion/otx.py">OTX</a> - an expansion module for <a href="https://otx.alienvault.com/">OTX</a>.</li>
<li><a href="misp_modules/modules/expansion/passivetotal.py">passivetotal</a> - a <a href="https://www.passivetotal.org/">passivetotal</a> module that queries a number of different PassiveTotal datasets.</li>
<li><a href="misp_modules/modules/expansion/pdf_enrich.py">pdf-enrich</a> - an enrichment module to extract text from PDF into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/pptx_enrich.py">pptx-enrich</a> - an enrichment module to get text out of PowerPoint document into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/qrcode.py">qrcode</a> - a module decode QR code, barcode and similar codes from an image and enrich with the decoded values.</li>
<li><a href="misp_modules/modules/expansion/rbl.py">rbl</a> - a module to get RBL (Real-Time Blackhost List) values from an attribute.</li>
<li><a href="misp_modules/modules/expansion/recordedfuture.py">recordedfuture</a> - a hover and expansion module for enriching MISP attributes with threat intelligence from Recorded Future.</li>
<li><a href="misp_modules/modules/expansion/reversedns.py">reversedns</a> - Simple Reverse DNS expansion service to resolve reverse DNS from MISP attributes.</li>
<li><a href="misp_modules/modules/expansion/securitytrails.py">securitytrails</a> - an expansion module for <a href="https://securitytrails.com/">securitytrails</a>.</li>
<li><a href="misp_modules/modules/expansion/shodan.py">shodan</a> - a minimal <a href="https://www.shodan.io/">shodan</a> expansion module.</li>
<li><a href="misp_modules/modules/expansion/sigma_queries.py">Sigma queries</a> - Experimental expansion module querying a sigma rule to convert it into all the available SIEM signatures.</li>
<li><a href="misp_modules/modules/expansion/sigma_syntax_validator.py">Sigma syntax validator</a> - Sigma syntax validator.</li>
<li><a href="misp_modules/modules/expansion/socialscan.py">Socialscan</a> - a hover module to check if an email address or a username is used on different online platforms, using the <a href="https://github.com/iojw/socialscan">socialscan</a> python library</li>
<li><a href="misp_modules/modules/expansion/sophoslabs_intelix.py">SophosLabs Intelix</a> - SophosLabs Intelix is an API for Threat Intelligence and Analysis (free tier available). <a href="https://aws.amazon.com/marketplace/pp/B07SLZPMCS">SophosLabs</a></li>
<li><a href="misp_modules/modules/expansion/sourcecache.py">sourcecache</a> - a module to cache a specific link from a MISP instance.</li>
<li><a href="misp_modules/modules/expansion/stix2_pattern_syntax_validator.py">STIX2 pattern syntax validator</a> - a module to check a STIX2 pattern syntax.</li>
<li><a href="misp_modules/modules/expansion/threatcrowd.py">ThreatCrowd</a> - an expansion module for <a href="https://www.threatcrowd.org/">ThreatCrowd</a>.</li>
<li><a href="misp_modules/modules/expansion/threatminer.py">threatminer</a> - an expansion module to expand from <a href="https://www.threatminer.org/">ThreatMiner</a>.</li>
<li><a href="misp_modules/modules/expansion/trustar_enrich.py">TruSTAR Enrich</a> - an expansion module to enrich MISP data with <a href="https://www.trustar.co/">TruSTAR</a>.</li>
<li><a href="misp_modules/modules/expansion/urlhaus.py">urlhaus</a> - Query urlhaus to get additional data about a domain, hash, hostname, ip or url.</li>
<li><a href="misp_modules/modules/expansion/urlscan.py">urlscan</a> - an expansion module to query <a href="https://urlscan.io">urlscan.io</a>.</li>
<li><a href="misp_modules/modules/expansion/virustotal.py">virustotal</a> - an expansion module to query the <a href="https://www.virustotal.com/gui/home">VirusTotal</a> API with a high request rate limit required. (More details about the API: <a href="https://developers.virustotal.com/reference">here</a>)</li>
<li><a href="misp_modules/modules/expansion/virustotal_public.py">virustotal_public</a> - an expansion module to query the <a href="https://www.virustotal.com/gui/home">VirusTotal</a> API with a public key and a low request rate limit. (More details about the API: <a href="https://developers.virustotal.com/reference">here</a>)</li>
<li><a href="misp_modules/modules/expansion/vmray_submit.py">VMray</a> - a module to submit a sample to VMray.</li>
<li><a href="misp_modules/modules/expansion/vulndb.py">VulnDB</a> - a module to query <a href="https://www.riskbasedsecurity.com/">VulnDB</a>.</li>
<li><a href="misp_modules/modules/expansion/vulners.py">Vulners</a> - an expansion module to expand information about CVEs using Vulners API.</li>
<li><a href="misp_modules/modules/expansion/whois.py">whois</a> - a module to query a local instance of <a href="https://github.com/rafiot/uwhoisd">uwhois</a>.</li>
<li><a href="misp_modules/modules/expansion/wiki.py">wikidata</a> - a <a href="https://www.wikidata.org">wikidata</a> expansion module.</li>
<li><a href="misp_modules/modules/expansion/xforceexchange.py">xforce</a> - an IBM X-Force Exchange expansion module.</li>
<li><a href="misp_modules/modules/expansion/xlsx_enrich.py">xlsx-enrich</a> - an enrichment module to get text out of an Excel document into MISP (using free-text parser).</li>
<li><a href="misp_modules/modules/expansion/yara_query.py">YARA query</a> - a module to create YARA rules from single hash attributes.</li>
<li><a href="misp_modules/modules/expansion/yara_syntax_validator.py">YARA syntax validator</a> - YARA syntax validator.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/farsight_passivedns.py">Farsight DNSDB Passive DNS</a> - a hover and expansion module to expand hostname and IP addresses with passive DNS information.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/geoip_country.py">GeoIP</a> - a hover and expansion module to get GeoIP information from geolite/maxmind.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/greynoise.py">Greynoise</a> - a hover to get information from greynoise.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/hashdd.py">hashdd</a> - a hover module to check file hashes against <a href="http://www.hashdd.com">hashdd.com</a> including NSLR dataset.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/hibp.py">hibp</a> - a hover module to lookup against Have I Been Pwned?</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/intel471.py">intel471</a> - an expansion module to get info from <a href="https://intel471.com">Intel471</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/ipasn.py">IPASN</a> - a hover and expansion to get the BGP ASN of an IP address.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/iprep.py">iprep</a> - an expansion module to get IP reputation from packetmail.net.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/joesandbox_submit.py">Joe Sandbox submit</a> - Submit files and URLs to Joe Sandbox.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/joesandbox_query.py">Joe Sandbox query</a> - Query Joe Sandbox with the link of an analysis and get the parsed data.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/macaddress_io.py">macaddress.io</a> - a hover module to retrieve vendor details and other information regarding a given MAC address or an OUI from <a href="https://macaddress.io">MAC address Vendor Lookup</a>. See <a href="https://macaddress.io/integrations/MISP-module">integration tutorial here</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/macvendors.py">macvendors</a> - a hover module to retrieve mac vendor information.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/ocr-enrich.py">ocr-enrich</a> - an enrichment module to get OCRized data from images into MISP.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/ods-enrich.py">ods-enrich</a> - an enrichment module to get text out of OpenOffice spreadsheet document into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/odt-enrich.py">odt-enrich</a> - an enrichment module to get text out of OpenOffice document into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/onyphe.py">onyphe</a> - a modules to process queries on Onyphe.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/onyphe_full.py">onyphe_full</a> - a modules to process full queries on Onyphe.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/otx.py">OTX</a> - an expansion module for <a href="https://otx.alienvault.com/">OTX</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/passivetotal.py">passivetotal</a> - a <a href="https://www.passivetotal.org/">passivetotal</a> module that queries a number of different PassiveTotal datasets.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/pdf-enrich.py">pdf-enrich</a> - an enrichment module to extract text from PDF into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/pptx-enrich.py">pptx-enrich</a> - an enrichment module to get text out of PowerPoint document into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/qrcode.py">qrcode</a> - a module decode QR code, barcode and similar codes from an image and enrich with the decoded values.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/rbl.py">rbl</a> - a module to get RBL (Real-Time Blackhost List) values from an attribute.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/reversedns.py">reversedns</a> - Simple Reverse DNS expansion service to resolve reverse DNS from MISP attributes.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/securitytrails.py">securitytrails</a> - an expansion module for <a href="https://securitytrails.com/">securitytrails</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/shodan.py">shodan</a> - a minimal <a href="https://www.shodan.io/">shodan</a> expansion module.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/sigma_queries.py">Sigma queries</a> - Experimental expansion module querying a sigma rule to convert it into all the available SIEM signatures.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/sigma_syntax_validator.py">Sigma syntax validator</a> - Sigma syntax validator.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/sourcecache.py">sourcecache</a> - a module to cache a specific link from a MISP instance.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/stix2_pattern_syntax_validator.py">STIX2 pattern syntax validator</a> - a module to check a STIX2 pattern syntax.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/threatcrowd.py">ThreatCrowd</a> - an expansion module for <a href="https://www.threatcrowd.org/">ThreatCrowd</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/threatminer.py">threatminer</a> - an expansion module to expand from <a href="https://www.threatminer.org/">ThreatMiner</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/urlhaus.py">urlhaus</a> - Query urlhaus to get additional data about a domain, hash, hostname, ip or url.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/urlscan.py">urlscan</a> - an expansion module to query <a href="https://urlscan.io">urlscan.io</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/virustotal.py">virustotal</a> - an expansion module to query the <a href="https://www.virustotal.com/gui/home">VirusTotal</a> API with a high request rate limit required. (More details about the API: <a href="https://developers.virustotal.com/reference">here</a>)</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/virustotal_public.py">virustotal_public</a> - an expansion module to query the <a href="https://www.virustotal.com/gui/home">VirusTotal</a> API with a public key and a low request rate limit. (More details about the API: <a href="https://developers.virustotal.com/reference">here</a>)</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/vmray_submit.py">VMray</a> - a module to submit a sample to VMray.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/vulndb.py">VulnDB</a> - a module to query <a href="https://www.riskbasedsecurity.com/">VulnDB</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/vulners.py">Vulners</a> - an expansion module to expand information about CVEs using Vulners API.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/whois.py">whois</a> - a module to query a local instance of <a href="https://github.com/rafiot/uwhoisd">uwhois</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/wiki.py">wikidata</a> - a <a href="https://www.wikidata.org">wikidata</a> expansion module.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/xforceexchange.py">xforce</a> - an IBM X-Force Exchange expansion module.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/xlsx-enrich.py">xlsx-enrich</a> - an enrichment module to get text out of an Excel document into MISP (using free-text parser).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/yara_query.py">YARA query</a> - a module to create YARA rules from single hash attributes.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/expansion/yara_syntax_validator.py">YARA syntax validator</a> - YARA syntax validator.</li>
</ul>
<h3 id="export-modules">Export modules<a class="headerlink" href="#export-modules" title="Permanent link">&para;</a></h3>
<ul>
<li><a href="misp_modules/modules/export_mod/cef_export.py">CEF</a> - module to export Common Event Format (CEF).</li>
<li><a href="misp_modules/modules/export_mod/cisco_firesight_manager_ACL_rule_export.py">Cisco FireSight Manager ACL rule</a> - module to export as rule for the Cisco FireSight manager ACL.</li>
<li><a href="misp_modules/modules/export_mod/goamlexport.py">GoAML export</a> - module to export in <a href="http://goaml.unodc.org/goaml/en/index.html">GoAML format</a>.</li>
<li><a href="misp_modules/modules/export_mod/liteexport.py">Lite Export</a> - module to export a lite event.</li>
<li><a href="misp_modules/modules/export_mod/pdfexport.py">PDF export</a> - module to export an event in PDF.</li>
<li><a href="misp_modules/modules/export_mod/mass_eql_export.py">Mass EQL Export</a> - module to export applicable attributes from an event to a mass EQL query.</li>
<li><a href="misp_modules/modules/export_mod/nexthinkexport.py">Nexthink query format</a> - module to export in Nexthink query format.</li>
<li><a href="misp_modules/modules/export_mod/osqueryexport.py">osquery</a> - module to export in <a href="https://osquery.io/">osquery</a> query format.</li>
<li><a href="misp_modules/modules/export_mod/threat_connect_export.py">ThreatConnect</a> - module to export in ThreatConnect CSV format.</li>
<li><a href="misp_modules/modules/export_mod/threatStream_misp_export.py">ThreatStream</a> - module to export in ThreatStream format.</li>
<li><a href="misp_modules/modules/export_mod/vt_graph.py">VirusTotal Graph</a> - Module to create a VirusTotal graph out of an event.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/cef_export.py">CEF</a> module to export Common Event Format (CEF).</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/cisco_firesight_manager_ACL_rule_export.py">Cisco FireSight Manager ACL rule</a> module to export as rule for the Cisco FireSight manager ACL.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/goamlexport.py">GoAML export</a> module to export in <a href="http://goaml.unodc.org/goaml/en/index.html">GoAML format</a>.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/liteexport.py">Lite Export</a> module to export a lite event.</li>
<li><a href="misp_modules/modules/export_mod/mass_eql_export.py">Mass EQL Export</a> module to export applicable attributes from an event to a mass EQL query.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/pdfexport.py">PDF export</a> module to export an event in PDF.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/nexthinkexport.py">Nexthink query format</a> module to export in Nexthink query format.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/osqueryexport.py">osquery</a> module to export in <a href="https://osquery.io/">osquery</a> query format.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/threat_connect_export.py">ThreatConnect</a> module to export in ThreatConnect CSV format.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/export_mod/threatStream_misp_export.py">ThreatStream</a> module to export in ThreatStream format.</li>
</ul>
<h3 id="import-modules">Import modules<a class="headerlink" href="#import-modules" title="Permanent link">&para;</a></h3>
<ul>
<li><a href="misp_modules/modules/import_mod/csvimport.py">CSV import</a> - Customizable CSV import module.</li>
<li><a href="misp_modules/modules/import_mod/cuckooimport.py">Cuckoo JSON</a> - Cuckoo JSON import.</li>
<li><a href="misp_modules/modules/import_mod/email_import.py">Email Import</a> - Email import module for MISP to import basic metadata.</li>
<li><a href="misp_modules/modules/import_mod/goamlimport.py">GoAML import</a> - Module to import <a href="http://goaml.unodc.org/goaml/en/index.html">GoAML</a> XML format.</li>
<li><a href="misp_modules/modules/import_mod/joe_import.py">Joe Sandbox import</a> - Parse data from a Joe Sandbox json report.</li>
<li><a href="misp_modules/modules/import_mod/lastline_import.py">Lastline import</a> - Module to import Lastline analysis reports.</li>
<li><a href="misp_modules/modules/import_mod/ocr.py">OCR</a> - Optical Character Recognition (OCR) module for MISP to import attributes from images, scan or faxes.</li>
<li><a href="misp_modules/modules/import_mod/openiocimport.py">OpenIOC</a> - OpenIOC import based on PyMISP library.</li>
<li><a href="misp_modules/modules/import_mod/threatanalyzer_import.py">ThreatAnalyzer</a> - An import module to process ThreatAnalyzer archive.zip/analysis.json sandbox exports.</li>
<li><a href="misp_modules/modules/import_mod/vmray_import.py">VMRay</a> - An import module to process VMRay export.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/csvimport.py">CSV import</a> Customizable CSV import module.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/cuckooimport.py">Cuckoo JSON</a> Cuckoo JSON import.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/email_import.py">Email Import</a> Email import module for MISP to import basic metadata.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/goamlimport.py">GoAML import</a> Module to import <a href="http://goaml.unodc.org/goaml/en/index.html">GoAML</a> XML format.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/joe_import.py">Joe Sandbox import</a> Parse data from a Joe Sandbox json report.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/ocr.py">OCR</a> Optical Character Recognition (OCR) module for MISP to import attributes from images, scan or faxes.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/openiocimport.py">OpenIOC</a> OpenIOC import based on PyMISP library.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/threatanalyzer_import.py">ThreatAnalyzer</a> - An import module to process ThreatAnalyzer archive.zip/analysis.json sandbox exports.</li>
<li><a href="https://github.com/MISP/misp-modules/tree/master/misp_modules/modules/import_mod/vmray_import.py">VMRay</a> - An import module to process VMRay export.</li>
</ul>
<h2 id="how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended">How to install and start MISP modules in a Python virtualenv? (recommended)<a class="headerlink" href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" title="Permanent link">&para;</a></h2>
<div class="codehilite"><pre><span></span>sudo apt-get install python3-dev python3-pip libpq5 libjpeg-dev tesseract-ocr libpoppler-cpp-dev imagemagick virtualenv libopencv-dev zbar-tools libzbar0 libzbar-dev libfuzzy-dev build-essential -y
sudo -u www-data virtualenv -p python3 /var/www/MISP/venv
<span class="nb">cd</span> /usr/local/src/
sudo chown -R www-data: .
sudo -u www-data git clone https://github.com/MISP/misp-modules.git
<span class="nb">cd</span> misp-modules
sudo -u www-data /var/www/MISP/venv/bin/pip install -I -r REQUIREMENTS
sudo -u www-data /var/www/MISP/venv/bin/pip install .
<span class="c1"># Start misp-modules as a service</span>
sudo cp etc/systemd/system/misp-modules.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl <span class="nb">enable</span> --now misp-modules
/var/www/MISP/venv/bin/misp-modules -l <span class="m">127</span>.0.0.1 -s <span class="p">&amp;</span> <span class="c1">#to start the modules</span>
</pre></div>
<h2 id="how-to-install-and-start-misp-modules-on-rhel-based-distributions">How to install and start MISP modules on RHEL-based distributions ?<a class="headerlink" href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" title="Permanent link">&para;</a></h2>
<p>As of this writing, the official RHEL repositories only contain Ruby 2.0.0 and Ruby 2.1 or higher is required. As such, this guide installs Ruby 2.2 from the <a href="https://access.redhat.com/documentation/en-us/red_hat_software_collections/3/html/3.2_release_notes/chap-installation#sect-Installation-Subscribe">SCL</a> repository. </p>
<div class="codehilite"><pre><span></span>sudo yum install rh-python36 rh-ruby22
sudo yum install openjpeg-devel
sudo yum install rubygem-rouge rubygem-asciidoctor zbar-devel opencv-devel gcc-c++ pkgconfig poppler-cpp-devel python-devel redhat-rpm-config
<span class="nb">cd</span> /var/www/MISP
git clone https://github.com/MISP/misp-modules.git
<span class="nb">cd</span> misp-modules
sudo -u apache /usr/bin/scl <span class="nb">enable</span> rh-python36 <span class="s2">&quot;virtualenv -p python3 /var/www/MISP/venv&quot;</span>
sudo -u apache /var/www/MISP/venv/bin/pip install -U -I -r REQUIREMENTS
sudo -u apache /var/www/MISP/venv/bin/pip install -U .
</pre></div>
<p>Create the service file /etc/systemd/system/misp-modules.service :
<div class="codehilite"><pre><span></span>echo &quot;[Unit]
Description=MISP&#39;s modules
After=misp-workers.service
[Service]
Type=simple
User=apache
Group=apache
ExecStart=/usr/bin/scl enable rh-python36 rh-ruby22 &#39;/var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s&#39;
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target&quot; | sudo tee /etc/systemd/system/misp-modules.service
</pre></div></p>
<p>The <code>After=misp-workers.service</code> must be changed or removed if you have not created a misp-workers service.
Then, enable the misp-modules service and start it:
<div class="codehilite"><pre><span></span>systemctl daemon-reload
systemctl <span class="nb">enable</span> --now misp-modules
</pre></div></p>
<h2 id="how-to-add-your-own-misp-modules">How to add your own MISP modules?<a class="headerlink" href="#how-to-add-your-own-misp-modules" title="Permanent link">&para;</a></h2>
<p>Create your module in <a href="misp_modules/modules/expansion/">misp_modules/modules/expansion/</a>, <a href="misp_modules/modules/export_mod/">misp_modules/modules/export_mod/</a>, or <a href="misp_modules/modules/import_mod/">misp_modules/modules/import_mod/</a>. The module should have at minimum three functions:</p>
<ul>
<li><strong>introspection</strong> function that returns a dict of the supported attributes (input and output) by your expansion module.</li>
<li><strong>handler</strong> function which accepts a JSON document to expand the values and return a dictionary of the expanded values.</li>
<li><strong>version</strong> function that returns a dict with the version and the associated meta-data including potential configurations required of the module.</li>
</ul>
<p>Don't forget to return an error key and value if an error is raised to propagate it to the MISP user-interface.</p>
<p>Your module's script name should also be added in the <code>__all__</code> list of <code>&lt;module type folder&gt;/__init__.py</code> in order for it to be loaded.</p>
<div class="codehilite"><pre><span></span><span class="o">...</span>
<span class="c1"># Checking for required value</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">request</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;ip-src&#39;</span><span class="p">):</span>
<span class="c1"># Return an error message</span>
<span class="k">return</span> <span class="p">{</span><span class="s1">&#39;error&#39;</span><span class="p">:</span> <span class="s2">&quot;A source IP is required&quot;</span><span class="p">}</span>
<span class="o">...</span>
</pre></div>
<h3 id="introspection">introspection<a class="headerlink" href="#introspection" title="Permanent link">&para;</a></h3>
<p>The function that returns a dict of the supported attributes (input and output) by your expansion module.</p>
<div class="codehilite"><pre><span></span><span class="n">mispattributes</span> <span class="o">=</span> <span class="p">{</span><span class="s1">&#39;input&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;link&#39;</span><span class="p">,</span> <span class="s1">&#39;url&#39;</span><span class="p">],</span>
<span class="s1">&#39;output&#39;</span><span class="p">:</span> <span class="p">[</span><span class="s1">&#39;attachment&#39;</span><span class="p">,</span> <span class="s1">&#39;malware-sample&#39;</span><span class="p">]}</span>
<span class="k">def</span> <span class="nf">introspection</span><span class="p">():</span>
<span class="k">return</span> <span class="n">mispattributes</span>
</pre></div>
<h3 id="version">version<a class="headerlink" href="#version" title="Permanent link">&para;</a></h3>
<p>The function that returns a dict with the version and the associated meta-data including potential configurations required of the module.</p>
<h3 id="additional-configuration-values">Additional Configuration Values<a class="headerlink" href="#additional-configuration-values" title="Permanent link">&para;</a></h3>
<p>If your module requires additional configuration (to be exposed via the MISP user-interface), you can define those in the moduleconfig value returned by the version function.</p>
<div class="codehilite"><pre><span></span><span class="c1"># config fields that your code expects from the site admin</span>
<span class="n">moduleconfig</span> <span class="o">=</span> <span class="p">[</span><span class="s2">&quot;apikey&quot;</span><span class="p">,</span> <span class="s2">&quot;event_limit&quot;</span><span class="p">]</span>
<span class="k">def</span> <span class="nf">version</span><span class="p">():</span>
<span class="n">moduleinfo</span><span class="p">[</span><span class="s1">&#39;config&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">moduleconfig</span>
<span class="k">return</span> <span class="n">moduleinfo</span>
</pre></div>
<p>When you do this a config array is added to the meta-data output containing all the potential configuration values:</p>
<div class="codehilite"><pre><span></span>&quot;meta&quot;: {
&quot;description&quot;: &quot;PassiveTotal expansion service to expand values with multiple Passive DNS sources&quot;,
&quot;config&quot;: [
&quot;username&quot;,
&quot;password&quot;
],
&quot;module-type&quot;: [
&quot;expansion&quot;,
&quot;hover&quot;
],
...
</pre></div>
<p>If you want to use the configuration values set in the web interface they are stored in the key <code>config</code> in the JSON object passed to the handler.</p>
<div class="codehilite"><pre><span></span>def handler(q=False):
# Check if we were given a configuration
config = q.get(&quot;config&quot;, {})
# Find out if there is a username field
username = config.get(&quot;username&quot;, None)
</pre></div>
<h3 id="handler">handler<a class="headerlink" href="#handler" title="Permanent link">&para;</a></h3>
<p>The function which accepts a JSON document to expand the values and return a dictionary of the expanded values.</p>
<div class="codehilite"><pre><span></span><span class="k">def</span> <span class="nf">handler</span><span class="p">(</span><span class="n">q</span><span class="o">=</span><span class="bp">False</span><span class="p">):</span>
<span class="s2">&quot;Fully functional rot-13 encoder&quot;</span>
<span class="k">if</span> <span class="n">q</span> <span class="ow">is</span> <span class="bp">False</span><span class="p">:</span>
<span class="k">return</span> <span class="bp">False</span>
<span class="n">request</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">q</span><span class="p">)</span>
<span class="n">src</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;ip-src&#39;</span><span class="p">)</span>
<span class="k">if</span> <span class="n">src</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="c1"># Return an error message</span>
<span class="k">return</span> <span class="p">{</span><span class="s1">&#39;error&#39;</span><span class="p">:</span> <span class="s2">&quot;A source IP is required&quot;</span><span class="p">}</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="p">{</span><span class="s1">&#39;results&#39;</span><span class="p">:</span>
<span class="n">codecs</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="n">src</span><span class="p">,</span> <span class="s2">&quot;rot-13&quot;</span><span class="p">)}</span>
</pre></div>
<h4 id="export-module">export module<a class="headerlink" href="#export-module" title="Permanent link">&para;</a></h4>
<p>For an export module, the <code>request["data"]</code> object corresponds to a list of events (dictionaries) to handle.</p>
<p>Iterating over events attributes is performed using their <code>Attribute</code> key.</p>
<div class="codehilite"><pre><span></span><span class="o">...</span>
<span class="k">for</span> <span class="n">event</span> <span class="ow">in</span> <span class="n">request</span><span class="p">[</span><span class="s2">&quot;data&quot;</span><span class="p">]:</span>
<span class="k">for</span> <span class="n">attribute</span> <span class="ow">in</span> <span class="n">event</span><span class="p">[</span><span class="s2">&quot;Attribute&quot;</span><span class="p">]:</span>
<span class="c1"># do stuff w/ attribute[&#39;type&#39;], attribute[&#39;value&#39;], ...</span>
<span class="o">...</span>
<span class="c1">### Returning Binary Data</span>
<span class="n">If</span> <span class="n">you</span> <span class="n">want</span> <span class="n">to</span> <span class="k">return</span> <span class="n">a</span> <span class="nb">file</span> <span class="ow">or</span> <span class="n">other</span> <span class="n">data</span> <span class="n">you</span> <span class="n">need</span> <span class="n">to</span> <span class="n">add</span> <span class="n">a</span> <span class="n">data</span> <span class="n">attribute</span><span class="o">.</span>
<span class="o">~~~</span><span class="n">python</span>
<span class="p">{</span><span class="s2">&quot;results&quot;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&quot;values&quot;</span><span class="p">:</span> <span class="s2">&quot;filename.txt&quot;</span><span class="p">,</span>
<span class="s2">&quot;types&quot;</span><span class="p">:</span> <span class="s2">&quot;attachment&quot;</span><span class="p">,</span>
<span class="s2">&quot;data&quot;</span> <span class="p">:</span> <span class="n">base64</span><span class="o">.</span><span class="n">b64encode</span><span class="p">(</span><span class="o">&lt;</span><span class="n">ByteIO</span><span class="o">&gt;</span><span class="p">)</span> <span class="c1"># base64 encode your data first</span>
<span class="s2">&quot;comment&quot;</span><span class="p">:</span> <span class="s2">&quot;This is an attachment&quot;</span><span class="p">}}</span>
</pre></div>
<p>If the binary file is malware you can use 'malware-sample' as the type. If you do this the malware sample will be automatically zipped and password protected ('infected') after being uploaded.</p>
<div class="codehilite"><pre><span></span><span class="p">{</span><span class="s2">&quot;results&quot;</span><span class="p">:</span> <span class="p">{</span><span class="s2">&quot;values&quot;</span><span class="p">:</span> <span class="s2">&quot;filename.txt&quot;</span><span class="p">,</span>
<span class="s2">&quot;types&quot;</span><span class="p">:</span> <span class="s2">&quot;malware-sample&quot;</span><span class="p">,</span>
<span class="s2">&quot;data&quot;</span> <span class="p">:</span> <span class="n">base64</span><span class="o">.</span><span class="n">b64encode</span><span class="p">(</span><span class="o">&lt;</span><span class="n">ByteIO</span><span class="o">&gt;</span><span class="p">)</span> <span class="c1"># base64 encode your data first</span>
<span class="s2">&quot;comment&quot;</span><span class="p">:</span> <span class="s2">&quot;This is an attachment&quot;</span><span class="p">}}</span>
</pre></div>
<p><a href="https://github.com/MISP/PyMISP/blob/4f230c9299ad9d2d1c851148c629b61a94f3f117/pymisp/mispevent.py#L185-L200">To learn more about how data attributes are processed you can read the processing code here.</a></p>
<h3 id="module-type">Module type<a class="headerlink" href="#module-type" title="Permanent link">&para;</a></h3>
<p>A MISP module can be of four types:</p>
<ul>
<li><strong>expansion</strong> - service related to an attribute that can be used to extend and update an existing event.</li>
<li><strong>hover</strong> - service related to an attribute to provide additional information to the users without updating the event.</li>
<li><strong>import</strong> - service related to importing and parsing an external object that can be used to extend an existing event.</li>
<li><strong>export</strong> - service related to exporting an object, event, or data.</li>
</ul>
<p>module-type is an array where the list of supported types can be added.</p>
<h2 id="testing-your-modules">Testing your modules?<a class="headerlink" href="#testing-your-modules" title="Permanent link">&para;</a></h2>
<p>MISP uses the <strong>modules</strong> function to discover the available MISP modules and their supported MISP attributes:</p>
<div class="codehilite"><pre><span></span>% curl -s http://127.0.0.1:6666/modules | jq .
[
{
&quot;name&quot;: &quot;passivetotal&quot;,
&quot;type&quot;: &quot;expansion&quot;,
&quot;mispattributes&quot;: {
&quot;input&quot;: [
&quot;hostname&quot;,
&quot;domain&quot;,
&quot;ip-src&quot;,
&quot;ip-dst&quot;
],
&quot;output&quot;: [
&quot;ip-src&quot;,
&quot;ip-dst&quot;,
&quot;hostname&quot;,
&quot;domain&quot;
]
},
&quot;meta&quot;: {
&quot;description&quot;: &quot;PassiveTotal expansion service to expand values with multiple Passive DNS sources&quot;,
&quot;config&quot;: [
&quot;username&quot;,
&quot;password&quot;
],
&quot;author&quot;: &quot;Alexandre Dulaunoy&quot;,
&quot;version&quot;: &quot;0.1&quot;
}
},
{
&quot;name&quot;: &quot;sourcecache&quot;,
&quot;type&quot;: &quot;expansion&quot;,
&quot;mispattributes&quot;: {
&quot;input&quot;: [
&quot;link&quot;
],
&quot;output&quot;: [
&quot;link&quot;
]
},
&quot;meta&quot;: {
&quot;description&quot;: &quot;Module to cache web pages of analysis reports, OSINT sources. The module returns a link of the cached page.&quot;,
&quot;author&quot;: &quot;Alexandre Dulaunoy&quot;,
&quot;version&quot;: &quot;0.1&quot;
}
},
{
&quot;name&quot;: &quot;dns&quot;,
&quot;type&quot;: &quot;expansion&quot;,
&quot;mispattributes&quot;: {
&quot;input&quot;: [
&quot;hostname&quot;,
&quot;domain&quot;
],
&quot;output&quot;: [
&quot;ip-src&quot;,
&quot;ip-dst&quot;
]
},
&quot;meta&quot;: {
&quot;description&quot;: &quot;Simple DNS expansion service to resolve IP address from MISP attributes&quot;,
&quot;author&quot;: &quot;Alexandre Dulaunoy&quot;,
&quot;version&quot;: &quot;0.1&quot;
}
}
]
</pre></div>
<p>The MISP module service returns the available modules in a JSON array containing each module name along with their supported input attributes.</p>
<p>Based on this information, a query can be built in a JSON format and saved as body.json:</p>
<div class="codehilite"><pre><span></span><span class="p">{</span>
<span class="nt">&quot;hostname&quot;</span><span class="p">:</span> <span class="s2">&quot;www.foo.be&quot;</span><span class="p">,</span>
<span class="nt">&quot;module&quot;</span><span class="p">:</span> <span class="s2">&quot;dns&quot;</span>
<span class="p">}</span>
</pre></div>
<p>Then you can POST this JSON format query towards the MISP object server:</p>
<div class="codehilite"><pre><span></span>curl -s http://127.0.0.1:6666/query -H <span class="s2">&quot;Content-Type: application/json&quot;</span> --data @body.json -X POST
</pre></div>
<p>The module should output the following JSON:</p>
<div class="codehilite"><pre><span></span><span class="p">{</span>
<span class="nt">&quot;results&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">&quot;types&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;ip-src&quot;</span><span class="p">,</span>
<span class="s2">&quot;ip-dst&quot;</span>
<span class="p">],</span>
<span class="nt">&quot;values&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;188.65.217.78&quot;</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</pre></div>
<p>It is also possible to restrict the category options of the resolved attributes by passing a list of categories along (optional):</p>
<div class="codehilite"><pre><span></span><span class="p">{</span>
<span class="nt">&quot;results&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">&quot;types&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;ip-src&quot;</span><span class="p">,</span>
<span class="s2">&quot;ip-dst&quot;</span>
<span class="p">],</span>
<span class="nt">&quot;values&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;188.65.217.78&quot;</span>
<span class="p">],</span>
<span class="nt">&quot;categories&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;Network activity&quot;</span><span class="p">,</span>
<span class="s2">&quot;Payload delivery&quot;</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</pre></div>
<p>For both the type and the category lists, the first item in the list will be the default setting on the interface.</p>
<h3 id="enable-your-module-in-the-web-interface">Enable your module in the web interface<a class="headerlink" href="#enable-your-module-in-the-web-interface" title="Permanent link">&para;</a></h3>
<p>For a module to be activated in the MISP web interface it must be enabled in the "Plugin Settings.</p>
<p>Go to "Administration &gt; Server Settings" in the top menu
- Go to "Plugin Settings" in the top "tab menu bar"
- Click on the name of the type of module you have created to expand the list of plugins to show your module.
- Find the name of your plugin's "enabled" value in the Setting Column.
"Plugin.[MODULE NAME]_enabled"
- Double click on its "Value" column</p>
<div class="codehilite"><pre><span></span>Priority Setting Value Description Error Message
Recommended Plugin.Import_ocr_enabled false Enable or disable the ocr module. Value not set.
</pre></div>
<ul>
<li>Use the drop-down to set the enabled value to 'true'</li>
</ul>
<div class="codehilite"><pre><span></span>Priority Setting Value Description Error Message
Recommended Plugin.Import_ocr_enabled true Enable or disable the ocr module. Value not set.
</pre></div>
<h3 id="set-any-other-required-settings-for-your-module">Set any other required settings for your module<a class="headerlink" href="#set-any-other-required-settings-for-your-module" title="Permanent link">&para;</a></h3>
<p>In this same menu set any other plugin settings that are required for testing.</p>
<h2 id="install-misp-module-on-an-offline-instance">Install misp-module on an offline instance.<a class="headerlink" href="#install-misp-module-on-an-offline-instance" title="Permanent link">&para;</a></h2>
<p>First, you need to grab all necessary packages for example like this :</p>
<p>Use pip wheel to create an archive
<div class="codehilite"><pre><span></span>mkdir misp-modules-offline
pip3 wheel -r REQUIREMENTS shodan --wheel-dir=./misp-modules-offline
tar -cjvf misp-module-bundeled.tar.bz2 ./misp-modules-offline/*
</pre></div>
On offline machine :
<div class="codehilite"><pre><span></span>mkdir misp-modules-bundle
tar xvf misp-module-bundeled.tar.bz2 -C misp-modules-bundle
cd misp-modules-bundle
ls -1|while read line; do sudo pip3 install --force-reinstall --ignore-installed --upgrade --no-index --no-deps ${line};done
</pre></div>
Next you can follow standard install procedure.</p>
<h2 id="how-to-contribute-your-own-module">How to contribute your own module?<a class="headerlink" href="#how-to-contribute-your-own-module" title="Permanent link">&para;</a></h2>
<p>Fork the project, add your module, test it and make a pull-request. Modules can be also private as you can add a module in your own MISP installation.</p>
<h2 id="tips-for-developers-creating-modules">Tips for developers creating modules<a class="headerlink" href="#tips-for-developers-creating-modules" title="Permanent link">&para;</a></h2>
<p>Download a pre-built virtual image from the <a href="https://www.circl.lu/services/misp-training-materials/">MISP training materials</a>.</p>
<ul>
<li>Create a Host-Only adapter in VirtualBox</li>
<li>Set your Misp OVA to that Host-Only adapter</li>
<li>Start the virtual machine</li>
<li>Get the IP address of the virtual machine</li>
<li>SSH into the machine (Login info on training page)</li>
<li>Go into the misp-modules directory</li>
</ul>
<div class="codehilite"><pre><span></span><span class="nb">cd</span> /usr/local/src/misp-modules
</pre></div>
<p>Set the git repo to your fork and checkout your development branch. If you SSH'ed in as the misp user you will have to use sudo.</p>
<div class="codehilite"><pre><span></span>sudo git remote set-url origin https://github.com/YourRepo/misp-modules.git
sudo git pull
sudo git checkout MyModBranch
</pre></div>
<p>Remove the contents of the build directory and re-install misp-modules.</p>
<div class="codehilite"><pre><span></span>sudo rm -fr build/*
sudo -u www-data /var/www/MISP/venv/bin/pip install --upgrade .
</pre></div>
<p>SSH in with a different terminal and run <code>misp-modules</code> with debugging enabled.</p>
<div class="codehilite"><pre><span></span><span class="c1"># In case misp-modules is not a service do:</span>
<span class="c1"># sudo killall misp-modules</span>
sudo systemctl disable --now misp-modules
sudo -u www-data /var/www/MISP/venv/bin/misp-modules -d
</pre></div>
<p>In your original terminal you can now run your tests manually and see any errors that arrive</p>
<div class="codehilite"><pre><span></span><span class="nb">cd</span> tests/
curl -s http://127.0.0.1:6666/query -H <span class="s2">&quot;Content-Type: application/json&quot;</span> --data @MY_TEST_FILE.json -X POST
<span class="nb">cd</span> ../
</pre></div>
<h2 id="documentation">Documentation<a class="headerlink" href="#documentation" title="Permanent link">&para;</a></h2>
<p>In order to provide documentation about some modules that require specific input / output / configuration, the <a href="doc">doc</a> directory contains detailed information about the general purpose, requirements, features, input and ouput of each of these modules:</p>
<ul>
<li>***description** - quick description of the general purpose of the module, as the one given by the moduleinfo</li>
<li><strong>requirements</strong> - special libraries needed to make the module work</li>
<li><strong>features</strong> - description of the way to use the module, with the required MISP features to make the module give the intended result</li>
<li><strong>references</strong> - link(s) giving additional information about the format concerned in the module</li>
<li><strong>input</strong> - description of the format of data used in input</li>
<li><strong>output</strong> - description of the format given as the result of the module execution</li>
</ul>
<p>Fork the project, add your module, test it and make a pull-request. Modules can be also private as you can add a module in your own MISP installation.
For further information please see <a href="contribute/">Contribute</a>.</p>
<h2 id="licenses">Licenses<a class="headerlink" href="#licenses" title="Permanent link">&para;</a></h2>
<p><a href="https://app.fossa.io/projects/git%2Bgithub.com%2FMISP%2Fmisp-modules?ref=badge_large"><img alt="FOSSA Status" src="https://app.fossa.io/api/projects/git%2Bgithub.com%MISP%2Fmisp-modules.svg?type=large" /></a></p>
<p>For further Information see also the <a href="license/">license file</a>.</p>

2
search/search_index.json
View File

File diff suppressed because one or more lines are too long

BIN
sitemap.xml.gz
View File

Binary file not shown.