MISP
/
misp-modules
mirror of https://github.com/MISP/misp-modules
2
0
Fork 0
Code Issues Releases Wiki Activity

Deployed feeeadd with MkDocs version: 1.6.0

gh-pages
Alexandre Dulaunoy 2024-08-19 18:30:41 +02:00
parent 370c05c271
commit c68c0ba3bd
5 changed files with 237 additions and 265 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
expansion/index.html
View File

@ -361,15 +361,6 @@
</span> </span>
</a> </a>
</li>
<li class="md-nav__item">
<a href="#bgp-ranking" class="md-nav__link">
<span class="md-ellipsis">
BGP Ranking
</span>
</a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
@ -1518,15 +1509,6 @@
</span> </span>
</a> </a>
</li>
<li class="md-nav__item">
<a href="#bgp-ranking" class="md-nav__link">
<span class="md-ellipsis">
BGP Ranking
</span>
</a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
@ -2717,42 +2699,6 @@
</li> </li>
</ul> </ul>
<hr /> <hr />
<h4 id="bgp-ranking"><a href="https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/bgpranking.py">BGP Ranking</a><a class="headerlink" href="#bgp-ranking" title="Permanent link">&para;</a></h4>
<p>Query BGP Ranking to get the ranking of an Autonomous System number.
[<a href="https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/bgpranking.py">source code</a>]</p>
<ul>
<li>
<p><strong>features</strong>:</p>
<blockquote>
<p>The module takes an AS number attribute as input and displays its description as well as its ranking position in BGP Ranking for a given day.</p>
</blockquote>
</li>
<li>
<p><strong>input</strong>:</p>
<blockquote>
<p>Autonomous system number.</p>
</blockquote>
</li>
<li>
<p><strong>output</strong>:</p>
<blockquote>
<p>An asn object with its related bgp-ranking object.</p>
</blockquote>
</li>
<li>
<p><strong>references</strong>:</p>
<blockquote>
<p><a href="https://github.com/D4-project/BGP-Ranking/">https://github.com/D4-project/BGP-Ranking/</a></p>
</blockquote>
</li>
<li>
<p><strong>requirements</strong>:</p>
<blockquote>
<p>pybgpranking python library</p>
</blockquote>
</li>
</ul>
<hr />
<h4 id="btc-scam-check"><a href="https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/btc_scam_check.py">BTC Scam Check</a><a class="headerlink" href="#btc-scam-check" title="Permanent link">&para;</a></h4> <h4 id="btc-scam-check"><a href="https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/btc_scam_check.py">BTC Scam Check</a><a class="headerlink" href="#btc-scam-check" title="Permanent link">&para;</a></h4>
<p><img src=../logos/bitcoin.png height=60></p> <p><img src=../logos/bitcoin.png height=60></p>
<p>An expansion hover module to query a special dns blacklist to check if a bitcoin address has been abused. <p>An expansion hover module to query a special dns blacklist to check if a bitcoin address has been abused.
@ -6718,7 +6664,7 @@ It is also possible to filter results on 1 db_source by configuring db_source_fi
<li> <li>
<p><strong>references</strong>:</p> <p><strong>references</strong>:</p>
<blockquote> <blockquote>
<p><a href="https://github.com/rafiot/uwhoisd">https://github.com/rafiot/uwhoisd</a></p> <p><a href="https://github.com/Lookyloo/uwhoisd">https://github.com/Lookyloo/uwhoisd</a></p>
</blockquote> </blockquote>
</li> </li>
<li> <li>

1
index.html
View File

@ -669,7 +669,6 @@ without modifying core components. The API is available via a simple REST API wh
<li><a href="https://misp.github.io/misp-modules/expansion/#assemblyline-query">AssemblyLine Query</a> - A module tu query the AssemblyLine API with a submission ID to get the submission report and parse it.</li> <li><a href="https://misp.github.io/misp-modules/expansion/#assemblyline-query">AssemblyLine Query</a> - A module tu query the AssemblyLine API with a submission ID to get the submission report and parse it.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#assemblyline-submit">AssemblyLine Submit</a> - A module to submit samples and URLs to AssemblyLine for advanced analysis, and return the link of the submission.</li> <li><a href="https://misp.github.io/misp-modules/expansion/#assemblyline-submit">AssemblyLine Submit</a> - A module to submit samples and URLs to AssemblyLine for advanced analysis, and return the link of the submission.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#backscatter.io">Backscatter.io</a> - Backscatter.io module to bring mass-scanning observations into MISP.</li> <li><a href="https://misp.github.io/misp-modules/expansion/#backscatter.io">Backscatter.io</a> - Backscatter.io module to bring mass-scanning observations into MISP.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#bgp-ranking">BGP Ranking</a> - Query BGP Ranking to get the ranking of an Autonomous System number.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#btc-scam-check">BTC Scam Check</a> - An expansion hover module to query a special dns blacklist to check if a bitcoin address has been abused.</li> <li><a href="https://misp.github.io/misp-modules/expansion/#btc-scam-check">BTC Scam Check</a> - An expansion hover module to query a special dns blacklist to check if a bitcoin address has been abused.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#btc-steroids">BTC Steroids</a> - An expansion hover module to get a blockchain balance from a BTC address in MISP.</li> <li><a href="https://misp.github.io/misp-modules/expansion/#btc-steroids">BTC Steroids</a> - An expansion hover module to get a blockchain balance from a BTC address in MISP.</li>
<li><a href="https://misp.github.io/misp-modules/expansion/#censys-enrich">Censys Enrich</a> - An expansion module to enrich attributes in MISP by quering the censys.io API</li> <li><a href="https://misp.github.io/misp-modules/expansion/#censys-enrich">Censys Enrich</a> - An expansion module to enrich attributes in MISP by quering the censys.io API</li>

425
install/index.html
View File

@ -76,7 +76,7 @@
<div data-md-component="skip"> <div data-md-component="skip">
<a href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" class="md-skip"> <a href="#install-from-pip" class="md-skip">
Skip to content Skip to content
</a> </a>
@ -378,55 +378,110 @@
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix> <ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" class="md-nav__link"> <a href="#install-from-pip" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to install and start MISP modules (in a Python virtualenv)? (recommended) Install from pip
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" class="md-nav__link"> <a href="#install-from-cloned-repository" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to install and start MISP modules on RHEL-based distributions ? Install from cloned repository
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-use-an-misp-modules-docker-container" class="md-nav__link"> <a href="#install-the-systemd-unit" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to use an MISP modules Docker container Install the systemd unit
</span> </span>
</a> </a>
<nav class="md-nav" aria-label="How to use an MISP modules Docker container"> </li>
<li class="md-nav__item">
<a href="#run-the-tests" class="md-nav__link">
<span class="md-ellipsis">
Run the tests
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#build-the-documentation" class="md-nav__link">
<span class="md-ellipsis">
Build the documentation
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#run-misp-modules" class="md-nav__link">
<span class="md-ellipsis">
Run MISP modules
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#run-misp-modules-in-docker" class="md-nav__link">
<span class="md-ellipsis">
Run MISP modules in Docker
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#install-misp-module-on-an-offline-instance" class="md-nav__link">
<span class="md-ellipsis">
Install misp-module on an offline instance
</span>
</a>
<nav class="md-nav" aria-label="Install misp-module on an offline instance">
<ul class="md-nav__list"> <ul class="md-nav__list">
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-build" class="md-nav__link"> <a href="#if-misp-modules-is-available-on-pypi" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker build If misp-modules is available on PyPI
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-run" class="md-nav__link"> <a href="#if-misp-modules-is-not-available-on-pypi" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker run If misp-modules is not available on PyPI
</span>
</a>
<nav class="md-nav" aria-label="If misp-modules is not available on PyPI">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#using-poetry-bundle" class="md-nav__link">
<span class="md-ellipsis">
Using poetry bundle
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-compose" class="md-nav__link"> <a href="#using-poetry-export" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker-compose Using poetry export
</span> </span>
</a> </a>
@ -437,12 +492,8 @@
</li> </li>
<li class="md-nav__item"> </ul>
<a href="#install-misp-module-on-an-offline-instance" class="md-nav__link"> </nav>
<span class="md-ellipsis">
Install misp-module on an offline instance.
</span>
</a>
</li> </li>
@ -562,55 +613,110 @@
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix> <ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" class="md-nav__link"> <a href="#install-from-pip" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to install and start MISP modules (in a Python virtualenv)? (recommended) Install from pip
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" class="md-nav__link"> <a href="#install-from-cloned-repository" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to install and start MISP modules on RHEL-based distributions ? Install from cloned repository
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#how-to-use-an-misp-modules-docker-container" class="md-nav__link"> <a href="#install-the-systemd-unit" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
How to use an MISP modules Docker container Install the systemd unit
</span> </span>
</a> </a>
<nav class="md-nav" aria-label="How to use an MISP modules Docker container"> </li>
<li class="md-nav__item">
<a href="#run-the-tests" class="md-nav__link">
<span class="md-ellipsis">
Run the tests
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#build-the-documentation" class="md-nav__link">
<span class="md-ellipsis">
Build the documentation
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#run-misp-modules" class="md-nav__link">
<span class="md-ellipsis">
Run MISP modules
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#run-misp-modules-in-docker" class="md-nav__link">
<span class="md-ellipsis">
Run MISP modules in Docker
</span>
</a>
</li>
<li class="md-nav__item">
<a href="#install-misp-module-on-an-offline-instance" class="md-nav__link">
<span class="md-ellipsis">
Install misp-module on an offline instance
</span>
</a>
<nav class="md-nav" aria-label="Install misp-module on an offline instance">
<ul class="md-nav__list"> <ul class="md-nav__list">
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-build" class="md-nav__link"> <a href="#if-misp-modules-is-available-on-pypi" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker build If misp-modules is available on PyPI
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-run" class="md-nav__link"> <a href="#if-misp-modules-is-not-available-on-pypi" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker run If misp-modules is not available on PyPI
</span>
</a>
<nav class="md-nav" aria-label="If misp-modules is not available on PyPI">
<ul class="md-nav__list">
<li class="md-nav__item">
<a href="#using-poetry-bundle" class="md-nav__link">
<span class="md-ellipsis">
Using poetry bundle
</span> </span>
</a> </a>
</li> </li>
<li class="md-nav__item"> <li class="md-nav__item">
<a href="#docker-compose" class="md-nav__link"> <a href="#using-poetry-export" class="md-nav__link">
<span class="md-ellipsis"> <span class="md-ellipsis">
Docker-compose Using poetry export
</span> </span>
</a> </a>
@ -621,12 +727,8 @@
</li> </li>
<li class="md-nav__item"> </ul>
<a href="#install-misp-module-on-an-offline-instance" class="md-nav__link"> </nav>
<span class="md-ellipsis">
Install misp-module on an offline instance.
</span>
</a>
</li> </li>
@ -650,179 +752,104 @@
<h1>Install Guides</h1> <h1>Install Guides</h1>
<h2 id="how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended">How to install and start MISP modules (in a Python virtualenv)? (recommended)<a class="headerlink" href="#how-to-install-and-start-misp-modules-in-a-python-virtualenv-recommended" title="Permanent link">&para;</a></h2> <h2 id="install-from-pip">Install from pip<a class="headerlink" href="#install-from-pip" title="Permanent link">&para;</a></h2>
<p><strong><em>Be sure to run the latest version of <code>pip</code></em></strong>. To install the latest version of pip, <code>pip install --upgrade pip</code> will do the job.</p> <p>It is strongly recommended to use a virtual environment (see here for instructions <a href="https://docs.python.org/3/tutorial/venv.html">https://docs.python.org/3/tutorial/venv.html</a>).</p>
<div class="highlight"><pre><span></span><code><span class="nv">SUDO_WWW</span><span class="o">=</span><span class="s2">&quot;sudo -u www-data&quot;</span> <p>Once the virtual environment is loaded just use the command:</p>
<div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>misp-modules
sudo<span class="w"> </span>apt-get<span class="w"> </span>install<span class="w"> </span>-y<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>python3-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>python3-pip<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>git<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libpq5<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libjpeg-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>tesseract-ocr<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libpoppler-cpp-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>imagemagick<span class="w"> </span>virtualenv<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libopencv-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>zbar-tools<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libzbar0<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libzbar-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libfuzzy-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>libcaca-dev<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>build-essential
<span class="c1"># BEGIN with virtualenv:</span>
<span class="nv">$SUDO_WWW</span><span class="w"> </span>virtualenv<span class="w"> </span>-p<span class="w"> </span>python3<span class="w"> </span>/var/www/MISP/venv
<span class="c1"># END with virtualenv</span>
<span class="nb">cd</span><span class="w"> </span>/usr/local/src/
<span class="c1"># Ideally you add your user to the staff group and make /usr/local/src group writeable, below follows an example with user misp</span>
sudo<span class="w"> </span>adduser<span class="w"> </span>misp<span class="w"> </span>staff
sudo<span class="w"> </span>chmod<span class="w"> </span><span class="m">2775</span><span class="w"> </span>/usr/local/src
sudo<span class="w"> </span>chown<span class="w"> </span>root:staff<span class="w"> </span>/usr/local/src
git<span class="w"> </span>clone<span class="w"> </span>https://github.com/MISP/misp-modules.git
git<span class="w"> </span>clone<span class="w"> </span>git://github.com/stricaud/faup.git<span class="w"> </span>faup
git<span class="w"> </span>clone<span class="w"> </span>git://github.com/stricaud/gtcaca.git<span class="w"> </span>gtcaca
<span class="c1"># Install gtcaca/faup</span>
<span class="nb">cd</span><span class="w"> </span>gtcaca
mkdir<span class="w"> </span>-p<span class="w"> </span>build
<span class="nb">cd</span><span class="w"> </span>build
cmake<span class="w"> </span>..<span class="w"> </span><span class="o">&amp;&amp;</span><span class="w"> </span>make
sudo<span class="w"> </span>make<span class="w"> </span>install
<span class="nb">cd</span><span class="w"> </span>../../faup
mkdir<span class="w"> </span>-p<span class="w"> </span>build
<span class="nb">cd</span><span class="w"> </span>build
cmake<span class="w"> </span>..<span class="w"> </span><span class="o">&amp;&amp;</span><span class="w"> </span>make
sudo<span class="w"> </span>make<span class="w"> </span>install
sudo<span class="w"> </span>ldconfig
<span class="nb">cd</span><span class="w"> </span>../../misp-modules
<span class="c1"># BEGIN with virtualenv:</span>
<span class="nv">$SUDO_WWW</span><span class="w"> </span>/var/www/MISP/venv/bin/pip<span class="w"> </span>install<span class="w"> </span>-I<span class="w"> </span>-r<span class="w"> </span>REQUIREMENTS
<span class="nv">$SUDO_WWW</span><span class="w"> </span>/var/www/MISP/venv/bin/pip<span class="w"> </span>install<span class="w"> </span>.
<span class="c1"># END with virtualenv</span>
<span class="c1"># BEGIN without virtualenv:</span>
sudo<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-I<span class="w"> </span>-r<span class="w"> </span>REQUIREMENTS
sudo<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>.
<span class="c1"># END without virtualenv</span>
<span class="c1"># Start misp-modules as a service</span>
sudo<span class="w"> </span>cp<span class="w"> </span>etc/systemd/system/misp-modules.service<span class="w"> </span>/etc/systemd/system/
sudo<span class="w"> </span>systemctl<span class="w"> </span>daemon-reload
sudo<span class="w"> </span>systemctl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>--now<span class="w"> </span>misp-modules
sudo<span class="w"> </span>service<span class="w"> </span>misp-modules<span class="w"> </span>start<span class="w"> </span><span class="c1"># or</span>
/var/www/MISP/venv/bin/misp-modules<span class="w"> </span>-l<span class="w"> </span><span class="m">127</span>.0.0.1<span class="w"> </span>-s<span class="w"> </span><span class="p">&amp;</span><span class="w"> </span><span class="c1"># to start the modules manually</span>
</code></pre></div> </code></pre></div>
<h2 id="how-to-install-and-start-misp-modules-on-rhel-based-distributions">How to install and start MISP modules on RHEL-based distributions ?<a class="headerlink" href="#how-to-install-and-start-misp-modules-on-rhel-based-distributions" title="Permanent link">&para;</a></h2> <p>Note: this install method might not yet be available.</p>
<p>As of this writing, the official RHEL repositories only contain Ruby 2.0.0 and Ruby 2.1 or higher is required. As such, this guide installs Ruby 2.2 from the <a href="https://access.redhat.com/documentation/en-us/red_hat_software_collections/3/html/3.2_release_notes/chap-installation#sect-Installation-Subscribe">SCL</a> repository.</p> <h2 id="install-from-cloned-repository">Install from cloned repository<a class="headerlink" href="#install-from-cloned-repository" title="Permanent link">&para;</a></h2>
<div class="highlight"><pre><span></span><code><span class="nv">SUDO_WWW</span><span class="o">=</span><span class="s2">&quot;sudo -u apache&quot;</span> <p>In this case the only requirement is to install <code>poetry</code>. Normally you just need to run <code>pip install poetry</code>, but see here for more alternatives <a href="https://python-poetry.org/docs/#installation">https://python-poetry.org/docs/#installation</a>.</p>
sudo<span class="w"> </span>yum<span class="w"> </span>install<span class="w"> </span><span class="se">\</span> <p>Once <code>poetry</code> is installed, you can clone the repository and install <code>misp-modules</code> as follows:</p>
<span class="w"> </span>rh-python36<span class="w"> </span><span class="se">\</span> <div class="highlight"><pre><span></span><code>git<span class="w"> </span>clone<span class="w"> </span>https://github.com/MISP/misp-modules.git<span class="w"> </span><span class="o">&amp;&amp;</span><span class="w"> </span><span class="nb">cd</span><span class="w"> </span>misp-modules
<span class="w"> </span>rh-ruby22<span class="w"> </span><span class="se">\</span> git<span class="w"> </span>submodule<span class="w"> </span>update<span class="w"> </span>--init
<span class="w"> </span>openjpeg-devel<span class="w"> </span><span class="se">\</span> poetry<span class="w"> </span>install
<span class="w"> </span>rubygem-rouge<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>rubygem-asciidoctor<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>zbar-devel<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>opencv-devel<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>gcc-c++<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>pkgconfig<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>poppler-cpp-devel<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>python-devel<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>redhat-rpm-config
<span class="nb">cd</span><span class="w"> </span>/var/www/MISP
<span class="nv">$SUDO_WWW</span><span class="w"> </span>git<span class="w"> </span>clone<span class="w"> </span>https://github.com/MISP/misp-modules.git
<span class="nb">cd</span><span class="w"> </span>misp-modules
<span class="nv">$SUDO_WWW</span><span class="w"> </span>/usr/bin/scl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>rh-python36<span class="w"> </span><span class="s2">&quot;virtualenv -p python3 /var/www/MISP/venv&quot;</span>
<span class="nv">$SUDO_WWW</span><span class="w"> </span>/var/www/MISP/venv/bin/pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>-I<span class="w"> </span>-r<span class="w"> </span>REQUIREMENTS
<span class="nv">$SUDO_WWW</span><span class="w"> </span>/var/www/MISP/venv/bin/pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>.
</code></pre></div> </code></pre></div>
<p>Create the service file /etc/systemd/system/misp-modules.service :</p> <p>Note that the dependencies will require a number of system packages installed. On Ubuntu these packages are <code>libpoppler-cpp-dev</code>, <code>libzbar0</code>, and <code>tesseract-ocr</code>. For an updated list, check the github action used to test the build inside <code>.github/workflows</code>.</p>
<div class="highlight"><pre><span></span><code><span class="nb">echo</span><span class="w"> </span><span class="s2">&quot;[Unit]</span> <h2 id="install-the-systemd-unit">Install the systemd unit<a class="headerlink" href="#install-the-systemd-unit" title="Permanent link">&para;</a></h2>
<span class="s2">Description=MISP&#39;s modules</span> <p>To run <code>misp-modules</code> as a service on a distribution based on systemd, you need to create the unit as follows and store it in a file <code>/etc/systemd/system/misp-modules.service</code>:</p>
<span class="s2">After=misp-workers.service</span> <div class="highlight"><pre><span></span><code><span class="o">[</span>Unit<span class="o">]</span>
<span class="nv">Description</span><span class="o">=</span>MISP<span class="w"> </span>modules
<span class="s2">[Service]</span> <span class="o">[</span>Service<span class="o">]</span>
<span class="s2">Type=simple</span> <span class="nv">Type</span><span class="o">=</span>simple
<span class="s2">User=apache</span> <span class="nv">User</span><span class="o">=</span>apache
<span class="s2">Group=apache</span> <span class="nv">Group</span><span class="o">=</span>apache
<span class="s2">ExecStart=/usr/bin/scl enable rh-python36 rh-ruby22 &#39;/var/www/MISP/venv/bin/misp-modules -l 127.0.0.1 -s&#39;</span> <span class="nv">ExecStart</span><span class="o">=</span><span class="s1">&#39;/path/to/venv/bin/misp-modules -l 127.0.0.1 -s&#39;</span>
<span class="s2">Restart=always</span> <span class="nv">Restart</span><span class="o">=</span>always
<span class="s2">RestartSec=10</span> <span class="nv">RestartSec</span><span class="o">=</span><span class="m">10</span>
<span class="s2">[Install]</span> <span class="o">[</span>Install<span class="o">]</span>
<span class="s2">WantedBy=multi-user.target&quot;</span><span class="w"> </span><span class="p">|</span><span class="w"> </span>sudo<span class="w"> </span>tee<span class="w"> </span>/etc/systemd/system/misp-modules.service <span class="nv">WantedBy</span><span class="o">=</span>multi-user.target
</code></pre></div> </code></pre></div>
<p>The <code>After=misp-workers.service</code> must be changed or removed if you have not created a misp-workers service. <p>Then, enable the misp-modules service and start it:
Then, enable the misp-modules service and start it:
<div class="highlight"><pre><span></span><code>systemctl<span class="w"> </span>daemon-reload <div class="highlight"><pre><span></span><code>systemctl<span class="w"> </span>daemon-reload
systemctl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>--now<span class="w"> </span>misp-modules systemctl<span class="w"> </span><span class="nb">enable</span><span class="w"> </span>--now<span class="w"> </span>misp-modules
</code></pre></div></p> </code></pre></div></p>
<h2 id="how-to-use-an-misp-modules-docker-container">How to use an MISP modules Docker container<a class="headerlink" href="#how-to-use-an-misp-modules-docker-container" title="Permanent link">&para;</a></h2> <h2 id="run-the-tests">Run the tests<a class="headerlink" href="#run-the-tests" title="Permanent link">&para;</a></h2>
<h3 id="docker-build">Docker build<a class="headerlink" href="#docker-build" title="Permanent link">&para;</a></h3> <p>To run tests you need to install misp-modules from the cloned repository, run the server, and then run the tests. You can do all these step with <code>poetry</code>.</p>
<div class="highlight"><pre><span></span><code>docker<span class="w"> </span>build<span class="w"> </span>-t<span class="w"> </span>misp-modules<span class="w"> </span><span class="se">\</span> <div class="highlight"><pre><span></span><code>poetry<span class="w"> </span>install
<span class="w"> </span>--build-arg<span class="w"> </span><span class="nv">BUILD_DATE</span><span class="o">=</span><span class="k">$(</span>date<span class="w"> </span>-u<span class="w"> </span>+<span class="s2">&quot;%Y-%m-%d&quot;</span><span class="k">)</span><span class="w"> </span><span class="se">\</span> poetry<span class="w"> </span>run<span class="w"> </span>misp-modules
<span class="w"> </span>docker/
</code></pre></div> </code></pre></div>
<h3 id="docker-run">Docker run<a class="headerlink" href="#docker-run" title="Permanent link">&para;</a></h3> <p>And in another terminal:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># Start Redis</span> <div class="highlight"><pre><span></span><code>poetry<span class="w"> </span>run<span class="w"> </span>pytest
docker<span class="w"> </span>run<span class="w"> </span>--rm<span class="w"> </span>-d<span class="w"> </span>--name<span class="o">=</span>misp-redis<span class="w"> </span>redis:alpine
<span class="c1"># Start MISP-modules</span>
docker<span class="w"> </span>run<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>--rm<span class="w"> </span>-d<span class="w"> </span>--name<span class="o">=</span>misp-modules<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-e<span class="w"> </span><span class="nv">REDIS_BACKEND</span><span class="o">=</span>misp-redis<span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-e<span class="w"> </span><span class="nv">REDIS_PORT</span><span class="o">=</span><span class="s2">&quot;6379&quot;</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-e<span class="w"> </span><span class="nv">REDIS_PW</span><span class="o">=</span><span class="s2">&quot;&quot;</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-e<span class="w"> </span><span class="nv">REDIS_DATABASE</span><span class="o">=</span><span class="s2">&quot;245&quot;</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>-e<span class="w"> </span><span class="nv">MISP_MODULES_DEBUG</span><span class="o">=</span><span class="s2">&quot;false&quot;</span><span class="w"> </span><span class="se">\</span>
<span class="w"> </span>dcso/misp-dockerized-misp-modules
</code></pre></div> </code></pre></div>
<h3 id="docker-compose">Docker-compose<a class="headerlink" href="#docker-compose" title="Permanent link">&para;</a></h3> <h2 id="build-the-documentation">Build the documentation<a class="headerlink" href="#build-the-documentation" title="Permanent link">&para;</a></h2>
<div class="highlight"><pre><span></span><code>services: <p>To build the documentation you can use the provided <code>Makefile</code>.
misp-modules: Inside you will find three targets:</p>
# https://hub.docker.com/r/dcso/misp-dockerized-misp-modules <ul>
image: dcso/misp-dockerized-misp-modules:3 <li>
<p><code>generate_docs</code>: install the depdendency and generate the documentation.</p>
# Local image: </li>
#image: misp-modules <li>
#build: <p><code>generate_docs</code>: build the documentation using <code>mkdocs</code>.</p>
# context: docker/ </li>
<li>
environment: <p><code>deploy</code>: deploy the documentation using <code>mkdocs gh-deploy</code>.</p>
# Redis </li>
REDIS_BACKEND: misp-redis <li>
REDIS_PORT: &quot;6379&quot; <p><code>test-docs</code>: run a local server exposing the newly built documentation.</p>
REDIS_DATABASE: &quot;245&quot; </li>
# System PROXY (OPTIONAL) </ul>
http_proxy: <p>Note: you can either run the targets using <code>poetry</code> (default), or using the Docker image <code>squidfunk/mkdocs-material</code> by setting the environment variable <code>USE_DOCKER=true</code>.</p>
https_proxy: <h2 id="run-misp-modules">Run MISP modules<a class="headerlink" href="#run-misp-modules" title="Permanent link">&para;</a></h2>
no_proxy: 0.0.0.0 <p>If you installed it using pip, you just need to execute the command <code>misp-modules</code> (source the virtual environment a second time to update the search paths). If you installed it from the cloned repository, just use poetry, i.e., <code>poetry run misp-modules</code>.</p>
# Timezone (OPTIONAL) <h2 id="run-misp-modules-in-docker">Run MISP modules in Docker<a class="headerlink" href="#run-misp-modules-in-docker" title="Permanent link">&para;</a></h2>
TZ: Europe/Berlin <p>You can find an up-to-date container image and related documentation at the following repository: <a href="https://github.com/MISP/misp-docker">https://github.com/MISP/misp-docker</a> .</p>
# MISP-Modules (OPTIONAL) <h2 id="install-misp-module-on-an-offline-instance">Install misp-module on an offline instance<a class="headerlink" href="#install-misp-module-on-an-offline-instance" title="Permanent link">&para;</a></h2>
MISP_MODULES_DEBUG: &quot;false&quot; <h3 id="if-misp-modules-is-available-on-pypi">If <code>misp-modules</code> is available on PyPI<a class="headerlink" href="#if-misp-modules-is-available-on-pypi" title="Permanent link">&para;</a></h3>
# Logging options (OPTIONAL) <p>Once <code>misp-modules</code> is available on PyPI, you can just download all the necessary packages:</p>
LOG_SYSLOG_ENABLED: &quot;no&quot; <div class="highlight"><pre><span></span><code>mkdir<span class="w"> </span>wheels
misp-redis: pip<span class="w"> </span>wheel<span class="w"> </span>misp-modules<span class="w"> </span>--no-cache-dir<span class="w"> </span>-w<span class="w"> </span>./wheels
# https://hub.docker.com/_/redis or alternative https://hub.docker.com/r/dcso/misp-dockerized-redis/
image: redis:alpine
</code></pre></div> </code></pre></div>
<h2 id="install-misp-module-on-an-offline-instance">Install misp-module on an offline instance.<a class="headerlink" href="#install-misp-module-on-an-offline-instance" title="Permanent link">&para;</a></h2> <p>Move the <code>wheels</code> directory to the target system, and install them there:</p>
<p>First, you need to grab all necessary packages for example like this :</p> <div class="highlight"><pre><span></span><code>pip<span class="w"> </span>install<span class="w"> </span>--no-cache-dir<span class="w"> </span>--use-deprecated<span class="o">=</span>legacy-resolver<span class="w"> </span>/wheels/*.whl
<p>Use pip wheel to create an archive
<div class="highlight"><pre><span></span><code>mkdir misp-modules-offline
pip3 wheel -r REQUIREMENTS shodan --wheel-dir=./misp-modules-offline
tar -cjvf misp-module-bundeled.tar.bz2 ./misp-modules-offline/*
</code></pre></div> </code></pre></div>
On offline machine : <p>Once again, using a virtual environment is recommended.</p>
<div class="highlight"><pre><span></span><code>mkdir misp-modules-bundle <h3 id="if-misp-modules-is-not-available-on-pypi">If <code>misp-modules</code> is not available on PyPI<a class="headerlink" href="#if-misp-modules-is-not-available-on-pypi" title="Permanent link">&para;</a></h3>
tar xvf misp-module-bundeled.tar.bz2 -C misp-modules-bundle <p>You have two choices, the first approach uses <code>poetry export</code> to export the entire virtual environment so you can copy and run it on the target system; the second one uses <code>poetry bundle</code> to export a <code>requirements.txt</code> file.</p>
cd misp-modules-bundle <h4 id="using-poetry-bundle">Using <code>poetry bundle</code><a class="headerlink" href="#using-poetry-bundle" title="Permanent link">&para;</a></h4>
ls -1|while read line; do sudo pip3 install --force-reinstall --ignore-installed --upgrade --no-index --no-deps ${line};done <p>This is quite straightforward but it assumes your target system is relatively similar (same distribution, architecture, libaries).</p>
<div class="highlight"><pre><span></span><code>poetry<span class="w"> </span>install
poetry<span class="w"> </span>self<span class="w"> </span>add<span class="w"> </span>poetry-plugin-bundle
poetry<span class="w"> </span>bundle<span class="w"> </span>venv<span class="w"> </span>/destination/path/
</code></pre></div>
<h4 id="using-poetry-export">Using <code>poetry export</code><a class="headerlink" href="#using-poetry-export" title="Permanent link">&para;</a></h4>
<p>This is a bit more convoluted and it is similar to how you would install <code>misp-modules</code> on an offline instance.</p>
<p>Just follow those instructions but replace the package <code>misp-modules</code> with <code>-r requirements.txt</code>.</p>
<p>Before doing so you need to generate the <code>requirements.txt</code> file. Due to the fact we are still supporting Python 3.8 and that Poetry still has some limitations (soon to be resolved) you need to need to replace the line <code>python = "&gt;=3.8.*,&lt;3.13"</code> inside <code>pyproject.toml</code> with your exact version (just run <code>python --version</code>).</p>
<p>The following <code>sed</code> command does everything for you.</p>
<div class="highlight"><pre><span></span><code>sed<span class="w"> </span>-i<span class="w"> </span><span class="s2">&quot;s/^python = .*/python = \&quot;</span><span class="k">$(</span>python<span class="w"> </span>-c<span class="w"> </span><span class="s1">&#39;import platform; print(platform.python_version())&#39;</span><span class="k">)</span><span class="s2">\&quot;/&quot;</span><span class="w"> </span>pyproject.toml
</code></pre></div>
<p>Then, run the following commands to generate your very own <code>requirements.txt</code>.</p>
<div class="highlight"><pre><span></span><code>poetry<span class="w"> </span>lock
poetry<span class="w"> </span>install
poetry<span class="w"> </span>self<span class="w"> </span>add<span class="w"> </span>poetry-plugin-export
poetry<span class="w"> </span><span class="nb">export</span><span class="w"> </span>--without-hashes<span class="w"> </span>-f<span class="w"> </span>requirements.txt<span class="w"> </span>-o<span class="w"> </span>requirements.txt
</code></pre></div>
<p>Note that <code>misp-modules</code> will not be part of the <code>requirements.txt</code> file and you will need to create the wheel yourself:</p>
<div class="highlight"><pre><span></span><code>poetry<span class="w"> </span>build<span class="w"> </span>--output<span class="w"> </span>./wheels
</code></pre></div> </code></pre></div>
Next you can follow standard install procedure.</p>

16
sitemap.xml
View File

@ -2,42 +2,42 @@
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url> <url>
<loc>https://www.misp-project.org/</loc> <loc>https://www.misp-project.org/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/action_mod/</loc> <loc>https://www.misp-project.org/action_mod/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/contribute/</loc> <loc>https://www.misp-project.org/contribute/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/expansion/</loc> <loc>https://www.misp-project.org/expansion/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/export_mod/</loc> <loc>https://www.misp-project.org/export_mod/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/import_mod/</loc> <loc>https://www.misp-project.org/import_mod/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/install/</loc> <loc>https://www.misp-project.org/install/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
<url> <url>
<loc>https://www.misp-project.org/license/</loc> <loc>https://www.misp-project.org/license/</loc>
<lastmod>2024-08-13</lastmod> <lastmod>2024-08-19</lastmod>
<changefreq>daily</changefreq> <changefreq>daily</changefreq>
</url> </url>
</urlset> </urlset>

BIN
sitemap.xml.gz
View File

Binary file not shown.