Merge branch 'main' of github.com:MISP/misp-book into main

appendices/README.md

@@ -563,6 +563,7 @@ This section lists some projects we know of but not officially support and rely
 | [volatility-misp](https://github.com/CIRCL/volatility-misp) | Volatility plugin to interface with MISP | Not tested by MISP core team |
 | [misp2bro](https://github.com/thnyheim/misp2bro) | Python script that gets IOC from MISP and converts it into BRO intel files. | Not tested by MISP core team |
 | [TA-misp](https://github.com/stricaud/TA-misp) | Splunk integration with MISP | Not tested by MISP core team |
+| [Tinycheck](https://github.com/KasperskyLab/TinyCheck) | TinyCheck integration with MISP | Not tested by MISP core team |
 | [MISP QRadar](https://github.com/karthikkbala/MISP-QRadar-Integration) | The Project can used to integrate QRadar with MISP Threat Sharing Platform | Not tested by MISP core team |
 | [pymisp-suricata_search](https://github.com/raw-data/pymisp-suricata_search) | Multi-threaded suricata search module for MISP | Not tested by MISP core team |
 | [MISP-ThreatExchange](https://github.com/EC-DIGIT-CSIRC/MISP-ThreatExchange) | Script to interface MISP with Facebook ThreatExchange | Not tested by MISP core team |
@@ -666,3 +667,165 @@ Because LDAP and MISP users are paired by e-mail address, it is possible to migr
 * Currently it is not possible to log in with both LDAP and local (MISP) accounts.
 * Admins can change users email address. But when `updateUser` is set to true, when the user will log in again, the e-mail address will be updated from LDAP.
 * `Security.require_password_confirmation` setting currently does not work with LDAP authentication. But on the other hand, since user cannot change e-mail address and password, this setting is not important.
+
+# Appendix G: SimpleBackgroundJobs Migration guide
+As of **MISP** version `2.4.151` we introduced a simpler way to handle background jobs without relying in CakeResque as this library is no longer mantained.
+
+For the time being both background jobs backends will be supported, but we plan to phase out the CakeResque one in a near future.
+
+The new backend requires [Supervisor](http://supervisord.org/) and some extra PHP packages.
+
+**This guide is intended for Ubuntu/Debian systems**
+
+## Install requirements
+Run on your MISP instance the following commands.
+
+1. Install **Supervisord**:
+    ```
+    sudo apt install supervisor -y
+    ```
+
+
+2. Install required PHP packages:
+    ```
+    cd /var/www/MISP/app
+    sudo -u www-data php composer.phar require --with-all-dependencies supervisorphp/supervisor:^4.0 \
+        guzzlehttp/guzzle \
+        php-http/message  \
+        lstrojny/fxmlrpc
+
+    ```
+
+3. Add the following settings at the bottom of the **Supervisord** conf file, usually located in:
+
+    `/etc/supervisor/supervisord.conf`
+    ```
+    [inet_http_server]
+    port=127.0.0.1:9001
+    username=supervisor
+    password=PWD_CHANGE_ME
+    ```
+
+4. Use the following configuration as a template for the services, usually located in:
+
+    `/etc/supervisor/conf.d/misp-workers.conf`
+    ```
+    [group:misp-workers]
+    programs=default,email,cache,prio,update
+
+    [program:default]
+    directory=/var/www/MISP
+    command=/var/www/MISP/app/Console/cake start_worker default
+    process_name=%(program_name)s_%(process_num)02d
+    numprocs=5
+    autostart=true
+    autorestart=true
+    redirect_stderr=false
+    stderr_logfile=/var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    stdout_logfile=/var/www/MISP/app/tmp/logs/misp-workers.log
+    directory=/var/www/MISP
+    user=www-data
+
+    [program:prio]
+    directory=/var/www/MISP
+    command=/var/www/MISP/app/Console/cake start_worker prio
+    process_name=%(program_name)s_%(process_num)02d
+    numprocs=5
+    autostart=true
+    autorestart=true
+    redirect_stderr=false
+    stderr_logfile=/var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    stdout_logfile=/var/www/MISP/app/tmp/logs/misp-workers.log
+    directory=/var/www/MISP
+    user=www-data
+
+    [program:email]
+    directory=/var/www/MISP
+    command=/var/www/MISP/app/Console/cake start_worker email
+    process_name=%(program_name)s_%(process_num)02d
+    numprocs=5
+    autostart=true
+    autorestart=true
+    redirect_stderr=false
+    stderr_logfile=/var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    stdout_logfile=/var/www/MISP/app/tmp/logs/misp-workers.log
+    directory=/var/www/MISP
+    user=www-data
+
+    [program:update]
+    directory=/var/www/MISP
+    command=/var/www/MISP/app/Console/cake start_worker update
+    process_name=%(program_name)s_%(process_num)02d
+    numprocs=1
+    autostart=true
+    autorestart=true
+    redirect_stderr=false
+    stderr_logfile=/var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    stdout_logfile=/var/www/MISP/app/tmp/logs/misp-workers.log
+    directory=/var/www/MISP
+    user=www-data
+
+    [program:cache]
+    directory=/var/www/MISP
+    command=/var/www/MISP/app/Console/cake start_worker cache
+    process_name=%(program_name)s_%(process_num)02d
+    numprocs=5
+    autostart=true
+    autorestart=true
+    redirect_stderr=false
+    stderr_logfile=/var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    stdout_logfile=/var/www/MISP/app/tmp/logs/misp-workers.log
+    user=www-data
+    ```
+
+5. Restart **Supervisord** to load the changes:
+    ```
+    sudo service supervisor restart
+    ```
+
+6. Check **Supervisord** workers are running:
+    ```
+    $ sudo supervisorctl status
+    misp-workers:cache_00            RUNNING   pid 1673228, uptime 1:37:54
+    misp-workers:cache_01            RUNNING   pid 1673225, uptime 1:37:54
+    misp-workers:cache_02            RUNNING   pid 1673375, uptime 1:37:53
+    misp-workers:cache_03            RUNNING   pid 1673398, uptime 1:37:52
+    misp-workers:cache_04            RUNNING   pid 1673303, uptime 1:37:53
+    misp-workers:default_00          RUNNING   pid 1673222, uptime 1:37:54
+    misp-workers:default_01          RUNNING   pid 1673385, uptime 1:37:52
+    misp-workers:default_02          RUNNING   pid 1673391, uptime 1:37:52
+    misp-workers:default_03          RUNNING   pid 1673223, uptime 1:37:54
+    misp-workers:default_04          RUNNING   pid 1673393, uptime 1:37:52
+    misp-workers:email_00            RUNNING   pid 1673394, uptime 1:37:52
+    misp-workers:email_01            RUNNING   pid 1673312, uptime 1:37:53
+    misp-workers:email_02            RUNNING   pid 1673224, uptime 1:37:54
+    misp-workers:email_03            RUNNING   pid 1673227, uptime 1:37:54
+    misp-workers:email_04            RUNNING   pid 1673333, uptime 1:37:53
+    misp-workers:prio_00             RUNNING   pid 1673279, uptime 1:37:54
+    misp-workers:prio_01             RUNNING   pid 1673304, uptime 1:37:53
+    misp-workers:prio_02             RUNNING   pid 1673305, uptime 1:37:53
+    misp-workers:prio_03             RUNNING   pid 1673232, uptime 1:37:54
+    misp-workers:prio_04             RUNNING   pid 1673319, uptime 1:37:53
+    misp-workers:update_00           RUNNING   pid 1673327, uptime 1:37:53
+    ```
+
+## MISP Config
+1. Go to your **MISP** instances `Server Settings & Maintenance` page, and then to the new [SimpleBackgroundJobs]((https://localhost/servers/serverSettings/SimpleBackgroundJobs)) tab.
+
+2. Update the `SimpleBackgroundJobs.supervisor_password` with the password you set in the _Install requirements_ section 3.
+
+3. Verify Redis and other settings are correct and then set `SimpleBackgroundJobs.enabled` to `true`.
+
+4. Use **MISP** normally and visit [Administration -> Jobs](/jobs/index) to check Jobs are running correctly. 
+    If there are any issues check the logs:
+    * /var/www/MISP/app/tmp/logs/misp-workers-errors.log
+    * /var/www/MISP/app/tmp/logs/misp-workers.log
+
+5. Once the new workers are functioning as expected, you can remove the previous workers service:
+    ```bash
+    $ sudo systemctl stop --now misp-workers
+    $ sudo systemctl disable --now misp-workers
+    ```
+
+### Notes
+Scheduled tasks (TasksController) are not supported with the new backend, however this feature is going to be deprecated, it is recommended to use cron jobs instead.

connectors/README.md

@@ -24,7 +24,7 @@ Before installing the sample:
 ## Getting Started
 After the prerequisites are installed or met, perform the following steps to use these scripts:
 
-1. Download or clone this repository.
+1. Download or clone [this repository](https://github.com/microsoftgraph/security-api-solutions/tree/master/Samples/MISP).
 1. Go to directory `security-api-solutions/Samples/MISP`
 1. Install dependencies.  In the command line, run `pip3 install requests requests-futures pymisp` 
 1. To run script, go to the root directory of misp-graph-script and enter `PYTHONHASHSEED=0 python3 script.py` in the command line. 

managing-feeds/README.md

@@ -54,6 +54,7 @@ Here you will have access to a dynamic form. Let's check each field by order. Th
 * Enabled: Is the feed active or not
 * Caching enabled: Should the feed data be cached
 * Lookup visible: If this is not checked, correlations will only show up for you; if checked, correlations are visible for other users as well
+* Disable correlation: If this is checked, correlations will be disabled for all events coming from this Feed
 * Name: Name to identify the feed; not required to be unique
 * Provider: Name of the content provider
 * Input Source: Where does the input come from