diff --git a/automation/README.md b/automation/README.md index 8a6e529..3aea3fd 100644 --- a/automation/README.md +++ b/automation/README.md @@ -82,6 +82,77 @@ curl --header "Authorization: YOUR API KEY " --header "Accept: application/json" {"name":"Not Found","message":"Not Found","url":"\/servers\/gaaa"} ~~~~ +## Search + +It is possible to search in the database for a list of attributes or events based on a list of criterias. + +To return attributes or events in a desired format, use the following URL and header settings: + +URL: +~~~~ +YOUR_MISP_URL/attributes/restSearch +YOUR_MISP_URL/events/restSearch +~~~~ + +Headers: +~~~~ +Accept: application/json +Content-type: application/json +Authorization: YOUR_API_KEY +~~~~ + +The next feature to take care of then is the body of the query. This is where you are going to put your filters. +As an example, if we want to export all the IP addresses that have a TLP marking and not marked as TLP:red, you can find below the corresponding filters to use: +~~~~json +{ + "returnFormat": "json", + "type": { + "OR": [ + "ip-src", + "ip-dst" + ] + }, + "tags": { + "NOT": [ + "tlp:red" + ], + "OR": [ + "tlp:%" + ] + } +} +~~~~ + +Find below a non exhaustive list of parameters that can be used to filter data in your search (some parameters specific to given export formats are not mentioned): +- **returnFormat**: Set the return format of the search (Currently supported: json, xml, openioc, suricata, snort - more formats are being moved to restSearch with the goal being that all searches happen through this API). Can be passed as the first parameter after restSearch or via the JSON payload. +- **limit**: Limit the number of results returned, depending on the scope (for example 10 attributes or 10 full events). +- **page**: If a limit is set, sets the page to be returned. page 3, limit 100 will return records 201->300). +- **value**: Search for the given value in the attributes' value field. +- **type**: The attribute type, any valid MISP attribute type is accepted. +- **category**: The attribute category, any valid MISP attribute category is accepted. +- **org**: Search by the creator organisation by supplying the organisation identifier. +- **tags**: To include a tag in the results just write its names into this parameter. To exclude a tag prepend it with a '!'. +- **quickfilter**: Enabling this (by passing "1" as the argument) will make the search ignore all of the other arguments, except for the auth key and value. MISP will return an xml / json (depending on the header sent) of all events that have a sub-string match on value in the event info, event orgc, or any of the attribute value1 / value2 fields, or in the attribute comment. +- **from**: Events with the date set to a date after the one specified in the from field (format: 2015-02-15). This filter will use the date of the event. +- **to**: Events with the date set to a date before the one specified in the to field (format: 2015-02-15). This filter will use the date of the event. +- **eventid**: The events that should be included / excluded from the search +- **withAttachments**: If set, encodes the attachments / zipped malware samples as base64 in the data field within each attribute +- **metadata**: Only the metadata (event, tags, relations) is returned, attributes and proposals are omitted. +- **uuid**: Restrict the results by uuid. +- **publish_timestamp**: Restrict the results by the timestamp of the last publishing of the event. The input can be a timetamp or a short-hand time description (7d or 24h for example). You can also pass a list with two values to set a time range (for example ["14d", "7d"]). +- **last**: (Deprecated synonym for publish_timestamp) Restrict the results by the timestamp of the last publishing of the event. The input can be a timetamp or a short-hand time description (7d or 24h for example). You can also pass a list with two values to set a time range (for example ["14d", "7d"]). +- **timestamp**: Restrict the results by the timestamp (last edit). Any event with a timestamp newer than the given timestamp will be returned. In case you are dealing with /attributes as scope, the attribute's timestamp will be used for the lookup. The input can be a timetamp or a short-hand time description (7d or 24h for example). You can also pass a list with two values to set a time range (for example ["14d", "7d"]). +- **published**: Set whether published or unpublished events should be returned. Do not set the parameter if you want both. +- **enforceWarninglist**: Remove any attributes from the result that would cause a hit on a warninglist entry. +- **to_ids**: By default (0) all attributes are returned that match the other filter parameters, irregardless of their to_ids setting. To restrict the returned data set to to_ids only attributes set this parameter to 1. You can only use the special "exclude" setting to only return attributes that have the to_ids flag disabled. +- **deleted**: If this parameter is set to 1, it will return soft-deleted attributes along with active ones. By using "only" as a parameter it will limit the returned data set to soft-deleted data only. +- **includeEventUuid**: Instead of just including the event ID, also include the event UUID in each of the attributes. +- **event_timestamp**: Only return attributes from events that have received a modification after the given timestamp. The input can be a timetamp or a short-hand time description (7d or 24h for example). You can also pass a list with two values to set a time range (for example ["14d", "7d"]). +- **sgReferenceOnly**: If this flag is set, sharing group objects will not be included, instead only the sharing group ID is set. +- **eventinfo**: Filter on the event's info field. +- **searchall**: Search for a full or a substring (delimited by % for substrings) in the event info, event tags, attribute tags, attribute values or attribute comment fields. +- **attackGalaxy**: Select the ATT&CK matrix like galaxy to use when using returnFormat = attack. Defaults to the Mitre ATT&CK library via mitre-attack-pattern. + ## Events management ### /events diff --git a/faq/README.md b/faq/README.md index f841f31..6daa5fe 100644 --- a/faq/README.md +++ b/faq/README.md @@ -902,6 +902,120 @@ Created symlink from /etc/systemd/system/multi-user.target.wants/php73-php-fpm.s A galaxy can be assigned like a tag. You can use the add tag function and copy the full conntector-tag. Example `misp-galaxy:ransomware=“Locky”`, which can be found in `/galaxy_clusters/view/` +## Updating PHP from 7.2 to 7.4.5 on Ubuntu 18.04 + +### Installation + +1. Disable and Uninstall Currently Installed SSDEEP +```bash +sudo phpdismod ssdeep +sudo pecl uninstall ssdeep +sudo apt purge ssdeep +sudo rm -rf /etc/php/7.2/mods-available/ssdeep.ini +``` + +2. Install PHP 7.4.5 +```bash +sudo apt install software-properties-common -qy +sudo add-apt-repository ppa:ondrej/php -y +sudo apt update +sudo apt install -qy \ + libapache2-mod-php7.4 \ + php7.4 \ + php7.4-cli \ + php7.4-dev \ + php7.4-json \ + php7.4-xml \ + php7.4-mysql \ + php7.4-opcache \ + php7.4-readline \ + php7.4-mbstring \ + php-redis \ + php-gnupg \ + php-gd +sudo apt update +sudo apt upgrade -y +``` + +3. Install SSDEEP +```bash +cd /usr/local/src +sudo rm -rf ssdeep-2.14.1.tar.gz ssdeep-2.14.1 +sudo wget https://github.com/ssdeep-project/ssdeep/releases/download/release-2.14.1/ssdeep-2.14.1.tar.gz +sudo tar zxvf ssdeep-2.14.1.tar.gz +cd ssdeep-2.14.1 +sudo ./configure --datadir=/usr --prefix=/usr --localstatedir=/var --sysconfdir=/etc +sudo make +sudo make install +``` + +4. Test SSDEEP +```bash +ssdeep -h +``` + +5. Install ssdeep_php +```bash + sudo pecl channel-update pecl.php.net + sudo pecl install ssdeep +``` + +6. Enable SSDEEP in both 7.2 and 7.4 (** as root** `sudo su`) +```bash +echo 'extension=ssdeep.so' > /etc/php/7.2/mods-available/ssdeep.ini +echo 'extension=ssdeep.so' > /etc/php/7.4/mods-available/ssdeep.ini +``` + +7. Enable SSDEEP PHP Mod +```bash +sudo phpenmod ssdeep +``` + +8. Set PHP 7.4.5 to default PHP +```bash +sudo a2dismod php7.2 +sudo a2enmod php7.4 +sudo update-alternatives --set php /usr/bin/php7.4 +``` + +9. [Optional] Set better values for defaults +```bash +sudo sed -i "s/max_execution_time = 30/max_execution_time = 300/" /etc/php/7.4/apache2/php.ini ; \ +sudo sed -i "s/memory_limit = 128M/memory_limit = 2048M/" /etc/php/7.4/apache2/php.ini ; \ +sudo sed -i "s/upload_max_filesize = 2M/upload_max_filesize = 500M/" /etc/php/7.4/apache2/php.ini ; \ +sudo sed -i "s/post_max_size = 8M/post_max_size = 500M/" /etc/php/7.4/apache2/php.ini ; \ +sudo sed -i "s/max_execution_time = 30/max_execution_time = 300/" /etc/php/7.4/cli/php.ini ; \ +sudo sed -i "s/upload_max_filesize = 2M/upload_max_filesize = 500M/" /etc/php/7.4/cli/php.ini ; \ +sudo sed -i "s/post_max_size = 8M/post_max_size = 5000M/" /etc/php/7.4/cli/php.ini ; +``` + +10. Restart Apache to implement changes +```bash +sudo sudo systemctl restart apache2 +``` + +### Verification of php 7.2 to 7.4 + +1. **Administration** > **Server Settings & Maintenance** + +2. **Diagnostics** + +3. Scroll down to the **PHP Settings** section and verify + + +### What are the required steps after a MISP installation to have a properly running instance? + +- First login with the installation credentials and change the password immediatly (especially if your instance is publicly accessible) +- Set the base_url to the hostname of your machine (apache virtualhost name) +- Create a new organisation which will be the host organisation running the MISP instance +- Set the new organisation in `MISP.host_org_id` to replace the default one +- Set messages like `MISP.footermidleft` and alike to a proper message to help your users +- Create a new user as `admin` role with the new organisation +- Log with the new user, if successful, remove the default user used during the installation such as `admin@admin.test` +- Select and enable required taxonomies for your sharing community +- Select and enable the external feeds (as caching only if you don't want full events but you can get the full feeds too) +- Select and enable the warning-list (if you don't know what to enable, select all) +- Add the remote MISP instances where you have access to (either caching only or full pull if you want the complete events)