MISP
/
misp-book
mirror of https://github.com/MISP/misp-book
2
0
Fork 0
Code Issues Releases Wiki Activity

Merge pull request #88 from SteveClement/master 

Glossary update, Typo fixeage, Trailing spaces removal
pull/89/head
Steve Clement 2018-02-02 12:15:40 +01:00 committed by GitHub
parent 36c92790a2 c3e3c29e4c
commit fcbaf10e5e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 166 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

102
GLOSSARY.md
View File

@ -1,13 +1,103 @@
# MISP Glossary in alphabetical order
This glossary is meant as a quick lookup document in case of any need of clarification of any threat sharing, threat-intel lingo.
In case you use any CCBYSA licensed content, or other pieces that are subject to licensing, make sure to add it at the end of this glossary.
# API
MISP makes extensive use of its RESTful API (Application programming interface) both internally and provides an external API for automation, synchronisation or any other tasks requiring a machine to machine interface.
In general terms, it is a set of clearly defined methods of communication between various software components. A good [API](https://en.wikipedia.org/wiki/Application_programming_interface) makes it easier to develop a computer program by providing all the building blocks, which are then put together by the programmer. An API may be for a web-based system, operating system, database system, computer hardware or software library.
The de-facto standard for talking to MISP via an API is [PyMISP](https://github.com/MISP/PyMISP).
# RESTful
Representational state transfer ([REST](https://en.wikipedia.org/wiki/Representational_state_transfer)) or RESTful web services are a way of providing interoperability between computer systems on the Internet. REST-compliant Web services allow requesting systems to access and manipulate textual representations of Web resources using a uniform and predefined set of stateless operations. Other forms of Web services exist which expose their own arbitrary sets of operations such as WSDL and SOAP.
# Sharing groups
Sharing groups in MISP are a more granular way to create re-usable distribution lists for events/attributes that allow users to include organisations from their own instance (local organisations) as well as organisations from directly, or indirectly connected instances (external organisations).
[More about creating and managing sharing groups](https://www.circl.lu/doc/misp/using-the-system#create-and-manage-sharing-groups)
# Site admin
As an admin (not to be confused with Org Admin), you can set up new accounts for users, edit user profiles, delete them, or just have a look at all the viewers' profiles.
Site admins have access to every administrator feature for all the data located on the system including global features such as the creation and modification of user roles and instance links.
# IOC
Indicator of compromise (IOC or IoC) is an artefact observed on a network or in an operating system or information channel that could reference an intrusion or a reference to a technique used by an attacker.§
# NIDS
Network Intrusion Detection System eg. Snort, Suricata
# MISP
Malware Information Sharing Platform and Threat Sharing. Commonly known simply as MISP.
# IOC
Indicator of compromise (IOC or IoC) is an artifact observed on a network or in an operating system or information channel that could reference an intrusion or a reference to a technique used by an attacker.
# MISP feeds
MISP includes a set of public OSINT feeds in its default configuration. The feeds can be used as a source of correlations for all of your events and attributes without the need to import them directly into your system. The MISP feed system allows for fast correlation but also a for quick comparisons of the feeds against one another.
[More](http://www.misp-project.org/feeds/)
# NIDS
Network Intrusion Detection System
# MISP format
MISP formats are described in specification document based on the current implementation of MISP core and PyMISP. These specifications are available for other developers willing to develop their own tools or software supporting the [MISP format](https://github.com/MISP/misp-rfc/blob/master/misp-core-format/raw.md.txt).
# MISP Galaxy Cluster
MISP galaxy is a simple method to express a large object called cluster that can be attached to MISP events or attributes. A cluster can be composed of one or more elements. Elements are expressed as key-values. There are default vocabularies available in MISP galaxy but those can be overwritten, replaced or updated as you wish. Existing clusters and vocabularies can be used as-is or as a template. MISP distribution can be applied to each cluster to permit a limited or broader distribution scheme. The following document is generated from the machine-readable JSON describing the MISP galaxy.
[More](https://www.misp-project.org/galaxy.html)
# MISP Instance
A MISP instance is an installation of the MISP software and the connected database. All the data visible to the users is stored locally in the database and data that is shareable (based on the distribution settings) can be synchronised with other instances via the Sync actions. The instance that you are reading this manual on will be referred to as "this instance" or "your instance". The instances that your instance synchronises with will be referred to as "remote instances".
# MISP Objects
MISP objects are used in MISP (starting from version 2.4.80) system and can be used by other information sharing tool. MISP objects are in addition to MISP attributes to allow advanced combinations of attributes. The creation of these objects and their associated attributes are based on real cyber security use-cases and existing practices in information sharing. The objects are just shared like any other attributes in MISP even if the other MISP instances don’t have the template of the object. The following document is generated from the machine-readable JSON describing the MISP objects.
[More](https://www.misp-project.org/objects.html)
# MISP PGP Key
or PGP instance key is the PGP (Pretty Good Privacy) key used by the MISP instance and which is only used to sign notification.
The PGP key used in the MISP instance must *not* be used anywhere else and should not be valuable.
or GnuPG instance key is the PGP (Pretty Good Privacy) key used by the MISP instance and which is only used to sign notification.
The GnuPG key used in the MISP instance must **not** be used anywhere else and should not be valuable.
# MISP Taxonomies
[Taxonomy](https://en.wikipedia.org/wiki/Taxonomy_(general)) is the practice and science of classification. The word is also used as a count noun: a taxonomy, or taxonomic scheme, is a particular classification. The word finds its roots in the Greek language τάξις, taxis (meaning 'order', 'arrangement') and νόμος, nomos ('law' or 'science').
Taxonomies that can be used in MISP (2.4) and other information sharing tool and expressed in Machine Tags (Triple Tags). A machine tag is composed of a namespace (MUST), a predicate (MUST) and an (OPTIONAL) value. Machine tags are often called triple tag due to their format.
For more details on taxonomies and classification [the documentation](https://www.circl.lu/doc/misp-taxonomies/) on [circl.lu](https://circl.lu) is best.
# Org Admin
Organisation admins (Org Admin) are restricted to executing site-admin actions exclusively within their own organisation’s users only.
They can administer users, events and logs of their own respective organisations.
# OSINT
[Open-source intelligence](https://en.wikipedia.org/wiki/Open-source_intelligence) (OSINT) is data collected from publicly available sources to be used in an intelligence context.[1] In the intelligence community, the term "open" refers to overt, publicly available sources (as opposed to covert or clandestine sources). It is not related to open-source software or public intelligence.
OSINT under one name or another has been around for hundreds of years. With the advent of instant communications and rapid information transfer, a great deal of actionable and predictive intelligence can now be obtained from public, unclassified sources.
# Pivot path
The (branching) path taken by a user from event to event while following correlation links. This is represented by the branching graph in the event view.
# Pivoting
The act of navigating from event to event through correlation links.
# Proposals
Each event can only be directly edited by users of the original creator organisation (and site admins). However, if another organisation would like to amend an event with extra information on an event, or if they'd like to correct a mistake in an attribute, they can create a Proposal. These proposals could then be accepted by the original creator organisation. These proposals can be pulled to another server, allowing users on connected instances to propose changes which then could be accepted by the original creators on another instance (and subsequently pushed back).
# Publishing
When an event is first created by a user, it is visible to everyone on the instance based on the access rights ("Your organisation only" events will not be visible to users of other organisations), but they will not be synchronised and they won't be exportable. For this, a user with publishing permission of the organisation that created the event has to publish the event. The system will then inform all the users of the instance that are subscribing to e-mail notifications and who have access to view the published event via an e-mail.
# Pull
Pulling is the process of using the configured sync user on a remote instance to REST GET all of the accessible data (based on the distribution rights) to your instance and store it.
# Push
Pushing is the process of using a configured instance link to send an event or all accessible events (limited by the distribution rights) through the REST interface to a remote instance.
# Scheduled Tasks
Certain common tasks can be scheduled for a later execution or for regular recurring executions. These tasks currently include caching all of the export formats, pulling from all eligible instances and pushing to all eligible instances.
# Sync User
A user of a role that grants sync permissions, these users (and their authentication keys) are used to serve as the points of connection between instances. Events pushed to an instance are pushed to a sync user, who then creates the events on the remote instance. Events pulled are added by the sync user that is used to connect the remote instance to your instance. As an administrator, keep in mind that a sync user needs auth key and publish permissions, has to have undergone the mandatory password change and has to have accepted the Terms of Use in order for the sync to work. Please make sure that all of these steps are taken before attempting to push or pull.
# Synchronisation
What we call synchronisation is an exchange of data between two (or more) MISP instances through our pull and push mechanisms.
# Tagging
Users with tagging rights can assigned various dynamically created tags to events, allowing an arbitrary link between events to be created. It is possible to filter events based on these tags and they can also be used to filter events for the automation.
# Templating
Users with templating rights can create easy to fill forms that help with the event creation process.
# Attributions & Licenses
## Parts of API is derived from the Wikipedia article ["API"](https://en.wikipedia.org/wiki/Application_programming_interface),
## Parts of OSINT is derived from the Wikipedia article ["Open-source intelligence"](https://en.wikipedia.org/wiki/Open-source_intelligence), ## Parts of RESTful is derived from the Wikipedia article ["REST"](https://en.wikipedia.org/wiki/Representational_state_transfer),
## Parts of Taxonomies is derived from the Wikipedia article ["Taxonomy_(general)"](https://en.wikipedia.org/wiki/Taxonomy_(general))
## which is under the Creative Commons Attribution-Share-Alike License 3.0 (https://creativecommons.org/licenses/by-sa/3.0/).

54
administration/README.md
View File

@ -19,12 +19,12 @@
- [x] Tools
- [ ] Server Settings
- [ ] Jobs
- [ ] Scheduled Tasks
- [ ] Scheduled Tasks
- - -
### Users
As an admin, you can set up new accounts for users, edit their user profiles, delete them, or just have a look at all the viewers’ profiles. Organisation admins are restricted to executing these actions exclusively within their own organisation’s users only.
As an admin (not to be confused with Org Admin), you can set up new accounts for users, edit user profiles, delete them, or just have a look at all the viewers' profiles. Organisation admins (Org Admin) are restricted to executing these actions exclusively within their own organisation’s users only.
#### Adding a new user:
@ -32,7 +32,7 @@ To add a new user, click on the Add User button in the administration menu to th
![Fill this form out to add a new user. Keep in mind that the drop-down menu titled "Role" controls privileges the user will have.](figures/add_user.png)
* **Email:** The user's e-mail address, this will be used as his/her login name and as an address to send all automatated e-mails as well as e-mails sent by contacting the user as the reporter of an event.
* **Email:** The user's e-mail address, this will be used as his/her login name and as an address to send all automated e-mails as well as e-mails sent by contacting the user as the reporter of an event.
* **Set password:** Tick the box if you want to define a temporary user-password for the user. If you don't, you should use the action button 'reset password' in the 'List Users' view to generate one and send it by email to the user.
* **Password:** *This textbox is displayed only when 'Set password' is ticked.* A Temporary password for the user that he/she should change after the first login. Ensure that the password is at least 6 characters long, includes a digit or a special character and contains at least one upper-case and at least one lower-case character.
* **Confirm Password:** *This textbox is displayed only when 'Set password' is ticked.* This should be an exact copy of the Password field.
@ -42,7 +42,7 @@ To add a new user, click on the Add User button in the administration menu to th
* **NIDS Sid:** ID of network intrusion detection systems.
* **Sync user for:** Use this option for granting the user the right to synchronize the event between MISP server. This option is available for admin, Org Admin and Sync user role.
* **Gpgkey:** The key used to encrypt e-mails sent through the system.
* **Fetch GPG key:** Fetch GPG public key.
* **Fetch GPG key:** Fetch GnuPG public key.
* **Receive alerts when events are published:** This option will subscribe the new user to automatically generated e-mails whenever an event is published.
* **Receive alerts from "contact reporter" requests:** This option will subscribe the new user to e-mails that are generated when another user tries to get in touch with an event's reporting organisation that matches that of the new user.
* **Disable this user account:** Tick it if you want to disable this user account.
@ -56,10 +56,10 @@ To list all current users of the system, just click on List Users under the admi
* **Id:** The user's automatically assigned ID number.
* **Org:** The organisation that the user belongs to.
* **Email:** The e-mail address (and login name) of the user.
* **Authkey:** Unique authentication key of the user.
* **Authkey:** Unique authentication key of the user.
* **Autoalert:** Shows whether the user has subscribed to auto-alerts and is continuing to receive mass-emails regarding newly published events that he/she is eligible for.
* **Contactalert:** Shows whether the user has the subscription to contact reporter e-mails directed at his/her organisation is turned on or off.
* **Gpgkey:** Shows whether the user has entered a Gpgkey yet.
* **Gpgkey:** Shows whether the user has entered a GnuPG key yet.
* **Nids Sid:** Displays the currently assigned NIDS ID.
* **Termsaccepted:** This flag indicates whether the user has accepted the terms of use or not.
* **Last login:** Date of last login.
@ -109,7 +109,7 @@ To add a new organisation, click on the "Add Organisation" button in the adminis
* **Uuid:** Unique identifier. If you want to share organisation between MISP multi-instance, use the same Uuid.
* **A brief description of the organisation:** A word for describing the organisation.
* **Nationality:** A drop-down list for selecting the country of organisation.
* **Sector:** Define the sector of organisation (financial, transport, telecom...)
* **Sector:** Define the sector of organisation (financial, transport, telecom…)
* **Type of organisation:** Define the type of the organisation.
* **Contacts:** You can add some contact details for the organisation.
@ -122,21 +122,21 @@ To list all current organisation of the system, just click on List Organisations
* **Id:** The organisation's automatically assigned ID number.
* **Logo:** Picture of the organisation.
* **Name:** Name of the organisation.
* **Uuid:** Unique identifier of orgnisation. Share this Uuid when using it between MISP's multi-instance.
* **Uuid:** Unique identifier of organisation. Share this Uuid when using it between MISP's multi-instance.
* **Description:** Description of the organisation.
* **Nationality:** Country of the organisation.
* **Sector:** Sector defined for the organisation.
* **Type:** Type of organisation.
* **Contacts:** Contacts of orgnisation.
* **Contacts:** Contacts of organisation.
* **Added by:** Login of the user who added the organisation
* **Local:** Flag defined if the organisation is local or remote.
* **Actions:** There are 3 options available: edit, delete or display an organisation's information. These options are also available on the left menu when you are on the display view.
* **Edit Organisation:** Same options of create organisation's view.
![Edit organisation.](figures/edit_org.png)
![Edit organisation.](figures/edit_org.png)
* **Delete Organisation:** Use this option for deleting organisation.<br />
![Delete organisation.](figures/delete_org.png)
![Delete organisation.](figures/delete_org.png)
* **View Organisation:** Use this option to display information about the selected organisation. In this view, you can display the user belongs to this organisation and events published by organisation.
![View organisation.](figures/view_org.png)
![View organisation.](figures/view_org.png)
#### Merge organisations:
Merge Organisation menu is available only in the organisation view, under the left menu. Merge one organisation to another will transfer all users and data from one to another. On the left the organisation to merge, on the right the target one.
@ -147,7 +147,7 @@ Merge Organisation menu is available only in the organisation view, under the le
### Roles
Privileges are assigned to users by assigning them to rule groups. Rule groups use one of four options determining what they can do with events as well as four additional privilege elevation settings. These are the four options to edit the full options availble in the Roles section: Read Only, Manage My Own Events, Manage Organisation Events, Manage & Publish Organisation Events. A short description is provided below:
Privileges are assigned to users by assigning them to rule groups. Rule groups use one of four options determining what they can do with events as well as four additional privilege elevation settings. These are the four options to edit the full options available in the Roles section: Read Only, Manage My Own Events, Manage Organisation Events, Manage & Publish Organisation Events. A short description is provided below:
* **Read Only:** This allows a user to browse events that his organisation has access to, but doesn't allow any changes to be made to the database.
* **Manage My Own Events:** The second option, gives its users the rights to create, modify or delete their own events, but they cannot publish them.
@ -162,7 +162,7 @@ The extra permissions are defined below:
* **Perm Sharing Group:** Grant access to edit or create sharing groups.
* **Perm Site Admin:** Gives the user full administrator privileges, this setting is used for site admins.
* **Perm Auth:** This setting enables the authentication key of the role's users to be used for rest requests.
* **Perm Tag Editor:** Grants access to edit or create tags.
* **Perm Tag Editor:** Grants access to edit or create tags.
* **Perm Delegate:** Grant access to delegate the publication of an event to a third-party organization.
* **Perm Sync:** This setting enables the users of the role to be used as a synchronisation user. The authentication key of this user can be handed out to the administrator of a remote MISP instance to allow the synchronisation features to work.
* **Perm Regexp Access:** Allows users who have this permission enabled to edit the regular expression table. Be careful when giving out this permission, incorrect regular expressions can be very harmful (infinite loops, loss of data, etc.).
@ -182,8 +182,8 @@ By clicking on the List Roles button, you can view a list of all currently regis
* **Id:** The role's automatically assigned ID number.
* **Name:** The name of role.
* **Permission:** One of the 4 permissions: Read Only, Manage My Own Events, Manage Organization Events, Manage & Publish Organisation Events.
* **Extra Permissions flag:** Flag for each extra permissions: Admin, Site Admin, Sync Actions, Audit Actions, Auth key access, Regex Actions, Tagger, Tag Editor, Template Editor, Sharing Group Editor, Deletagions Access.
* **Permission:** One of the 4 permissions: Read Only, Manage My Own Events, Manage Organization Events, Manage & Publish Organisation Events.
* **Extra Permissions flag:** Flag for each extra permissions: Admin, Site Admin, Sync Actions, Audit Actions, Auth key access, Regex Actions, Tagger, Tag Editor, Template Editor, Sharing Group Editor, Delegations Access.
* **Action Buttons:** There are 2 options available: Edit Role or Delete it.
* **Edit Role:** Same options of create role's view.<br />
![Edit Role.](figures/edit_roles.png)
@ -214,11 +214,11 @@ Since version 2.3, MISP has a settings and diagnostics tool that allows site-adm
The settings and diagnostics tool is split up into several aspects, all accessible via the tabs on top of the tool. For any unset or incorrectly set setting, or failed diagnostic a number next to the tab name will indicate the number and severity of the issues. If the number is written with a red font, it means that the issue is critical. First, let's look at the various tabs:
* **Overview**: General overview of the current state of your MISP installation
* **MISP settings**: Basic MISP settings. This includes the way MISP handles the default settings for distribution settings, whether background jobs are enabled, etc
* **GnuPG settings**: GPG related settings.
* **GnuPG settings**: GnuPG related settings.
* **Proxy settings**: HTTP proxy related settings.
* **Security settings**: Settings controlling brute-force protection and the application's salt key.
* **Misc settings**: Settings controlling debug options, please ensure that debug is always disabled on a production system.
* **Diagnostics**: The diagnostics tool checks if all directories that MISP uses to store data are writeable by the apache user. Also, the tool checks whether the STIX libraries and GPG are working as intended.
* **Diagnostics**: The diagnostics tool checks if all directories that MISP uses to store data are writeable by the apache user. Also, the tool checks whether the STIX libraries and GnuPG are working as intended.
* **Workers**: Shows the background workers (if enabled) and shows a warning if they are not running. Admins can also restart the workers here.
* **Download report**: Download a report in JSON format, compiled of all of the settings visible in the tool.
@ -305,9 +305,9 @@ Listing all the log entries will display the following columns generated by the
* **Action:** The action's type. This can include: login/logout for users, add, edit, delete for events, attributes, users and servers.
* **Title:** The title of an event always includes the target type (Event, User, Attribute, Server), the target's ID and the target's name (for example: e-mail address for users, event description for events).
* **Change:** This field is only populated for entries with "add" or "edit" actions. The changes are detailed in the following format:
__variable (initial_value)_ => _(new_value)_,...
__variable (initial_value)_ => _(new_value)_,…
When the entry is about the creation of a new item (such as adding a new event) then the change will look like this for example:
_org()_ => _(ADMIN)_, _date()_ => _(20012-10-19)_,...
_org()_ => _(ADMIN)_, _date()_ => _(20012-10-19)_,…
![You can search logs using this form, narrow down your search by populating several fields.](figures/search_log.png)
@ -318,7 +318,7 @@ Another way to browse the logs is to search it by filtering the results accordin
* **Email:** By searching by Email, it is possible to view the log entries of a single user.
* **Org:** Searching for an organisation allows you to see all actions taken by any member of the organisation.
* **Action:** With the help of this drop down menu, you can search for various types of actions taken (such as logins, deletions, etc).
* **Title:** There are several ways in which to use this field, since the title fields contain several bits of information and the search searches for any substrings contained within the field, it is possible to just search for the ID number of a logged event, the username / server's name / event's name / attribute's name of the event target.
* **Title:** There are several ways in which to use this field, since the title fields contain several bits of information and the search searches for any substrings contained within the field, it is possible to just search for the ID number of a logged event, the username / server's name / event's name / attributes name of the event target.
* **Change:** With the help of this field, you can search for various specific changes or changes to certain variables (Ex.: using "Published" as the search term for and find all log entries where an event has been "Published", ip-src will find all attributes where a source IP address has been entered / edited, etc).
@ -368,7 +368,7 @@ Apart from off-loading long-lasting jobs to the background workers, there is a s
* **Description**: A brief description of the task.
* **Message**: This field shows when the job was queued by the scheduler for execution.
![Site administrators can schedule reccuring tasks on this page.](figures/schedule.png)
![Site administrators can schedule recurring tasks on this page.](figures/schedule.png)
### Various administration tips & tricks
@ -468,7 +468,7 @@ Increase to the following values (or more if you want to)
; is disabled through enable_post_data_reading.
; http://php.net/post-max-size
post_max_size = 256M
[...]
[…]
; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 1024M
@ -484,7 +484,7 @@ service apache2 restart
The preferred method for support & feature requests is to use the [GitHub ticketing system](https://github.com/MISP/MISP/issues).
If you want to discuss something related to MISP, want some help from the community, etc... You have
If you want to discuss something related to MISP, want some help from the community, etc… You have
the [MISP Users mailing list](https://groups.google.com/forum/#!forum/misp-users) and the [MISP developers mailing list](https://groups.google.com/forum/#!forum/misp-devel).
A number of companies offer custom development, consulting, and support around MISP, please check [the support page of the MISP Project website](http://www.misp-project.org/#support).
@ -494,7 +494,7 @@ A number of companies offer custom development, consulting, and support around M
The setting MISP.extended_alert_subject allows you to have an extended subject. One word of warning though. If you’re using encryption : the subject will not be encrypted. Be aware that you might leak some sensitive information this way. Below is an example how the two subject types look like. First with the option disabled, then with the option enabled.
```
Event 7 - Low - TLP Amber
Event 8 - OSINT - Dissecting XXX... - Low - TLP Amber
Event 8 - OSINT - Dissecting XXX… - Low - TLP Amber
```
Taken from [Koen Van Impe's blog](http://www.vanimpe.eu/2015/05/31/getting-started-misp-malware-information-sharing-platform-threat-sharing-part-3/)
@ -525,8 +525,8 @@ By default, MISP has several layers of logs that can be used to trouble-shoot an
* **Apache access logs**: Rotating logs generated by apache, logging each request, by default (on Ubuntu) they are found in /var/log/apache2/misp.local\_access.log. The location can be changed via the apache conf file
* **Apache error logs**: Rotating logs generated by apache, logging error messages, by default (on Ubuntu) they are found in /var/log/apache2/misp.local\_error.log. This error log file will generally not be used by MISP, however, if there is a PHP level error that prevents MISP from functionining you might have relevant entries here.
* **MISP error log**: Generated by MISP, logging any exceptions that occur during usage. These can be found in /var/www/MISP/app/tmp/logs/error.log (assuming default installation path). If you see errors in here and are stuck with an issue let us know via github!
* **MISP debug log**: Generated by MISP, any debug messages and Notice level messages will be sent to this file. Generally less interesting, but can be helpful during debuging sessions. It should not be necesary to monitor this under normal usage. The file can be found in /var/www/MISP/app/tmp/logs/debug.log (assuming default installation path).
* **MISP error log**: Generated by MISP, logging any exceptions that occur during usage. These can be found in /var/www/MISP/app/tmp/logs/error.log (assuming default installation path). If you see errors in here and are stuck with an issue [let us know via GitHub](https://github.com/MISP/MISP/issues/)!
* **MISP debug log**: Generated by MISP, any debug messages and Notice level messages will be sent to this file. Generally less interesting, but can be helpful during debugging sessions. It should not be necessary to monitor this under normal usage. The file can be found in /var/www/MISP/app/tmp/logs/debug.log (assuming default installation path).
* **MISP worker error log**: Generated by MISP background workers, logging any exceptions generated during a background job. It is the equivalent of the MISP error log for background jobs, so if scheduled tasks, synchronisation or e-mailing with the workers enabled are causing issues, this is the place to check. It can normally be found at /var/www/MISP/app/tmp/logs/resque-worker-error.log
* **MISP worker logs**: Rotating logs generated by MISP background workers, logging any jobs executed by workers. This is part of the normal operation of background workers and doesn't have to be monitored, though it can help when debugging issues. Normally found at /var/www/MISP/app/tmp/logs/resque-[current date].log
* **MISP scheduler error log**: Generated by MISP scheduler worker, logging any exceptions generated during the scheduling of a background job. It is the equivalent of the MISP error log for scheduled jobs. It can normally be found at /var/www/MISP/app/tmp/logs/resque-scheduler-error.log

32
automation/README.md
View File

@ -2,15 +2,15 @@
# Automation API
Automation functionality is designed to automatically generate signatures for intrusion detection systems. To enable signature generation for a given attribute, Signature field of this attribute must be set to Yes. Note that not all attribute types are applicable for signature generation, currently we only support NIDS signature generation for IP, domains, host names, user agents etc., and hash list generation for MD5/SHA1 values of file artifacts. Support for more attribute types is planned. To to make this functionality available for automated tools an authentication key is used. This makes it easier for your tools to access the data without further form-based-authentication.
Automation functionality is designed to automatically generate signatures for intrusion detection systems. To enable signature generation for a given attribute, Signature field of this attribute must be set to Yes. Note that not all attribute types are applicable for signature generation, currently we only support NIDS signature generation for IP, domains, host names, user agents etc., and hash list generation for MD5/SHA1 values of file artefacts. Support for more attribute types is planned. To to make this functionality available for automated tools an authentication key is used. This makes it easier for your tools to access the data without further form-based-authentication.
## General
### Automation URL
The documentation will include a default MISP url in the examples. Don't forget to replace it with your MISP url.
The documentation will include a default MISP URL in the examples. Don't forget to replace it with your MISP URL.
Default MISP url in the documentation:
Default MISP URL in the documentation:
~~~~
https://<misp url>/
@ -20,7 +20,7 @@ https://<misp url>/
The authentication of the automation is performed via a secure key available in the MISP UI interface. Make sure you keep that key secret as it gives access to the entire database! The API key is available in the event actions menu under automation.
Since version 2.2 the usage of the authentication key in the url is deprecated. Instead, pass the auth key in an Authorization header in the request. The legacy option of having the auth key in the url is temporarily still supported but not recommended.
Since version 2.2 the usage of the authentication key in the URL is deprecated. Instead, pass the auth key in an Authorization header in the request. The legacy option of having the auth key in the URL is temporarily still supported but not recommended.
The authorization is performed by using the following header:
@ -331,7 +331,7 @@ Usage of the API:
https://<misp url>/events/stix/download
~~~~
Search parameters can be passed to the function via url parameters or by POSTing an xml or json object (depending on the return type). The following parameters can be passed to the STIX export tool: id, withAttachments, tags. Both id and tags can use the && (and) and ! (not) operators to build queries. Using the url parameters, the syntax is as follows:
Search parameters can be passed to the function via URL parameters or by POSTing an xml or json object (depending on the return type). The following parameters can be passed to the STIX export tool: id, withAttachments, tags. Both id and tags can use the && (and) and ! (not) operators to build queries. Using the URL parameters, the syntax is as follows:
~~~~
https://<misp url>/events/stix/download/[id]/[withAttachments]/[tags]/[from]/[to]/[last]
@ -370,7 +370,7 @@ https://<misp url>/events/stix/download.json
~~~~
~~~~json
{"request": {"id":["!51","!62"],"withAttachment":false,"tags":["APT1","!OSINT"],"from":false,"to":"2015-02-15"}}
{"request": {"id":["!51","!62"],"withAttachment":false,"tags":["APT1","!OSINT"],"from":false,"to":"2015-02-15"}}
~~~~
If you use XML query objects:
@ -928,7 +928,7 @@ To query the add or edit APIs for the valid parameters, simply send a GET reques
~~~
{
"name": "\/admin\/organisations\/add API description",
"description": "POST an Organisation object in JSON format to this API to create a new organsiation.",
"description": "POST an Organisation object in JSON format to this API to create a new organisation.",
"mandatory_fields": [
"name"
],
@ -1127,7 +1127,7 @@ https://<misp-instance>/events/csv/download/<event-id>?attributes=timestamp,type
The order of columns will be honoured including those related to object level information.
To select object level columns, simply pre-pend the given object column’s name by object_, such as:
To select object level columns, simply prepend the given object column's name by object_, such as:
~~~~
https://<misp-instance>/events/csv/download/<event-id>?attributes=timestamp,type,uuid,value&object_attributes=uuid,name
@ -1152,7 +1152,7 @@ It is possible to further restrict the exported values using the following filte
<dl>
<dt>tags</dt>
<dd>To include a tag in the results just write its names into this parameter. To exclude a tag prepend it with a '!'. You can also chain several tag
commands together with the '&&' operator. Please be aware the colons (:) cannot be used in the tag search when passed through the url. Use semicolons
commands together with the '&&' operator. Please be aware the colons (:) cannot be used in the tag search when passed through the URL. Use semicolons
instead (the search will automatically search for colons instead).</dd>
<dt>id</dt>
<dd>The event's ID</dd>
@ -1178,7 +1178,7 @@ MISP will inject header values into the zone file as well as define the action t
|RPZ_ns_alt||
|RPZ_email| root.localhost|
To override the above values, either use the url parameters as described below:
To override the above values, either use the URL parameters as described below:
~~~~
https://<misp url>/attributes/rpz/download/[tags]/[eventId]/[from]/[to]/[policy]/[walled_garden]/[ns]/[ns_alt]/[email]/[serial]/[refresh]/[retry]/[expiry]/[minim
@ -1300,7 +1300,7 @@ For example, to retrieve all attributes for event #5, including non IDS marked a
~~~~
https://<misp url>/attributes/text/download/all/null/5/true
~~~~
## RESTful searches with JSON result
It is possible to search the database for attributes based on a list of criteria
@ -1311,7 +1311,7 @@ To return an event with all of its attributes, relations, shadowAttributes, use
https://<misp url>/attributes/restSearch/json/[value]/[type]/[category]/[org]/[tag]/[quickfilter]/[from]/[to]/[last]/[eventid]/[withAttachments]/[metadata]/[uuid]
~~~~
## RESTful searches with XML result export
@ -1509,7 +1509,7 @@ You can also download samples by knowing its MD5 hash. Simply pass the hash alon
You can also use this API to get all samples from events that contain the passed hash. For this functionality, just pass the "allSamples" flag along.
Note that if you are getting all samples from matching events, you can use all supported hash types (md5, sha1, sha256) for the lookup.
You can also get all the samples from an event with a given event ID, by passing along the eventID parameter. Make sure that either an event ID or a hash is passed along, otherwise an error message will be returned. Also, if no hash is set, the allSamples flag will get set automatically.
You can also get all the samples from an event with a given event ID, by passing along the eventID parameter. Make sure that either an event ID or a hash is passed along, otherwise an error message will be returned. Also, if no hash is set, the allSamples flag will get set automatically.
https://<misp url>/attributes/downloadSample/[hash]/[allSamples]/[eventID]
@ -1629,13 +1629,13 @@ Based on the API key used, the list of visible sharing groups will be returned i
The MISP feeds can be enabled via the API.
A feed can be enabled by POSTing on the following url (feed_id is the id of the feed):
A feed can be enabled by POSTing on the following URL (feed_id is the id of the feed):
~~~~
/feeds/enable/feed_id
~~~~
A feed can be disabled by POSTing on the following url (feed_id is the id of the feed):
A feed can be disabled by POSTing on the following URL (feed_id is the id of the feed):
~~~~
/feeds/disable/feed_id
@ -1699,7 +1699,7 @@ It is also possible to POST a STIX indicator with sighting data to the following
https://<misp url>/sightings/add/stix
~~~~
MISP will use the sighting's related observables to gather all values and create sightings for each attribute that matches any of the values. If no related observables are provided in the Sighting object, then MISP will fall back to the Indicator itself and use its observables' values to create the sightings. The time of the sighting is the current time, unless the timestamp attribute is set on the Sightings object, in which case that is taken.
MISP will use the sightings related observables to gather all values and create sightings for each attribute that matches any of the values. If no related observables are provided in the Sighting object, then MISP will fall back to the Indicator itself and use its observables' values to create the sightings. The time of the sighting is the current time, unless the timestamp attribute is set on the Sightings object, in which case that is taken.
An example STIX sightings document:

8
general-concepts/README.md
View File

@ -8,11 +8,11 @@ There are two types of admins in MISP: Admins (also referred to as org admins) a
### Background Jobs
A lot of the heavier tasks are a burden to users, in that their actions can cause long delays (and in some cases timeouts) while the application logic is executing. To alleviate this, long processes have been (if enabled) moved to background jobs, meaning that their execution happens asynchronously in the background, allowing the user to freely interact with the platform whilst the request is being processed.
### MISP Instance
A MISP instance is an installation of the MISP software and the connected database. All the data visible to the users is stored locally in the database and data that is shareable (based on the distribution settings) can be synchronised with other instances via the Sync actions. The instance that you are reading this manual on will be refered to as "this instance" or "your instance". The instances that your instance synchronises with will be refered to as "remote instances".
A MISP instance is an installation of the MISP software and the connected database. All the data visible to the users is stored locally in the database and data that is shareable (based on the distribution settings) can be synchronised with other instances via the Sync actions. The instance that you are reading this manual on will be referred to as "this instance" or "your instance". The instances that your instance synchronises with will be referred to as "remote instances".
### Organisation administrators and Site administrators
We have two types of administrators, site and organisation admins. The former has access to every administrator feature for all the data located on the system including global features such as the creation and modification of user roles and instance links, whilst organisation admins can administer users, events and logs of their own respective organisations.<br />
### Pivot path
The (branching) path taken by a user from event to event while following correlation links. This is represented by the branching graph in the event view.
The (branching) path taken by a user from event to event while following correlation links. This is represented by the branching graph in the event view.
### Pivoting
The act of navigating from event to event through correlation links.
### Proposals
@ -24,13 +24,13 @@ Pulling is the process of using the configured sync user on a remote instance to
### Push
Pushing is the process of using a configured instance link to send an event or all accessible events (limited by the distribution rights) through the REST interface to a remote instance.
### Scheduled Tasks
Certain common tasks can be scheduled for a later execution or for regular recurring executions. These tasks currently include caching all of the export formats, pulling from all eligible instances and pushing to all eligible instances.
Certain common tasks can be scheduled for a later execution or for regular recurring executions. These tasks currently include caching all of the export formats, pulling from all eligible instances and pushing to all eligible instances.
### Sync User
A user of a role that grants sync permissions, these users (and their authentication keys) are used to serve as the points of connection between instances. Events pushed to an instance are pushed to a sync user, who then creates the events on the remote instance. Events pulled are added by the sync user that is used to connect the remote instance to your instance. As an administrator, keep in mind that a sync user needs auth key and publish permissions, has to have undergone the mandatory password change and has to have accepted the Terms of Use in order for the sync to work. Please make sure that all of these steps are taken before attempting to push or pull.
### Synchronisation
What we call synchronisation is an exchange of data between two (or more) MISP instances through our pull and push mechanisms.
### Tagging
Users with tagging rights can assigned various dynamically created tags to events, allowing an arbitrary link between events to be created. It is possible to filter events based on these tags and they can also be used to filter events for the automation.
Users with tagging rights can assigned various dynamically created tags to events, allowing an arbitrary link between events to be created. It is possible to filter events based on these tags and they can also be used to filter events for the automation.
### Templating
Users with templating rights can create easy to fill forms that help with the event creation process.

42
pymisp/README.md
View File

@ -18,19 +18,19 @@ Note that you need to have Auth Key access in your MISP instance to use PyMISP
* Proposals: add, edit, accept, discard
* Full text search and search by attributes
* Get STIX event
* Export statistics
* Export statistics
And even more, just look at the api.py file
### Installation
You can install PyMISP by either using pip or by getting the last version from the [GitHub repository](https://github.com/MISP/PyMISP)
You can install PyMISP by either using pip or by getting the last version from the [GitHub repository](https://github.com/MISP/PyMISP)
#### Install from pip
~~~~
pip install pymisp
~~~~
#### Install the lastest version from repo
#### Install the latest version from the repository
~~~~
git clone https://github.com/MISP/PyMISP.git && cd PyMISP
python setup.py install
@ -79,7 +79,7 @@ from keys import misp_url, misp_key
import argparse
~~~~
First of all, it is obvious that we need to import PyMISP.
Then we also need to know both the instance with which we will work and the API key to use: Both should be stored in the keys.py file.
Then we also need to know both the instance with which we will work and the API key to use: Both should be stored in the keys.py file.
Finally we import argparse library so the script can handle arguments.
~~~~python
# For python2 & 3 compat, a bit dirty, but it seems to be the least bad one
@ -88,13 +88,13 @@ try:
except NameError:
pass
~~~~
Just a few lines to be sure that pyhon 2 and 3 are supported
Just a few lines to be sure that python 2 and 3 are supported
~~~~python
def init(url, key):
return PyMISP(url, key, True, 'json', debug=True)
~~~~
This function will create a PyMISP object that will be used later to interact with the MISP instance.
As seen in the [api.py](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L85), a PyMISP object need to know both the url of the MISP instance and the API key to use. It can also take additionnal and not mandatory data, such as the use or not of SSL or the name of the export format.
As seen in the [api.py](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L85), a PyMISP object need to know both the URL of the MISP instance and the API key to use. It can also take additional and not mandatory data, such as the use or not of SSL or the name of the export format.
~~~~python
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Create an event on MISP.')
@ -105,7 +105,7 @@ if __name__ == '__main__':
~~~~
Then the function starts by preparing the awaited arguments:
* event: The event that will get a new attribute
* type: The type of the attribute that will be added. See [here](../categories-and-types/README.md) for more information
* type: The type of the attribute that will be added. See [here](../categories-and-types/README.md) for more information
* value: The value of the new attribute
~~~~python
misp = init(misp_url, misp_key)
@ -115,7 +115,7 @@ Thanks to the previously created function, we create a PyMISP object.
event = misp.get_event(args.event)
event = misp.add_named_attribute(event, args.type, args.value)
~~~~
In order to add the new argument, we first need to fetch the event in the MISP database using the [get\_event](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L223) function which only need the event\_id. Then only once we have it, we can call the function [add\_named\_attribute](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L372) that will add the argument.
In order to add the new argument, we first need to fetch the event in the MISP database using the [get\_event](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L223) function which only need the event\_id. Then only once we have it, we can call the function [add\_named\_attribute](https://github.com/CIRCL/PyMISP/blob/master/pymisp/api.py#L372) that will add the argument.
~~~~python
print(event)
~~~~
@ -164,7 +164,7 @@ Arguments:
#### del.py
Delete an event or an attribute from a MISP instance. The event has the priority: if both are set, only the event will be deleted.
Delete an event or an attribute from a MISP instance. The event has the priority: if both are set, only the event will be deleted.
Arguments:
* **event**: Event ID to delete.
@ -172,7 +172,7 @@ Arguments:
#### delete_user.py
Delete the user with the given id. Keep in mind that disabling users (by setting the disabled flag via an edit) is always prefered to keep user associations to events intact.
Delete the user with the given id. Keep in mind that disabling users (by setting the disabled flag via an edit) is always preferred to keep user associations to events intact.
Arguments:
* **user_id**: The id of the user you want to delete.
@ -220,7 +220,7 @@ Arguments:
#### sharing_groups.py
Get a list of the sharing groups from the MISP instance.
Get a list of the sharing groups from the MISP instance.
No argument.
#### sighting.py
@ -232,7 +232,7 @@ Arguments:
#### stats.py
Output attributes statistics from a MISP instance.
Output attributes statistics from a MISP instance.
No argument.
#### suricata.py
@ -245,7 +245,7 @@ Arguments:
#### tags.py
Get tags from MISP instance.
Get tags from MISP instance.
No argument.
#### tagstatistics.py
@ -273,15 +273,15 @@ Arguments:
* **event**: Not supplying an event ID will cause MISP to create a single new event for all of the POSTed malware samples.
* **distrib**: The distribution setting used for the attributes and for the newly created event, if relevant. [0-3].
* **ids**: You can flag all attributes created during the transaction to be marked as \"to_ids\" or not.
* **categ**: The category that will be assigned to the uploaded samples. Valid options are: Payload delivery, Artifacts dropped, Payload Installation, External Analysis.
* **categ**: The category that will be assigned to the uploaded samples. Valid options are: Payload delivery, Artefacts dropped, Payload Installation, External Analysis.
* **info**: Used to populate the event info field if no event ID supplied.
* **analysis**: The analysis level of the newly created event, if applicatble. [0-2]
* **threat**: The threat level ID of the newly created event, if applicatble. [1-4]
* **analysis**: The analysis level of the newly created event, if applicable. [0-2]
* **threat**: The threat level ID of the newly created event, if applicable. [1-4]
* **comment**: Comment for the uploaded file(s).
#### users_list.py
#### users_list.py
Get a list of the sharing groups from the MISP instance.
Get a list of the sharing groups from the MISP instance.
No argument.
### Going further
@ -300,7 +300,7 @@ outputdir = 'output'
# filters = {'tag' : 'tlp : white|feed-export|!privint', 'org':'CIRCL'}
filters = {}
valid_attribute_distribution_levels = ['0', '1', '2', '3', '4', '5']
valid_attribute_distribution_levels = ['0', '1', '2', '3', '4', '5']
~~~~
@ -329,9 +329,9 @@ for uri in osintcircl.json():
#### ioc-2-misp
Allow to import OpenIOC files into MISP easily. It is also possible to set specific tags on these events.
#### Situational Awareness
* attribute_treemap.py generate a treemap showing the distribution of the attributes on the misp instance.
* attribute_treemap.py generate a tree-map showing the distribution of the attributes on the MISP instance.
* tags_* : these functions help having statistics and graphs about the tag repartition.

4
user-management/README.md
View File

@ -5,10 +5,10 @@
### First run of the system
When first logging into MISP with the username and password provided by your administrator, there are a number of things that need to be done, before you can start using the system.
* **Acceping the Terms of use:** The terms of use are shown immediately after logging in for the first time, make sure to read through this page before clicking "Accept Terms" at the bottom of the page.
* **Accepting the Terms of use:** The terms of use are shown immediately after logging in for the first time, make sure to read through this page before clicking "Accept Terms" at the bottom of the page.
* **Changing the password:** After accepting the ToU, you'll be prompted to change your password, but keep in mind that it has to be at least 6 characters long, it has to include at least one upper-case and one lower-case character in addition to a digit or a special character. Enter the same password into the confirm password field, before clicking submit to finalise the change.
![Changing the password](figures/password.png)
* **Setting up the GPG Key:** In order for the system to be able to encrypt the messages that you send through it, it needs to know your GPG key. Navigate to the Edit profile view (My Profile on the left -> Edit profile in the top right corner). Paste the key into the Gpgkey field and click submit.
* **Setting up the GnuPG Key:** In order for the system to be able to encrypt the messages that you send through it, it needs to know your GnuPG key. Navigate to the Edit profile view (My Profile on the left -> Edit profile in the top right corner). Paste the key into the GnuPG Key field and click submit.
* **Subscribing to Auto-alerts:** Turning auto-alerts on will allow the system to send you e-mail notifications about any new public events entered into the system by other users and private events added by members of your organisation. To turn this on, navigate to the Edit profile view (My profile on the left navigation menu -> Edit profile in the top right corner). Tick the auto-alert checkbox and click submit to enable this feature.
![Use these checkboxes to subscribe to auto-alerts and contact reporter e-mails.](figures/alerts.png)
* **Subscribing to e-mails sent via the "Contact Reporter" functionality:** This feature is turned on right below the autoalerts and will allow you to receive e-mails addressed to your organisation whenever a user tries to ask about an event that was posted by a user of your organisation. Keep in mind that you can still be addressed by such a request even when this setting is turned off, if someone tries to contact you as the event creator directly or your organisation for an event that you personally have created then you will be notified.