Added the rest of the documentation

- this is still WIP
- images need replacing
- categories and types are a bit weird still

SUMMARY.md

@@ -4,4 +4,6 @@
 * [General Layout](general-layout/README.md)
 * [General Concepts](general-concepts/README.md)
 * [User Management and Global Actions](user-management/README.md)
-* [Using the system](using-the-system/README.md)
+* [Using the System](using-the-system/README.md)
+* [Administration](administration/README.md)
+* [Categories and Types](categories-and-types/README.md)

administration/README.md

@@ -0,0 +1,266 @@
+## Administration
+
+### Server settings and diagnostics
+
+Since version 2.3, MISP has a settings and diagnostics tool that allows site-admins to manage and diagnose their MISP installation. You can access this by navigating to Administration - Server settings
+
+![Server settings overview with all of the tabs explained.](figures/settings_1.png)
+
+The settings and diagnostics tool is split up into several aspects, all accessible via the tabs ontop of the tool. For any unset or incorrectly set setting, or failed diagnostic a number next to the tab name will indicate the number and severity of the issues. If the number is written with a red font, it means that the issue is critical. First, let's look at the various tabs:
+*   **Overview**: General overview of the current state of your MISP installation
+*   **MISP settings**: Basic MISP settings. This includes the way MISP handles the default settings for distribution settings, whether background jobs are enabled, etc
+*   **GnuPG settings**: GPG related settings.
+*   **Proxy settings**: HTTP proxy related settings.
+*   **Security settings**: Settings controlling the brute-force protection and the application's salt key.
+*   **Misc settings**: You change the debug options here, but make sure that debug is always disabled on a production system.
+*   **Diagnostics**: The diagnostics tool checks if all directories that MISP uses to store data are writeable by the apache user. Also, the tool checks whether the STIX libraries and GPG are working as intended.
+*   **Workers**: Shows the background workers (if enabled) and shows a warning if they are not running. Admins can also restart the workers here.
+*   **Download report**: Download a report in JSON format, compiled of all of the settings visible in the tool.
+	
+![The settings tabs explained.](figures/settings_2.png)
+
+Each of the setting pages is a table with each row representing a setting. Coloured rows indicate that the setting is incorrect / not set and the colour determines the severity (red = critical, yellow = recommended, green = optional). The columns are as follows:
+*   **Priority**: The severity of the setting.
+*   **Setting**: The setting name.
+*   **Value**: The current value of the setting.
+*   **Description**: A description of what the setting does.
+*   **Error Message**: If the setting is incorrect / not set, then this field will let the user know what is wrong.
+
+![The workers tab.](figures/settings_3.png)	
+
+The workers tab shows a list of the workers that MISP can use. You can restart the workers using the restart all workers, If the button doesn't work, make sure that the workers were started using the apache user. This can however only be done using the command line, refer to the INSTALL.txt documentation on how to let the workers automatically start on each boot.
+	
+*   **Worker Type**: The worker type is determined by the queue it monitors. MISP currently has 4 queues (cache, default, email and a special _schdlr_ queue).
+*   **Worker Id**: The ID is made up of the machine name, the PID of the worker and the queue it monitors.
+*   **Status**: Displays OK if the worker is running. If the _schdlr_ worker is the only one not running make sure that you copy the config file into the cakeresque directory as described in the INSTALL.txt documentation.
+	
+	
+### Import Blacklist
+
+It is possible to ban certain values from ever being entered into the system via an event info field or an attribute value. This is done by blacklisting the value in this section.
+
+#### Adding and modifying entries
+
+Administrators can add, edit or delete blacklisted items by using the appropriate functions in the list's action menu and the menu on the left.
+	
+### Import Regexp
+
+The system allows administrators to set up rules for regular expressions that will automatically alter newly entered or imported events (from GFI Sandbox).
+
+#### The purpose of Import Regexp entries
+
+They can be used for several things, such as unifying the capitalisation of file paths for more accurate event correlation or to automatically censor the usernames and use system path variable names (changing C:\Users\UserName\Appdata\Roaming\file.exe to %APPDATA%\file.exe).
+The second use is blocking, if a regular expression is entered with a blank replacement, any event info or attribute value containing the expression will not be added. Please make sure the entered regexp expression follows the preg_replace pattern rules as described [here](http://php.net/manual/en/function.preg-replace.php)
+
+#### Adding and modifying entries
+
+Administrators can add, edit or delete regular expression rules, which are made up of a regex pattern that the system searches for and a replacement for the detected pattern.
+
+![Add, edit or remove Regexp entries that will affect all newly created attributes here.](figures/regexp.png)
+	
+### Managing the Signature whitelist
+
+The signature whitelist view, accessible through the administration menu on the left, allows administrators to create and maintain a list of addresses that are whitelisted from ever being added to the NIDS signatures. Addresses listed here will be commented out when exporting the NIDS list.
+
+#### Whitelisting an address:
+
+While in the whitelist view, click on New Whitelist on the left to bring up the add whitelist view to add a new address. 
+
+#### Managing the list:
+
+When viewing the list of whitelisted addresses, the following pieces of information are shown: The ID of the whitelist entry (assigned automatically when a new address is added), the address itself that is being whitelisted and a set of controls allowing you to delete the entry or edit the address.
+
+![You can edit or delete currently white-listed addresses using the action buttons on this list.](figures/whitelist.png)
+	
+### Managing the users:
+
+As an admin, you can set up new accounts for users, edit the profiles of users, delete them, or just have a look at all the viewers' profiles. Organisation admins are restricted to executing the same actions on their organisation's users only.
+
+#### Adding a new user:
+
+To add a new user, click on the New User button in the administration menu to the left and fill out the following fields in the view that is loaded:
+
+![Fill this form out to add a new user. Keep in mind that the drop-down menu titled Role controls the privileges the user will have.](figures/add_user.png)
+	
+*   **Email:** The user's e-mail address, this will be used as his/her login name and as an address to send all the automatic e-mails and e-mails sent by contacting the user as the reporter of an event.
+*   **Password:** A temporary password for the user that he/she should change after the first login. Make sure that it is at least 6 characters long, includes a digit or a special character and contains at least one upper-case and at least one lower-case character.
+*   **Confirm Password:** This should be an exact copy of the Password field.
+*   **Org:**The organisation of the user. Entering ADMIN into this field will give administrator privileges to the user. If you are an organisation admin, then this field will be unchangeable and be set to your own organisation.
+*   **Roles:** A drop-down list allows you to choose a role-group that the user should belong to. Roles define the privileges of the user. To learn more about roles, [click here](#managing-the-roles).
+*   **Receive alerts when events are published:** This option will subscribe the new user to automatically generated e-mails whenever an event is published.
+*   **Receive alerts from "contact reporter" requests:** This option will subscribe the new user to e-mails that are generated when another user tries to get in touch with an event's reporting organisation that matches that of the new user.
+*   **Authkey:** This is assigned automatically and is the unique authentication key of the user (he/she will be able to reset this and receive a new key). It is used for exports and for connecting one server to another, but it requires the user to be assigned to a role that has auth permission enabled.
+*   **NIDS Sid:** Nids ID, not yet implemented.
+*   **Gpgkey:** The key used for encrypting e-mails sent through the system. 
+	
+#### Listing all users:
+
+To list all current users of the system, just click on List Users under the administration menu to the left. A view will be loaded with a list of all users and the following columns of information:
+
+![View, Edit or Delete a user using the action buttons to the right.](figures/list_users.png)
+	
+*   **Id:** The user's automatically assigned ID number.
+*   **Org:** The organisation that the user belongs to.
+*   **Email:** The e-mail address (and login name) of the user.
+*   **Autoalert:** Shows whether the user has subscribed to auto-alerts and is always receiving the mass-emails regarding newly published events that he/she is eligible for.
+*   **ontactalert:** Shows whether the user has the subscription to contact reporter e-mails directed at his/her organisation turned on or off.
+*   **Gpgkey:** Shows whether the user has entered a Gpgkey yet.
+*   **Nids Sid:** Shows the currently assigned NIDS ID.
+*   **Termsaccepted:** This flag indicates whether the user has accepted the terms of use or not.
+*   **Newsread:** The last point in time when the user has looked at the news section of the system.
+*   **Action Buttons:** Here you can view a detailed view of a user, edit the basic details of a user (same view as the one used for creating a new user, but all the fields come filled out by default) or remove a user completely. 
+	
+#### Editing a user:
+
+To add a new user, click on the New User button in the administration menu to the left and fill out the following fields in the view that is loaded:
+	
+*   **Email:** The user's e-mail address, this will be used as his/her login name and as an address to send all the automatic e-mails and e-mails sent by contacting the user as the reporter of an event.
+*   **Password:** It is possible to assign a new password manually for a user. For example, in case that he/she forgot the old one a new temporary one can be assigned. Make sure to check the "Change password" field if you do give out a temporary password, so that the user will be forced to change it after login.
+*   **Confirm Password:** This should be an exact copy of the Password field.
+*   **Org:**The organisation of the user. Entering ADMIN into this field will give administrator privileges to the user. If you are an organisation admin, then this field will be unchangeable and be set to your own organisation.
+*   **Roles:** A drop-down list allows you to choose a role-group that the user should belong to. Roles define the privileges of the user. To learn more about roles, [click here](#managing-the-roles).
+*   **Receive alerts when events are published:** This option will subscribe the user to automatically generated e-mails whenever an event is published.
+*   **Receive alerts from "contact reporter" requests:** This option will subscribe the user to e-mails that are generated when another user tries to get in touch with an event's reporting organisation that matches that of the user.
+*   **Authkey:** It is possible to request a new authentication key for the user. 
+*   **NIDS Sid:** Nids ID, not yet implemented.
+*   **Termsaccepted:** Indicates whether the user has accepted the terms of use already or not.
+*   **Change Password:** Setting this flag will require the user to change password after the next login.
+*   **Gpgkey:** The key used for encrypting e-mails sent through the system. 
+	
+#### Contacting a user:
+
+Site admins can use the "Contact users" feature to send all or an individual user an e-mail. Users that have a PGP key set will receive their e-mails encrypted. When clicking this button on the left, you'll be presented with a form that allows you to specify the type of the e-mail, who it should reach and what the content is using the following options:
+
+![Contact your users here.](figures/contact.png)
+	
+*   **Action:** This defines the type of the e-mail, which can be a custom message or a password reset. Password resets automatically include a new temporary password at the bottom of the message and will automatically change the user's password accordingly.
+*   **Recipient:** The recipient toggle lets you contact all your users, a single user (which creates a second drop-down list with all the e-mail addresses of the users) and potential future users (which opens up a text field for the e-mail address and a text area field for a PGP public key).
+*   **Subject:** In the case of a custom e-mail, you can enter a subject line here.
+*   **Subject:** In the case of a custom e-mail, you can enter a subject line here.
+*   **Custom message checkbox:** This is available for password resets, you can either write your own message (which will be appended with a temporary key and the signature), or let the system generate one automatically.
+	
+Keep in mind that all e-mails sent through this system will, in addition to your own message, will be signed in the name of the instance's host organisation's support team, will include the e-mail address of the instance's support (if the contact field is set in the bootstrap file), and will include the instance's PGP signature for users that have a PGP key set (and thus are eligible for an encrypted e-mail).
+	
+### Managing the roles
+
+Privileges are assigned to users by assigning them to rule groups, which use one of four options determining what they can do with events and four additional privilege elevating settings. The four options for event manipulation are: Read Only, Manage My Own Events, Manage Organisation Events, Manage & Publish Organisation Events. The extra privileges are admin, sync, authentication key usage and audit permission
+	
+*   **Read Only:** This allows the user to browse events that his organisation has access to, but doesn't allow any changes to be made to the database.
+*   **Manage My Own Events:** The second option, gives its users rights to create, modify or delete their own events, but they cannot publish them.
+*   **Manage Organization Events:** allows users to create events or modify and delete events created by a member of their organisation.
+*   **Manage & Publish Organisation Events:** This last setting, gives users the right to do all of the above and also to publish the events of their organisation.
+*   **Perm sync:** This setting allows the users of the role to be used as a synchronisation user. The authentication key of this user can be handed out to the administrator of a remote MISP instance to allow the synchronisation features to work.
+*   **Perm auth:** This setting enables the authentication key of the role's users to be used for rest requests. 
+*   **Perm admin:** Gives the user limited administrator privileges, this setting is used for the organisation admins. 
+*   **Perm site admin:** Gives the user full administrator privileges, this setting is used for the site admins. 
+*   **Perm audit:** Grants access to the logs. With the exception of site admins, only logs generated by the user's own org are visible. 
+*   **Perm regexp access:** Allows the users with this permission enabled to edit the regular expression table. Be careful when giving out this permission, incorrect regular expressions can be very harmful (infinite loops, loss of data, etc.).
+*   **Perm tagger:** Allows the user with this permission to create custom tags and assign them to events. 
+	
+#### Creating roles:
+
+When creating a new role, you will have to enter a name for the role to be created and set up the permissions (as described above) using the radio toggle and the four check-boxes.
+
+#### Listing roles:
+
+By clicking on the List Roles button, you can view a list of all the currently registered roles and a list of the permission flags turned on for each. In addition, you can find buttons that allow you to edit and delete the roles. Keep in mind that you will need to first remove every member from a role before you can delete it.
+
+![You can View, Edit or Delete roles using the action buttons to the right in each row. Keep in mind that a role has to be devoid of members before it can be deleted.](figures/list_groups.png)
+	
+### Using the logs of MISP
+
+Users with audit permissions are able to browse or search the logs that MISP automatically appends each time certain actions are taken (actions that modify data or if a user logs in and out).
+Generally, the following actions are logged:
+	
+*   **User:** Creation, deletion, modification, Login / Logout
+*   **Event:**Creation, deletion, modification, publishing
+*   **Attribute:** Creation, deletion, modification
+*   **ShadowAttribute:** Creation, deletion, Accept, Discard
+*   **Roles:** Creation, deletion, modification
+*   **Blacklist:** Creation, deletion, modification
+*   **Whitelist:** Creation, deletion, modification
+*   **Regexp:** Creation, deletion, modification
+	
+	
+#### Browsing the logs:
+
+Listing all the log entries will show the following columns generated by the users of your organisation (or all organisations in the case of site admins):
+
+![Here you can view a list of all logged actions.](figures/list_logs.png)
+	
+*   **Id:** The automatically assigned ID number of the entry.
+*   **Email:** The e-mail address of the user whose actions triggered the entry.
+*   **Org:** The organisation of the above mentioned user.
+*   **Created:** The date and time when the entry originated.
+*   **Action:** The action's type. This can include: login/logout for users, add, edit, delete for events, attributes, users and servers.
+*   **Title:** The title of an event always includes the target type (Event, User, Attribute, Server), the target's ID and the target's name (for example: e-mail address for users, event description for events).
+*   **Change:** This field is only filled out for entries with the action being add or edit. The changes are detailed in the following format:
+   __variable (initial_value)_ => _(new_value)_,...
+   When the entry is about the creation of a new item (such as adding a new event) then the change will look like this for example:
+   _org()_ => _(ADMIN)_, _date()_ => _(20012-10-19)_,... 
+	
+![You can search the logs using this form, narrow down your search by filling out several fields.](figures/search_log.png)
+
+#### Searching the Logs:
+
+Another way to browse the logs is to search it by filtering the results according to the following fields (the search is a sub-string search, the sub-string has to be an exact match for the entry in the field that is being searched for):
+	
+*   **Email:** By searching by Email, it is possible to view the log entries of a single user.
+*   **Org:** Searching for an organisation allows you to see all actions taken by any member of the organisation.
+*   **Action:** With the help of this drop down menu, you can search for various types of actions taken (such as logins, deletions, etc).
+*   **Title:** There are several ways in which to use this field, since the title fields contain several bits of information and the search searches for any substrings contained within the field, it is possible to just search for the ID number of a logged event, the username / server's name / event's name / attribute's name of the event target.
+*   **Change:** With the help of this field, you can search for various specific changes or changes to certain variables (such as published will find all the log entries where an event has gotten published, ip-src will find all attributes where a source IP address has been entered / edited, etc).
+	
+	
+### Administrative Tools
+
+MISP has a couple of administrative tools that help administrators keep their instance up to date and healthy. The list of these small tools can change rapidly with each new version, but they should be self-explanatory. Make sure to check this section after upgrading to a new version, just in case there is a new upgrade script in there - though if this is the case it will be mentioned in the upgrade instructions.
+	
+### Background Processing
+
+If enabled, MISP can delegate a lot of the time intensive tasks to the background workers. These will then be executed in order, allowing the users of the instance to keep using the system without a hiccup and without having to wait for the process to finish. It also allows for certain tasks to be scheduled and automated.
+
+#### Command Line Tools for the Background Workers
+
+The background workers are powered by [CakeResque](https://github.com/kamisama/Cake-Resque), so all of the CakeResque commands work. 
+To start all of the workers needed by MISP go to your `/var/www/MISP/app/Console/worker` (assuming a standard installation path) and execute start.sh.
+To interact with the workers, here is a list of useful commands. Go to your `/var/www/MISP/app/Console` (assuming a standard installation path) and execute one of the following commands as a parameter to `./cake CakeResque.CakeResque` (for example: `./cake CakeResque.CakeResque tail`):
+	
+*   **tail**: tail the various log files that CakeResque creates, just choose the one from the list that you are interested in.
+*   **cleanup**: terminate the job that a worker is working on immediately. You will be presented with a choice of workers to choose from when executing this command.
+*   **clear**: Clear the queue of a worker immediately.
+*   **stats**: shows some statistics about your workers including the count of successful and failed jobs.
+	
+The other commands should not be needed, instead of starting / stopping or restarting workers use the supplied start.sh (it stops all workers and starts them all up again). For further instructions on how to use the console commands for the workers, visit the [CakeResque list of commands](http://cakeresque.kamisama.me/commands#cleanup).
+
+#### Monitoring the Background Processes
+
+The "Jobs" menu item within the Administration menu allows site admins to get an overview of all of the currently and in the past scheduled jobs. Admins can see the status of each job, and what the queued job is trying to do. If a job fails, it will try to set an error message here too. The following columns are shown in the jobs table:
+	
+*   **Id**: The job's ID (this is the ID of the job's metadata stored in the default datastore, not to be confused with the process ID stored in the redis database and used by the workers)
+*   **Process**: The process's ID.
+*   **Worker**: The name of the worker queue. There are 3+1 workers running if background jobs are enabled: default, cache, email, and a special Scheduler (this should never show up in the jobs table).
+*   **Job Type**: The name of the queued job.
+*   **Input**: Shows a basic input handled by the job - such as "Event:50" for a publish email alert job for event 50.
+*   **Message**: This will show what the job is currently doing or alternatively an error message describing why a job failed. 
+*   **Org**: The string identifier of the organisation that has scheduled the job. 
+*   **Status**: The status reported by the worker.
+*   **Retries**: Currently unused, it is planned to introduced automatic delayed retries for the background processing to add resilience.
+*   **Progress**: A progress bar showing how the job is coming along.
+	
+![Site administrators can monitor the process of all queued jobs here.](figures/jobs.png)
+
+#### Scheduling Jobs and Recurring Jobs
+
+Apart from off-loading long-lasting jobs to the background workers, there is a second major benefit of enabling the background workers: Site-administrators can schedule recurring tasks for the jobs that generally take the longest to execute. At the moment this includes pushing / pulling other instances and generating a full export cache for every organisation and export type. MISP comes with these 3 tasks pre-defined, but further tasks are planned. The following fields make up the scheduled tasks table: 
+	
+*   **Id**: The ID of the task.
+*   **Type**: The type of the task.
+*   **Frequency (h)**: This number sets how often the job should be executed in hours. Setting this to 168 and picking the next execution on Sunday at 01:00 would execute the task every Sunday at 1 AM. Setting this value to 0 will make the task only run once on the scheduled date / time without rescheduling it afterwards.
+*   **Scheduled Time**: The time (in 24h format) when the task should be executed the next time it runs (and all consecutive times if a multiple of 24 is chosen for frequency).
+*   **Next Run**: The date on which the task should be executed.
+*   **Description**: A brief description of the task.
+*   **Message**: This field shows when the job was queued by the scheduler for execution. 
+	
+![Site administrators can schedule reccuring tasks on this page.](figures/schedule.png)
+

categories-and-types/README.md

@@ -0,0 +1,148 @@
+### Attribute Categories vs Types
+
+|Category| Internal reference | Targeting data | Antivirus detection | Payload delivery | Artifacts dropped | Payload installation |
+| --- |:---:|:---:|:---:|:---:|:---:|:---:|
+|md5| | | | X | X | X |
+|sha1| | | | X | X | X |
+|sha256| | | | X | X | X |
+|filename| | | | X | X | X |
+|filename|md5| | | | X | X | X |
+|filename|sha1| | | | X | X | X |
+|filename|sha256| | | | X | X | X |
+|ip-src| | | | X | | |
+|ip-dst| | | | X | | |
+|hostname| | | | X | | |
+|domain| | | | X | | |
+|email-src| | | | X | | |
+|email-dst| | | | X | | |
+|email-subject| | | | X | | |
+|email-attachment| | | | X | | |
+|url| | | | X | | |
+|http-method| | | | | | |
+|user-agent| | | | X | | |
+|regkey| | | | | X | |
+|regkey|value| | | | | X | |
+|AS| | | | X | | |
+|snort| | | | | | |
+|pattern-in-file| | | | X | X | X |
+|pattern-in-traffic| | | | X | | X |
+|pattern-in-memory| | | | | X | X |
+|yara| | | | X | X | X |
+|vulnerability| | | | X | | X |
+|attachment| | | X | X | X | X |
+|malware-sample| | | | X | X | X |
+|link| X | | X | X | | |
+|comment| X | X | X | X | X | X |
+|text| X | | X | X | X | X |
+|other| X | | X | X | X | X |
+|named pipe| | | | | X | |
+|mutex| | | | | X | |
+|target-user| | X | | | | |
+|target-email| | X | | | | |
+|target-machine| | X | | | | |
+|target-org| | X | | | | |
+|target-location| | X | | | | |
+|target-external| | X | | | | |
+
+|Category| Persistence mechanism | Network activity | Payload type | Attribution | External analysis | Other |
+| --- |:---:|:---:|:---:|:---:|:---:|:---:|
+|md5| | | | | X | |
+|sha1| | | | | X | |
+|sha256| | | | | X | |
+|filename| X | | | | X | |
+|filename|md5| | | | | X | |
+|filename|sha1| | | | | X | |
+|filename|sha256| | | | | X | |
+|ip-src| | X | | | X | |
+|ip-dst| | X | | | X | |
+|hostname| | X | | | X | |
+|domain| | X | | | X | |
+|email-src| | | | | | |
+|email-dst| | X | | | | |
+|email-subject| | | | | | |
+|email-attachment| | | | | | |
+|url| | X | | | X | |
+|http-method| | X | | | | |
+|user-agent| | X | | | X | |
+|regkey| X | | | | X | |
+|regkey|value| X | | | | X | |
+|AS| | X | | | X | |
+|snort| | X | | | X | |
+|pattern-in-file| | X | | | X | |
+|pattern-in-traffic| | X | | | X | |
+|pattern-in-memory| | | | | X | |
+|yara| | | | | | |
+|vulnerability| | | | | X | |
+|attachment| | X | | | X | |
+|malware-sample| | | | | X | |
+|link| | | | | X | |
+|comment| X | X | X | X | X | X |
+|text| X | X | X | X | X | X |
+|other| X | X | X | X | X | X |
+|named pipe| | | | | | |
+|mutex| | | | | | |
+|target-user| | | | | | |
+|target-email| | | | | | |
+|target-machine| | | | | | |
+|target-org| | | | | | |
+|target-location| | | | | | |
+|target-external| | | | | | |
+
+### Categories
+
+*   **Internal reference**: Reference used by the publishing party (e.g. ticket number)
+*   **Targeting data**: Targeting information to include recipient email, infected machines, department, and or locations.<br/>
+*   **Antivirus detection**: List of anti-virus vendors detecting the malware or information on detection performance (e.g. 13/43 or 67%). Attachment with list of detection or link to VirusTotal could be placed here as well.
+*   **Payload delivery**: Information about the way the malware payload is initially delivered, for example information about the email or web-site, vulnerability used, originating IP etc. Malware sample itself should be attached here.
+*   **Artifacts dropped**: Any artifact (files, registry keys etc.) dropped by the malware or other modifications to the system
+*   **Payload installation**: Location where the payload was placed in the system and the way it was installed. For example, a filename|md5 type attribute can be added here like this: c:\windows\system32\malicious.exe|41d8cd98f00b204e9800998ecf8427e.
+*   **Persistence mechanism**: Mechanisms used by the malware to start at boot. This could be a registry key, legitimate driver modification, LNK file in startup
+*   **Network activity**: Information about network traffic generated by the malware
+*   **Payload type**: Information about the final payload(s). Can contain a function of the payload, e.g. keylogger, RAT, or a name if identified, such as Poison Ivy.
+*   **Attribution**: Identification of the group, organisation, or country behind the attack
+*   **External analysis**: Any other result from additional analysis of the malware like tools output Examples: pdf-parser output, automated sandbox analysis, reverse engineering report.
+*   **Other**: Attributes that are not part of any other category
+
+### Types
+
+*   **md5**: You are encouraged to use filename|md5 instead. A checksum in md5 format, only use this if you don't know the correct filename
+*   **sha1**: You are encouraged to use filename|sha1 instead. A checksum in sha1 format, only use this if you don't know the correct filename
+*   **sha256**: You are encouraged to use filename|sha256 instead. A checksum in sha256 format, only use this if you don't know the correct filename
+*   **filename**: Filename
+*   **filename|md5**: A filename and an md5 hash separated by a | (no spaces)
+*   **filename|sha1**: A filename and an sha1 hash separated by a | (no spaces)
+*   **filename|sha256**: A filename and an sha256 hash separated by a | (no spaces)
+*   **ip-src**: A source IP address of the attacker
+*   **ip-dst**: A destination IP address of the attacker or C&C server. Also set the IDS flag on when this IP is hardcoded in malware
+*   **hostname**: A full host/dnsname of an attacker. Also set the IDS flag on when this hostname is hardcoded in malware
+*   **domain**: A domain name used in the malware. Use this instead of hostname when the upper domain is important or can be used to create links between events.
+*   **email-src**: The email address (or domainname) used to send the malware.
+*   **email-dst**: A recipient email address that is not related to your constituency.
+*   **email-subject**: The subject of the email
+*   **email-attachment**: File name of the email attachment.
+*   **url**: url
+*   **http-method**: HTTP method used by the malware (e.g. POST, GET, ...).
+*   **user-agent**: The user-agent used by the malware in the HTTP request.
+*   **regkey**: Registry key or value
+*   **regkey|value**: Registry value + data separated by |
+*   **AS**: Autonomous system
+*   **snort**: An IDS rule in Snort rule-format. This rule will be automatically rewritten in the NIDS exports.
+*   **pattern-in-file**: Pattern in file that identifies the malware
+*   **pattern-in-traffic**: Pattern in network traffic that identifies the malware
+*   **pattern-in-memory**: Pattern in memory dump that identifies the malware
+*   **yara**: Yara signature
+*   **vulnerability**: A reference to the vulnerability used in the exploit
+*   **attachment**: Please upload files using the <em>Upload Attachment</em> button.
+*   **malware-sample**: Please upload files using the <em>Upload Attachment</em> button.
+*   **link**: Link to an external information
+*   **comment**: Comment or description in a human language. This will not be correlated with other attributes (NOT IMPLEMENTED YET)
+*   **text**: Name, ID or a reference
+*   **other**: Other attribute
+*   **named pipe**: Named pipe, use the format \.\pipe\<PipeName>
+*   **mutex**: Mutex, use the format \BaseNamedObjects\<Mutex>
+*   **target-user**: Attack Targets Username(s)
+*   **target-email**: Attack Targets Email(s)
+*   **target-machine**: Attack Targets Machine Name(s)
+*   **target-org**: Attack Targets Department or Orginization(s)
+*   **target-location**: Attack Targets Physical Location(s)
+*   **target-external**: External Target Orginizations Affected by this Attack

using-the-system/README.md

@@ -270,3 +270,349 @@ The last option is a checkbox that restricts all of the results to attributes th
 
 !["You can view the event that an attribute belongs to with the view button, or you can edit/delete the attribute via the buttons on the right."](figures/search_attribute_result.png)
 
+
+## Updating and modifying events and attributes:
+
+Every event and attribute can easily be edited. First of all it is important to find the event or attribute that is to be edited, using any of the methods mentioned in the section on [browsing past events](#browsing_events).
+Once it is found, the edit button (whether it be under actions when events/attributes get listed or simply on the event view) will bring up the same screen as what is used to create the entry of the same type (for an event it would be the event screen as [seen here](#Creating an event), for an attribute the attribute screen as [described here](#add-attributes-to-the-event)).
+Keep in mind that editing any event (either directly or indirectly through an attribute) will unpublish it, meaning that you'll have to publish it (through the event view) again once you are done.
+
+## Tagging:
+
+As described earlier, users with tagging rights can arbitrarily tag events using tags chosen from a pool of available options. If you have tagging privileges and would like to create a new tag, navigate to Event Actions - Add Tag. You'll be presented with the following form:
+
+![Enter a name for the tag and click on the color field to be able to pick a colour for it.](figures/tag.png)
+
+Fill out the following fields:
+*   **Name**: Pick a name for the tag. Try to use consistent naming conventions across your instance, to avoid confusion.
+*   **Colour**: You can choose a colour for the tag by clicking on the colour field and using the colour picker tool. Try to avoid having duplicate or similar looking colours to help avoid confusion.
+
+## Templating:
+
+Newer users can easily be overwhelmed by having to manually populate events with attributes without any guidance. What sort of information should go into the event? What should be the category and type of a C2 IP? Templates allow users to use simple forms to populate events.
+Even though MISP ships with a few default templates, it is possible for users (with the appropriate templating privilege) to create new templates for their users or for all users of the instance. Let's look at how you can create a template.
+First go to Event Actions - Add Template to go to the event creation view.
+
+![Fill in the generic information about the template.](figures/create_template.png)
+
+The following fields have to be filled out:
+*   **Name**: The name of the template should describe what type of an event it should be used to generate attributes.
+*   **Tags**: You can attach tags to the template - an event populated using the template would automatically receive the tag(s). Add new tags using the + button. If you chnage your mind about a tag you can remove it with the cross next to the tag name.
+*   **Event Description**: A short description about the events that this template should be used for.
+*   **Share this template with others**: The template can be set to be usable by any organisation on the instance or only by the one that has created it.
+
+Once the skeleton template is created, you can start populating the template with data. There are 3 types of elements that can be used during the creation of a template: attribute, file and text elements. Text elements divide the template into sections with an information field, followed by all of the attribute/file fields until a new text field is read. Don't worry about the order of the elements during creation, they can be re-arranged using drag & drop. Let's look at the 3 element types:
+
+**Attribute Element**
+
+![This element will generate regular attributes based on user entry.](figures/template_attribute.png)
+
+The following fields have to be filled out:
+*   **Name**: The field name that will be presented to the user.
+*   **Description**: A brief description of the element. Make sure that you provide sufficient information to the user to make it obvious what is expected.
+*   **Category**: The category used for any attributes created using this template element.
+*   **Type**: The type or complex type used for any attributes created using this template element. Complex types allow for several related types to be used on data entry. For example, a "file" complex type element allows for filenames and hashes.
+*   **Use Complex types**: If the category permits it, switch to a complex type using this checkbox.
+*   **Automatically mark for IDS**: If checked, any attributes generated using this element will be marked for IDS exporting.
+*   **Mandatory element**: If the elemnt is marked as mandatory, then the template form can only be submitted by users if this field is filled out.
+*   **Batch import element**: Allow for multiple values to be entered (separated by line breaks).
+
+**File Element**
+
+![This element will generate attachments based on user entry.](figures/template_file.png)
+
+The following fields have to be filled out:
+*   **Name**: The field name that will be presented to the user.
+*   **Description**: A brief description of the element. Make sure that you provide sufficient information to the user to make it obvious what is expected.
+*   **Category**: The category to be used by all attachments uploaded through this element. 
+*   **Malware**: If the uploaded files are malicious and should be encrypted and password protected, mark this checkbox.
+*   **Mandatory element**: If it should be required to upload an attachment, check this checkbox.
+*   **Batch import element**: Ticking this checkbox allows users to upload several files using this element.
+
+**Text Element**
+
+![This element will start a section in the template, which continues until the next text element or the end of the template.](figures/template_text.png)
+
+The following fields have to be filled out:
+*   **Name**: The name of the section that will be presented to the user.
+*   **Text**: The description of the section. Explain briefly to the user what the following attribute/file elements will be dealing with. There are several ways to split a template into sections, try to have ease of use in mind while creating it.
+
+## Contacting the reporter:
+
+To get in touch with the reporter of a previously registered event, just find the event for which you would like to contact the reporter by either finding it on the list of events, by finding it through one of its attributes or by finding it through a related event.
+Once the event is found and the event view opened, click the button titled "Contact Reporter". This will bring up a view where you can enter your message that is to be e-mailed to all members of the reporting organisation that subscribe to receiving such reports or the reporting user himself. Along with your message, the detailed information about the event in question will be included in the e-mail.
+
+![Enter your message to the reporter and choose whether his/her entire organisation should get the message or not by ticking the check-box.](figures/contact_reporter.png)
+
+By default, the message will be sent to every member of the organisation that posted the event in the first place, but if you tick the check-box below the message field before sending the mail, only the person that reported the event will get e-mailed. 
+
+## Automation:
+It is possible to quickly and conveniently export the data contained within the system using the automation features located in the main menu on the left (available to users with authentication key access only). There are various sets of data that can be exported, by using the authentication key provided by the system (also shown on the export page). If for whatever reason you would need to invalidate your current key and get a new one instead (for example due to the old one becoming compromised) just hit the reset link next to the authentication key in the export view or in your "my profile" view.
+To find out about the various export formats and the usage within the automation functions, please read the page on the [API's usage](#api).
+
+## Exporting data:
+
+For users that do not have authentication key access, an alternate export feature is available that relies on your interactive login to the site. To access these, just use the export menu button to the left and you'll be presented with a list of export options.<br/> 
+Depending on your server's configuration, you will be presented with one of two possible pages, depending on whether you have background processing enabled or not.
+
+#### Export page with background jobs disabled
+
+The page will list a set of export formats that you can immediately download as a file. Just click on the desired export format and MISP will start collecting all the data that you will receive in a file. Keep in mind that this can be a lengthy process. To avoid having to wait, consult with your instance's site administrator about enabling the background processing.
+
+![Use the export features here to quickly download data in various formats](figures/export.png)
+
+#### Export page with background jobs enabled
+
+If the background jobs are enabled, you'll be redirected to a different version of the export page. Here you will see a table with all of the major export formats and the current status of the cached export files. Keep in mind that these are generated on an organisation by organisation basis, so even though others have generated newer export caches your organisation may have an outdated cache. You can simply issue a generate command (by clicking the "Generate" button) on the desired export type and the background workers will start fetching and assembling your cache. A progress bar will show the progress of the export process.
+Once done, you can click "Download" to download the freshly generated cache file. If the cache is already up to date from before, then you don't have to regenerate the cache, just click on the "download" button.
+You may have noticed that the TEXT export only has a generate button - this is because TEXT exports are made up of a lot of types of exports, all of which get generated together. To download any of these files, just click on any of the attribute types at the bottom of the table.
+A quick description of each of the fields in the table:
+*   **Type**: The type of the export (such as XML, Suricata, MD5, etc.).
+*   **Last Update**: The generation date of the current cache for the given export type.
+*   **Description**: A description of the export format. 
+*   **Outdated**: This compares the cache generation date to the last timestamp when an event was updated and lets you know whether the cache is outdated or not. 
+*   **Progress**: Shows the progress of the last initiated generation process.
+*   **Actions**: Download or Generate the given cache with these buttons. 
+
+![Use the export features here to quickly download data in various formats](figures/export_bg.png)
+
+#### Exporting search results and individual events
+Apart from the options offered by the export pages, it's also possible to export all events involved in a search attribute result table, by using the "Download results as XML" button on the left menu bar.
+
+![Download a .xml from all the events that are shown through an attribute in the search results.](figures/export_search.png) 
+
+Each event's view has its own export feature, both as an XML export and as a .ioc file. To reach these features, just navigate to an event and use the appropriate buttons on the right side.
+
+![Download a .xml or a .ioc of the event.](figures/export_search.png)
+
+## Connecting to other instances:
+
+Apart from being a self contained repository of attacks/malware, one of the main features of MISP is its ability to connect to other instances and share (parts of) its information. The following options allow you to set up and maintain such connections.
+
+### Setting up a connection to another server:
+
+In order to share data with a remote server via pushes and pulls, you need to request a valid authentication key from the hosting organisation of the remote instance. When clicking on List Servers and then on New Server, a form comes up that needs to be filled out in order for your instance to connect to it. The following fields need to be filled out:
+
+![Make sure that you enter the authentication key that you have been given by the hosting organisation of the remote instance, instead of the one you have gotten from this one.](figures/add_server.png)
+
+*   **Base URL:** The URL of the remote server.
+*   **Organization:** The organisation that runs the remote server. It is very impoportant that this setting is filled out exactly as the organisation name set up in the bootstrap file of the remote instance.
+*   **Authkey:** The authentication key that you have received from the hosting organisation of the remote instance.
+*   **Push:** This check-box controls whether your server is allowed to push to the remote instance.
+*   **Pull:** This check-box controls whether your server can request to pull all data from the remote instance.
+*   **Self Signed:** Ticking this checkbox will allow syncing with instances using self-signed certificates.
+*   **Certificate File:** If the instance that you want to connect to has their entire own certificate chain, you can use this to import a .pem file with it and override CakePHP's standard root CA file.
+
+**If you are an administrator**, trying to allow another instance to connect to your own, it is vital that two rules are followed when setting up a synchronisation account: 
+*   The synchronisation user has to have the sync permission and full read/write/publish privileges turned on
+*   Both the sync user and the organisation setting in your instance's Config/bootstrap.php file have to match the organisation identifier of the hosting organisation.
+
+### Browsing the currently set up server connections and interacting with them:
+
+If you ever need to change the data about the linked servers or remove any connections, you have the following options to view and manipulate the server connections, when clicking on List Servers: (you will be able to see a list of all servers that your server connects to, including the base address, the organisation running the server the last pushed and pulled event IDs and the control buttons.).
+
+![Apart from editing / deleting the link to the remote server, you can issue a push all or pull all command from here.](figures/list_servers.png)
+
+*   **Editing the connection to the:** By clicking edit a view, [that is identical to the new instance view](#setting-up-a-connection-to-another-server), is loaded, with all the current information of the instance pre-entered.
+*   **Deleting the connection to the instance:** Clicking the delete button will delete the link to the instance.
+*   **Push all:** By clicking this button, all events that are eligible to be pushed on the instance you are on will start to be pushed to the remote instance. Events and attributes that exist on the far end will be updated.
+*   **Pull all:** By clicking this button, all events that are set to be pull-able or full access on the remote server will be copied to this instance. Existing events will not be updated.
+
+## Rest API:
+
+The platform is also [RESTfull](http://en.wikipedia.org/wiki/Representational_state_transfer), so this means that you can use structured format (XML or JSON) to access Events data.
+
+### Requests
+
+Use any HTTP compliant library to perform requests.
+You can choose which format you would like to use as input/output for the REST calls by specifying the Accept and Content-Type headers.
+
+The following headers are required if you wish to recieve / push XML data:
+**Authorization**: _your authorisation key_
+**Accept**: _application/xml_
+**Content-Type**: _application/xml_
+
+The following headers are required if you wish to recieve / push JSON data:
+**Authorization**: _your authorisation key_
+**Accept**: _application/json_
+**Content-Type**: _application/json_
+The following table shows the relation of the request type and the resulting action:
+
+| HTTP format       | URL            | Controller action invoked     |
+| ----------------- | -------------- | ----------------------------- |
+| GET               | /events        | EventsController::index()     |
+| GET               | /events/123    | EventsController::view(123)   |
+| POST              | /events        | EventsController::add()       |
+| PUT               | /events/123    | EventsController::edit(123)   |
+| DELETE            | /events/123    | EventsController::delete(123) |
+| POST              | /events/123    | EventsController::edit(123)   |
+
+*Attachments are included using base64 encoding below the `data` tag.
+<br/>
+
+### Example - Get single Event
+
+In this example we fetch the details of a single Event (and thus also his Attributes).
+The request should be:
+
+`GET https://your_misp_url/events/123`
+
+And with the HTTP Headers:
+`Accept: application/xml`
+`Authorization: your_api_key`
+
+The response you're going to get is the following data:
+
+```xml
+<pre><?xml version="1.0" encoding="UTF-8" standalone="no"?>;
+<response>
+	<Event>
+		<id>57</id>
+		<org>NCIRC</org>
+		<date>2014-03-04</date>
+		<threat_level_id>1</threat_level_id>
+		<info>Code monkey doing code monkey stuff</info>
+		<published>1</published>
+		<uuid>50aa54aa-f7a0-4d74-910d-10f0ff32448e</uuid>
+		<attribute_count>1</attribute_count>
+		<analysis>1</analysis>
+		<timestamp>1393327600</timestamp>
+		<distribution>1</distribution>
+		<proposal_email_lock>0</proposal_email_lock>
+		<orgc>Iglocska</orgc>
+		<locked>0</locked>
+		<publish_timestamp>1393327600</publish_timestamp>
+		<Attribute>
+			<id>9577</id>
+			<type>other</type>
+			<category>Artifacts dropped</category>
+			<to_ids>1</to_ids>
+			<uuid>50aa54bd-adec-4544-b494-10f0ff32448e</uuid>
+			<event_id>57</event_id>
+			<distribution>1</distribution>
+			<timestamp>1393327600</timestamp>
+			<comment>This is an Attribute</comment>
+			<value>Some_attribute</value>
+			<ShadowAttribute />
+		</Attribute>
+		<ShadowAttribute />
+		<RelatedEvent />
+	</Event>
+	<xml_version>2.2.0</xml_version>
+</response>
+```
+
+
+#### Example - Add new Event
+
+In this example we want to add a single Event.
+The request should be:
+
+```
+POST https://your_misp_url/events
+Accept: application/xml
+Authorization: your_api_key
+```
+
+And the request body:
+
+```xml
+<Event>
+	<date>2014-03-04</date>
+	<threat_level_id>1</threat_level_id>
+	<info>Something concise</info>
+	<published>1</published>
+	<analysis>1</analysis>
+	<distribution>1</distribution>
+	<Attribute>
+		<type>other</type>
+		<category>Artifacts dropped</category>
+		<to_ids>1</to_ids>
+		<distribution>1</distribution>
+		<comment>This is an Attribute</comment>
+		<value>Some_attribute</value>
+	</Attribute>
+</Event>
+```
+
+The response you're going to get is the following data:
+
+```
+HTTP/1.1 100 Continue
+HTTP/1.1 200 Continue
+Date: Tue, 04-Mar-2014 15:00:00
+Server: Apache/2.2.22 (Ubuntu) PHP/5.4.9-4ubuntu2.3
+X-Powered-By: PHP/5.4.9-4ubuntu2.3
+Set-Cookie: CAKEPHP=deleted; expires=Wed, 05-Mar-2014 15:00:00 GMT; path=/
+Set-Cookie: CAKEPHP=a4ok3lr5p9n5drqj27025i4le3; expires Tue, 04-Mar-2014 15:00:00 GMT; path=/; HttpOnly
+Content-Length: 1 kB
+Content-Type: application/xml
+```
+
+```xml
+<?xml version = "1.0" encoding = "UTF-8"?>
+<response>
+	<Event>
+		<id>76</id>
+		<org>NCIRC</org>
+		<date>2014-03-04</date>
+		<threat_level_id>1</threat_level_id>
+		<info>Something concise</info>
+		<published>1</published>
+		<uuid>50aa54aa-f7a0-4d74-920d-10f0ff32448e</uuid>
+		<attribute_count>1</attribute_count>
+		<analysis>1</analysis>
+		<timestamp>1393328991</timestamp>
+		<distribution>1</distribution>
+		<proposal_email_lock>0</proposal_email_lock>
+		<orgc>Iglocska</orgc>
+		<locked>0</locked>
+		<publish_timestamp>1393947960</publish_timestamp>
+		<Attribute>
+			<id>10462</id>
+			<type>other</type>
+			<category>Artifacts dropped</category>
+			<to_ids>1</to_ids>
+			<uuid>50aa54bd-adec-4544-b412-10f0ff32448e</uuid>
+			<event_id>76</event_id>
+			<distribution>1</distribution>
+			<timestamp>1393328991</timestamp>
+			<comment/>
+			<value>Some_attribute</value>
+			<ShadowAttribute/>
+		</Attribute>
+		<ShadowAttribute/>
+		<RelatedEvent>
+			<id>75</id>
+			<org>NCIRC</org>
+			<date>2012-11-19</date>
+			<info>Code monkey doing code monkey stuff</info>
+			<uuid>50aa54aa-f7a0-4d74-910d-10f0ff32448e</uuid>
+			<published>1</published>
+			<analysis>1</analysis>
+			<attribute_count>1</attribute_count>
+			<orgc>Iglocska</orgc>
+			<timestamp>1393327600</timestamp>
+			<distribution>1</distribution>
+			<proposal_email_lock>0</proposal_email_lock>
+			<locked>0</locked>
+			<threat_level_id>1</threat_level_id>
+			<publish_timestamp>1393947655</publish_timestamp>
+		</RelatedEvent>
+	</Event>
+	<xml_version>2.2.0</xml_version>
+</response>
+```
+
+The respone from requesting an invalid page
+
+```xml
+<?xml version = "1.0" encoding = "UTF-8"?>
+<response>
+	<name>Not Found</name>
+	<url>/The_meaning_of_life</url>
+</response>
+```
+
+
+