From eb69a6e78f52b680393383580483192b839f0c1a Mon Sep 17 00:00:00 2001 From: iglocska Date: Thu, 6 Mar 2014 09:20:05 +0100 Subject: [PATCH] User guide and UI changes - first set of changes to the user guide, still missing updated images - some UI changes to make the looks a bit more appealing --- app/View/Elements/footer.ctp | 5 +- app/View/Elements/global_menu.ctp | 67 +++-- app/View/Layouts/default.ctp | 14 +- app/View/Pages/doc/administration.ctp | 59 +++- app/View/Pages/doc/concepts.ctp | 32 ++- app/View/Pages/doc/general.ctp | 10 +- app/View/Pages/doc/user_management.ctp | 13 +- app/View/Pages/doc/using_the_system.ctp | 353 +++++++++++++++--------- app/webroot/css/main.css | 15 +- tools/curl/addEvent.sh | 4 +- 10 files changed, 388 insertions(+), 184 deletions(-) diff --git a/app/View/Elements/footer.ctp b/app/View/Elements/footer.ctp index bf2ed717d..179aa72fe 100644 --- a/app/View/Elements/footer.ctp +++ b/app/View/Elements/footer.ctp @@ -1,6 +1,7 @@
-
-
+
+
+
"> +
+
-
+
+ + + + + +
- -
-
- 0): ?> - events - - events - - Malware - Information - Sharing - Platform -
-
-
\ No newline at end of file +
+ \ No newline at end of file diff --git a/app/View/Layouts/default.ctp b/app/View/Layouts/default.ctp index 410b4fce6..c4f73f921 100755 --- a/app/View/Layouts/default.ctp +++ b/app/View/Layouts/default.ctp @@ -8,7 +8,6 @@ Html->meta('icon'); - // echo $this->Html->css('cake.generic'); echo $this->Html->css('roboto'); echo $this->Html->css('bootstrap'); // see http://twitter.github.io/bootstrap/base-css.html @@ -33,6 +32,7 @@
element('global_menu'); + //throw new Exception(serialize($this->Session->read('Message.flash.message'))); if ($debugMode == 'debugOff') { ?>
@@ -50,7 +50,11 @@
Session->check('Message'); + if (Configure::read('debug') == 0 && $this->Session->check('Message')) { + echo 'class="topGap"'; + } ?> > fetch('content'); ?> @@ -70,10 +74,16 @@
Loading
+ + diff --git a/app/View/Pages/doc/administration.ctp b/app/View/Pages/doc/administration.ctp index f1b27aa60..97db7849d 100755 --- a/app/View/Pages/doc/administration.ctp +++ b/app/View/Pages/doc/administration.ctp @@ -13,6 +13,8 @@
  • User Management
  • Role Management
  • Logging
    • +
  • Administrative Tools
    • +
  • Background Processing
  • Html->link('Categories and Types', array('controller' => 'pages', 'action' => 'display', 'doc', 'categories_and_types')); ?>
    • @@ -109,9 +111,12 @@ Privileges are assigned to users by assigning them to rule groups, which use one Manage Organization Events: allows users to create events or modify and delete events created by a member of their organisation.
    Manage & Publish Organisation Events: This last setting, gives users the right to do all of the above and also to publish the events of their organisation.
    Perm sync: This setting allows the users of the role to be used as a synchronisation user. The authentication key of this user can be handed out to the administrator of a remote MISP instance to allow the synchronisation features to work.
    -Perm admin: Gives the user administrator privileges, this setting is used for the organisation admins.
    -Perm audit: Grants access to the logs. With the exception of site admins, only logs generated by the user's own org are visible.
    Perm auth: This setting enables the authentication key of the role's users to be used for rest requests.
    +Perm admin: Gives the user limited administrator privileges, this setting is used for the organisation admins.
    +Perm site admin: Gives the user full administrator privileges, this setting is used for the site admins.
    +Perm audit: Grants access to the logs. With the exception of site admins, only logs generated by the user's own org are visible.
    +Perm regexp access: Allows the users with this permission enabled to edit the regular expression table. Be careful when giving out this permission, incorrect regular expressions can be very harmful (infinite loops, loss of data, etc.).
    +Perm tagger: Allows the user with this permission to create custom tags and assign them to events.

    Creating roles:

    When creating a new role, you will have to enter a name for the role to be created and set up the permissions (as described above) using the radio toggle and the four check-boxes.

    Listing roles:

    @@ -120,11 +125,12 @@ By clicking on the List Roles button, you can view a list of all the currently r

    Using the logs of MISP

    Users with audit permissions are able to browse or search the logs that MISP automatically appends each time certain actions are taken (actions that modify data or if a user logs in and out).
    -Generally, the following actions are logged:
    +Generally, the following actions are logged:

    • User: Creation, deletion, modification, Login / Logout
    • Event:Creation, deletion, modification, publishing
    • Attribute: Creation, deletion, modification
      • +
    • ShadowAttribute: Creation, deletion, Accept, Discard
    • Roles: Creation, deletion, modification
    • Blacklist: Creation, deletion, modification
    • Whitelist: Creation, deletion, modification
      • @@ -133,7 +139,7 @@ Generally, the following actions are logged:

      Browsing the logs:

      Listing all the log entries will show the following columns generated by the users of your organisation (or all organisations in the case of site admins):
      -List logs
      +List logs

      • Id: The automatically assigned ID number of the entry.
      • Email: The e-mail address of the user whose actions triggered the entry.
        • @@ -148,7 +154,7 @@ Listing all the log entries will show the following columns generated by the use
      Search log

      Searching the Logs:

      -Another way to browse the logs is to search it by filtering the results according to the following fields (the search is a sub-string search, the sub-string has to be an exact match for the entry in the field that is being searched for):
      +Another way to browse the logs is to search it by filtering the results according to the following fields (the search is a sub-string search, the sub-string has to be an exact match for the entry in the field that is being searched for):

      • Email: By searching by Email, it is possible to view the log entries of a single user.
      • Org: Searching for an organisation allows you to see all actions taken by any member of the organisation.
        • @@ -156,5 +162,46 @@ Another way to browse the logs is to search it by filtering the results accordin
      • Title: There are several ways in which to use this field, since the title fields contain several bits of information and the search searches for any substrings contained within the field, it is possible to just search for the ID number of a logged event, the username / server's name / event's name / attribute's name of the event target.
      • Change: With the help of this field, you can search for various specific changes or changes to certain variables (such as published will find all the log entries where an event has gotten published, ip-src will find all attributes where a source IP address has been entered / edited, etc).
      - +
      +

      Administrative Tools

      +MISP has a couple of administrative tools that help administrators keep their instance up to date and healthy. The list of these small tools can change rapidly with each new version, but they should be self-explanatory. Make sure to check this section after upgrading to a new version, just in case there is a new upgrade script in there - though if this is the case it will be mentioned in the upgrade instructions.

      +
      +

      Background Processing

      +If enabled, MISP can delegate a lot of the time intensive tasks to the background workers. These will then be executed in order, allowing the users of the instance to keep using the system without a hiccup and without having to wait for the process to finish. It also allows for certain tasks to be scheduled and automated. +

      Command Line Tools for the Background Workers

      +The background workers are powered by CakeResque, so all of the CakeResque commands work. +To start all of the workers needed by MISP go to your /var/www/MISP/app/Console/worker (assuming a standard installation path) and execute start.sh. +To interact with the workers, here is a list of useful commands. Go to your /var/www/MISP/app/Console (assuming a standard installation path) and execute one of the following commands as a parameter to ./cake CakeResque.CakeResque (for example: ./cake CakeResque.CakeResque tail):

      +
        +
      • tail: tail the various log files that CakeResque creates, just choose the one from the list that you are interested in.
        • +
      • cleanup: terminate the job that a worker is working on immediately. You will be presented with a choice of workers to choose from when executing this command.
        • +
      • clear: Clear the queue of a worker immediately.
        • +
      • stats: shows some statistics about your workers including the count of successful and failed jobs.
        • +
      +The other commands should not be needed, instead of starting / stopping or restarting workers use the supplied start.sh (it stops all workers and starts them all up again). For further instructions on how to use the console commands for the workers, visit the CakeResque list of commands.
      +

      Monitoring the Background Processes

      +The "Jobs" menu item within the Administration menu allows site admins to get an overview of all of the currently and in the past scheduled jobs. Admins can see the status of each job, and what the queued job is trying to do. If a job fails, it will try to set an error message here too. The following columns are shown in the jobs table:

      +
        +
      • Id: The job's ID (this is the ID of the job's metadata stored in the default datastore, not to be confused with the process ID stored in the redis database and used by the workers)
        • +
      • Process: The process's ID.
        • +
      • Worker: The name of the worker queue. There are 3+1 workers running if background jobs are enabled: default, cache, email, and a special Scheduler (this should never show up in the jobs table).
        • +
      • Job Type: The name of the queued job.
        • +
      • Input: Shows a basic input handled by the job - such as "Event:50" for a publish email alert job for event 50.
        • +
      • Message: This will show what the job is currently doing or alternatively an error message describing why a job failed.
        • +
      • Org: The string identifier of the organisation that has scheduled the job.
        • +
      • Status: The status reported by the worker.
        • +
      • Retries: Currently unused, it is planned to introduced automatic delayed retries for the background processing to add resilience.
        • +
      • Progress: A progress bar showing how the job is coming along.
        • +
      +

      Scheduling Jobs and Recurring Jobs

      +Apart from off-loading long-lasting jobs to the background workers, there is a second major benefit of enabling the background workers: Site-administrators can schedule recurring tasks for the jobs that generally take the longest to execute. At the moment this includes pushing / pulling other instances and generating a full export cache for every organisation and export type. MISP comes with these 3 tasks pre-defined, but further tasks are planned. The following fields make up the scheduled tasks table:

      +
        +
      • Id: The ID of the task.
        • +
      • Type: The type of the task.
        • +
      • Frequency (h): This number sets how often the job should be executed in hours. Setting this to 168 and picking the next execution on Sunday at 01:00 would execute the task every Sunday at 1 AM. Setting this value to 0 will make the task only run once on the scheduled date / time without rescheduling it afterwards.
        • +
      • Scheduled Time: The time (in 24h format) when the task should be executed the next time it runs (and all consecutive times if a multiple of 24 is chosen for frequency).
        • +
      • Next Run: The date on which the task should be executed.
        • +
      • Description: A brief description of the task.
        • +
      • Message: This field shows when the job was queued by the scheduler for execution.

        • +
    \ No newline at end of file diff --git a/app/View/Pages/doc/concepts.ctp b/app/View/Pages/doc/concepts.ctp index 0b2c749a2..194957012 100644 --- a/app/View/Pages/doc/concepts.ctp +++ b/app/View/Pages/doc/concepts.ctp @@ -12,22 +12,32 @@

    General Concepts

    +

    Admins and Site Admins

    +There are two types of admins in MISP: Admins (also refered to as org admins) and Site Admins. Whilst the former can only do some limited administration of users of his/her own organisation, site admins have access to all of the features and data of the system. They are in charge of making sure that the system runs correctly and the maintenance of MISP. +

    Background Jobs

    +A lot of the heavier tasks are a burden to users, in that their actions can cause long delays (and in some cases timeouts) while the application logic is executing. To alleviate this, long processes have been (if enabled) moved to background jobs, meaning that their execution happens asynchronously in the background, allowing the user to freely interact with the platform whilst the request is being processed. +

    MISP Instance

    +A MISP instance is an installation of the MISP software and the connected database. All the data visible to the users is stored locally in the database and data that is shareable (based on the distribution settings) can be synchronised with other instances via the Sync actions. The instance that you are reading this manual on will be refered to as "this instance" or "your instance". The instances that your instance synchronises with will be refered to as "remote instances".

    Organisation administrators and Site administrators

    We have two types of administrators, site and organisation admins. The former has access to every administrator feature for all the data located on the system including global features such as the creation and modification of user roles and instance links, whilst organisation admins can administer users, events and logs of their own respective organisations.
    -

    Synchronisation

    -What we call synchronisation is an exchange of data between two (or more) MISP instances through our pull and push mechanisms. +

    Pivot path

    +The (branching) path taken by a user from event to event while following correlation links. This is represented by the branching graph in the event view. +

    Pivoting

    +The act of navigating from event to event through correlation links. +

    Proposals

    +Each event can only be directly edited by users of the original creator organisation (and site admins). However, if another organisation would like to amend an event with extra information on an event, or if they'd like to correct a mistake in an attribute, they can create a Proposal. These proposals could then be accepted by the original creator organisation. These proposals can be pulled to another server, allowing users on connected instances to propose changes which then could be accepted by the original creators on another instance (and subsequently pushed back). +

    Publishing

    +When an event is first created by a user, it is visible to everyone on the instance based on the access rights ("Your organisation only" events will not be visible to users of other organisations), but they will not be synchronised and they won't be exportable. For this, a user with publishing permission of the organisation that created the event has to publish the event. The system will then inform all the users of the instance that are subscribing to e-mail notifications and who have access to view the published event via an e-mail.

    Pull

    Pulling is the process of using the configured sync user on a remote instance to REST GET all of the accessible data (based on the distribution rights) to your instance and store it.

    Push

    Pushing is the process of using a configured instance link to send an event or all accessible events (limited by the distribution rights) through the REST interface to a remote instance. -

    Publishing

    -When an event is first created by a user, it is visible to everyone on the instance based on the access rights ("Your organisation only" events will not be visible to users of other organisations), but they will not be synchronised and they won't be exportable. For this, a user with publishing permission of the organisation that created the event has to publish the event. The system will then inform all the users of the instance that are subscribing to e-mail notifications and who have access to view the published event via an e-mail. -

    MISP Instance

    -A MISP instance is an installation of the MISP software and the connected database. All the data visible to the users is stored locally in the database and data that is shareable (based on the distribution settings) can be synchronised with other instances via the Sync actions. The instance that you are reading this manual on will be refered to as "this instance" or "your instance". The instances that your instance synchronises with will be refered to as "remote instances". -

    Sync user

    +

    Scheduled Tasks

    +Certain common tasks can be scheduled for a later execution or for regular recurring executions. These tasks currently include caching all of the export formats, pulling from all eligible instances and pushing to all eligible instances. +

    Sync User

    A user of a role that grants sync permissions, these users (and their authentication keys) are used to serve as the points of connection between instances. Events pushed to an instance are pushed to a sync user, who then creates the events on the remote instance. Events pulled are added by the sync user that is used to connect the remote instance to your instance. As an administrator, keep in mind that a sync user needs auth key and publish permissions, has to have undergone the mandatory password change and has to have accepted the Terms of Use in order for the sync to work. Please make sure that all of these steps are taken before attempting to push or pull. -

    Admins and Site Admins

    -There are two types of admins in MISP: Admins (also refered to as org admins) and Site Admins. Whilst the former can only do some limited administration of users of his/her own organisation, site admins have access to all of the features and data of the system. They are in charge of making sure that the system runs correctly and the maintenance of MISP. -

    Background jobs

    -A lot of the heavier tasks are a burden to users, in that their actions can cause long delays (and in some cases timeouts) while the application logic is executing. To alleviate this, long processes have been (if enabled) moved to background jobs, meaning that their execution happens asynchronously in the background, allowing the user to freely interact with the platform whilst the request is being processed. +

    Synchronisation

    +What we call synchronisation is an exchange of data between two (or more) MISP instances through our pull and push mechanisms. +

    Tagging

    +Users with tagging rights can assigned various dynamically created tags to events, allowing an arbitrary link between events to be created. It is possible to filter events based on these tags and they can also be used to filter events for the automation.
    \ No newline at end of file diff --git a/app/View/Pages/doc/general.ctp b/app/View/Pages/doc/general.ctp index 416e7a6ad..62380d1af 100644 --- a/app/View/Pages/doc/general.ctp +++ b/app/View/Pages/doc/general.ctp @@ -23,6 +23,8 @@
  • Sync Actions: With administrator access rights, shows a list of the connected instances and allows the initiation of a push and a pull (more about the synchronisation mechanisms later).
  • Administration: Administrators can add, edit or remove user accounts and user roles. Roles define the access rights to certain features such as publishing of events, usage of the REST interface or synchronisation of any user belonging to the given role. Site administrators can also access a contact form, through which it is possible to reset the passwords of users, or to just get in touch with them via encrypted e-mails.
  • Audit: If you have audit permissions, you can view the logs for your organisation (or for site admins for the entire system) here or even search the logs if you are interested in something specific.
    • +
  • Discussions: Link to the discussion threads.
    • +
  • Proposal Notifications: This shows how many proposals your organisation has received and across how many events they are spread out. Clicking this will take you to the list of proposals.
  • Log out: Logs you out of the system.
    • @@ -35,16 +37,16 @@
  • Search Attributes: You can set search terms for a filtered attribute index view here.
  • View Proposals: Shows a list of all proposals that you are eligible to see.
  • Events with proposals: Shows all of the events created by your organsiation that has pending proposals.
    • -
  • List Tags:List all the tags that have been created by users with tag creation rights on this instance
    • -
  • Add Tag:Create a new tag
    • +
  • List Tags:List all the tags that have been created by users with tag creation rights on this instance.
    • +
  • Add Tag:Create a new tag.
  • Export: Export the data accessible to you in various formats.
  • Automation: If you have authentication key access, you can view how to use your key to use the REST interface for automation here.
    • Input filters
      -
    • Import Whitelist: You can view the whitelist rules, which contain the values that are blocked from being used for exports and automation on this instance. Site administrators have access to editing this list.
    • Import Regexp: You can view the Regular Expression rules, which modify the data that can be entered into the system. This can and should be used to help filter out personal information from automatic imports (such as removing the username from windows file paths), having unified representation for certain common values for easier correlation or simply standardising certain input. It is also possible to block certain values from being inserted. As a site administrator or a user with regex permission, you can also edit these rules.
      • +
    • Signature Whitelist: You can view the whitelist rules, which contain the values that are blocked from being used for exports and automation on this instance. Site administrators have access to editing this list.
    Global Actions
    @@ -55,7 +57,7 @@
  • Role Permissions: You can view the role permissions here.
  • User Guide: A link to this user guide.
  • Terms & Conditions: View the terms & conditions again.
    • -
  • Statistics: View a series of statistics about the users and the data on this instane
    • +
  • Statistics: View a series of statistics about the users and the data on this instance.
  • Log out: Logs the current user out.
    • diff --git a/app/View/Pages/doc/user_management.ctp b/app/View/Pages/doc/user_management.ctp index 5a3a182e8..e6d17d37f 100644 --- a/app/View/Pages/doc/user_management.ctp +++ b/app/View/Pages/doc/user_management.ctp @@ -52,10 +52,19 @@ MISP also provides its users with some information about itself and its users th
  • Member statistics: By using the Members List menu button on the left, you can get a quick overview over how many users each organisation has registered on your server, and a histogram, depicting the distribution of attribute types created by each organisation.

  • User Guide: The user guide is also accessible via the Global Actions menu. You can find out more about how to use the system by reading this.

  • Terms & Conditions: It is possible to review the terms & conditions that were shown during the first run of the system by clicking on the terms & conditions link in the Global Actions menu.

    • -
  • Statistics: View statistics about the users and the data contained within this instance.
    • +
  • Statistics: View statistics about the users and the data contained within this instance. +
      +
    • + General Statistics: View a set of statistics such as the number of Events and Attributes currently in existance on the platform. The number in the bracket shows the number of new items added during this week. +
      • +
    • + Activity Heatmap: This graph shows a heatmap of all activity related to creating event related data on a day by day basis. By default, the graph shows the sum of the contributions of all organisations, but using the buttons representing each organisation in existance on the platform you can switch to the activity heatmap of a single organisation. If you'd like to see the activity further back in the past, just use the arrow buttons to navigate the heatmap. +
      • +
    +

    • Inspecting the input filters:

    -All the events and attributes that get entered into MISP will be run through a series of input filters. These are defined by the site administrators, but every user can take a look at the currently active lists.

    +All the events and attributes that get entered into MISP will be run through a series of input filters. These are defined by the site administrators or users with special privileges to edit the filters, but every user can take a look at the currently active lists.