From 6aa91e5cfcddc70250e72f46d95750f4bcfc3468 Mon Sep 17 00:00:00 2001 From: Camille Schneider Date: Mon, 2 Apr 2018 17:07:27 +0200 Subject: [PATCH 1/5] Add an illustration to the MISP synchronisation mechanism --- sharing/README.md | 43 +- sharing/figures/misp-sync-servers.svg | 1621 +++++++++++++++++++++++++ 2 files changed, 1652 insertions(+), 12 deletions(-) create mode 100644 sharing/figures/misp-sync-servers.svg diff --git a/sharing/README.md b/sharing/README.md index d80f288..45a72e4 100644 --- a/sharing/README.md +++ b/sharing/README.md @@ -4,7 +4,7 @@ * [Explanation](#users) * [Setup](#setup) -* [Roles](#roles) +* [Rules](#rules) * [Tools](#tools) * [Server Settings](#server-settings) * [Events](#events) @@ -15,15 +15,34 @@ * Quick benefit without the obligation to contribute * Low barrier access to get acquainted to the system -##Concept +## Concept The following figure shows the concept how different MISP instances could tie together. ![Scenario example](figures/MISP_scenario_example.png) -##Setup +## Setup +### Introduction -###Adding a server +In MISP, two ways exist to get events: + +* **Use case 1**: From another MISP server (also called MISP instance), by synchronising two MISP servers. +* **Use case 2**: From a link, by using [Feeds](../managing-feeds). + +The example below illustrate the synchronisation between two MISP servers (use case 1). +An organisation B (OrgB) wants to synchronise its MISP server, called ServerB, with the MISP server of an organisation A (Org A), called ServerA. The following steps can be taken to syncronise ServerB with ServerA: + +

+ Synchronisation between two MISP servers +

+ +* **Step 1**: Add OrgB as a local organisation on ServerA (OrgB.ServerA). +* **Step 2**: Add a Sync User (syncuser@OrgB.ServerA) in the organisation OrgB.ServerA on the MISP ServerA. +* **Step 3**: [Set up a sync server](###adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP serverA. + +For additional information on the synchronisation process, refer to the [MISP GitHub issues](https://github.com/MISP/MISP/issues), for example [issue 2595](https://github.com/MISP/MISP/issues/2595). + +### Adding a server Servers can be added by users via @@ -89,15 +108,15 @@ https:///servers/add You can also upload a certificate file if the instance you are trying to connect to has its own signing authority. (*.pem) -###Test connection +### Test connection Test connection can be used to test the connection to the remote server and will give a feedback about local and remote version of MISP. -###Rules +### Rules Rules are used to limit sharing to e.g. events with a given tag, or disabling sharing for events containing a certain Tag. -###Troubleshooting +### Troubleshooting If you have issues connecting to a remote servers try to do the following things: @@ -106,7 +125,7 @@ If you have issues connecting to a remote servers try to do the following things - with connection issues do a package capture to find out more - if you have a SSL connection issue to a remote server with a signed by a CA that is not included in OS, make sure the whole certificate path is included in the path. -##Collaboration +## Collaboration ### Proposals @@ -128,7 +147,7 @@ https:///threads/index ![Discussions](figures/discussions.png) -####Create a new Topic +#### Create a new Topic To create a new topic @@ -138,7 +157,7 @@ https:///posts/add ![Start a topic](figures/discussions_start_topic.png) -####Comment a topic +#### Comment a topic A topic can be commented by any user @@ -183,7 +202,7 @@ This will describe what to do within events to be shared. There is an article about sharing groups in [here](../using-the-system/#create-and-manage-sharing-groups) -#Recommendation +# Recommendation The following section will describe what is the best practice how many MISP instances that showed to be good for orgs. Of course depending on your specific requirements an architecture could be more spread or simplified. @@ -200,4 +219,4 @@ It will download all events and do enrichment between these events. This system is the main system used by human analysts. It will it is not linked to any external MISP instance other then the Staging System. -To publish events to the community assign the right tags to match your push [Rules](#rules) and **publish the event** +To publish events to the community assign the right tags to match your push [Rules](#rules) and **publish the event** \ No newline at end of file diff --git a/sharing/figures/misp-sync-servers.svg b/sharing/figures/misp-sync-servers.svg new file mode 100644 index 0000000..e8356b2 --- /dev/null +++ b/sharing/figures/misp-sync-servers.svg @@ -0,0 +1,1621 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + Organsiation + + OrgA.SeverA + + + + + User + + + + + + User + + + + + + User + + OrgB.ServerA + + + + Sync User + + + + + User + + + + + + User + + + OrgB.ServerB + + + + + User + + + + + + User + + + + + + User + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Operatedby B + MISPServerB + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Operatedby A + MISPServerA + + + + + + + Legend + + + + + + + + Synchronisation between twoMISP servers + Organisation in the MISP databaseof a MISP server Organisation User of an organisation in the MISPdatabase of a MISP server + + + + + + + + + + + + + + + + + + + + + + + + + + + MISP server (also called MISP instance) + + + + + + 1 + 1 + + + 2 + + 2 + + + + 3 + 3 + + + From 5d11506b9b4b7a7f55b34d346a0eb7b6d4336889 Mon Sep 17 00:00:00 2001 From: Camille Schneider Date: Mon, 2 Apr 2018 18:04:06 +0200 Subject: [PATCH 2/5] fix broken links and bugs in svg figure --- sharing/README.md | 9 +- sharing/figures/misp-sync-servers.svg | 1620 +++++++++++++------------ 2 files changed, 855 insertions(+), 774 deletions(-) diff --git a/sharing/README.md b/sharing/README.md index 45a72e4..ff593ac 100644 --- a/sharing/README.md +++ b/sharing/README.md @@ -2,14 +2,13 @@ ## Sharing / Synchronisation -* [Explanation](#users) +* [Explanation](#concept) * [Setup](#setup) * [Rules](#rules) -* [Tools](#tools) -* [Server Settings](#server-settings) +* [Server Settings](#adding-a-server) * [Events](#events) * [Sharing groups](#sharing-groups) -* [Recommendations](#recommendations) +* [Recommendations](#recommendation) * MISP's core functionality is sharing where everyone can be a consumer and/or a contributor/producer. * Quick benefit without the obligation to contribute @@ -38,7 +37,7 @@ An organisation B (OrgB) wants to synchronise its MISP server, called ServerB, w * **Step 1**: Add OrgB as a local organisation on ServerA (OrgB.ServerA). * **Step 2**: Add a Sync User (syncuser@OrgB.ServerA) in the organisation OrgB.ServerA on the MISP ServerA. -* **Step 3**: [Set up a sync server](###adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP serverA. +* **Step 3**: [Set up a sync server](#adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP serverA. For additional information on the synchronisation process, refer to the [MISP GitHub issues](https://github.com/MISP/MISP/issues), for example [issue 2595](https://github.com/MISP/MISP/issues/2595). diff --git a/sharing/figures/misp-sync-servers.svg b/sharing/figures/misp-sync-servers.svg index e8356b2..7b53b7a 100644 --- a/sharing/figures/misp-sync-servers.svg +++ b/sharing/figures/misp-sync-servers.svg @@ -17,7 +17,7 @@ version="1.1" id="svg8" inkscape:version="0.92.2 (5c3e80d, 2017-08-06)" - sodipodi:docname="misp-sync-server.svg"> + sodipodi:docname="misp-sync-servers.svg"> Organsiation - - OrgA.SeverA - - - - - User + id="g1977"> - - - + id="g1881"> - + User + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333311px;line-height:1.25;font-family:sans-serif;-inkscape-font-specification:'sans-serif, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + x="161.33043" + y="81.417549" + id="text1647">OrgA.SeverA + + + + + + User + + + + + + + User + + + + + + + User + + - - - + id="g1914"> - + User - - OrgB.ServerA - - - - Sync User - - - - - User - - - - - - User + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:4.23333311px;line-height:1.25;font-family:Arial;-inkscape-font-specification:'Arial, Normal';font-variant-ligatures:normal;font-variant-caps:normal;font-variant-numeric:normal;font-feature-settings:normal;text-align:start;letter-spacing:0px;word-spacing:0px;writing-mode:lr-tb;text-anchor:start;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332" + x="109.96315" + y="81.53093" + id="text1577">OrgB.ServerA + + + + + + Sync User + + + + + + + User + + + + + + + User + + - - OrgB.ServerB - - - - - User - - - + id="g2007"> - User - - - - - - User + y="75.436554" + x="8.9473619" + height="29.261854" + width="45.227303" + id="rect1128-1-6-0-0" + style="fill:#ffffff;fill-opacity:1;stroke:#000000;stroke-width:0.14214589;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1" /> + OrgB.ServerB + + + + + + User + + + + + + + User + + + + + + + User + + + id="g5-1-2-2-2-8" + transform="matrix(0.14965509,0,0,0.14965509,134.52582,35.687092)"> + id="g7-5-0-5-0-1"> + id="g9-9-3-6-5-6"> - - - - - + id="g11-8-3-5-4-8"> + + + + + + - - - - - - - - - - - - - - Operatedby A - MISPServerA - + + + + + + + + + Operatedby A + MISPServerA + - Legend - - - - - - - Synchronisation between twoMISP servers - Organisation in the MISP databaseof a MISP server Organisation User of an organisation in the MISPdatabase of a MISP server + id="g2062"> + Legend: + + - - - - - - - - - - - - - - - - - - - - - - + transform="translate(0,4.2333335)" + id="g3331"> + id="path1088-9-7-2-9-0" + cx="17.094486" + cy="162.6606" + style="fill:#000000;fill-opacity:1;stroke:#000000;stroke-width:0.36006752;stroke-opacity:1" + r="3.1367824" /> + + + + Synchronisation between twoMISP servers + + Organisation in the MISP databaseof a MISP server + Organisation + User of an organisation in the MISPdatabase of a MISP server + + + + + + + + + + + + + + + + + + + + + + + + + + + + MISP server (also called MISP instance) - MISP server (also called MISP instance) @@ -1563,59 +1649,55 @@ id="tspan1036-6" sodipodi:role="line" /> + id="g2082" + transform="matrix(1.2581629,0,0,1.2581629,-27.394347,-22.067732)"> 1 + transform="matrix(1.137605,0,0,1.0962957,61.193194,74.546053)">1 1 + x="97.914543" + y="78.779259">1 + id="g2086" + transform="matrix(1.2025094,0,0,1.2025094,-17.77181,-24.996134)"> 2 - - 2 - + transform="matrix(1.137605,0,0,1.0962957,62.769371,97.821058)">2 + 2 + id="g2090" + transform="matrix(1.2947565,0,0,1.2947565,-22.289826,-37.258286)"> 3 + transform="matrix(1.137605,0,0,1.0962957,33.762862,122.74571)">3 3 + x="70.318848" + y="126.50646">3 From 6e0c676ce3ea5e5e29eea31d71b2e5cb401fbd1b Mon Sep 17 00:00:00 2001 From: Camille Schneider Date: Mon, 2 Apr 2018 18:12:26 +0200 Subject: [PATCH 3/5] fix typos --- sharing/README.md | 2 +- sharing/figures/misp-sync-servers.svg | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/sharing/README.md b/sharing/README.md index ff593ac..f31d334 100644 --- a/sharing/README.md +++ b/sharing/README.md @@ -37,7 +37,7 @@ An organisation B (OrgB) wants to synchronise its MISP server, called ServerB, w * **Step 1**: Add OrgB as a local organisation on ServerA (OrgB.ServerA). * **Step 2**: Add a Sync User (syncuser@OrgB.ServerA) in the organisation OrgB.ServerA on the MISP ServerA. -* **Step 3**: [Set up a sync server](#adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP serverA. +* **Step 3**: [Set up a sync server](#adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP ServerA. For additional information on the synchronisation process, refer to the [MISP GitHub issues](https://github.com/MISP/MISP/issues), for example [issue 2595](https://github.com/MISP/MISP/issues/2595). diff --git a/sharing/figures/misp-sync-servers.svg b/sharing/figures/misp-sync-servers.svg index 7b53b7a..11684a6 100644 --- a/sharing/figures/misp-sync-servers.svg +++ b/sharing/figures/misp-sync-servers.svg @@ -582,8 +582,8 @@ inkscape:pageopacity="0.0" inkscape:pageshadow="2" inkscape:zoom="1.4" - inkscape:cx="417.0469" - inkscape:cy="245.36859" + inkscape:cx="369.66654" + inkscape:cy="237.43961" inkscape:document-units="mm" inkscape:current-layer="layer1" showgrid="false" @@ -1356,7 +1356,7 @@ inkscape:connector-curvature="0" /> From 84939e9daa81b80b94d60313f22c0cdcc2626c9c Mon Sep 17 00:00:00 2001 From: Camille Schneider Date: Mon, 2 Apr 2018 23:57:54 +0200 Subject: [PATCH 4/5] Re-organise & add a paragraph on distribution --- sharing/README.md | 88 ++++++++++++++++++++++++++++++++--------------- 1 file changed, 60 insertions(+), 28 deletions(-) diff --git a/sharing/README.md b/sharing/README.md index f31d334..7f73c94 100644 --- a/sharing/README.md +++ b/sharing/README.md @@ -2,28 +2,18 @@ ## Sharing / Synchronisation -* [Explanation](#concept) -* [Setup](#setup) -* [Rules](#rules) -* [Server Settings](#adding-a-server) -* [Events](#events) -* [Sharing groups](#sharing-groups) -* [Recommendations](#recommendation) - * MISP's core functionality is sharing where everyone can be a consumer and/or a contributor/producer. * Quick benefit without the obligation to contribute * Low barrier access to get acquainted to the system -## Concept +## Synchronisation +### Concept The following figure shows the concept how different MISP instances could tie together. ![Scenario example](figures/MISP_scenario_example.png) -## Setup -### Introduction - -In MISP, two ways exist to get events: +In MISP, two ways exist to get events from remote sources: * **Use case 1**: From another MISP server (also called MISP instance), by synchronising two MISP servers. * **Use case 2**: From a link, by using [Feeds](../managing-feeds). @@ -32,14 +22,15 @@ The example below illustrate the synchronisation between two MISP servers (use c An organisation B (OrgB) wants to synchronise its MISP server, called ServerB, with the MISP server of an organisation A (Org A), called ServerA. The following steps can be taken to syncronise ServerB with ServerA:

- Synchronisation between two MISP servers + Synchronisation between two MISP servers
+ FIGURE: Illustration of the synchronisation between two MISP servers

* **Step 1**: Add OrgB as a local organisation on ServerA (OrgB.ServerA). * **Step 2**: Add a Sync User (syncuser@OrgB.ServerA) in the organisation OrgB.ServerA on the MISP ServerA. * **Step 3**: [Set up a sync server](#adding-a-server) on MISP ServerB using the key (called Authkey) from the sync user (syncuser@OrgB.ServerA) created on MISP ServerA. -For additional information on the synchronisation process, refer to the [MISP GitHub issues](https://github.com/MISP/MISP/issues), for example [issue 2595](https://github.com/MISP/MISP/issues/2595). +For additional information on the synchronisation process, refer to the [MISP GitHub issues](https://github.com/MISP/MISP/issues), for example, [issue 2595](https://github.com/MISP/MISP/issues/2595). ### Adding a server @@ -113,7 +104,7 @@ Test connection can be used to test the connection to the remote server and will ### Rules -Rules are used to limit sharing to e.g. events with a given tag, or disabling sharing for events containing a certain Tag. +Rules are used to limit sharing when synchronising events and attributes, to e.g. events with a given tag, or disabling sharing for events containing a certain Tag. ### Troubleshooting @@ -124,6 +115,56 @@ If you have issues connecting to a remote servers try to do the following things - with connection issues do a package capture to find out more - if you have a SSL connection issue to a remote server with a signed by a CA that is not included in OS, make sure the whole certificate path is included in the path. +## Sharing and distribution + +The following section describes how distribution mechanisms of events and attributes work. + +### Distribution settings + +The below five distribution settings are available for events and attributes. Descriptions of those settings can be found [here](../using-the-system/#creating-an-event). + +* Your organisation only +* This community only +* Connected communities +* All communities +* Sharing group + +Events that are not published are only distributed/shared to the local organisations on the same MISP server/instance (within the limit of the distribution model). +Only events that are **published** will be shared with remote organisations on other MISP servers via push/pull mechanisms. +More details on publishing events [here](../using-the-system/#publish-an-event). + +### Community + +A community is composed of the local organisations on a MISP server and the remote organisations connected by the sync users. For more information on the concept of community, refer to an [article on MISP information sharing following ISO/IEC 27010](https://github.com/MISP/misp-compliance/blob/master/ISO_IEC_27010/misp-sharing-information-following-ISO-IEC-27010.md), explaining the concept of community. + +Specifically, communities are not reversible. Taking the example of the above figure, illustrating the synchronisation between two MISP servers, OrgB.ServerB is part of the MISP ServerA community but OrgB.ServerA is not part of MISP ServerB community. + +### Distribution mechanisms + +The distribution level of an event is automatically decreased as it is synchronised with other MISP instances, when it was originally set to: + +* Community only (to organisation only) +* Connected community (to community only) + +It is not decreased when it was originally set to: + +* Organisation only +* All communities +* Sharing group + +[!] This rule does not apply if “Internal instance” has been checked when creating the server. + +As an example, the figure below illustrates two events **e** and **e'** created by OrgA and respectively shared as "This community only" and "Connected communities" and how they propagate in an illustrative MISP set of intances synchronised with each others. + +

+ Illustration of MISP organisations and community interactions
+ FIGURE: Illustration of MISP organisations and community interactions +

+ +### Sharing-groups + +There is an article about sharing groups in [here](../using-the-system/#create-and-manage-sharing-groups) + ## Collaboration ### Proposals @@ -191,29 +232,20 @@ These E-Mail alerts are an opt-in feature ![Change user settings](figures/profile_receive_alerts.png) -# Events -This will describe what to do within events to be shared. - -* Only events that are **published** will be shared - -# Sharing-groups - -There is an article about sharing groups in [here](../using-the-system/#create-and-manage-sharing-groups) - -# Recommendation +## Recommendation The following section will describe what is the best practice how many MISP instances that showed to be good for orgs. Of course depending on your specific requirements an architecture could be more spread or simplified. The architecture is divided into several systems / stages beginning with: -## MISP Staging System +### MISP Staging System This systems purpose is to be linked to all available external MISP systems that you have access to. It will download all events and do enrichment between these events. -## MISP SECOps System +### MISP SECOps System This system is the main system used by human analysts. It will it is not linked to any external MISP instance other then the Staging System. From 4fd744c2c3e4b63865f95a08f56b4da8d03e5816 Mon Sep 17 00:00:00 2001 From: Camille Schneider Date: Tue, 3 Apr 2018 00:05:06 +0200 Subject: [PATCH 5/5] fix broken links --- sharing/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sharing/README.md b/sharing/README.md index 7f73c94..e2962e5 100644 --- a/sharing/README.md +++ b/sharing/README.md @@ -135,9 +135,9 @@ More details on publishing events [here](../using-the-system/#publish-an-event). ### Community -A community is composed of the local organisations on a MISP server and the remote organisations connected by the sync users. For more information on the concept of community, refer to an [article on MISP information sharing following ISO/IEC 27010](https://github.com/MISP/misp-compliance/blob/master/ISO_IEC_27010/misp-sharing-information-following-ISO-IEC-27010.md), explaining the concept of community. +A community is composed of the local organisations on a MISP server and the remote organisations connected by the sync users. For more information on the concept of community, refer to an [article on MISP information sharing following ISO/IEC 27010](https://github.com/MISP/misp-compliance/blob/master/ISO_IEC_27010/misp-sharing-information-following-ISO-IEC-27010.md#suitable-data-model), explaining the concept of community. -Specifically, communities are not reversible. Taking the example of the above figure, illustrating the synchronisation between two MISP servers, OrgB.ServerB is part of the MISP ServerA community but OrgB.ServerA is not part of MISP ServerB community. +Specifically, communities are not reversible. Taking the example of the above figure, illustrating the synchronisation between two MISP servers, OrgB.ServerB is part of the MISP ServerA community but OrgB.ServerA is not part of MISP ServerB community. ### Distribution mechanisms @@ -250,4 +250,4 @@ It will download all events and do enrichment between these events. This system is the main system used by human analysts. It will it is not linked to any external MISP instance other then the Staging System. -To publish events to the community assign the right tags to match your push [Rules](#rules) and **publish the event** \ No newline at end of file +To publish events to the community assign the right tags to match your push [Rules](#rules) and **publish the event**