merge + update medusalocker

2022-07-06 09:28:46 +02:00 · 2022-07-06 09:28:46 +02:00 · 7e37fa0cdd
parent c2e7ef4fab 4638dbde86
commit 7e37fa0cdd
20 changed files with 75945 additions and 58223 deletions
--- a/README.md
+++ b/README.md
@ -19,8 +19,6 @@ to localized information (which is not shared) or additional information (that c

 # Available Galaxy - clusters

-
-
 ## Android

 [Android](https://www.misp-project.org/galaxy.html#_android) - Android malware galaxy based on multiple open sources.
@ -65,7 +63,7 @@ Category: *mobile* - source: *https://arxiv.org/pdf/2005.05110.pdf* - total: *47

 [Botnet](https://www.misp-project.org/galaxy.html#_botnet) - botnet galaxy

-Category: *tool* - source: *MISP Project* - total: *63* elements
+Category: *tool* - source: *MISP Project* - total: *71* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_botnet)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/botnet.json)]

@ -361,7 +359,7 @@ Category: *measure* - source: *MISP Project* - total: *20* elements

 [Ransomware](https://www.misp-project.org/galaxy.html#_ransomware) - Ransomware galaxy based on https://docs.google.com/spreadsheets/d/1TWS238xacAto-fLKh1n5uTsdijWdCEsGIM0Y0Hvmc5g/pubhtml and http://pastebin.com/raw/GHgpWjar

-Category: *tool* - source: *Various* - total: *1602* elements
+Category: *tool* - source: *Various* - total: *1608* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_ransomware)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/ransomware.json)]

@ -369,7 +367,7 @@ Category: *tool* - source: *Various* - total: *1602* elements

 [RAT](https://www.misp-project.org/galaxy.html#_rat) - remote administration tool or remote access tool (RAT), also called sometimes remote access trojan, is a piece of software or programming that allows a remote "operator" to control a system as if they have physical access to that system.

-Category: *tool* - source: *MISP Project* - total: *262* elements
+Category: *tool* - source: *MISP Project* - total: *264* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_rat)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/rat.json)]

@ -425,7 +423,7 @@ Category: *tool* - source: *Open Sources* - total: *6* elements

 [Surveillance Vendor](https://www.misp-project.org/galaxy.html#_surveillance_vendor) - List of vendors selling surveillance technologies including malware, interception devices or computer exploitation services.

-Category: *actor* - source: *MISP Project* - total: *13* elements
+Category: *actor* - source: *MISP Project* - total: *14* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_surveillance_vendor)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/surveillance-vendor.json)]

@ -457,7 +455,7 @@ Category: *tea-matrix* - source: ** - total: *7* elements

 [Threat Actor](https://www.misp-project.org/galaxy.html#_threat_actor) - Known or estimated adversary groups targeting organizations and employees. Adversary groups are regularly confused with their initial operation or campaign. threat-actor-classification meta can be used to clarify the understanding of the threat-actor if also considered as operation, campaign or activity group.

-Category: *actor* - source: *MISP Project* - total: *366* elements
+Category: *actor* - source: *MISP Project* - total: *379* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_threat_actor)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/threat-actor.json)]

@ -465,7 +463,7 @@ Category: *actor* - source: *MISP Project* - total: *366* elements

 [Tool](https://www.misp-project.org/galaxy.html#_tool) - threat-actor-tools is an enumeration of tools used by adversaries. The list includes malware but also common software regularly used by the adversaries.

-Category: *tool* - source: *MISP Project* - total: *530* elements
+Category: *tool* - source: *MISP Project* - total: *533* elements

 [[HTML](https://www.misp-project.org/galaxy.html#_tool)] - [[JSON](https://github.com/MISP/misp-galaxy/blob/main/clusters/tool.json)]

--- a/clusters/android.json
+++ b/clusters/android.json
@ -4653,7 +4653,18 @@
      },
      "uuid": "aef548fb-76f5-4eb9-9942-f189cb0d16f6",
      "value": "Razdel"
+    },
+    {
+      "description": "Vulture is an Android banking trojan found in Google Play by ThreatFabric. It uses screen recording and keylogging as main strategy to harvest login credentials.",
+      "meta": {
+        "refs": [
+          "https://www.threatfabric.com/blogs/vultur-v-for-vnc.html",
+          "https://twitter.com/_icebre4ker_/status/1485651238175846400"
+        ]
+      },
+      "uuid": "66026639-132f-436e-8348-1219714e9f62",
+      "value": "Vulture"
    }
  ],
-  "version": 20
+  "version": 21
 }
--- a/clusters/backdoor.json
+++ b/clusters/backdoor.json
@ -172,7 +172,20 @@
      ],
      "uuid": "16902832-0118-40f2-b29e-eaba799b2bf4",
      "value": "SUNBURST"
+    },
+    {
+      "description": "BPFDoor is a passive backdoor used by a China-based threat actor. This backdoor supports multiple protocols for communicating with a C2 including TCP, UDP, and ICMP allowing the threat actor a variety of mechanisms to interact with the implant",
+      "meta": {
+        "refs": [
+          "https://troopers.de/troopers22/talks/7cv8pz/",
+          "https://doublepulsar.com/bpfdoor-an-active-chinese-global-surveillance-tool-54b078f1a896?gi=1effe9eb6507",
+          "https://twitter.com/cyb3rops/status/1523227511551033349",
+          "https://twitter.com/CraigHRowland/status/1523266585133457408"
+        ]
+      },
+      "uuid": "0c3b1aa5-3a33-493e-9126-28ebced4ed09",
+      "value": "BPFDoor"
    }
  ],
-  "version": 11
+  "version": 12
 }
--- a/clusters/cryptominers.json
+++ b/clusters/cryptominers.json
@ -42,7 +42,27 @@
      },
      "uuid": "20e563b0-f0c9-4253-aedd-a4542d6689ed",
      "value": "WannaMine"
+    },
+    {
+      "description": "Blue Mockingbird Crypto miner is a crypto-mining payload within DLLs on Windows Systems.",
+      "meta": {
+        "refs": [
+          "https://redcanary.com/blog/blue-mockingbird-cryptominer/"
+        ]
+      },
+      "uuid": "3dd091c9-608f-44d6-ac0c-5dfdf9bb4518",
+      "value": "Blue Mockingbird Cryptominer"
+    },
+    {
+      "description": "The Krane malware uses SSH brute-force techniques to drop the XMRig cryptominer on the target to mine for the Hashvault pool.",
+      "meta": {
+        "refs": [
+          "https://cujo.com/threat-alert-krane-malware/"
+        ]
+      },
+      "uuid": "a0c0ab05-c390-425c-9311-f64bf7ca9145",
+      "value": "Krane"
    }
  ],
-  "version": 1
+  "version": 2
 }
--- a/clusters/handicap.json
+++ b/clusters/handicap.json
@ -0,0 +1,276 @@
+{
+  "authors": [
+    "Agathe MANGEOT",
+    "Cyril BURTIN "
+  ],
+  "category": "med-bdm-it",
+  "description": "Liste des maladies invalidantes reconnues comme handicap",
+  "name": "handicap",
+  "source": "MDPH /caf",
+  "type": "Handicap",
+  "uuid": "84310ba3-fa6a-44aa-b378-b9e3271c7777",
+  "values": [
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Accident vasculaire cérébral invalidant",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "edbfa998-9ec8-418c-b1c1-3007fd244e16",
+      "value": "Accident vasculaire cérébral invalidant"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Insuffisances médullaires et autres cytopénies chroniques",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "f666a569-c2a9-42d5-beaa-b8787be0acb5",
+      "value": "Insuffisances médullaire"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Artériopathies chroniques avec manisfestations ischémiques",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "d2e8b63b-df68-4006-98bf-9953cdf276b1",
+      "value": "Artériopathies chroniques"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "non",
+        "Nom": " Bilharziose compliquée",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "1910135a-a28f-4c21-a07a-83ca25ce40e1",
+      "value": "Bilharziose compliquée"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Insuffisances cardiaque grave, troubles du rythme graves, cardiopathies valvulaires graves, cardiopathies congénitales graves",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "f5f29cfd-a890-417a-b0ab-b256524db0ae",
+      "value": "Insuffisances cardiaque"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Maladies chroniques actives du foie (hépatite B ou C) et cirrhoses",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "14bfd714-804a-414a-badb-9ea326de62d2",
+      "value": "Hépatite B ou C"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Déficit immunitaire primitif grave nécessitant un traitement prolongé, infection par le virus de l'immuno-déficience humaine (VIH)",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "a988a347-c1db-4d17-a221-ce997dbaf67b",
+      "value": "Déficit immunitaire primitif grave"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Diabète de type 1 et diabète de type 2 et d l'adulte ou de l'enfant",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "34ace033-af1e-4d3d-9cd9-5f12b555a32e",
+      "value": "Diabète"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Formes graves des affections neurologiques et musculaires(dont myopathie), epilepsie grave",
+        "Type d'affection": "affection de longue durée "
+      },
+      "uuid": "ce6ebe05-893a-4d0a-a366-4bee63fc6ca5",
+      "value": "Affections neurologiques et musculaire"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Hemoglopathies, hémolyses, chroniques constitutionelles et acquises sévères",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "0cd69cd5-3d56-475d-b188-22acc887249e",
+      "value": "Hemoglopathies, hémolyses"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Hémophilies et affections constitutionnelles de l'hémostase graves",
+        "Type d'affection": "affection de ongue durée"
+      },
+      "uuid": "4ee033ec-26cb-4315-a80a-f98f3de478b8",
+      "value": "Hémophilies"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "non",
+        "Nom": "Maladie coronaire : infarctus du myocarde",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "9db8ea00-37ce-4c0d-a902-f36124087231",
+      "value": "Infarctus du myocarde"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Insuffisance respiratoire chronique grave(exemple: asthme grave)",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "8de7bdba-cf58-494e-b596-b66fda2b22ae",
+      "value": "Insuffisance respiratoire"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "non",
+        "Nom": "Maladie d'Alzheimer et autres démences",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "80ce126d-692c-45f0-ba06-fded443b93fb",
+      "value": "Maladie d'Alzheimer et autres démences"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "non",
+        "Nom": "Maladies de Parkinson",
+        "Type d'affection": "Maladie dégénérative"
+      },
+      "uuid": "5a2783cb-53e4-451a-9487-e558894bcd13",
+      "value": "Parkinson"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Maladie métabolique héréditaire nécessitant un traitement prolongé spécialisé",
+        "Type d'affection": "affection de longue durée"
+      },
+      "uuid": "d1a3b820-0d25-4e5a-9f3a-1471b06856c3",
+      "value": "Maladies métabolque héréditaire"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Mucovicidose",
+        "Type d'affection": "Maladie dégénérative"
+      },
+      "uuid": "54c9d607-f434-416b-acfa-07609fefe8db",
+      "value": "Mucovisidose"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Néphropatie chronique grave et syndrome néphrotique primitif(insufficance rénale) ",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "c5070703-8c73-49b7-a8b7-a8b0e8ee53a6",
+      "value": "Néphropatie"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "oui",
+        "Nom": "Paraplégie",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "9851ce51-b95f-41bf-8e22-2c83751c78ad",
+      "value": "Paraplégie"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Vascularites, lupus érythémateux systémique, sclérodermie systémique",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "6ff9e37c-04dd-4d9c-ac23-fe23ad9ef581",
+      "value": "Vascularites"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Polyarthrite rhumathoïde évolutive",
+        "Type d'affection": "Affection évolutive "
+      },
+      "uuid": "6bacef11-be44-42d2-a653-ac5df3aa46ac",
+      "value": "Polyarthrite"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Affections psychiatrique de longue durée (exemple: dépression récurrente, troubles bipolaires)",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "fbe1be61-bf2b-4547-9c3e-24f9e113537a",
+      "value": "Affection Psychiatrique"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Rectolite hémorragique et maladie de Crohn évolutives",
+        "Type d'affection": "Maladie évolutive"
+      },
+      "uuid": "fbb79ba0-08f2-40f4-b562-863edd5b3137",
+      "value": "Rectolite"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Sclérose en plaques",
+        "Type d'affection": "Maladie dégénérative"
+      },
+      "uuid": "fbb79ba0-08f2-40f4-b562-861edd5b3135",
+      "value": "Sclérose en plaque"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Scoliose idiopathique structurale évolutive",
+        "Type d'affection": "Maladie évolutive "
+      },
+      "uuid": "0a97945d-4fc7-4a09-bb73-760891adea52",
+      "value": "Scoliose idiopathique"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Spondylarthrite grave",
+        "Type d'affection": "Maladie évolutive"
+      },
+      "uuid": "7958ba36-084f-4c60-9b5c-1ece7c839734",
+      "value": "Spondylarthrite"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Suite de transplantation d'organe",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "29fd13a3-4d8b-4e9f-8253-374f8ac07b9c",
+      "value": "Transplantation d'organe"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Tuberculose active, lèpre",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "211a6aab-415f-4e9d-ad9b-199aca6bd33d",
+      "value": "Tuberculose/lèpre"
+    },
+    {
+      "meta": {
+        "Aménagement(prothèse, orthèse, pompe à insuline,...)": "possible",
+        "Nom": "Tumeur maligne(cancer), affection maligne du tissu lymphatique ou hématopoïétique(exemple: lymphome)",
+        "Type d'affection": "Affection de longue durée"
+      },
+      "uuid": "9eb1eacc-3996-4c8b-ad4e-8a4e168c415e",
+      "value": "Tumeur maligne"
+    }
+  ],
+  "version": 1
+}
--- a/clusters/malpedia.json
+++ b/clusters/malpedia.json
--- a/clusters/mitre-attack-pattern.json
+++ b/clusters/mitre-attack-pattern.json
--- a/clusters/mitre-course-of-action.json
+++ b/clusters/mitre-course-of-action.json
--- a/clusters/mitre-intrusion-set.json
+++ b/clusters/mitre-intrusion-set.json
--- a/clusters/mitre-malware.json
+++ b/clusters/mitre-malware.json
--- a/clusters/mitre-tool.json
+++ b/clusters/mitre-tool.json
--- a/clusters/ransomware.json
+++ b/clusters/ransomware.json
@ -21647,7 +21647,7 @@
      "value": "MBR-ONI"
    },
    {
-      "description": "ransomware",
+      "description": "Observed as recently as May 2022, MedusaLocker actors predominantly rely on vulnerabilities in Remote Desktop Protocol (RDP) to access victims’ networks. The MedusaLocker actors encrypt the victim's data and leave a ransom note with communication instructions in every folder containing an encrypted file. The note directs victims to provide ransomware payments to a specific Bitcoin wallet address. MedusaLocker appears to operate as a Ransomware-as-a-Service (RaaS) model based on the observed split of ransom payments. Typical RaaS models involve the ransomware developer and various affiliates that deploy the ransomware on victim systems. MedusaLocker ransomware payments appear to be consistently split between the affiliate, who receives 55 to 60 percent of the ransom; and the developer, who receives the remainder.",
      "meta": {
        "extensions": [
          ".1btc",
@ -21701,6 +21701,10 @@
          "recovery_instructions.html",
          "HOW_TO_RECOVER_DATA.html",
          "recovery_instruction.html"
+        ],
+        "refs": [
+          "https://www.cisa.gov/uscert/ncas/alerts/aa22-181a",
+          "https://www.cisa.gov/uscert/sites/default/files/publications/AA22-181A_stopransomware_medusalocker.pdf"
        ]
      },
      "uuid": "627d603a-906f-4fbf-b922-f03eea4578fe",
@ -23633,6 +23637,19 @@
    },
    {
      "description": "ransomware",
+      "meta": {
+        "attribution-confidence": "100",
+        "country": "RU",
+        "extensions": [
+          ".conti"
+        ],
+        "ransomnotes": [
+          "All of your files are currently encrypted by CONTI ransomware."
+        ],
+        "refs": [
+          "https://www.cyber.gov.au/acsc/view-all-content/advisories/2021-010-acsc-ransomware-profile-conti"
+        ]
+      },
      "uuid": "201eff54-d41e-4f70-916c-5dfb9301730a",
      "value": "Conti"
    },
@ -24296,10 +24313,12 @@
          "https://www.sentinelone.com/labs/blackcat-ransomware-highly-configurable-rust-driven-raas-on-the-prowl-for-victims/",
          "https://www.varonis.com/blog/alphv-blackcat-ransomware",
          "https://www.intrinsec.com/alphv-ransomware-gang-analysis",
-          "https://unit42.paloaltonetworks.com/blackcat-ransomware/"
+          "https://unit42.paloaltonetworks.com/blackcat-ransomware/",
+          "https://www.cyber.gov.au/acsc/view-all-content/advisories/2022-004-acsc-ransomware-profile-alphv-aka-blackcat"
        ],
        "synonyms": [
-          "ALPHV"
+          "ALPHV",
+          "Noberus"
        ]
      },
      "related": [
@ -24520,5 +24539,5 @@
      "value": "Rook"
    }
  ],
-  "version": 101
+  "version": 102
 }
--- a/clusters/surveillance-vendor.json
+++ b/clusters/surveillance-vendor.json
@ -197,7 +197,19 @@
      },
      "uuid": "4dbfa61e-0cf5-4142-babf-3cdce348568d",
      "value": "Cytrox"
+    },
+    {
+      "description": "RCS Lab S.p.A., Italian vendor likely using Tykelab Srl as a front company.",
+      "meta": {
+        "refs": [
+          "https://www.rcslab.it/en/index.html",
+          "https://www.lookout.com/blog/hermit-spyware-discovery",
+          "https://www.vice.com/en/article/nz75wd/european-surveillance-companies-agt-rcs-sell-syria-tools-of-oppression"
+        ]
+      },
+      "uuid": "28ed79b6-a11d-4e41-af80-ece8f0e0c2d3",
+      "value": "RCSLab"
    }
  ],
-  "version": 2
+  "version": 3
 }
--- a/clusters/threat-actor.json
+++ b/clusters/threat-actor.json
--- a/galaxies/handicap.json
+++ b/galaxies/handicap.json
@ -0,0 +1,9 @@
+{
+  "description": "Handicap classifying",
+  "icon": "android",
+  "name": "handicap",
+  "namespace": "misp",
+  "type": "Handi",
+  "uuid": "84310ba3-fa6a-44aa-b378-b9e3271c7777",
+  "version": 1
+}
--- a/tools/adoc_galaxy.py
+++ b/tools/adoc_galaxy.py
@ -30,12 +30,15 @@ clusters = []
 pathClusters = os.path.join(thisDir, '../clusters')
 pathGalaxies = os.path.join(thisDir, '../galaxies')

+skip_list = ["cancer.json", "handicap.json"]
+
 for f in os.listdir(pathGalaxies):
    if '.json' in f:
        with open(os.path.join(pathGalaxies, f), 'r') as f_in:
            galaxy_data = json.load(f_in)
            if galaxy_data.get('namespace') != 'deprecated':
-                clusters.append(f)
+                if f not in skip_list:
+                    clusters.append(f)

 clusters.sort()

--- a/tools/del_duplicate_refs.py
+++ b/tools/del_duplicate_refs.py
@ -0,0 +1,17 @@
+#!/usr/bin/env python3
+# coding=utf-8
+"""
+    Tool to remove duplicates in cluster references
+"""
+import sys
+import json
+
+with open(sys.argv[1], 'r') as f:
+    data = json.load(f)
+
+for c in data['values']:
+    c['meta']['refs'] = list(dict.fromkeys(c['meta']['refs']))
+
+with open(sys.argv[1], 'w') as f:
+    json.dump(data, f)
+
--- a/tools/fetch_malpedia.sh
+++ b/tools/fetch_malpedia.sh
@ -0,0 +1,6 @@
+#!/bin/bash
+cd "${0%/*}"
+wget -O malpedia.json https://malpedia.caad.fkie.fraunhofer.de/api/get/misp
+mv malpedia.json ../clusters/malpedia.json
+./del_duplicate_refs.py ../clusters/malpedia.json
+(cd ..; ./jq_all_the_things.sh)
--- a/tools/generate-index.py
+++ b/tools/generate-index.py
@ -14,7 +14,7 @@ def gen_galaxy_tag(galaxy_name, cluster_name):
    return '{}={}'.format(galaxy_name, cluster_name)

 galaxies_fnames = []
-files_to_ignore = []
+files_to_ignore = ["cancer.json", "handicap.json"]
 pathClusters = '../clusters'

 for f in os.listdir(pathClusters):
--- a/tools/mitre-cti/v2.0/create_mitre-galaxy.py
+++ b/tools/mitre-cti/v2.0/create_mitre-galaxy.py
@ -177,7 +177,16 @@ for t in types:
        item_2.pop('type', None)
        file_data['values'].append(item_2)

-    file_data['values'] = sorted(file_data['values'], key=lambda x: sorted(x['value']))  # FIXME the sort algo needs to be further improved
+    # FIXME the sort algo needs to be further improved, potentially with a recursive deep sort
+    file_data['values'] = sorted(file_data['values'], key=lambda x: sorted(x['value']))
+    for item in file_data['values']:
+        if 'related' in item:
+            item['related'] = sorted(item['related'], key=lambda x: x['dest-uuid'])
+        if 'meta' in item:
+            if 'refs' in item['meta']:
+                item['meta']['refs'] = sorted(item['meta']['refs'])
+            if 'mitre_data_sources' in item['meta']:
+                item['meta']['mitre_data_sources'] = sorted(item['meta']['mitre_data_sources'])
    file_data['version'] += 1
    with open(fname, 'w') as f:
        json.dump(file_data, f, indent=2, sort_keys=True, ensure_ascii=False)