MISP
/
misp-training
mirror of https://github.com/MISP/misp-training
2
0
Fork 0
Code Issues Releases Wiki Activity
misp-training/a.7-rest-API/Introduction_to_PyMISP.ipynb

1271 lines
36 KiB
Plaintext
Raw Permalink Normal View History

add: [a.7-rest-API] Added notebook with a quick introduction to PyMISP
2024-07-10 00:20:47 +02:00
{
"cells": [
{
"cell_type": "markdown",
"id": "b915b20d-c8c0-4124-bb53-82640beaf973",
"metadata": {},
"source": [
"# Initializing your environment\n",
"\n",
"## Setting up a virtual environment\n",
"(Optional but recommended)\n",
"\n",
"```bash\n",
"virtualenv -p python3 venv\n",
"source venv/bin/activate\n",
"```\n",
"(Use `deactivate` to exit from `source` once you are done)\n",
"\n",
"Alternatively you can also prefix all your `python` and `pip` commands with `./venv/bin/` (e.g: `./venv/bin/pip3 install -U pip`)\n",
"\n",
"\n",
"\n",
"## Setting up Jupyter\n",
"\n",
"In order to follow along on your computer:\n",
"\n",
"```bash\n",
"pip3 install notebook\n",
"jupyter-notebook\n",
"```\n",
"\n",
"## Installation of PyMISP\n",
"\n",
"#### Make sure the submodules are up-to-date and cloned\n",
"\n",
"```bash\n",
"git submodule update --init --recursive PyMISP/\n",
"```\n",
"\n",
"#### Install PyMISP with the developer options\n",
"\n",
"```bash\n",
"cd PyMISP\n",
"pip3 install -e .\n",
"```\n",
"\n",
"#### To be able to use the additional PyMISP helpers\n",
"\n",
"```bash\n",
"# Make sure the package required for pydeep is installed\n",
"sudo apt-get install -y libfuzzy-dev\n",
"\n",
"pip3 install python-magic, lief, git+https://github.com/kbandla/pydeep.git\n",
"```\n",
"\n",
"# Using the PyMISP objects\n",
"\n",
"PyMISP is the python library used to deal with MISP format so you do not have to deal with the JSON blob yourself without knowing about the required and optional fields.\n",
"\n",
"## MISPEvent\n",
"\n",
"MISPEvent is the main class to use when you want to create/update events on a MISP instance."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7705fb68-c34c-47cc-9184-12bbe3d5f4be",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPEvent\n",
"\n",
"event = MISPEvent()\n",
"\n",
"event.info = 'A fancy MISP Event' # Required\n",
"event.distribution = 0 # Optional, sets the distribution level to \"Your Organisation only\"\n",
"event.threat_level_id = 2 # Optional, sets the threat level to \"Medium\"\n",
"event.analysis = 1 # Optional, set the analysis to \"Ongoing\"\n",
"\n",
"print(event.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "5e41ead0-cc18-4f93-8f4d-e352e985772e",
"metadata": {},
"source": [
"### Set the event date\n",
"\n",
"The date can be in many different formats. PyMISP normalizes it in a way that will be understood by your MISP instance: a date in the `YYYY-MM-DD` format."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "32ec2d30-3363-408d-baac-41d77de0ff5a",
"metadata": {},
"outputs": [],
"source": [
"event.set_date('2022-01-06')\n",
"print(f'From a text date: {event.date}')\n",
"\n",
"from datetime import date\n",
"d = date.today()\n",
"event.set_date(d)\n",
"print(f'From a datetime.date date: {event.date}')\n",
"\n",
"from datetime import datetime\n",
"d = datetime.now()\n",
"event.set_date(d)\n",
"# MISP expects a day, so the Hour will be dropped\n",
"print(f'From a datetime.datetime date: {event.date}')"
]
},
{
"cell_type": "markdown",
"id": "8200bdea-6b14-42e5-9dd4-3d2430bf4f5d",
"metadata": {},
"source": [
"### Tag an Event\n",
"\n",
"An easy way to tag an Event"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c94a1de0-e7b7-4dea-a33b-4f9370f1473e",
"metadata": {},
"outputs": [],
"source": [
"event.add_tag('tlp:white')\n",
"\n",
"print(event.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "bb01abe3-6bf6-4667-bcbb-f41cd59455eb",
"metadata": {},
"source": [
"## MISP Attribute\n",
"\n",
"MISP Attributes are the raw pieces of data that can either be indicators of compromise (IoCs) or observed data.\n",
"\n",
"They are defined with a triplet of required values being a `type`, a `category` and a `value`, and a bunch of optional fields.\n",
"\n",
"The Attributes parameters are the following:\n",
"- **type** (required)\n",
"- **value** (required)\n",
"- **category**: the type of information (if not set, the default one for the given `type` is used)\n",
"- **to_ids**: defines whether the attribute defines some malicious data that should be blocked, or not (if not set, the default value for the given `type` is used)\n",
"- **distribution**: defaults to inherit from parent (event or object)\n",
"- **disable_correlation**: if you want to avoid correlations between events on that specific value\n",
"- **data**: for `malware-sample` and `attachment` types, BytesIO object of the file.\n",
"\n",
"### A minimal and quick way of adding an attribute"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e9bdcf4c-88e8-43c2-a31b-e99c739fbc14",
"metadata": {},
"outputs": [],
"source": [
"attribute1 = event.add_attribute('ip-dst', '8.8.8.8')\n",
"\n",
"print(attribute1.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "ec564fe5-65f3-41cc-a839-5b36050ae425",
"metadata": {},
"source": [
"### Set inline parameters"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b3a3d404-5165-41e6-b94b-dbbda2c573b0",
"metadata": {},
"outputs": [],
"source": [
"attribute2 = event.add_attribute('ip-dst', '8.8.8.9', to_ids=False, disable_correlation=True)\n",
"\n",
"print(attribute2.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "f23e353e-6c3e-4ab0-9992-081d9e535d81",
"metadata": {},
"source": [
"### Modify an existing attribute\n",
"\n",
"Every parameter can be modified in a pythonic way"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "55c2f3d9-60df-4c43-ac3b-d422225d0b01",
"metadata": {},
"outputs": [],
"source": [
"attribute1.to_ids = False\n",
"\n",
"print(attribute1.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "fde0520b-1031-44f9-91fe-049317ff4453",
"metadata": {},
"source": [
"### Soft delete an attribute\n",
"\n",
"The default approach on MISP is to soft delete data, which means it will not be displayed in the dafult view on MISP. \n",
"The reason for doing this is to allow to push delete updates to instances we synchronise with."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6652598b-9896-4376-92ab-f14ac3eee31a",
"metadata": {},
"outputs": [],
"source": [
"attribute2.delete()\n",
"\n",
"print(event.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "9475f08f-1227-4cee-a266-da9789a8b2d9",
"metadata": {},
"source": [
"### A more advanced way of passing the different parameters at once"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3212f832-db60-47a4-941a-9fc88b71614b",
"metadata": {},
"outputs": [],
"source": [
"from uuid import uuid4\n",
"\n",
"attribute_uuid = uuid4()\n",
"print(attribute_uuid)\n",
"\n",
"kwargs = {\n",
" 'to_ids': False,\n",
" 'disable_correlation': True,\n",
" 'category': 'Network activity',\n",
" 'uuid': attribute_uuid\n",
"}\n",
"attribute = event.add_attribute('ip-src', '1.1.1.1', **kwargs)\n",
"\n",
"print(attribute.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "a3daccce-9913-47cc-9f69-05c3677931b4",
"metadata": {},
"source": [
"### Using the MISPAttribute class\n",
"\n",
"Allows you to play with the attribute before adding it to the event.\n",
"\n",
"It is then possible to load the attribute from a JSON or from a dict"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "014e72b9-5194-424f-9b8d-7e0eeda19fc2",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPAttribute\n",
"\n",
"attribute = MISPAttribute()\n",
"attribute.type = 'domain'\n",
"attribute.value = 'circl.lu'\n",
"\n",
"print(attribute.to_json(indent=4))\n",
"print(event.add_attribute(**attribute).to_json(indent=4))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b860c96-f6d0-4583-af98-cb567701ead8",
"metadata": {},
"outputs": [],
"source": [
"# Loaded from a JSON\n",
"json = '''{\n",
" \"type\": \"domain\",\n",
" \"value\": \"circl.lu\",\n",
" \"to_ids\": false\n",
"}'''\n",
"\n",
"json_attribute = MISPAttribute()\n",
"json_attribute.from_json(json)\n",
"\n",
"print(json_attribute.to_json(indent=4))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9b7c30d0-0d12-4d5c-83e4-6b3838804666",
"metadata": {},
"outputs": [],
"source": [
"# Loaded from a python dict\n",
"_dict = {\n",
" 'type': 'domain',\n",
" 'value': 'circl.lu',\n",
" 'to_ids': False\n",
"}\n",
"\n",
"dict_attribute = MISPAttribute()\n",
"dict_attribute.from_dict(**_dict)\n",
"\n",
"print(dict_attribute.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "71fbfefc-7b2b-4cb7-81b6-fb54c72d29eb",
"metadata": {},
"source": [
"### Tag an Attribute\n",
"\n",
"The same way to tag events applies for attributes"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e181f80-7d06-4647-8b3a-dbe4e8b24050",
"metadata": {},
"outputs": [],
"source": [
"dict_attribute.add_tag('tlp:clear')\n",
"\n",
"print(dict_attribute.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "98e577d2-57aa-4d89-89aa-d94d482e9715",
"metadata": {},
"source": [
"## MISP Object\n",
"\n",
"MISP Objects are containers to group attributes in a way that makes sense. The objects are based on templates that are bundled in the library itself, but you can also use your own templates."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d9fefc03-10d8-41f7-923a-3c41f26ddac7",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPObject\n",
"\n",
"misp_object = MISPObject('domain-ip')\n",
"misp_object.comment = 'My fancy new object'\n",
"\n",
"object_attribute = misp_object.add_attribute('domain', value='circl.lu')\n",
"object_attribute.add_tag('tlp:green')\n",
"misp_object.add_attribute('ip', value='149.13.33.14')\n",
"misp_object.add_attribute('first-seen', value='2022-12-31')\n",
"misp_object.add_attribute('last-seen', value='2023-01-06')\n",
"\n",
"print(misp_object.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "f3b951d7-e684-4939-bf91-205bacc29c06",
"metadata": {},
"source": [
"### Short version to add an object to a MISPEvent\n",
"\n",
"You can also add the object directly in a MISP event"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f39a277b-dbf3-4ce2-bb43-96cc91c89e1f",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPObject\n",
"\n",
"misp_object = event.add_object(name='domain-ip', comment='My fancy new object')\n",
"\n",
"object_attribute = misp_object.add_attribute('domain', value='circl.lu')\n",
"object_attribute.add_tag('tlp:green')\n",
"misp_object.add_attribute('ip', value='149.13.33.14', to_ids=False)\n",
"misp_object.add_attribute('first-seen', value='2022-12-31', disable_correlation=False)\n",
"misp_object.add_attribute('last-seen', value='2023-01-06')\n",
"\n",
"misp_object.add_reference(attribute1.uuid, 'connects-to')\n",
"\n",
"print(event.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "9fc9233f-271f-4c24-aa02-8f3d80b28193",
"metadata": {},
"source": [
"# Helpers for MISPObjects\n",
"\n",
"For some objects, we have helpers in order to make your life easier. The most relevant example is the file object: when you have a file to push on MISP, there are plenty of indicators you can extract at once, and it is pretty simple to automate, so we made it a oneliner.\n",
"\n",
"**Note**: This requires a few more dependencies to get the full power of the script: \n",
"* `lief` to extract indicators out of PE/ELF/MachO files, and soon Android binaries.\n",
"* `python-magic` to get the mime type\n",
"* `pydeep` to compute the ssdeep of the binary whenever possible\n",
"\n",
"\n",
"```bash\n",
"pip install lief python-magic git+https://github.com/kbandla/pydeep.git\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bbe1c6ac-ea0f-48f7-aa56-99e36d13f1b0",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"test_path = Path().resolve() / 'test_files'\n",
"\n",
"from pymisp.tools import FileObject\n",
"\n",
"file_object = FileObject(\n",
" filepath=test_path / 'EICAR.com',\n",
" standalone=False\n",
")\n",
"print(file_object.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "8791d585-ac01-4153-bab3-0cf3e3a902d5",
"metadata": {},
"source": [
"## Playing with a malware sample\n",
"\n",
"The data you receive out of the JSON dump from a MISP instance is a base64 encoded zip with `infected` as a password. \n",
"The zip file contains 2 files, one containing the original file name of the uploaded file, and the other one is the binary.\n",
"\n",
"This is pretty much a pain to use as-is, so there is an helper for that!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b0ac3d7d-945e-4f17-b87c-47e90ce04af6",
"metadata": {},
"outputs": [],
"source": [
"sample = file_object.get_attributes_by_relation('malware-sample')[0]\n",
"print(sample)\n",
"print('File name --->', sample.malware_filename)\n",
"print(sample.malware_binary)\n",
"print('Content of the malware (in bytes) ----->', sample.malware_binary.getvalue())## Use lief to extract indicators out of binaries\n",
"\n",
"Another cool hepler allows you to pass the path of a binary. If the binary's format is supported by `lief`, you get the file object, the binary definition (PE, ELf or Mach-o) and the relevant sections.\n",
"\n",
"If it is anything else, it will simply generate a file object."
]
},
{
"cell_type": "markdown",
"id": "09fcdb92-0e69-4c2c-a4a7-60ab839df08e",
"metadata": {},
"source": [
"## Use lief to extract indicators out of binaries\n",
"\n",
"Another cool hepler allows you to pass the path of a binary. If the binary's format is supported by `lief`, you get the file object, the binary definition (PE, ELf or Mach-o) and the relevant sections.\n",
"\n",
"If it is anything else, it will simply generate a file object."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "caae39c4-3f44-4cce-ba46-7279771ab20b",
"metadata": {},
"outputs": [],
"source": [
"from pymisp.tools import make_binary_objects\n",
"\n",
"misp_event = MISPEvent()\n",
"misp_event.info = 'Test with binary file'\n",
"\n",
"filepath = test_path / 'whoami.exe'\n",
"file_obj, bin_obj, sections = make_binary_objects(\n",
" filepath=filepath.as_posix(),\n",
" standalone=False\n",
")\n",
"\n",
"misp_event.add_object(file_obj)\n",
"if bin_obj:\n",
" misp_event.add_object(bin_obj)\n",
" for section in sections:\n",
" misp_event.add_object(section)"
]
},
{
"cell_type": "markdown",
"id": "48fbad25-55e0-435c-ba2e-1b30e08d393d",
"metadata": {},
"source": [
"The references between the different objects are also set by default with the `make_binary_objects` method."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5b63c77a-e5b7-485f-9994-8a44245c9cd8",
"metadata": {},
"outputs": [],
"source": [
"for misp_object in misp_event.objects:\n",
" print(misp_object)"
]
},
{
"cell_type": "markdown",
"id": "8ac123a9-adc5-4775-96a5-867d43076e3d",
"metadata": {},
"source": [
"## Generic helper\n",
"\n",
"This helper can be used when you already have a script that does the mapping between your own code and a MISPObject template."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "528b5342-68d9-4aa0-8502-b8164b02e70e",
"metadata": {},
"outputs": [],
"source": [
"from pymisp.tools import GenericObjectGenerator\n",
"\n",
"attributes_as_dict = [\n",
" {\n",
" 'filename': 'shell1.exe',\n",
" 'sha1': {\n",
" 'value': 'b7afa7acf1b7ded2c4e3d0884b5cdaa230d9f82e',\n",
" 'to_ids': False\n",
" },\n",
" 'size-in-bytes': {\n",
" 'value': 24576,\n",
" 'disable_correlation': True\n",
" }\n",
" }\n",
"]\n",
"\n",
"misp_object = GenericObjectGenerator('file', strict=True)\n",
"misp_object.generate_attributes(attributes_as_dict)\n",
"\n",
"print(misp_object.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "feeaf3d7-e44f-4209-9abd-819f958188d6",
"metadata": {},
"source": [
"### User defined objects\n",
"\n",
"The Generic helper can also be used to define your own object template."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73f62d17-ce6e-4942-9bc0-ea3c31592699",
"metadata": {},
"outputs": [],
"source": [
"attributes_as_dict = [\n",
" {\n",
" 'MyCoolAttribute': {\n",
" 'value': 'critical thing',\n",
" 'type': 'text'\n",
" },\n",
" 'MyCoolerAttribute': {\n",
" 'value': 'even worse',\n",
" 'type': 'text'\n",
" }\n",
" }\n",
"]\n",
"\n",
"# We cannot use `strict=True` here\n",
"misp_object = GenericObjectGenerator('my-cool-template')\n",
"misp_object.generate_attributes(attributes_as_dict)\n",
"\n",
"print(misp_object.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "c05b4256-3999-4b37-85f4-504b5a0855c7",
"metadata": {},
"source": [
"PyMISP is OK with this generic object and won't complain if you set the required fields.\n",
"\n",
"Nonetheless, before pushing such event to MISP, we want to set a few additional fields:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "470c893b-d94e-4fd4-9560-0055a28a12e7",
"metadata": {},
"outputs": [],
"source": [
"from uuid import uuid4\n",
"\n",
"misp_object.template_uuid = uuid4()\n",
"misp_object.template_id = 1\n",
"misp_object.description = 'foo'\n",
"setattr(misp_object, 'meta-category', 'bar')\n",
"\n",
"print(misp_object.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "c704fb73-8864-4136-b346-51db59e3ea3e",
"metadata": {},
"source": [
"### Use locally defined object templates\n",
"\n",
"**Important**: The path you pass as parameter for `misp_objects_path_custom` needs to contain a directory equals to the value of the parameter `name` (same structure as the content of the `misp-object` repository)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e462ec5b-4d88-4bf0-b011-29f8e23c58a8",
"metadata": {},
"outputs": [],
"source": [
"user_defined_obj = MISPObject(\n",
" name='test_object_template', strict=True,\n",
" misp_objects_path_custom=test_path\n",
")\n",
"\n",
"user_defined_obj.add_attribute('member1', 'foo')\n",
"user_defined_obj.add_attribute('member2', value='bar', to_ids=True)\n",
"user_defined_obj.add_attribute('member3', **{'value': 'baz'})\n",
"\n",
"print(user_defined_obj.to_json(indent=4))"
]
},
{
"cell_type": "markdown",
"id": "f2cc273a-c641-44ce-901f-d379cc675a73",
"metadata": {},
"source": [
"# Interacting with your MISP instance\n",
"\n",
"### Recovering your API key\n",
"\n",
"- Go to `Global Actions` then `My profile`\n",
"- Alternatively, access the `/users/view/me` URL of your MISP instance\n",
"\n",
"## Initializing the variables\n",
"\n",
"We need to set a few variables:\n",
"- The URL of the MISP instance\n",
"- Your API key\n",
"- The certificate verification bool variable"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0af58de8-3686-472b-a007-c1f6e80215a1",
"metadata": {},
"outputs": [],
"source": [
"# The URL of the MISP instance to connect to\n",
"misp_url = 'https://training.misp-community.org'\n",
"\n",
"# The authentication key (can be found in the MISP\n",
"# web interface under _misp_url_/users/view/me -> Authkey)\n",
"misp_key = '_YOUR_AUTHENTICATION_KEY_' # CHANGE ACCORDINGLY \n",
"\n",
"# Should PyMISP verify the MISP certificate\n",
"misp_verifycert = True"
]
},
{
"cell_type": "markdown",
"id": "48a23673-41a7-4465-880c-ca2f7898bc28",
"metadata": {},
"source": [
"Now we can use the API class"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "de6b8d18-8d84-4ae2-8a2d-1740667e67a0",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import PyMISP\n",
"\n",
"misp = PyMISP(misp_url, misp_key, misp_verifycert)"
]
},
{
"cell_type": "markdown",
"id": "e7e71f2a-b15a-43ad-b1aa-b13653accb39",
"metadata": {},
"source": [
"## Creating an Event\n",
"\n",
"### Directly\n",
"\n",
"```python\n",
"event1 = misp.add_event(\n",
" {\n",
" 'info': 'CryptoLocker ransomware infection via e-mail',\n",
" 'distribution': 3,\n",
" 'analysis': 2,\n",
" 'threat_level_id': 1\n",
" }\n",
")\n",
"\n",
"print(f'Event id of the created event: {event1.id}')\n",
"```\n",
"\n",
"### Using the MISPEvent constructor"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8935636a-d0ed-4ed0-a6da-22886ac3ccec",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPAttribute, MISPEvent\n",
"\n",
"event = MISPEvent()\n",
"event.info = 'Ransomware infection via e-mail'\n",
"event.distribution = 0 # My organisation only\n",
"event.threat_level_id = 3 # High\n",
"event.analysis = 1 # Ongoing"
]
},
{
"cell_type": "markdown",
"id": "857b6374-8d1d-4492-aef1-66ffcaaf7738",
"metadata": {},
"source": [
"### Add Attributes"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "45339ef1-13f0-4676-b81a-f0122011e959",
"metadata": {},
"outputs": [],
"source": [
"# Add the btc address attribute\n",
"btc_attribute = event.add_attribute('btc', '1J7fmT3Wv62u4p6FuwbVJQXhQEjw3jzV63',\n",
" comment='BTC address to pay the ransom')\n",
"print(btc_attribute)"
]
},
{
"cell_type": "markdown",
"id": "aafa6d20-d408-4d4d-b80a-998761717ea1",
"metadata": {},
"source": [
"### Add the Event"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af236f21-aa64-4e27-8e80-21650d36f325",
"metadata": {},
"outputs": [],
"source": [
"misp_event = misp.add_event(event, pythonify=True)\n",
"print(f'Event id of the created event: {misp_event.id}')"
]
},
{
"cell_type": "markdown",
"id": "276bb88c-08ed-43ad-b05f-5410984fdc6d",
"metadata": {},
"source": [
"## Updating an existing Event\n",
"\n",
"### Fetching the Event"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41f7c76d-a65d-419f-a390-0da7db8f6a4e",
"metadata": {},
"outputs": [],
"source": [
"# Pick an Event id - like the one you just printed above\n",
"event_id = 121\n",
"\n",
"event = misp.get_event(event_id, pythonify=True)\n",
"print(event)"
]
},
{
"cell_type": "markdown",
"id": "ba6df21f-f3c0-46bb-8bf0-8e2253db960a",
"metadata": {},
"source": [
"### Add some Objects\n",
"\n",
"We want to push a file directly in our Event"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "76cdb5eb-ddd1-45a0-9cfa-be84ce027b25",
"metadata": {},
"outputs": [],
"source": [
"from pathlib import Path\n",
"sample_path = '/Users/chrisr3d/Downloads/cryptolocker.exe'\n",
"\n",
"from pymisp.tools import FileObject\n",
"\n",
"file_object = FileObject(\n",
" filepath=Path(sample_path).resolve(),\n",
" standalone=False\n",
")\n",
"added_object = event.add_object(file_object)\n",
"for attribute in added_object.attributes:\n",
" print(attribute.object_relation, attribute.value)"
]
},
{
"cell_type": "markdown",
"id": "5dd0e2b3-0ab5-4ba9-8bc0-4c6101429ee5",
"metadata": {},
"source": [
"### Add the IP address attribute"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3f2d202a-4dbb-494a-ad8f-3b170a569c1e",
"metadata": {},
"outputs": [],
"source": [
"ip_attribute = MISPAttribute()\n",
"ip_attribute.from_dict(\n",
" **{\n",
" 'type': 'ip-dst',\n",
" 'value': '81.177.170.166',\n",
" 'comment': 'IP address of a C2 server distributing the ransomware'\n",
" }\n",
")\n",
"added_ip = event.add_attribute(**ip_attribute)\n",
"print(added_ip)"
]
},
{
"cell_type": "markdown",
"id": "5e26982d-2230-4700-a187-de36aef6361f",
"metadata": {},
"source": [
"### Add new Objects"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "57d50934-188c-4df3-a8fe-3e7a879ae676",
"metadata": {},
"outputs": [],
"source": [
"from pymisp import MISPObject\n",
"\n",
"# Encode the email received by the victim\n",
"email = MISPObject('email')\n",
"email.add_attribute('from', 'andrew_ryan@rindustries.rp')\n",
"email.add_attribute('subject', 'Report for case 4829-2375')\n",
"email.add_attribute('email-body', 'Please see the attached Iolta report for 4829-2375.\\r\\n\\r\\nWe received a check request in the amount of $19,637.28 for the above referenced file. However, the attached report refects a $0 balance. At your earliest convenience, please advise how this request is to be funded.\\r\\n\\r\\nThanks.\\r\\n\\r\\nAndrew_Ryan *\\r\\nAccounts Payable\\r\\n\\r\\nRyan Industries\\r\\n42, Central Control Hephaestus - Rapture\\r\\nwww.rindustries.rp\\r\\n\\r\\n*Not licensed to practise law.\\r\\n\\r\\nThis communication contains information that is intended only for the recipient named and may be privileged, confidential, subject to the attorney-client privilege, and/or exempt from disclosure under applicable law. If you are not the intended recipient or agent responsible for delivering this communication to the intended recipient, you are hereby notified that you have received this communication in error, and that any review, disclosure, dissemination, distribution, use, or copying of this communication is STRICTLY PROHIBITED. If you have received this communication in error, please notify us immediately by telephone at 1-800-766-7751 or 1-972-643-6600 and destroy the material in its entirety, whether in electronic or hard copy format.')\n",
"email_object = event.add_object(email)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "edfe35db-20f0-42a0-a8b4-9e14e81f99bb",
"metadata": {},
"outputs": [],
"source": [
"# Encode the Identity of the sender\n",
"person = MISPObject('person')\n",
"person.comment = 'The sender of the email'\n",
"person.add_attribute('full-name', 'Andrew Ryan')\n",
"person.add_attribute('e-mail', 'andrew_ryan@rindustries.rp')\n",
"person.add_attribute('role', 'Suspect')\n",
"person_object = event.add_object(person)"
]
},
{
"cell_type": "markdown",
"id": "1b436530-cfb1-485e-a609-b126498dbb88",
"metadata": {},
"source": [
"### Add references between objects"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0ab639e1-422f-4b9f-8274-752f546c28c0",
"metadata": {},
"outputs": [],
"source": [
"# From the email object, add a reference to the person object\n",
"email_object.add_reference(person_object.uuid, 'sent-by')\n",
"\n",
"# From the file object, add a reference to the email object\n",
"added_object.add_reference(email_object.uuid, 'dropped-by')\n",
"\n",
"# From the file_object, add a reference to the IP address Attribute\n",
"added_object.add_reference(ip_attribute.uuid, 'connects-to')"
]
},
{
"cell_type": "markdown",
"id": "e12e186c-b5bd-4c60-9493-b078773aa0a3",
"metadata": {},
"source": [
"### Add some tags"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "489b8f6e-6bd0-4b35-affc-c34d80ddf555",
"metadata": {},
"outputs": [],
"source": [
"# Adding tags to the event\n",
"event.add_tag('tlp:green')\n",
"event.add_tag('ransomware:infection=\"phishing-e=mails\"')\n",
"event.add_tag('malware_classification:malware-category=\"Ransomware\"')\n",
"\n",
"# Adding a tag to the IP address attribute\n",
"added_ip.add_tag('adversary:infrastructure-type=\"C2\"')\n",
"\n",
"# Adding a Galaxy Cluster to the event\n",
"event.add_tag('misp-galaxy:ransomware=\"CryptoLocker\"')"
]
},
{
"cell_type": "markdown",
"id": "9995e281-02fd-4c9b-b096-32717e6791a1",
"metadata": {},
"source": [
"### Update the Event\n",
"\n",
"We previously pushed an event to MISP directly, but we also did local changes. \n",
"We need then to update the Event on MISP."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9addc88c-cabe-4ca9-abe2-2df1b84435ac",
"metadata": {},
"outputs": [],
"source": [
"event.analysis = 2 # Completed\n",
"event.distribution = 3 # All communities\n",
"event.publish()\n",
"\n",
"misp.update_event(event)"
]
},
{
"cell_type": "markdown",
"id": "c6ace256-3c60-4388-a3ba-2e17677a726c",
"metadata": {},
"source": [
"## API operations\n",
"\n",
"Here are some other useful operations available with the `PyMISP` constructor\n",
"\n",
"### Direct call (no validation)\n",
"\n",
"Since most of the operations are also available with direct calls:\n",
"\n",
"```python\n",
"misp.direct_call(f'attributes/add/{event_id}', {'type': 'ip-dst', 'value': '10.9.8.7'})\n",
"```\n",
"\n",
"### Searches"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5a87f664-8d2f-4c48-9b9e-234f4f51295e",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"\n",
"# Searching the Event index\n",
"endpoint = 'events/index'\n",
"\n",
"body = {\n",
" 'org': 'CIRCL',\n",
" 'published': True,\n",
" 'publish_timestamp': '1d'\n",
"}\n",
"\n",
"response = misp.direct_call(endpoint, body)\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"id": "45edd2a2-aa54-4fbf-b88d-62355eaa8882",
"metadata": {},
"source": [
"### RestSearch"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a1d86889-1056-4f1b-901c-2600a6504c4c",
"metadata": {},
"outputs": [],
"source": [
"endpoint = '/attributes/restSearch/'\n",
"\n",
"body = {\n",
" 'returnFormat': 'json',\n",
" 'type': ['ip-src', 'ip-dst'],\n",
" 'timestamp': '1d'\n",
"}\n",
"\n",
"response = misp.direct_call(endpoint, body)\n",
"print(f\"count: {len(response['Attribute'])}\")\n",
"print(response)"
]
},
{
"cell_type": "markdown",
"id": "098f74ba-3b2e-4339-a909-8f1d54b738a8",
"metadata": {},
"source": [
"## Using the search method"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d440506-7285-4342-8c04-1955365621d5",
"metadata": {},
"outputs": [],
"source": [
"# Search for Events published by the organisation 'ORGNAME' in the last 2 days\n",
"response = misp.search(org='CIRCL', published=True, publish_timestamp='2d', pythonify=True)\n",
"print(response)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83788062-75fa-4480-ab94-3588c5dd8ad8",
"metadata": {},
"outputs": [],
"source": [
"# Search for Events containing Attributes with a specific value\n",
"response = misp.search(value='81.177.170.166', pythonify=True)\n",
"for event in response:\n",
" print(event)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c68929a5-5029-4f46-9e93-8ccb79d0b56f",
"metadata": {},
"outputs": [],
"source": [
"# Search for published Events from the last 2 days tagged with tlp:clear\n",
"events = misp.search(publish_timestamp='2d', tags=['tlp:clear'], pythonify=True)\n",
"print(events)"
]
},
{
"cell_type": "markdown",
"id": "c56e8e9d-852d-4164-bebc-2de52f691fda",
"metadata": {},
"source": [
"### Search for attributes"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfc9f38c-670d-448a-ba21-3e3b8ad87a11",
"metadata": {},
"outputs": [],
"source": [
"# Search for IP addresses from the last 2 days\n",
"attributes = misp.search(\n",
" controller='attributes', type_attribute=['ip-src', 'ip-dst'], to_ids=1, timestamp='2d', pythonify=True\n",
")\n",
"for attribute in attributes:\n",
" print(attribute)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "03975b73-8131-4532-b7f1-a566da886fd7",
"metadata": {},
"outputs": [],
"source": [
"# Search for Attributes with TLP tag, but not tlp:amber nor tlp:red\n",
"attributes = misp.search(\n",
" controller='attributes', tags=['tlp:%', '!tlp:amber', '!tlp:red'],\n",
" published=True, publish_timestamp='2d', pythonify=True\n",
")\n",
"for attribute in attributes:\n",
" print(attribute)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1be120eb-56a2-4232-aa04-99e1fd211f51",
"metadata": {},
"outputs": [],
"source": [
"# Paginate on the results above\n",
"attributes = misp.search(\n",
" controller='attributes', tags=['tlp:%', '!tlp:amber', '!tlp:red'],\n",
" published=True, limit=5, page=2, publish_timestamp='2d', pythonify=True\n",
")\n",
"for attribute in attributes:\n",
" print(attribute)"
]
},
{
"cell_type": "markdown",
"id": "fe4580e6-982b-4c27-9bef-dd96920191a3",
"metadata": {},
"source": [
"### Trying different output formats"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5c51a9d-c193-415c-bf47-23a0d7d818e9",
"metadata": {},
"outputs": [],
"source": [
"# Returning the attributes in CSV\n",
"csv_attributes = misp.search(\n",
" controller='attributes', type_attribute=['ip-src', 'ip-dst'], to_ids=1, last='2d', return_format='csv'\n",
")\n",
"print(csv_attributes)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5577b220-f665-47fc-8f86-43252c0cc0c4",
"metadata": {},
"outputs": [],
"source": [
"# Converting data into STIX\n",
"import json\n",
"stix2_content = misp.search(eventid=event_id, return_format=\"stix2\")\n",
"print(json.dumps(stix2_content, indent=2))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d4844d38-0392-4ff9-b6a1-75df734cf798",
"metadata": {},
"outputs": [],
"source": [
"# Converting the Attributes only\n",
"stix2_content = misp.search(\n",
" controller='attributes', type_attribute=[\"md5\", \"sha1\", \"sha256\"], eventid=event_id,\n",
" return_format=\"stix2\"\n",
")\n",
"print(json.dumps(stix2_content, indent=2))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "79c7b073-2cef-4400-919d-f0e54b41b8ab",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}