{ "cells": [ { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "# Delete this cell to re-enable tracebacks\n", "import sys\n", "ipython = get_ipython()\n", "\n", "def hide_traceback(exc_tuple=None, filename=None, tb_offset=None,\n", " exception_only=False, running_compiled_code=False):\n", " etype, value, tb = sys.exc_info()\n", " value.__cause__ = None # suppress chained exceptions\n", " return ipython._showtraceback(etype, value, ipython.InteractiveTB.get_exception_only(etype, value))\n", "\n", "#ipython.showtraceback = hide_traceback" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "# JSON output syntax highlighting\n", "from __future__ import print_function\n", "from pygments import highlight\n", "from pygments.lexers import JsonLexer, TextLexer\n", "from pygments.formatters import HtmlFormatter\n", "from IPython.display import display, HTML\n", "from IPython.core.interactiveshell import InteractiveShell\n", "\n", "InteractiveShell.ast_node_interactivity = \"all\"\n", "\n", "def json_print(inpt):\n", " string = str(inpt)\n", " formatter = HtmlFormatter()\n", " if string[0] == '{':\n", " lexer = JsonLexer()\n", " else:\n", " lexer = TextLexer()\n", " return HTML('{}'.format(\n", " formatter.get_style_defs('.highlight'),\n", " highlight(string, lexer, formatter)))\n", "\n", "globals()['print'] = json_print" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TAXIICollection\n", "\n", "The TAXIICollection suite contains [TAXIICollectionStore](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionStore), [TAXIICollectionSource](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionSource), and [TAXIICollectionSink](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionSink). [TAXIICollectionStore](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionStore) pushes and retrieves STIX content to local/remote TAXII Collection(s). [TAXIICollectionSource](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionSource) retrieves STIX content from local/remote TAXII Collection(s). [TAXIICollectionSink](../api/datastore/stix2.datastore.taxii.rst#stix2.datastore.taxii.TAXIICollectionSink) pushes STIX content to local/remote TAXII Collection(s). Each of the interfaces is designed to be bound to a Collection from the [taxii2client](https://github.com/oasis-open/cti-taxii-client) library (taxii2client.Collection), where all [TAXIICollection](../api/datastore/stix2.datastore.taxii.rst) API calls will be executed through that Collection instance.\n", "\n", "A note on TAXII2 searching/filtering of STIX content: TAXII2 server implementations natively support searching on the STIX2 object properties: id, type and version; API requests made to TAXII2 can contain filter arguments for those 3 properties. However, the [TAXIICollection](../api/datastore/stix2.datastore.taxii.rst) suite supports searching on all STIX2 common object properties (see [Filters](../api/datastore/stix2.datastore.filters.rst) documentation for full listing). This works simply by augmenting the filtering that is done remotely at the TAXII2 server instance. [TAXIICollection](../api/datastore/stix2.datastore.taxii.rst) will seperate any supplied queries into TAXII supported filters and non-supported filters. During a [TAXIICollection](../api/datastore/stix2.datastore.taxii.rst) API call, TAXII2 supported filters get inserted into the TAXII2 server request (to be evaluated at the server). The rest of the filters are kept locally and then applied to the STIX2 content that is returned from the TAXII2 server, before being returned from the [TAXIICollection](../api/datastore/stix2.datastore.taxii.rst) API call. \n", "\n", "### TAXIICollection API\n", "\n", "### TAXIICollection Examples\n", "\n", "#### TAXIICollectionSource" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
{\n",
       "    "type": "malware",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec",\n",
       "    "created": "2017-01-27T13:49:53.997Z",\n",
       "    "modified": "2017-01-27T13:49:53.997Z",\n",
       "    "name": "Poison Ivy",\n",
       "    "description": "Poison Ivy",\n",
       "    "malware_types": [\n",
       "        "remote-access-trojan"\n",
       "    ],\n",
       "    "is_family": true\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
-------\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--6770298f-0fd8-471a-ab8c-1c658a46574e",\n",
       "    "created": "2016-11-03T12:30:59.000Z",\n",
       "    "modified": "2016-11-03T12:30:59.000Z",\n",
       "    "name": "Malicious site hosting downloader",\n",
       "    "description": "Accessing this url will infect your machine with malware.",\n",
       "    "indicator_types": [\n",
       "        "url-watchlist"\n",
       "    ],\n",
       "    "pattern": "[url:value = 'http://z4z10farb.cn/4712']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2017-01-27T13:49:53.935382Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--6770298f-0fd8-471a-ab8c-1c658a46574e",\n",
       "    "created": "2016-11-03T12:30:59.000Z",\n",
       "    "modified": "2016-12-25T12:30:59.444Z",\n",
       "    "name": "Malicious site hosting downloader",\n",
       "    "description": "Accessing this url will infect your machine with malware. Updated indicator",\n",
       "    "indicator_types": [\n",
       "        "url-watchlist"\n",
       "    ],\n",
       "    "pattern": "[url:value = 'http://x4z9arb.cn/4712']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2017-01-27T13:49:53.935382Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--6770298f-0fd8-471a-ab8c-1c658a46574e",\n",
       "    "created": "2016-11-03T12:30:59.000Z",\n",
       "    "modified": "2017-01-27T13:49:53.935Z",\n",
       "    "name": "Malicious site hosting downloader",\n",
       "    "description": "Accessing this url will infect your machine with malware. This is the last updated indicator",\n",
       "    "indicator_types": [\n",
       "        "url-watchlist"\n",
       "    ],\n",
       "    "pattern": "[url:value = 'http://x4z9arb.cn/4712']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2016-11-03T12:30:59Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from stix2 import TAXIICollectionSource\n", "from taxii2client import Collection\n", "\n", "# establish TAXII2 Collection instance\n", "collection = Collection(\"http://127.0.0.1:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/\", user=\"admin\", password=\"Password0\")\n", "# supply the TAXII2 collection to TAXIICollection\n", "tc_source = TAXIICollectionSource(collection)\n", "\n", "#retrieve STIX objects by id\n", "stix_obj = tc_source.get(\"malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec\")\n", "stix_obj_versions = tc_source.all_versions(\"indicator--6770298f-0fd8-471a-ab8c-1c658a46574e\")\n", "\n", "#for visual purposes\n", "print(stix_obj)\n", "print(\"-------\")\n", "for so in stix_obj_versions:\n", " print(so)\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--cd981c25-8042-4166-8945-51178443bdac",\n",
       "    "created": "2014-05-08T09:00:00.000Z",\n",
       "    "modified": "2014-05-08T09:00:00.000Z",\n",
       "    "name": "File hash for Poison Ivy variant",\n",
       "    "indicator_types": [\n",
       "        "file-hash-watchlist"\n",
       "    ],\n",
       "    "pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2014-05-08T09:00:00Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--6770298f-0fd8-471a-ab8c-1c658a46574e",\n",
       "    "created": "2016-11-03T12:30:59.000Z",\n",
       "    "modified": "2017-01-27T13:49:53.935Z",\n",
       "    "name": "Malicious site hosting downloader",\n",
       "    "description": "Accessing this url will infect your machine with malware. This is the last updated indicator",\n",
       "    "indicator_types": [\n",
       "        "url-watchlist"\n",
       "    ],\n",
       "    "pattern": "[url:value = 'http://x4z9arb.cn/4712']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2016-11-03T12:30:59Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--d8f573d9-5796-4d3f-98fd-d3b953738520",\n",
       "    "created": "2020-06-26T19:04:05.608201Z",\n",
       "    "modified": "2020-06-26T19:04:05.608201Z",\n",
       "    "description": "Smokey Bear implant",\n",
       "    "pattern": "[file:hashes.'SHA-256' = '09c7e05a39a59428743635242e4a867c932140a909f12a1e54fa7ee6a440c73b']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2020-06-26T19:04:05.608201Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" }, { "data": { "text/html": [ "
{\n",
       "    "type": "indicator",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "indicator--acd03fef-b8df-4c45-bd89-1ede90c39959",\n",
       "    "created": "2020-06-26T19:04:09.476525Z",\n",
       "    "modified": "2020-06-26T19:04:09.476525Z",\n",
       "    "description": "Smokey Bear implant",\n",
       "    "pattern": "[file:hashes.'SHA-256' = '09c7e05a39a59428743635242e4a867c932140a909f12a1e54fa7ee6a440c73b']",\n",
       "    "pattern_type": "stix",\n",
       "    "pattern_version": "2.1",\n",
       "    "valid_from": "2020-06-26T19:04:09.476525Z"\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from stix2 import Filter\n", "\n", "# retrieve multiple object from TAXIICollectionSource\n", "# by using filters\n", "f1 = Filter(\"type\",\"=\", \"indicator\")\n", "\n", "indicators = tc_source.query([f1])\n", "\n", "#for visual purposes\n", "for indicator in indicators:\n", " print(indicator)" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "#### TAXIICollectionSink" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from stix2 import TAXIICollectionSink, ThreatActor\n", "\n", "#create TAXIICollectionSINK and push STIX content to it\n", "tc_sink = TAXIICollectionSink(collection)\n", "\n", "# create new STIX threat-actor\n", "ta = ThreatActor(name=\"Teddy Bear\",\n", " threat_actor_types=[\"nation-state\"],\n", " sophistication=\"innovator\",\n", " resource_level=\"government\",\n", " goals=[\n", " \"compromising environment NGOs\",\n", " \"water-hole attacks geared towards energy sector\",\n", " ])\n", "\n", "tc_sink.add(ta)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### TAXIICollectionStore" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
{\n",
       "    "type": "malware",\n",
       "    "spec_version": "2.1",\n",
       "    "id": "malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec",\n",
       "    "created": "2017-01-27T13:49:53.997Z",\n",
       "    "modified": "2017-01-27T13:49:53.997Z",\n",
       "    "name": "Poison Ivy",\n",
       "    "description": "Poison Ivy",\n",
       "    "malware_types": [\n",
       "        "remote-access-trojan"\n",
       "    ],\n",
       "    "is_family": true\n",
       "}\n",
       "
\n" ], "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from stix2 import TAXIICollectionStore\n", "\n", "# create TAXIICollectionStore - note the same collection instance can\n", "# be used for the store\n", "tc_store = TAXIICollectionStore(collection)\n", "\n", "# retrieve STIX object by id from TAXII Collection through\n", "# TAXIICollectionStore\n", "stix_obj2 = tc_source.get(\"malware--c0931cc6-c75e-47e5-9036-78fabc95d4ec\")\n", "\n", "print(stix_obj2)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from stix2 import Indicator\n", "\n", "# add STIX object to TAXIICollectionStore\n", "ind = Indicator(description=\"Smokey Bear implant\",\n", " pattern_type=\"stix\",\n", " pattern=\"[file:hashes.'SHA-256' = '09c7e05a39a59428743635242e4a867c932140a909f12a1e54fa7ee6a440c73b']\")\n", "\n", "tc_store.add(ind)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Bug and Workaround\n", "\n", "You may get an error similar to the following when adding STIX objects to a TAXIICollectionStore or TAXIICollectionSink:\n", "\n", "```\n", "TypeError: Object of type ThreatActor is not JSON serializable\n", "```\n", "\n", "This is a known bug and we are working to fix it. For more information, see [this GitHub issue](https://github.com/oasis-open/cti-python-stix2/issues/125) In the meantime, try this workaround:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "tc_sink.add(json.loads(Bundle(ta).serialize()))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Or bypass the TAXIICollection altogether and interact with the collection itself:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "collection.add_objects(json.loads(Bundle(ta).serialize()))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.0a6" } }, "nbformat": 4, "nbformat_minor": 2 }