diff --git a/docs/guide/patterns.ipynb b/docs/guide/patterns.ipynb index 87bcba7..c963407 100644 --- a/docs/guide/patterns.ipynb +++ b/docs/guide/patterns.ipynb @@ -4,282 +4,28 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# STIX Patterns\n", + "# STIX2 Patterns\n", "\n", - "Python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. Python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher].(https://github.com/oasis-open/cti-pattern-matcher)\n", + "python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher](https://github.com/oasis-open/cti-pattern-matcher).\n", "\n", - "## Intro\n", "\n", - "Python-stix2 patterns are built compositely from the bottom up, creating components at sublevels before those at higher levels.\n", + "python-stix2 patterns are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.\n", "\n", - "## Examples\n", + "## API Tips\n", "\n", - "### Comparison Expressions" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Equality Comparison Expressions:\n", - "\n", - "\t[domain-name:value = 'site.of.interest.zaz']\n", - "\n", - "\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n", - "\n", - "\n", - "Greater-than Comparison Expression:\n", - "\n", - "\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n", - "\n", - "\n", - "Is-Subset Comparison Expression:\n", - "\n", - "\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n", - "\n" - ] - } - ], - "source": [ - "from stix2 import DomainName, File, IPv4Address\n", - "from stix2 import (ObjectPath, EqualityComparisonExpression, GreaterThanComparisonExpression,\n", - " IsSubsetComparisonExpression, FloatConstant, StringConstant)\n", + "### ObservationExpression\n", "\n", - "# ---- Equality Comparison expressions\n", - "print(\"Equality Comparison Expressions:\\n\")\n", + "Within the STIX2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets(```[]```) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.\n", "\n", - "lhs = ObjectPath(\"domain-name\", [\"value\"])\n", - "ece_1 = EqualityComparisonExpression(lhs, \"site.of.interest.zaz\")\n", - "print(\"\\t{}\\n\".format(ece_1.to_pattern()))\n", + "This requirement manifests itself within python-stix2 via ```ObservationExpression```. When creating STIX2 observation expressions, whenever the current expression is complete, wrap it with ```ObservationExpression()```. This allows the complete pattern expression - no matter complexity - to be rendered as a proper specification adhering string. *__Of which, when pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations__* . While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings. \n", "\n", - "lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n", - "ece_2 = EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\")\n", - "print(\"\\t{}\\n\".format(ece_2.to_pattern()))\n", + "In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ```ObservationExpression()```.\n", "\n", - "# Greater-than Comparison expressions\n", - "print(\"\\nGreater-than Comparison Expression:\\n\")\n", + "### ParentheticalExpression\n", "\n", - "lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n", - "gte = GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\"))\n", - "print(\"\\t{}\\n\".format(gte.to_pattern()))\n", + "Do not be confused by the ```ParentheticalExpression``` object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ```ObservationExpression```, wrapping completed subcomponent expressions with ```ParentheticalExpression()``` if explicit ordering is required. For usage examples with ```ParentheticalExpression```'s, see [here](#Compound-Observation-Expressions).\n", "\n", - "# IsSubset Comparison expressions\n", - "print(\"\\nIs-Subset Comparison Expression:\\n\")\n", - "\n", - "lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n", - "iss = IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\"))\n", - "print(\"\\t{}\\n\".format(iss.to_pattern()))\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Observation Expressions" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Observation Expressions:\n", - "\n", - "(AND)\n", - "[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n", - "\n", - "(OR)\n", - "[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n", - "\n", - "(OR,AND)\n", - "[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n", - "\n", - "(AND,OR,OR)\n", - "([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n", - "\n", - "(FollowedBy)\n", - "[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n", - "\n" - ] - } - ], - "source": [ - "from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n", - " EqualityComparisonExpression, AndBooleanExpression,\n", - " OrBooleanExpression, ParentheticalExpression,\n", - " AndObservationExpression, OrObservationExpression,\n", - " FollowedByObservationExpression)\n", - "\n", - "# ---- Observation expressions\n", - "print(\"Observation Expressions:\\n\")\n", - "\n", - "# AND boolean\n", - "ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n", - "ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n", - "abe = AndBooleanExpression([ece3, ece4])\n", - "print(\"(AND)\\n{}\\n\".format(abe.to_pattern()))\n", - "\n", - "# OR boolean\n", - "ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n", - "ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n", - "obe = OrBooleanExpression([ece5, ece6])\n", - "print(\"(OR)\\n{}\\n\".format(obe.to_pattern()))\n", - "\n", - "# ( OR ) AND boolean\n", - "ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n", - "ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n", - "ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n", - "obe1 = OrBooleanExpression([ece7, ece8])\n", - "pobe = ParentheticalExpression(obe1)\n", - "abe1 = AndBooleanExpression([pobe, ece9])\n", - "print(\"(OR,AND)\\n{}\\n\".format(abe1.to_pattern()))\n", - "\n", - "# ( AND ) OR ( OR ) observation\n", - "ece20 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n", - "ece21 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n", - "ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n", - "ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n", - "# NOTE: we need to use AND/OR observation expression instead of just boolean \n", - "# expressions as the operands are not on the same object-type\n", - "aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n", - "obe2 = OrBooleanExpression([ece22, ece23])\n", - "ooe = OrObservationExpression([aoe, obe2])\n", - "print(\"(AND,OR,OR)\\n{}\\n\".format(ooe.to_pattern()))\n", - "\n", - "# FOLLOWED-BY\n", - "ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n", - "ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n", - "fbe = FollowedByObservationExpression([ece10, ece11])\n", - "print(\"(FollowedBy)\\n{}\\n\".format(fbe.to_pattern()))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "__See [note](#BooleanExpressions-vs-CompoundObservationExpressions) on when to use BooleanExpressions vs CompoundObservationExpressions__" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Qualified Observation Expressions" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Qualified Observation Expressions:\n", - "\n", - "(WITHIN)\n", - "[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar'] WITHIN 300 SECONDS\n", - "\n", - "(REPEAT, WITHIN)\n", - "[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n", - "\n", - "(START-STOP)\n", - "[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n", - "\n" - ] - } - ], - "source": [ - "from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n", - " AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n", - " QualifiedObservationExpression, FollowedByObservationExpression)\n", - "\n", - "# Qualified Observation Expressions\n", - "print(\"Qualified Observation Expressions:\\n\")\n", - "\n", - "# WITHIN\n", - "ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n", - "ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n", - "fbe = FollowedByObservationExpression([ece10, ece11])\n", - "qoe = QualifiedObservationExpression(fbe, WithinQualifier(300))\n", - "print(\"(WITHIN)\\n{}\\n\".format(qoe.to_pattern()))\n", - "\n", - "# REPEATS, WITHIN\n", - "ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n", - "ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n", - "abe2 = AndBooleanExpression([ece12, ece13])\n", - "qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n", - "print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1.to_pattern()))\n", - "\n", - "# START, STOP\n", - "ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n", - "ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n", - "qoe2 = QualifiedObservationExpression(ece14, ssq)\n", - "print(\"(START-STOP)\\n{}\\n\".format(qoe2.to_pattern()))\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Attaching patterns to STIX2 Domain objects\n", - "\n", - "As seen in the previous examples, the *__to_pattern()__* utility is used to display the pattern. However, the true purpose of this utility is to be able to seamlessly add a constructed pattern to an python-stix2 object.\n", - "\n", - "The *__to_pattern()__* utility constructs a string version of the pattern that adheres to the STIX2 patterning language specification.\n", - "\n", - "### Example" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{\n", - " \"type\": \"indicator\",\n", - " \"id\": \"indicator--9f2d8014-3c2f-4127-bc2c-a209913404a5\",\n", - " \"created\": \"2018-08-27T18:57:00.762Z\",\n", - " \"modified\": \"2018-08-27T18:57:00.762Z\",\n", - " \"name\": \"Cryptotorch\",\n", - " \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n", - " \"valid_from\": \"2018-08-27T18:57:00.762972Z\",\n", - " \"labels\": [\n", - " \"malware\",\n", - " \"ransomware\"\n", - " ]\n", - "}\n" - ] - } - ], - "source": [ - "from stix2 import Indicator, EqualityComparisonExpression\n", - "\n", - "ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\")\n", - "ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=\"[{}]\".format(ece14))\n", - "print(ind)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## BooleanExpressions vs CompoundObservationExpressions\n", + "### BooleanExpressions vs CompoundObservationExpressions\n", "\n", "Be careful to note the difference between these two very similar pattern components. \n", "\n", @@ -304,7 +50,262 @@ " ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n", " \n", " __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n", - " \n" + "\n", + "\n", + "\n", + "## Examples\n", + "\n", + "### Comparison Expressions" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--- Equality Comparison Expressions\n", + "\n", + "\t[domain-name:value = 'site.of.interest.zaz']\n", + "\n", + "\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n", + "\n", + "\n", + "--- Greater-than Comparison Expression\n", + "\n", + "\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n", + "\n", + "\n", + "--- Is-Subset Comparison Expression\n", + "\n", + "\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n", + "\n" + ] + } + ], + "source": [ + "from stix2 import DomainName, File, IPv4Address\n", + "from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,\n", + " GreaterThanComparisonExpression, IsSubsetComparisonExpression,\n", + " FloatConstant, StringConstant)\n", + "\n", + "# ---- Equality Comparison expressions\n", + "print(\"--- Equality Comparison Expressions\\n\")\n", + "\n", + "lhs = ObjectPath(\"domain-name\", [\"value\"])\n", + "ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, \"site.of.interest.zaz\"))\n", + "print(\"\\t{}\\n\".format(ece_1))\n", + "\n", + "lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n", + "ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\"))\n", + "print(\"\\t{}\\n\".format(ece_2))\n", + "\n", + "# Greater-than Comparison expressions\n", + "print(\"\\n--- Greater-than Comparison Expression\\n\")\n", + "\n", + "lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n", + "gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\")))\n", + "print(\"\\t{}\\n\".format(gte))\n", + "\n", + "# IsSubset Comparison expressions\n", + "print(\"\\n--- Is-Subset Comparison Expression\\n\")\n", + "\n", + "lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n", + "iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\")))\n", + "print(\"\\t{}\\n\".format(iss))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Compound Observation Expressions" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--- Compound Observation Expressions\n", + "\n", + "(AND)\n", + "[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n", + "\n", + "(OR)\n", + "[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n", + "\n", + "(OR,AND)\n", + "[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n", + "\n", + "(AND,OR,OR)\n", + "([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n", + "\n", + "(FollowedBy)\n", + "[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n", + "\n" + ] + } + ], + "source": [ + "from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n", + " EqualityComparisonExpression, AndBooleanExpression,\n", + " OrBooleanExpression, ParentheticalExpression,\n", + " AndObservationExpression, OrObservationExpression,\n", + " FollowedByObservationExpression, ObservationExpression)\n", + "\n", + "# ---- Observation expressions\n", + "print(\"--- Compound Observation Expressions\\n\")\n", + "\n", + "# AND boolean\n", + "ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n", + "ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n", + "abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))\n", + "print(\"(AND)\\n{}\\n\".format(abe))\n", + "\n", + "# OR boolean\n", + "ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n", + "ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n", + "obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))\n", + "print(\"(OR)\\n{}\\n\".format(obe))\n", + "\n", + "# ( OR ) AND boolean\n", + "ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n", + "ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n", + "ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n", + "obe1 = OrBooleanExpression([ece7, ece8])\n", + "pobe = ParentheticalExpression(obe1)\n", + "abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))\n", + "print(\"(OR,AND)\\n{}\\n\".format(abe1))\n", + "\n", + "# ( AND ) OR ( OR ) observation\n", + "ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n", + "ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n", + "ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n", + "ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n", + "# NOTE: we need to use AND/OR observation expression instead of just boolean \n", + "# expressions as the operands are not on the same object-type\n", + "aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n", + "obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))\n", + "ooe = OrObservationExpression([aoe, obe2])\n", + "print(\"(AND,OR,OR)\\n{}\\n\".format(ooe))\n", + "\n", + "# FOLLOWED-BY\n", + "ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n", + "ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n", + "fbe = FollowedByObservationExpression([ece10, ece11])\n", + "print(\"(FollowedBy)\\n{}\\n\".format(fbe))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Qualified Observation Expressions" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "--- Qualified Observation Expressions\n", + "\n", + "(WITHIN)\n", + "([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS\n", + "\n", + "(REPEAT, WITHIN)\n", + "[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n", + "\n", + "(START-STOP)\n", + "[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n", + "\n" + ] + } + ], + "source": [ + "from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n", + " AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n", + " QualifiedObservationExpression, FollowedByObservationExpression,\n", + " ParentheticalExpression, ObservationExpression)\n", + "\n", + "# Qualified Observation Expressions\n", + "print(\"--- Qualified Observation Expressions\\n\")\n", + "\n", + "# WITHIN\n", + "ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n", + "ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n", + "fbe = FollowedByObservationExpression([ece10, ece11])\n", + "par = ParentheticalExpression(fbe)\n", + "qoe = QualifiedObservationExpression(par, WithinQualifier(300))\n", + "print(\"(WITHIN)\\n{}\\n\".format(qoe))\n", + "\n", + "# REPEATS, WITHIN\n", + "ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n", + "ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n", + "abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))\n", + "qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n", + "print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1))\n", + "\n", + "# START, STOP\n", + "ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n", + "ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n", + "qoe2 = QualifiedObservationExpression(ece14, ssq)\n", + "print(\"(START-STOP)\\n{}\\n\".format(qoe2))\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Attaching patterns to STIX2 Domain objects\n", + "\n", + "\n", + "### Example" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{\n", + " \"type\": \"indicator\",\n", + " \"id\": \"indicator--219bc5fc-fdbf-4b54-a2fc-921be7ab3acb\",\n", + " \"created\": \"2018-08-29T23:58:00.548Z\",\n", + " \"modified\": \"2018-08-29T23:58:00.548Z\",\n", + " \"name\": \"Cryptotorch\",\n", + " \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n", + " \"valid_from\": \"2018-08-29T23:58:00.548391Z\",\n", + " \"labels\": [\n", + " \"malware\",\n", + " \"ransomware\"\n", + " ]\n", + "}\n" + ] + } + ], + "source": [ + "from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression\n", + "\n", + "ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\"))\n", + "ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=ece14)\n", + "print(ind)" ] } ], diff --git a/stix2/patterns.py b/stix2/patterns.py index 59bd2bf..146ec04 100644 --- a/stix2/patterns.py +++ b/stix2/patterns.py @@ -313,10 +313,7 @@ class ObjectPath(object): class _PatternExpression(object): - - def to_pattern(self): - """return a properly formatted string of the pattern expression""" - return "[{}]".format(self.__str__()) + pass class _ComparisonExpression(_PatternExpression): @@ -445,7 +442,6 @@ class MatchesComparisonExpression(_ComparisonExpression): rhs (ObjectPath OR str): object path of right-hand-side component of expression negated (bool): comparison expression negated. Default: False """ - def __init__(self, lhs, rhs, negated=False): super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated) @@ -458,7 +454,6 @@ class IsSubsetComparisonExpression(_ComparisonExpression): rhs (ObjectPath OR str): object path of right-hand-side component of expression negated (bool): comparison expression negated. Default: False """ - def __init__(self, lhs, rhs, negated=False): super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated) @@ -471,7 +466,6 @@ class IsSupersetComparisonExpression(_ComparisonExpression): rhs (ObjectPath OR str): object path of right-hand-side component of expression negated (bool): comparison expression negated. Default: False """ - def __init__(self, lhs, rhs, negated=False): super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated) @@ -498,12 +492,13 @@ class _BooleanExpression(_PatternExpression): def __str__(self): sub_exprs = [] for o in self.operands: - sub_exprs.append("%s" % o) + sub_exprs.append(str(o)) return (" " + self.operator + " ").join(sub_exprs) class AndBooleanExpression(_BooleanExpression): - """'AND' Boolean Pattern Expression + """'AND' Boolean Pattern Expression. Only use if both operands are of + the same root object. Args: operands (list): AND operands @@ -513,7 +508,7 @@ class AndBooleanExpression(_BooleanExpression): class OrBooleanExpression(_BooleanExpression): - """'OR' Boolean Pattern Expression + """'OR' Boolean Pattern Expression. Only use if both operands are of the same root object Args: operands (list): OR operands @@ -552,11 +547,6 @@ class _CompoundObservationExpression(_PatternExpression): sub_exprs.append("%s" % o) return (" " + self.operator + " ").join(sub_exprs) - def to_pattern(self): - return "{0} {1} {2}".format(self.operands[0].to_pattern(), - self.operator, - self.operands[1].to_pattern()) - class AndObservationExpression(_CompoundObservationExpression): """'AND' Compound Observation Pattern Expression @@ -564,7 +554,6 @@ class AndObservationExpression(_CompoundObservationExpression): Args: operands (str): compound observation operands """ - def __init__(self, operands): super(AndObservationExpression, self).__init__("AND", operands) @@ -588,9 +577,6 @@ class FollowedByObservationExpression(_CompoundObservationExpression): def __init__(self, operands): super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands) - def to_pattern(self): - return "[{}] {} [{}]".format(self.operands[0], "FOLLOWEDBY", self.operands[1]) - class ParentheticalExpression(_PatternExpression): """Pattern Parenthetical Observation Expression @@ -606,9 +592,6 @@ class ParentheticalExpression(_PatternExpression): def __str__(self): return "(%s)" % self.expression - def to_pattern(self): - return "({})".format(self.expression.to_pattern()) - class _ExpressionQualifier(_PatternExpression): pass @@ -688,6 +671,3 @@ class QualifiedObservationExpression(_PatternExpression): def __str__(self): return "%s %s" % (self.observation_expression, self.qualifier) - - def to_pattern(self): - return "{} {}".format(self.observation_expression.to_pattern(), self.qualifier) diff --git a/stix2/test/test_pattern_expressions.py b/stix2/test/test_pattern_expressions.py index 14e3774..a4d0a5c 100644 --- a/stix2/test/test_pattern_expressions.py +++ b/stix2/test/test_pattern_expressions.py @@ -9,6 +9,7 @@ def test_create_comparison_expression(): exp = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'", stix2.HashConstant("aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "SHA-256")) # noqa + assert str(exp) == "file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f'" @@ -18,6 +19,7 @@ def test_boolean_expression(): exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name", stix2.StringConstant("^Final Report.+\\.exe$")) exp = stix2.AndBooleanExpression([exp1, exp2]) + assert str(exp) == "email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$'" # noqa @@ -66,9 +68,8 @@ def test_file_observable_expression(): "aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", 'SHA-256')) exp2 = stix2.EqualityComparisonExpression("file:mime_type", stix2.StringConstant("application/x-pdf")) - bool_exp = stix2.AndBooleanExpression([exp1, exp2]) - exp = stix2.ObservationExpression(bool_exp) - assert str(exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa + bool_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2])) + assert str(bool_exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa @pytest.mark.parametrize("observation_class, op", [ @@ -109,14 +110,12 @@ def test_artifact_payload(): "application/vnd.tcpdump.pcap") exp2 = stix2.MatchesComparisonExpression("artifact:payload_bin", stix2.StringConstant("\\xd4\\xc3\\xb2\\xa1\\x02\\x00\\x04\\x00")) - and_exp = stix2.AndBooleanExpression([exp1, exp2]) - exp = stix2.ObservationExpression(and_exp) - assert str(exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa + and_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2])) + assert str(and_exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa def test_greater_than_python_constant(): - exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", - 7.0) + exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", 7.0) exp = stix2.ObservationExpression(exp1) assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]" @@ -129,14 +128,14 @@ def test_greater_than(): def test_less_than(): - exp = stix2.LessThanComparisonExpression("file:size", - 1024) + exp = stix2.LessThanComparisonExpression("file:size", 1024) assert str(exp) == "file:size < 1024" def test_greater_than_or_equal(): exp = stix2.GreaterThanEqualComparisonExpression("file:size", 1024) + assert str(exp) == "file:size >= 1024" @@ -261,7 +260,7 @@ def test_invalid_integer_constant(): def test_invalid_timestamp_constant(): with pytest.raises(ValueError) as excinfo: stix2.TimestampConstant('foo') - assert 'must be a datetime object or timestamp string' in str(excinfo) + assert 'Must be a datetime object or timestamp string' in str(excinfo) def test_invalid_float_constant():