MISP
/
cti-python-stix2
mirror of https://github.com/MISP/cti-python-stix2
2
0
Fork 0
Code Issues Releases Wiki Activity

added to pattern guide; reverting pattern str conversion (as was already proper)

stix2.0
mbastian1135 2018-08-30 11:57:20 -04:00
parent 27dbaa5a65
commit 5648d2bc45
3 changed files with 282 additions and 302 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

533
docs/guide/patterns.ipynb
View File

@ -4,282 +4,28 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# STIX Patterns\n", "# STIX2 Patterns\n",
"\n", "\n",
"Python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. Python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher].(https://github.com/oasis-open/cti-pattern-matcher)\n", "python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher](https://github.com/oasis-open/cti-pattern-matcher).\n",
"\n",
"## Intro\n",
"\n",
"Python-stix2 patterns are built compositely from the bottom up, creating components at sublevels before those at higher levels.\n",
"\n",
"## Examples\n",
"\n",
"### Comparison Expressions"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Equality Comparison Expressions:\n",
"\n",
"\t[domain-name:value = 'site.of.interest.zaz']\n",
"\n",
"\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
"\n", "\n",
"\n", "\n",
"Greater-than Comparison Expression:\n", "python-stix2 patterns are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.\n",
"\n", "\n",
"\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n", "## API Tips\n",
"\n", "\n",
"### ObservationExpression\n",
"\n", "\n",
"Is-Subset Comparison Expression:\n", "Within the STIX2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets(```[]```) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.\n",
"\n", "\n",
"\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n", "This requirement manifests itself within python-stix2 via ```ObservationExpression```. When creating STIX2 observation expressions, whenever the current expression is complete, wrap it with ```ObservationExpression()```. This allows the complete pattern expression - no matter complexity - to be rendered as a proper specification adhering string. *__Of which, when pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations__* . While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings. \n",
"\n"
]
}
],
"source": [
"from stix2 import DomainName, File, IPv4Address\n",
"from stix2 import (ObjectPath, EqualityComparisonExpression, GreaterThanComparisonExpression,\n",
" IsSubsetComparisonExpression, FloatConstant, StringConstant)\n",
"\n", "\n",
"# ---- Equality Comparison expressions\n", "In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ```ObservationExpression()```.\n",
"print(\"Equality Comparison Expressions:\\n\")\n",
"\n", "\n",
"lhs = ObjectPath(\"domain-name\", [\"value\"])\n", "### ParentheticalExpression\n",
"ece_1 = EqualityComparisonExpression(lhs, \"site.of.interest.zaz\")\n",
"print(\"\\t{}\\n\".format(ece_1.to_pattern()))\n",
"\n", "\n",
"lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n", "Do not be confused by the ```ParentheticalExpression``` object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ```ObservationExpression```, wrapping completed subcomponent expressions with ```ParentheticalExpression()``` if explicit ordering is required. For usage examples with ```ParentheticalExpression```'s, see [here](#Compound-Observation-Expressions).\n",
"ece_2 = EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\")\n",
"print(\"\\t{}\\n\".format(ece_2.to_pattern()))\n",
"\n", "\n",
"# Greater-than Comparison expressions\n", "### BooleanExpressions vs CompoundObservationExpressions\n",
"print(\"\\nGreater-than Comparison Expression:\\n\")\n",
"\n",
"lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
"gte = GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\"))\n",
"print(\"\\t{}\\n\".format(gte.to_pattern()))\n",
"\n",
"# IsSubset Comparison expressions\n",
"print(\"\\nIs-Subset Comparison Expression:\\n\")\n",
"\n",
"lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
"iss = IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\"))\n",
"print(\"\\t{}\\n\".format(iss.to_pattern()))\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Observation Expressions:\n",
"\n",
"(AND)\n",
"[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
"\n",
"(OR)\n",
"[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
"\n",
"(OR,AND)\n",
"[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
"\n",
"(AND,OR,OR)\n",
"([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
"\n",
"(FollowedBy)\n",
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
"\n"
]
}
],
"source": [
"from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
" EqualityComparisonExpression, AndBooleanExpression,\n",
" OrBooleanExpression, ParentheticalExpression,\n",
" AndObservationExpression, OrObservationExpression,\n",
" FollowedByObservationExpression)\n",
"\n",
"# ---- Observation expressions\n",
"print(\"Observation Expressions:\\n\")\n",
"\n",
"# AND boolean\n",
"ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
"ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
"abe = AndBooleanExpression([ece3, ece4])\n",
"print(\"(AND)\\n{}\\n\".format(abe.to_pattern()))\n",
"\n",
"# OR boolean\n",
"ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
"ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
"obe = OrBooleanExpression([ece5, ece6])\n",
"print(\"(OR)\\n{}\\n\".format(obe.to_pattern()))\n",
"\n",
"# ( OR ) AND boolean\n",
"ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
"ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
"ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
"obe1 = OrBooleanExpression([ece7, ece8])\n",
"pobe = ParentheticalExpression(obe1)\n",
"abe1 = AndBooleanExpression([pobe, ece9])\n",
"print(\"(OR,AND)\\n{}\\n\".format(abe1.to_pattern()))\n",
"\n",
"# ( AND ) OR ( OR ) observation\n",
"ece20 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n",
"ece21 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
"ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
"ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
"# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
"# expressions as the operands are not on the same object-type\n",
"aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
"obe2 = OrBooleanExpression([ece22, ece23])\n",
"ooe = OrObservationExpression([aoe, obe2])\n",
"print(\"(AND,OR,OR)\\n{}\\n\".format(ooe.to_pattern()))\n",
"\n",
"# FOLLOWED-BY\n",
"ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n",
"ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"print(\"(FollowedBy)\\n{}\\n\".format(fbe.to_pattern()))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"__See [note](#BooleanExpressions-vs-CompoundObservationExpressions) on when to use BooleanExpressions vs CompoundObservationExpressions__"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Qualified Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Qualified Observation Expressions:\n",
"\n",
"(WITHIN)\n",
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar'] WITHIN 300 SECONDS\n",
"\n",
"(REPEAT, WITHIN)\n",
"[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
"\n",
"(START-STOP)\n",
"[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
"\n"
]
}
],
"source": [
"from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
" AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
" QualifiedObservationExpression, FollowedByObservationExpression)\n",
"\n",
"# Qualified Observation Expressions\n",
"print(\"Qualified Observation Expressions:\\n\")\n",
"\n",
"# WITHIN\n",
"ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n",
"ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"qoe = QualifiedObservationExpression(fbe, WithinQualifier(300))\n",
"print(\"(WITHIN)\\n{}\\n\".format(qoe.to_pattern()))\n",
"\n",
"# REPEATS, WITHIN\n",
"ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
"ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
"abe2 = AndBooleanExpression([ece12, ece13])\n",
"qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
"print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1.to_pattern()))\n",
"\n",
"# START, STOP\n",
"ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n",
"ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
"qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
"print(\"(START-STOP)\\n{}\\n\".format(qoe2.to_pattern()))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Attaching patterns to STIX2 Domain objects\n",
"\n",
"As seen in the previous examples, the *__to_pattern()__* utility is used to display the pattern. However, the true purpose of this utility is to be able to seamlessly add a constructed pattern to an python-stix2 object.\n",
"\n",
"The *__to_pattern()__* utility constructs a string version of the pattern that adheres to the STIX2 patterning language specification.\n",
"\n",
"### Example"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"type\": \"indicator\",\n",
" \"id\": \"indicator--9f2d8014-3c2f-4127-bc2c-a209913404a5\",\n",
" \"created\": \"2018-08-27T18:57:00.762Z\",\n",
" \"modified\": \"2018-08-27T18:57:00.762Z\",\n",
" \"name\": \"Cryptotorch\",\n",
" \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
" \"valid_from\": \"2018-08-27T18:57:00.762972Z\",\n",
" \"labels\": [\n",
" \"malware\",\n",
" \"ransomware\"\n",
" ]\n",
"}\n"
]
}
],
"source": [
"from stix2 import Indicator, EqualityComparisonExpression\n",
"\n",
"ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\")\n",
"ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=\"[{}]\".format(ece14))\n",
"print(ind)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## BooleanExpressions vs CompoundObservationExpressions\n",
"\n", "\n",
"Be careful to note the difference between these two very similar pattern components. \n", "Be careful to note the difference between these two very similar pattern components. \n",
"\n", "\n",
@ -304,7 +50,262 @@
" ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n", " ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n",
" \n", " \n",
" __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n", " __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n",
" \n" "\n",
"\n",
"\n",
"## Examples\n",
"\n",
"### Comparison Expressions"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Equality Comparison Expressions\n",
"\n",
"\t[domain-name:value = 'site.of.interest.zaz']\n",
"\n",
"\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
"\n",
"\n",
"--- Greater-than Comparison Expression\n",
"\n",
"\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n",
"\n",
"\n",
"--- Is-Subset Comparison Expression\n",
"\n",
"\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n",
"\n"
]
}
],
"source": [
"from stix2 import DomainName, File, IPv4Address\n",
"from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,\n",
" GreaterThanComparisonExpression, IsSubsetComparisonExpression,\n",
" FloatConstant, StringConstant)\n",
"\n",
"# ---- Equality Comparison expressions\n",
"print(\"--- Equality Comparison Expressions\\n\")\n",
"\n",
"lhs = ObjectPath(\"domain-name\", [\"value\"])\n",
"ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, \"site.of.interest.zaz\"))\n",
"print(\"\\t{}\\n\".format(ece_1))\n",
"\n",
"lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n",
"ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\"))\n",
"print(\"\\t{}\\n\".format(ece_2))\n",
"\n",
"# Greater-than Comparison expressions\n",
"print(\"\\n--- Greater-than Comparison Expression\\n\")\n",
"\n",
"lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
"gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\")))\n",
"print(\"\\t{}\\n\".format(gte))\n",
"\n",
"# IsSubset Comparison expressions\n",
"print(\"\\n--- Is-Subset Comparison Expression\\n\")\n",
"\n",
"lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
"iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\")))\n",
"print(\"\\t{}\\n\".format(iss))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Compound Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Compound Observation Expressions\n",
"\n",
"(AND)\n",
"[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
"\n",
"(OR)\n",
"[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
"\n",
"(OR,AND)\n",
"[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
"\n",
"(AND,OR,OR)\n",
"([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
"\n",
"(FollowedBy)\n",
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
"\n"
]
}
],
"source": [
"from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
" EqualityComparisonExpression, AndBooleanExpression,\n",
" OrBooleanExpression, ParentheticalExpression,\n",
" AndObservationExpression, OrObservationExpression,\n",
" FollowedByObservationExpression, ObservationExpression)\n",
"\n",
"# ---- Observation expressions\n",
"print(\"--- Compound Observation Expressions\\n\")\n",
"\n",
"# AND boolean\n",
"ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
"ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
"abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))\n",
"print(\"(AND)\\n{}\\n\".format(abe))\n",
"\n",
"# OR boolean\n",
"ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
"ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
"obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))\n",
"print(\"(OR)\\n{}\\n\".format(obe))\n",
"\n",
"# ( OR ) AND boolean\n",
"ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
"ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
"ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
"obe1 = OrBooleanExpression([ece7, ece8])\n",
"pobe = ParentheticalExpression(obe1)\n",
"abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))\n",
"print(\"(OR,AND)\\n{}\\n\".format(abe1))\n",
"\n",
"# ( AND ) OR ( OR ) observation\n",
"ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
"ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
"ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
"# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
"# expressions as the operands are not on the same object-type\n",
"aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
"obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))\n",
"ooe = OrObservationExpression([aoe, obe2])\n",
"print(\"(AND,OR,OR)\\n{}\\n\".format(ooe))\n",
"\n",
"# FOLLOWED-BY\n",
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"print(\"(FollowedBy)\\n{}\\n\".format(fbe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Qualified Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--- Qualified Observation Expressions\n",
"\n",
"(WITHIN)\n",
"([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS\n",
"\n",
"(REPEAT, WITHIN)\n",
"[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
"\n",
"(START-STOP)\n",
"[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
"\n"
]
}
],
"source": [
"from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
" AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
" QualifiedObservationExpression, FollowedByObservationExpression,\n",
" ParentheticalExpression, ObservationExpression)\n",
"\n",
"# Qualified Observation Expressions\n",
"print(\"--- Qualified Observation Expressions\\n\")\n",
"\n",
"# WITHIN\n",
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"par = ParentheticalExpression(fbe)\n",
"qoe = QualifiedObservationExpression(par, WithinQualifier(300))\n",
"print(\"(WITHIN)\\n{}\\n\".format(qoe))\n",
"\n",
"# REPEATS, WITHIN\n",
"ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
"ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
"abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))\n",
"qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
"print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1))\n",
"\n",
"# START, STOP\n",
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
"ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
"qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
"print(\"(START-STOP)\\n{}\\n\".format(qoe2))\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Attaching patterns to STIX2 Domain objects\n",
"\n",
"\n",
"### Example"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"type\": \"indicator\",\n",
" \"id\": \"indicator--219bc5fc-fdbf-4b54-a2fc-921be7ab3acb\",\n",
" \"created\": \"2018-08-29T23:58:00.548Z\",\n",
" \"modified\": \"2018-08-29T23:58:00.548Z\",\n",
" \"name\": \"Cryptotorch\",\n",
" \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
" \"valid_from\": \"2018-08-29T23:58:00.548391Z\",\n",
" \"labels\": [\n",
" \"malware\",\n",
" \"ransomware\"\n",
" ]\n",
"}\n"
]
}
],
"source": [
"from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression\n",
"\n",
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\"))\n",
"ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=ece14)\n",
"print(ind)"
] ]
} }
], ],

30
stix2/patterns.py
View File

@ -313,10 +313,7 @@ class ObjectPath(object):
class _PatternExpression(object): class _PatternExpression(object):
pass
def to_pattern(self):
"""return a properly formatted string of the pattern expression"""
return "[{}]".format(self.__str__())
class _ComparisonExpression(_PatternExpression): class _ComparisonExpression(_PatternExpression):
@ -445,7 +442,6 @@ class MatchesComparisonExpression(_ComparisonExpression):
rhs (ObjectPath OR str): object path of right-hand-side component of expression rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False negated (bool): comparison expression negated. Default: False
""" """
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated) super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated)
@ -458,7 +454,6 @@ class IsSubsetComparisonExpression(_ComparisonExpression):
rhs (ObjectPath OR str): object path of right-hand-side component of expression rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False negated (bool): comparison expression negated. Default: False
""" """
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated) super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
@ -471,7 +466,6 @@ class IsSupersetComparisonExpression(_ComparisonExpression):
rhs (ObjectPath OR str): object path of right-hand-side component of expression rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False negated (bool): comparison expression negated. Default: False
""" """
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated) super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
@ -498,12 +492,13 @@ class _BooleanExpression(_PatternExpression):
def __str__(self): def __str__(self):
sub_exprs = [] sub_exprs = []
for o in self.operands: for o in self.operands:
sub_exprs.append("%s" % o) sub_exprs.append(str(o))
return (" " + self.operator + " ").join(sub_exprs) return (" " + self.operator + " ").join(sub_exprs)
class AndBooleanExpression(_BooleanExpression): class AndBooleanExpression(_BooleanExpression):
"""'AND' Boolean Pattern Expression """'AND' Boolean Pattern Expression. Only use if both operands are of
the same root object.
Args: Args:
operands (list): AND operands operands (list): AND operands
@ -513,7 +508,7 @@ class AndBooleanExpression(_BooleanExpression):
class OrBooleanExpression(_BooleanExpression): class OrBooleanExpression(_BooleanExpression):
"""'OR' Boolean Pattern Expression """'OR' Boolean Pattern Expression. Only use if both operands are of the same root object
Args: Args:
operands (list): OR operands operands (list): OR operands
@ -552,11 +547,6 @@ class _CompoundObservationExpression(_PatternExpression):
sub_exprs.append("%s" % o) sub_exprs.append("%s" % o)
return (" " + self.operator + " ").join(sub_exprs) return (" " + self.operator + " ").join(sub_exprs)
def to_pattern(self):
return "{0} {1} {2}".format(self.operands[0].to_pattern(),
self.operator,
self.operands[1].to_pattern())
class AndObservationExpression(_CompoundObservationExpression): class AndObservationExpression(_CompoundObservationExpression):
"""'AND' Compound Observation Pattern Expression """'AND' Compound Observation Pattern Expression
@ -564,7 +554,6 @@ class AndObservationExpression(_CompoundObservationExpression):
Args: Args:
operands (str): compound observation operands operands (str): compound observation operands
""" """
def __init__(self, operands): def __init__(self, operands):
super(AndObservationExpression, self).__init__("AND", operands) super(AndObservationExpression, self).__init__("AND", operands)
@ -588,9 +577,6 @@ class FollowedByObservationExpression(_CompoundObservationExpression):
def __init__(self, operands): def __init__(self, operands):
super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands) super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands)
def to_pattern(self):
return "[{}] {} [{}]".format(self.operands[0], "FOLLOWEDBY", self.operands[1])
class ParentheticalExpression(_PatternExpression): class ParentheticalExpression(_PatternExpression):
"""Pattern Parenthetical Observation Expression """Pattern Parenthetical Observation Expression
@ -606,9 +592,6 @@ class ParentheticalExpression(_PatternExpression):
def __str__(self): def __str__(self):
return "(%s)" % self.expression return "(%s)" % self.expression
def to_pattern(self):
return "({})".format(self.expression.to_pattern())
class _ExpressionQualifier(_PatternExpression): class _ExpressionQualifier(_PatternExpression):
pass pass
@ -688,6 +671,3 @@ class QualifiedObservationExpression(_PatternExpression):
def __str__(self): def __str__(self):
return "%s %s" % (self.observation_expression, self.qualifier) return "%s %s" % (self.observation_expression, self.qualifier)
def to_pattern(self):
return "{} {}".format(self.observation_expression.to_pattern(), self.qualifier)

21
stix2/test/test_pattern_expressions.py
View File

@ -9,6 +9,7 @@ def test_create_comparison_expression():
exp = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'", exp = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
stix2.HashConstant("aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "SHA-256")) # noqa stix2.HashConstant("aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "SHA-256")) # noqa
assert str(exp) == "file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f'" assert str(exp) == "file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f'"
@ -18,6 +19,7 @@ def test_boolean_expression():
exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name", exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name",
stix2.StringConstant("^Final Report.+\\.exe$")) stix2.StringConstant("^Final Report.+\\.exe$"))
exp = stix2.AndBooleanExpression([exp1, exp2]) exp = stix2.AndBooleanExpression([exp1, exp2])
assert str(exp) == "email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$'" # noqa assert str(exp) == "email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$'" # noqa
@ -66,9 +68,8 @@ def test_file_observable_expression():
"aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f",
'SHA-256')) 'SHA-256'))
exp2 = stix2.EqualityComparisonExpression("file:mime_type", stix2.StringConstant("application/x-pdf")) exp2 = stix2.EqualityComparisonExpression("file:mime_type", stix2.StringConstant("application/x-pdf"))
bool_exp = stix2.AndBooleanExpression([exp1, exp2]) bool_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
exp = stix2.ObservationExpression(bool_exp) assert str(bool_exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
assert str(exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
@pytest.mark.parametrize("observation_class, op", [ @pytest.mark.parametrize("observation_class, op", [
@ -109,14 +110,12 @@ def test_artifact_payload():
"application/vnd.tcpdump.pcap") "application/vnd.tcpdump.pcap")
exp2 = stix2.MatchesComparisonExpression("artifact:payload_bin", exp2 = stix2.MatchesComparisonExpression("artifact:payload_bin",
stix2.StringConstant("\\xd4\\xc3\\xb2\\xa1\\x02\\x00\\x04\\x00")) stix2.StringConstant("\\xd4\\xc3\\xb2\\xa1\\x02\\x00\\x04\\x00"))
and_exp = stix2.AndBooleanExpression([exp1, exp2]) and_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
exp = stix2.ObservationExpression(and_exp) assert str(and_exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa
assert str(exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa
def test_greater_than_python_constant(): def test_greater_than_python_constant():
exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", 7.0)
7.0)
exp = stix2.ObservationExpression(exp1) exp = stix2.ObservationExpression(exp1)
assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]" assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]"
@ -129,14 +128,14 @@ def test_greater_than():
def test_less_than(): def test_less_than():
exp = stix2.LessThanComparisonExpression("file:size", exp = stix2.LessThanComparisonExpression("file:size", 1024)
1024)
assert str(exp) == "file:size < 1024" assert str(exp) == "file:size < 1024"
def test_greater_than_or_equal(): def test_greater_than_or_equal():
exp = stix2.GreaterThanEqualComparisonExpression("file:size", exp = stix2.GreaterThanEqualComparisonExpression("file:size",
1024) 1024)
assert str(exp) == "file:size >= 1024" assert str(exp) == "file:size >= 1024"
@ -261,7 +260,7 @@ def test_invalid_integer_constant():
def test_invalid_timestamp_constant(): def test_invalid_timestamp_constant():
with pytest.raises(ValueError) as excinfo: with pytest.raises(ValueError) as excinfo:
stix2.TimestampConstant('foo') stix2.TimestampConstant('foo')
assert 'must be a datetime object or timestamp string' in str(excinfo) assert 'Must be a datetime object or timestamp string' in str(excinfo)
def test_invalid_float_constant(): def test_invalid_float_constant():