added to pattern guide; reverting pattern str conversion (as was already proper)
mbastian1135
parent
27dbaa5a65
commit
5648d2bc45
|
|
@ -4,282 +4,28 @@
|
|||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# STIX Patterns\n",
|
||||
"# STIX2 Patterns\n",
|
||||
"\n",
|
||||
"Python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. Python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher].(https://github.com/oasis-open/cti-pattern-matcher)\n",
|
||||
"\n",
|
||||
"## Intro\n",
|
||||
"\n",
|
||||
"Python-stix2 patterns are built compositely from the bottom up, creating components at sublevels before those at higher levels.\n",
|
||||
"\n",
|
||||
"## Examples\n",
|
||||
"\n",
|
||||
"### Comparison Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Equality Comparison Expressions:\n",
|
||||
"\n",
|
||||
"\t[domain-name:value = 'site.of.interest.zaz']\n",
|
||||
"\n",
|
||||
"\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
|
||||
"python-stix2 supports STIX2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX2 specification. python-stix2 does not evaluate patterns against STIX2 content, for that functionality see [cti-pattern-matcher](https://github.com/oasis-open/cti-pattern-matcher).\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Greater-than Comparison Expression:\n",
|
||||
"python-stix2 patterns are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.\n",
|
||||
"\n",
|
||||
"\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n",
|
||||
"## API Tips\n",
|
||||
"\n",
|
||||
"### ObservationExpression\n",
|
||||
"\n",
|
||||
"Is-Subset Comparison Expression:\n",
|
||||
"Within the STIX2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets(```[]```) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.\n",
|
||||
"\n",
|
||||
"\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import DomainName, File, IPv4Address\n",
|
||||
"from stix2 import (ObjectPath, EqualityComparisonExpression, GreaterThanComparisonExpression,\n",
|
||||
" IsSubsetComparisonExpression, FloatConstant, StringConstant)\n",
|
||||
"This requirement manifests itself within python-stix2 via ```ObservationExpression```. When creating STIX2 observation expressions, whenever the current expression is complete, wrap it with ```ObservationExpression()```. This allows the complete pattern expression - no matter complexity - to be rendered as a proper specification adhering string. *__Of which, when pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations__* . While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings. \n",
|
||||
"\n",
|
||||
"# ---- Equality Comparison expressions\n",
|
||||
"print(\"Equality Comparison Expressions:\\n\")\n",
|
||||
"In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ```ObservationExpression()```.\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"domain-name\", [\"value\"])\n",
|
||||
"ece_1 = EqualityComparisonExpression(lhs, \"site.of.interest.zaz\")\n",
|
||||
"print(\"\\t{}\\n\".format(ece_1.to_pattern()))\n",
|
||||
"### ParentheticalExpression\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n",
|
||||
"ece_2 = EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\")\n",
|
||||
"print(\"\\t{}\\n\".format(ece_2.to_pattern()))\n",
|
||||
"Do not be confused by the ```ParentheticalExpression``` object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ```ObservationExpression```, wrapping completed subcomponent expressions with ```ParentheticalExpression()``` if explicit ordering is required. For usage examples with ```ParentheticalExpression```'s, see [here](#Compound-Observation-Expressions).\n",
|
||||
"\n",
|
||||
"# Greater-than Comparison expressions\n",
|
||||
"print(\"\\nGreater-than Comparison Expression:\\n\")\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
|
||||
"gte = GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\"))\n",
|
||||
"print(\"\\t{}\\n\".format(gte.to_pattern()))\n",
|
||||
"\n",
|
||||
"# IsSubset Comparison expressions\n",
|
||||
"print(\"\\nIs-Subset Comparison Expression:\\n\")\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
|
||||
"iss = IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\"))\n",
|
||||
"print(\"\\t{}\\n\".format(iss.to_pattern()))\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Observation Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Observation Expressions:\n",
|
||||
"\n",
|
||||
"(AND)\n",
|
||||
"[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
|
||||
"\n",
|
||||
"(OR)\n",
|
||||
"[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
|
||||
"\n",
|
||||
"(OR,AND)\n",
|
||||
"[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
|
||||
"\n",
|
||||
"(AND,OR,OR)\n",
|
||||
"([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
|
||||
"\n",
|
||||
"(FollowedBy)\n",
|
||||
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
|
||||
" EqualityComparisonExpression, AndBooleanExpression,\n",
|
||||
" OrBooleanExpression, ParentheticalExpression,\n",
|
||||
" AndObservationExpression, OrObservationExpression,\n",
|
||||
" FollowedByObservationExpression)\n",
|
||||
"\n",
|
||||
"# ---- Observation expressions\n",
|
||||
"print(\"Observation Expressions:\\n\")\n",
|
||||
"\n",
|
||||
"# AND boolean\n",
|
||||
"ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
|
||||
"ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
|
||||
"abe = AndBooleanExpression([ece3, ece4])\n",
|
||||
"print(\"(AND)\\n{}\\n\".format(abe.to_pattern()))\n",
|
||||
"\n",
|
||||
"# OR boolean\n",
|
||||
"ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
|
||||
"ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
|
||||
"obe = OrBooleanExpression([ece5, ece6])\n",
|
||||
"print(\"(OR)\\n{}\\n\".format(obe.to_pattern()))\n",
|
||||
"\n",
|
||||
"# ( OR ) AND boolean\n",
|
||||
"ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
|
||||
"ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
|
||||
"ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
|
||||
"obe1 = OrBooleanExpression([ece7, ece8])\n",
|
||||
"pobe = ParentheticalExpression(obe1)\n",
|
||||
"abe1 = AndBooleanExpression([pobe, ece9])\n",
|
||||
"print(\"(OR,AND)\\n{}\\n\".format(abe1.to_pattern()))\n",
|
||||
"\n",
|
||||
"# ( AND ) OR ( OR ) observation\n",
|
||||
"ece20 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n",
|
||||
"ece21 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
|
||||
"ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
|
||||
"ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
|
||||
"# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
|
||||
"# expressions as the operands are not on the same object-type\n",
|
||||
"aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
|
||||
"obe2 = OrBooleanExpression([ece22, ece23])\n",
|
||||
"ooe = OrObservationExpression([aoe, obe2])\n",
|
||||
"print(\"(AND,OR,OR)\\n{}\\n\".format(ooe.to_pattern()))\n",
|
||||
"\n",
|
||||
"# FOLLOWED-BY\n",
|
||||
"ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n",
|
||||
"ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
|
||||
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
|
||||
"print(\"(FollowedBy)\\n{}\\n\".format(fbe.to_pattern()))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"__See [note](#BooleanExpressions-vs-CompoundObservationExpressions) on when to use BooleanExpressions vs CompoundObservationExpressions__"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Qualified Observation Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Qualified Observation Expressions:\n",
|
||||
"\n",
|
||||
"(WITHIN)\n",
|
||||
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar'] WITHIN 300 SECONDS\n",
|
||||
"\n",
|
||||
"(REPEAT, WITHIN)\n",
|
||||
"[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
|
||||
"\n",
|
||||
"(START-STOP)\n",
|
||||
"[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
|
||||
" AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
|
||||
" QualifiedObservationExpression, FollowedByObservationExpression)\n",
|
||||
"\n",
|
||||
"# Qualified Observation Expressions\n",
|
||||
"print(\"Qualified Observation Expressions:\\n\")\n",
|
||||
"\n",
|
||||
"# WITHIN\n",
|
||||
"ece10 = EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\"))\n",
|
||||
"ece11 = EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\")\n",
|
||||
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
|
||||
"qoe = QualifiedObservationExpression(fbe, WithinQualifier(300))\n",
|
||||
"print(\"(WITHIN)\\n{}\\n\".format(qoe.to_pattern()))\n",
|
||||
"\n",
|
||||
"# REPEATS, WITHIN\n",
|
||||
"ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
|
||||
"ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
|
||||
"abe2 = AndBooleanExpression([ece12, ece13])\n",
|
||||
"qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
|
||||
"print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1.to_pattern()))\n",
|
||||
"\n",
|
||||
"# START, STOP\n",
|
||||
"ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\")\n",
|
||||
"ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
|
||||
"qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
|
||||
"print(\"(START-STOP)\\n{}\\n\".format(qoe2.to_pattern()))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Attaching patterns to STIX2 Domain objects\n",
|
||||
"\n",
|
||||
"As seen in the previous examples, the *__to_pattern()__* utility is used to display the pattern. However, the true purpose of this utility is to be able to seamlessly add a constructed pattern to an python-stix2 object.\n",
|
||||
"\n",
|
||||
"The *__to_pattern()__* utility constructs a string version of the pattern that adheres to the STIX2 patterning language specification.\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{\n",
|
||||
" \"type\": \"indicator\",\n",
|
||||
" \"id\": \"indicator--9f2d8014-3c2f-4127-bc2c-a209913404a5\",\n",
|
||||
" \"created\": \"2018-08-27T18:57:00.762Z\",\n",
|
||||
" \"modified\": \"2018-08-27T18:57:00.762Z\",\n",
|
||||
" \"name\": \"Cryptotorch\",\n",
|
||||
" \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
|
||||
" \"valid_from\": \"2018-08-27T18:57:00.762972Z\",\n",
|
||||
" \"labels\": [\n",
|
||||
" \"malware\",\n",
|
||||
" \"ransomware\"\n",
|
||||
" ]\n",
|
||||
"}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import Indicator, EqualityComparisonExpression\n",
|
||||
"\n",
|
||||
"ece14 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\")\n",
|
||||
"ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=\"[{}]\".format(ece14))\n",
|
||||
"print(ind)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## BooleanExpressions vs CompoundObservationExpressions\n",
|
||||
"### BooleanExpressions vs CompoundObservationExpressions\n",
|
||||
"\n",
|
||||
"Be careful to note the difference between these two very similar pattern components. \n",
|
||||
"\n",
|
||||
|
|
@ -304,7 +50,262 @@
|
|||
" ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n",
|
||||
" \n",
|
||||
" __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n",
|
||||
" \n"
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"## Examples\n",
|
||||
"\n",
|
||||
"### Comparison Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"--- Equality Comparison Expressions\n",
|
||||
"\n",
|
||||
"\t[domain-name:value = 'site.of.interest.zaz']\n",
|
||||
"\n",
|
||||
"\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"--- Greater-than Comparison Expression\n",
|
||||
"\n",
|
||||
"\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"--- Is-Subset Comparison Expression\n",
|
||||
"\n",
|
||||
"\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import DomainName, File, IPv4Address\n",
|
||||
"from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,\n",
|
||||
" GreaterThanComparisonExpression, IsSubsetComparisonExpression,\n",
|
||||
" FloatConstant, StringConstant)\n",
|
||||
"\n",
|
||||
"# ---- Equality Comparison expressions\n",
|
||||
"print(\"--- Equality Comparison Expressions\\n\")\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"domain-name\", [\"value\"])\n",
|
||||
"ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, \"site.of.interest.zaz\"))\n",
|
||||
"print(\"\\t{}\\n\".format(ece_1))\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n",
|
||||
"ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\"))\n",
|
||||
"print(\"\\t{}\\n\".format(ece_2))\n",
|
||||
"\n",
|
||||
"# Greater-than Comparison expressions\n",
|
||||
"print(\"\\n--- Greater-than Comparison Expression\\n\")\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
|
||||
"gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\")))\n",
|
||||
"print(\"\\t{}\\n\".format(gte))\n",
|
||||
"\n",
|
||||
"# IsSubset Comparison expressions\n",
|
||||
"print(\"\\n--- Is-Subset Comparison Expression\\n\")\n",
|
||||
"\n",
|
||||
"lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
|
||||
"iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\")))\n",
|
||||
"print(\"\\t{}\\n\".format(iss))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Compound Observation Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"--- Compound Observation Expressions\n",
|
||||
"\n",
|
||||
"(AND)\n",
|
||||
"[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
|
||||
"\n",
|
||||
"(OR)\n",
|
||||
"[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
|
||||
"\n",
|
||||
"(OR,AND)\n",
|
||||
"[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
|
||||
"\n",
|
||||
"(AND,OR,OR)\n",
|
||||
"([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
|
||||
"\n",
|
||||
"(FollowedBy)\n",
|
||||
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
|
||||
" EqualityComparisonExpression, AndBooleanExpression,\n",
|
||||
" OrBooleanExpression, ParentheticalExpression,\n",
|
||||
" AndObservationExpression, OrObservationExpression,\n",
|
||||
" FollowedByObservationExpression, ObservationExpression)\n",
|
||||
"\n",
|
||||
"# ---- Observation expressions\n",
|
||||
"print(\"--- Compound Observation Expressions\\n\")\n",
|
||||
"\n",
|
||||
"# AND boolean\n",
|
||||
"ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
|
||||
"ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
|
||||
"abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))\n",
|
||||
"print(\"(AND)\\n{}\\n\".format(abe))\n",
|
||||
"\n",
|
||||
"# OR boolean\n",
|
||||
"ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
|
||||
"ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
|
||||
"obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))\n",
|
||||
"print(\"(OR)\\n{}\\n\".format(obe))\n",
|
||||
"\n",
|
||||
"# ( OR ) AND boolean\n",
|
||||
"ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
|
||||
"ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
|
||||
"ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
|
||||
"obe1 = OrBooleanExpression([ece7, ece8])\n",
|
||||
"pobe = ParentheticalExpression(obe1)\n",
|
||||
"abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))\n",
|
||||
"print(\"(OR,AND)\\n{}\\n\".format(abe1))\n",
|
||||
"\n",
|
||||
"# ( AND ) OR ( OR ) observation\n",
|
||||
"ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
|
||||
"ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
|
||||
"ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
|
||||
"ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
|
||||
"# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
|
||||
"# expressions as the operands are not on the same object-type\n",
|
||||
"aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
|
||||
"obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))\n",
|
||||
"ooe = OrObservationExpression([aoe, obe2])\n",
|
||||
"print(\"(AND,OR,OR)\\n{}\\n\".format(ooe))\n",
|
||||
"\n",
|
||||
"# FOLLOWED-BY\n",
|
||||
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
|
||||
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
|
||||
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
|
||||
"print(\"(FollowedBy)\\n{}\\n\".format(fbe))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Qualified Observation Expressions"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"--- Qualified Observation Expressions\n",
|
||||
"\n",
|
||||
"(WITHIN)\n",
|
||||
"([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS\n",
|
||||
"\n",
|
||||
"(REPEAT, WITHIN)\n",
|
||||
"[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
|
||||
"\n",
|
||||
"(START-STOP)\n",
|
||||
"[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
|
||||
" AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
|
||||
" QualifiedObservationExpression, FollowedByObservationExpression,\n",
|
||||
" ParentheticalExpression, ObservationExpression)\n",
|
||||
"\n",
|
||||
"# Qualified Observation Expressions\n",
|
||||
"print(\"--- Qualified Observation Expressions\\n\")\n",
|
||||
"\n",
|
||||
"# WITHIN\n",
|
||||
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
|
||||
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
|
||||
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
|
||||
"par = ParentheticalExpression(fbe)\n",
|
||||
"qoe = QualifiedObservationExpression(par, WithinQualifier(300))\n",
|
||||
"print(\"(WITHIN)\\n{}\\n\".format(qoe))\n",
|
||||
"\n",
|
||||
"# REPEATS, WITHIN\n",
|
||||
"ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
|
||||
"ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
|
||||
"abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))\n",
|
||||
"qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
|
||||
"print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1))\n",
|
||||
"\n",
|
||||
"# START, STOP\n",
|
||||
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
|
||||
"ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
|
||||
"qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
|
||||
"print(\"(START-STOP)\\n{}\\n\".format(qoe2))\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Attaching patterns to STIX2 Domain objects\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Example"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{\n",
|
||||
" \"type\": \"indicator\",\n",
|
||||
" \"id\": \"indicator--219bc5fc-fdbf-4b54-a2fc-921be7ab3acb\",\n",
|
||||
" \"created\": \"2018-08-29T23:58:00.548Z\",\n",
|
||||
" \"modified\": \"2018-08-29T23:58:00.548Z\",\n",
|
||||
" \"name\": \"Cryptotorch\",\n",
|
||||
" \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
|
||||
" \"valid_from\": \"2018-08-29T23:58:00.548391Z\",\n",
|
||||
" \"labels\": [\n",
|
||||
" \"malware\",\n",
|
||||
" \"ransomware\"\n",
|
||||
" ]\n",
|
||||
"}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression\n",
|
||||
"\n",
|
||||
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\"))\n",
|
||||
"ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=ece14)\n",
|
||||
"print(ind)"
|
||||
]
|
||||
}
|
||||
],
|
||||
|
|
|
|||
|
|
@ -313,10 +313,7 @@ class ObjectPath(object):
|
|||
|
||||
|
||||
class _PatternExpression(object):
|
||||
|
||||
def to_pattern(self):
|
||||
"""return a properly formatted string of the pattern expression"""
|
||||
return "[{}]".format(self.__str__())
|
||||
pass
|
||||
|
||||
|
||||
class _ComparisonExpression(_PatternExpression):
|
||||
|
|
@ -445,7 +442,6 @@ class MatchesComparisonExpression(_ComparisonExpression):
|
|||
rhs (ObjectPath OR str): object path of right-hand-side component of expression
|
||||
negated (bool): comparison expression negated. Default: False
|
||||
"""
|
||||
|
||||
def __init__(self, lhs, rhs, negated=False):
|
||||
super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated)
|
||||
|
||||
|
|
@ -458,7 +454,6 @@ class IsSubsetComparisonExpression(_ComparisonExpression):
|
|||
rhs (ObjectPath OR str): object path of right-hand-side component of expression
|
||||
negated (bool): comparison expression negated. Default: False
|
||||
"""
|
||||
|
||||
def __init__(self, lhs, rhs, negated=False):
|
||||
super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
|
||||
|
||||
|
|
@ -471,7 +466,6 @@ class IsSupersetComparisonExpression(_ComparisonExpression):
|
|||
rhs (ObjectPath OR str): object path of right-hand-side component of expression
|
||||
negated (bool): comparison expression negated. Default: False
|
||||
"""
|
||||
|
||||
def __init__(self, lhs, rhs, negated=False):
|
||||
super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
|
||||
|
||||
|
|
@ -498,12 +492,13 @@ class _BooleanExpression(_PatternExpression):
|
|||
def __str__(self):
|
||||
sub_exprs = []
|
||||
for o in self.operands:
|
||||
sub_exprs.append("%s" % o)
|
||||
sub_exprs.append(str(o))
|
||||
return (" " + self.operator + " ").join(sub_exprs)
|
||||
|
||||
|
||||
class AndBooleanExpression(_BooleanExpression):
|
||||
"""'AND' Boolean Pattern Expression
|
||||
"""'AND' Boolean Pattern Expression. Only use if both operands are of
|
||||
the same root object.
|
||||
|
||||
Args:
|
||||
operands (list): AND operands
|
||||
|
|
@ -513,7 +508,7 @@ class AndBooleanExpression(_BooleanExpression):
|
|||
|
||||
|
||||
class OrBooleanExpression(_BooleanExpression):
|
||||
"""'OR' Boolean Pattern Expression
|
||||
"""'OR' Boolean Pattern Expression. Only use if both operands are of the same root object
|
||||
|
||||
Args:
|
||||
operands (list): OR operands
|
||||
|
|
@ -552,11 +547,6 @@ class _CompoundObservationExpression(_PatternExpression):
|
|||
sub_exprs.append("%s" % o)
|
||||
return (" " + self.operator + " ").join(sub_exprs)
|
||||
|
||||
def to_pattern(self):
|
||||
return "{0} {1} {2}".format(self.operands[0].to_pattern(),
|
||||
self.operator,
|
||||
self.operands[1].to_pattern())
|
||||
|
||||
|
||||
class AndObservationExpression(_CompoundObservationExpression):
|
||||
"""'AND' Compound Observation Pattern Expression
|
||||
|
|
@ -564,7 +554,6 @@ class AndObservationExpression(_CompoundObservationExpression):
|
|||
Args:
|
||||
operands (str): compound observation operands
|
||||
"""
|
||||
|
||||
def __init__(self, operands):
|
||||
super(AndObservationExpression, self).__init__("AND", operands)
|
||||
|
||||
|
|
@ -588,9 +577,6 @@ class FollowedByObservationExpression(_CompoundObservationExpression):
|
|||
def __init__(self, operands):
|
||||
super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands)
|
||||
|
||||
def to_pattern(self):
|
||||
return "[{}] {} [{}]".format(self.operands[0], "FOLLOWEDBY", self.operands[1])
|
||||
|
||||
|
||||
class ParentheticalExpression(_PatternExpression):
|
||||
"""Pattern Parenthetical Observation Expression
|
||||
|
|
@ -606,9 +592,6 @@ class ParentheticalExpression(_PatternExpression):
|
|||
def __str__(self):
|
||||
return "(%s)" % self.expression
|
||||
|
||||
def to_pattern(self):
|
||||
return "({})".format(self.expression.to_pattern())
|
||||
|
||||
|
||||
class _ExpressionQualifier(_PatternExpression):
|
||||
pass
|
||||
|
|
@ -688,6 +671,3 @@ class QualifiedObservationExpression(_PatternExpression):
|
|||
|
||||
def __str__(self):
|
||||
return "%s %s" % (self.observation_expression, self.qualifier)
|
||||
|
||||
def to_pattern(self):
|
||||
return "{} {}".format(self.observation_expression.to_pattern(), self.qualifier)
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ def test_create_comparison_expression():
|
|||
|
||||
exp = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
|
||||
stix2.HashConstant("aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "SHA-256")) # noqa
|
||||
|
||||
assert str(exp) == "file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f'"
|
||||
|
||||
|
||||
|
|
@ -18,6 +19,7 @@ def test_boolean_expression():
|
|||
exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name",
|
||||
stix2.StringConstant("^Final Report.+\\.exe$"))
|
||||
exp = stix2.AndBooleanExpression([exp1, exp2])
|
||||
|
||||
assert str(exp) == "email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$'" # noqa
|
||||
|
||||
|
||||
|
|
@ -66,9 +68,8 @@ def test_file_observable_expression():
|
|||
"aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f",
|
||||
'SHA-256'))
|
||||
exp2 = stix2.EqualityComparisonExpression("file:mime_type", stix2.StringConstant("application/x-pdf"))
|
||||
bool_exp = stix2.AndBooleanExpression([exp1, exp2])
|
||||
exp = stix2.ObservationExpression(bool_exp)
|
||||
assert str(exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
|
||||
bool_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
|
||||
assert str(bool_exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
|
||||
|
||||
|
||||
@pytest.mark.parametrize("observation_class, op", [
|
||||
|
|
@ -109,14 +110,12 @@ def test_artifact_payload():
|
|||
"application/vnd.tcpdump.pcap")
|
||||
exp2 = stix2.MatchesComparisonExpression("artifact:payload_bin",
|
||||
stix2.StringConstant("\\xd4\\xc3\\xb2\\xa1\\x02\\x00\\x04\\x00"))
|
||||
and_exp = stix2.AndBooleanExpression([exp1, exp2])
|
||||
exp = stix2.ObservationExpression(and_exp)
|
||||
assert str(exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa
|
||||
and_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
|
||||
assert str(and_exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa
|
||||
|
||||
|
||||
def test_greater_than_python_constant():
|
||||
exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy",
|
||||
7.0)
|
||||
exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", 7.0)
|
||||
exp = stix2.ObservationExpression(exp1)
|
||||
assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]"
|
||||
|
||||
|
|
@ -129,14 +128,14 @@ def test_greater_than():
|
|||
|
||||
|
||||
def test_less_than():
|
||||
exp = stix2.LessThanComparisonExpression("file:size",
|
||||
1024)
|
||||
exp = stix2.LessThanComparisonExpression("file:size", 1024)
|
||||
assert str(exp) == "file:size < 1024"
|
||||
|
||||
|
||||
def test_greater_than_or_equal():
|
||||
exp = stix2.GreaterThanEqualComparisonExpression("file:size",
|
||||
1024)
|
||||
|
||||
assert str(exp) == "file:size >= 1024"
|
||||
|
||||
|
||||
|
|
@ -261,7 +260,7 @@ def test_invalid_integer_constant():
|
|||
def test_invalid_timestamp_constant():
|
||||
with pytest.raises(ValueError) as excinfo:
|
||||
stix2.TimestampConstant('foo')
|
||||
assert 'must be a datetime object or timestamp string' in str(excinfo)
|
||||
assert 'Must be a datetime object or timestamp string' in str(excinfo)
|
||||
|
||||
|
||||
def test_invalid_float_constant():
|
||||
|
|
|
|||
Loading…
Reference in New Issue