MISP
/
cti-python-stix2
mirror of https://github.com/MISP/cti-python-stix2
2
0
Fork 0
Code Issues Releases Wiki Activity

Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1

stix2.1
Emmanuelle Vargas-Gonzalez 2018-10-17 07:30:23 -04:00
parent dec75082df d69cc53dd2
commit b2ef77b322
9 changed files with 763 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.travis.yml
View File

@ -6,6 +6,11 @@ python:
- "3.4" - "3.4"
- "3.5" - "3.5"
- "3.6" - "3.6"
matrix:
include:
- python: 3.7 # https://github.com/travis-ci/travis-ci/issues/9069#issuecomment-425720905
dist: xenial
sudo: true
install: install:
- pip install -U pip setuptools - pip install -U pip setuptools
- pip install tox-travis pre-commit - pip install tox-travis pre-commit

2
README.rst
View File

@ -125,8 +125,6 @@ select additional or substitute Maintainers, per `consensus agreements
**Current Maintainers of this TC Open Repository** **Current Maintainers of this TC Open Repository**
- `Greg Back <mailto:gback@mitre.org>`__; GitHub ID:
https://github.com/gtback/; WWW: `MITRE Corporation <http://www.mitre.org/>`__
- `Chris Lenk <mailto:clenk@mitre.org>`__; GitHub ID: - `Chris Lenk <mailto:clenk@mitre.org>`__; GitHub ID:
https://github.com/clenk/; WWW: `MITRE Corporation <http://www.mitre.org/>`__ https://github.com/clenk/; WWW: `MITRE Corporation <http://www.mitre.org/>`__

2
docs/guide/creating.ipynb
View File

@ -881,7 +881,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.6.3" "version": "3.6.5"
} }
}, },
"nbformat": 4, "nbformat": 4,

509
docs/guide/patterns.ipynb Normal file
View File

@ -0,0 +1,509 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# STIX2 Patterns\n",
"\n",
"The Python ``stix2`` library supports STIX 2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX 2 specification. ``stix2`` does not evaluate patterns against STIX 2 content; for that functionality see [cti-pattern-matcher](https://github.com/oasis-open/cti-pattern-matcher).\n",
"\n",
"Patterns in the ``stix2`` library are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.\n",
"\n",
"## API Tips\n",
"\n",
"### ObservationExpression\n",
"\n",
"Within the STIX 2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets(```[]```) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.\n",
"\n",
"This requirement manifests itself within the ``stix2`` library via ```ObservationExpression```. When creating STIX 2 observation expressions, whenever the current expression is complete, wrap it with ```ObservationExpression()```. This allows the complete pattern expression - no matter its complexity - to be rendered as a proper specification-adhering string. __*Note: When pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations*__. While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings. \n",
"\n",
"In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ```ObservationExpression()```.\n",
"\n",
"### ParentheticalExpression\n",
"\n",
"Do not be confused by the ```ParentheticalExpression``` object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ```ObservationExpression```, wrapping completed subcomponent expressions with ```ParentheticalExpression()``` if explicit ordering is required. For usage examples with ```ParentheticalExpression```'s, see [here](#Compound-Observation-Expressions).\n",
"\n",
"### BooleanExpressions vs CompoundObservationExpressions\n",
"\n",
"Be careful to note the difference between these two very similar pattern components. \n",
"\n",
"__BooleanExpressions__\n",
"\n",
" - [AndBooleanExpression](../api/stix2.patterns.rst#stix2.patterns.AndBooleanExpression)\n",
" - [OrbooleanExpression](../api/stix2.patterns.rst#stix2.patterns.OrBooleanExpression)\n",
" \n",
" __Usage__: When the boolean sub-expressions refer to the *same* root object \n",
"\n",
" __Example__:\n",
" ```[domain-name:value = \"www.5z8.info\" AND domain-name:resolvess_to_refs[*].value = \"'198.51.100.1/32'\"]```\n",
" \n",
" __Rendering__: when pattern is rendered, brackets or parenthesis will encapsulate boolean expression\n",
" \n",
"__CompoundObservationExpressions__\n",
"\n",
" - [AndObservationExpression](../api/stix2.patterns.rst#stix2.patterns.AndObservationExpression)\n",
" - [OrObservationExpression](../api/stix2.patterns.rst#stix2.patterns.OrObservationExpression)\n",
" \n",
" __Usage__: When the boolean sub-expressions refer to *different* root objects\n",
"\n",
" __Example__:\n",
" ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n",
" \n",
" __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n",
"\n",
"\n",
"\n",
"## Examples\n",
"\n",
"### Comparison Expressions"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"from stix2 import DomainName, File, IPv4Address\n",
"from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,\n",
" GreaterThanComparisonExpression, IsSubsetComparisonExpression,\n",
" FloatConstant, StringConstant)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Equality Comparison expressions"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\t[domain-name:value = 'site.of.interest.zaz']\n",
"\n",
"\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
"\n"
]
}
],
"source": [
"lhs = ObjectPath(\"domain-name\", [\"value\"])\n",
"ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, \"site.of.interest.zaz\"))\n",
"print(\"\\t{}\\n\".format(ece_1))\n",
"\n",
"lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n",
"ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\"))\n",
"print(\"\\t{}\\n\".format(ece_2))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Greater-than Comparison expressions"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n",
"\n"
]
}
],
"source": [
"lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
"gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\")))\n",
"print(\"\\t{}\\n\".format(gte))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### IsSubset Comparison expressions"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n",
"\n"
]
}
],
"source": [
"lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
"iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\")))\n",
"print(\"\\t{}\\n\".format(iss))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Compound Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
" EqualityComparisonExpression, AndBooleanExpression,\n",
" OrBooleanExpression, ParentheticalExpression,\n",
" AndObservationExpression, OrObservationExpression,\n",
" FollowedByObservationExpression, ObservationExpression)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### AND boolean"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(AND)\n",
"[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
"\n"
]
}
],
"source": [
"ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
"ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
"abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))\n",
"print(\"(AND)\\n{}\\n\".format(abe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### OR boolean"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(OR)\n",
"[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
"\n"
]
}
],
"source": [
"ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
"ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
"obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))\n",
"print(\"(OR)\\n{}\\n\".format(obe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### ( OR ) AND boolean"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(OR,AND)\n",
"[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
"\n"
]
}
],
"source": [
"ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
"ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
"ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
"obe1 = OrBooleanExpression([ece7, ece8])\n",
"pobe = ParentheticalExpression(obe1)\n",
"abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))\n",
"print(\"(OR,AND)\\n{}\\n\".format(abe1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### ( AND ) OR ( OR ) observation"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(AND,OR,OR)\n",
"([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
"\n"
]
}
],
"source": [
"ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
"ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
"ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
"# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
"# expressions as the operands are not on the same object-type\n",
"aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
"obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))\n",
"ooe = OrObservationExpression([aoe, obe2])\n",
"print(\"(AND,OR,OR)\\n{}\\n\".format(ooe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### FOLLOWED-BY"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(FollowedBy)\n",
"[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
"\n"
]
}
],
"source": [
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"print(\"(FollowedBy)\\n{}\\n\".format(fbe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Qualified Observation Expressions"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
" AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
" QualifiedObservationExpression, FollowedByObservationExpression,\n",
" ParentheticalExpression, ObservationExpression)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### WITHIN"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(WITHIN)\n",
"([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS\n",
"\n"
]
}
],
"source": [
"ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
"ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
"fbe = FollowedByObservationExpression([ece10, ece11])\n",
"par = ParentheticalExpression(fbe)\n",
"qoe = QualifiedObservationExpression(par, WithinQualifier(300))\n",
"print(\"(WITHIN)\\n{}\\n\".format(qoe))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### REPEATS, WITHIN"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(REPEAT, WITHIN)\n",
"[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
"\n"
]
}
],
"source": [
"ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
"ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
"abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))\n",
"qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
"print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### START, STOP"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(START-STOP)\n",
"[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
"\n"
]
}
],
"source": [
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
"ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
"qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
"print(\"(START-STOP)\\n{}\\n\".format(qoe2))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Attaching patterns to STIX2 Domain objects\n",
"\n",
"\n",
"### Example"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"type\": \"indicator\",\n",
" \"id\": \"indicator--219bc5fc-fdbf-4b54-a2fc-921be7ab3acb\",\n",
" \"created\": \"2018-08-29T23:58:00.548Z\",\n",
" \"modified\": \"2018-08-29T23:58:00.548Z\",\n",
" \"name\": \"Cryptotorch\",\n",
" \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
" \"valid_from\": \"2018-08-29T23:58:00.548391Z\",\n",
" \"labels\": [\n",
" \"malware\",\n",
" \"ransomware\"\n",
" ]\n",
"}\n"
]
}
],
"source": [
"from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression\n",
"\n",
"ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\"))\n",
"ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=ece14)\n",
"print(ind)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

1
setup.py
View File

@ -43,6 +43,7 @@ setup(
'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
], ],
keywords='stix stix2 json cti cyber threat intelligence', keywords='stix stix2 json cti cyber threat intelligence',
packages=find_packages(exclude=['*.test']), packages=find_packages(exclude=['*.test']),

5
stix2/base.py
View File

@ -334,3 +334,8 @@ class _Extension(_STIXBase):
def _check_object_constraints(self): def _check_object_constraints(self):
super(_Extension, self)._check_object_constraints() super(_Extension, self)._check_object_constraints()
self._check_at_least_one_property() self._check_at_least_one_property()
def _cls_init(cls, obj, kwargs):
if getattr(cls, '__init__', object.__init__) is not object.__init__:
cls.__init__(obj, **kwargs)

246
stix2/patterns.py
View File

@ -18,6 +18,11 @@ class _Constant(object):
class StringConstant(_Constant): class StringConstant(_Constant):
"""Pattern string constant
Args:
value (str): string value
"""
def __init__(self, value): def __init__(self, value):
self.value = value self.value = value
@ -26,17 +31,27 @@ class StringConstant(_Constant):
class TimestampConstant(_Constant): class TimestampConstant(_Constant):
"""Pattern timestamp constant
Args:
value (datetime.datetime OR str): if string, must be a timestamp string
"""
def __init__(self, value): def __init__(self, value):
try: try:
self.value = parse_into_datetime(value) self.value = parse_into_datetime(value)
except Exception: except Exception:
raise ValueError("must be a datetime object or timestamp string.") raise ValueError("Must be a datetime object or timestamp string.")
def __str__(self): def __str__(self):
return "t%s" % repr(self.value) return "t%s" % repr(self.value)
class IntegerConstant(_Constant): class IntegerConstant(_Constant):
"""Pattern interger constant
Args:
value (int): integer value
"""
def __init__(self, value): def __init__(self, value):
try: try:
self.value = int(value) self.value = int(value)
@ -59,6 +74,13 @@ class FloatConstant(_Constant):
class BooleanConstant(_Constant): class BooleanConstant(_Constant):
"""Pattern boolean constant
Args:
value (str OR int):
(str) 'true', 't' for True; 'false', 'f' for False
(int) 1 for True; 0 for False
"""
def __init__(self, value): def __init__(self, value):
if isinstance(value, bool): if isinstance(value, bool):
self.value = value self.value = value
@ -106,6 +128,15 @@ _HASH_REGEX = {
class HashConstant(StringConstant): class HashConstant(StringConstant):
"""Pattern hash constant
Args:
value (str): hash value
type (str): hash algorithm name. Supported hash algorithms:
"MD5", "MD6", RIPEMD160", "SHA1", "SHA224", "SHA256",
"SHA384", "SHA512", "SHA3224", "SHA3256", "SHA3384",
"SHA3512", "SSDEEP", "WHIRLPOOL"
"""
def __init__(self, value, type): def __init__(self, value, type):
key = type.upper().replace('-', '') key = type.upper().replace('-', '')
if key in _HASH_REGEX: if key in _HASH_REGEX:
@ -116,7 +147,11 @@ class HashConstant(StringConstant):
class BinaryConstant(_Constant): class BinaryConstant(_Constant):
"""Pattern binary constant
Args:
value (str): base64 encoded string value
"""
def __init__(self, value): def __init__(self, value):
try: try:
base64.b64decode(value) base64.b64decode(value)
@ -129,6 +164,11 @@ class BinaryConstant(_Constant):
class HexConstant(_Constant): class HexConstant(_Constant):
"""Pattern hexadecimal constant
Args:
value (str): hexadecimal value
"""
def __init__(self, value): def __init__(self, value):
if not re.match('^([a-fA-F0-9]{2})+$', value): if not re.match('^([a-fA-F0-9]{2})+$', value):
raise ValueError("must contain an even number of hexadecimal characters") raise ValueError("must contain an even number of hexadecimal characters")
@ -139,6 +179,11 @@ class HexConstant(_Constant):
class ListConstant(_Constant): class ListConstant(_Constant):
"""Pattern list constant
Args:
value (list): list of values
"""
def __init__(self, values): def __init__(self, values):
self.value = values self.value = values
@ -147,6 +192,12 @@ class ListConstant(_Constant):
def make_constant(value): def make_constant(value):
"""Convert value to Pattern constant, best effort attempt
at determining root value type and corresponding conversion
Args:
value: value to convert to Pattern constant
"""
if isinstance(value, _Constant): if isinstance(value, _Constant):
return value return value
@ -182,6 +233,16 @@ class _ObjectPathComponent(object):
class BasicObjectPathComponent(_ObjectPathComponent): class BasicObjectPathComponent(_ObjectPathComponent):
"""Basic object path component (for an observation or expression)
By "Basic", implies that the object path component is not a
list, object reference or futher referenced property, i.e. terminal
component
Args:
property_name (str): object property name
is_key (bool): is dictionary key, default: False
"""
def __init__(self, property_name, is_key=False): def __init__(self, property_name, is_key=False):
self.property_name = property_name self.property_name = property_name
# TODO: set is_key to True if this component is a dictionary key # TODO: set is_key to True if this component is a dictionary key
@ -192,6 +253,12 @@ class BasicObjectPathComponent(_ObjectPathComponent):
class ListObjectPathComponent(_ObjectPathComponent): class ListObjectPathComponent(_ObjectPathComponent):
"""List object path component (for an observation or expression)
Args:
property_name (str): list object property name
index (int): index of the list property's value that is specified
"""
def __init__(self, property_name, index): def __init__(self, property_name, index):
self.property_name = property_name self.property_name = property_name
self.index = index self.index = index
@ -201,6 +268,11 @@ class ListObjectPathComponent(_ObjectPathComponent):
class ReferenceObjectPathComponent(_ObjectPathComponent): class ReferenceObjectPathComponent(_ObjectPathComponent):
"""Reference object path component (for an observation or expression)
Args:
reference_property_name (str): reference object property name
"""
def __init__(self, reference_property_name): def __init__(self, reference_property_name):
self.property_name = reference_property_name self.property_name = reference_property_name
@ -209,6 +281,12 @@ class ReferenceObjectPathComponent(_ObjectPathComponent):
class ObjectPath(object): class ObjectPath(object):
"""Pattern operand object (property) path
Args:
object_type_name (str): name of object type for corresponding object path component
property_path (_ObjectPathComponent OR str): object path
"""
def __init__(self, object_type_name, property_path): def __init__(self, object_type_name, property_path):
self.object_type_name = object_type_name self.object_type_name = object_type_name
self.property_path = [ self.property_path = [
@ -221,11 +299,17 @@ class ObjectPath(object):
return "%s:%s" % (self.object_type_name, ".".join(["%s" % x for x in self.property_path])) return "%s:%s" % (self.object_type_name, ".".join(["%s" % x for x in self.property_path]))
def merge(self, other): def merge(self, other):
"""Extend the object property with that of the supplied object property path"""
self.property_path.extend(other.property_path) self.property_path.extend(other.property_path)
return self return self
@staticmethod @staticmethod
def make_object_path(lhs): def make_object_path(lhs):
"""Create ObjectPath from string encoded object path
Args:
lhs (str): object path of left-hand-side component of expression
"""
path_as_parts = lhs.split(":") path_as_parts = lhs.split(":")
return ObjectPath(path_as_parts[0], path_as_parts[1].split(".")) return ObjectPath(path_as_parts[0], path_as_parts[1].split("."))
@ -235,6 +319,14 @@ class _PatternExpression(object):
class _ComparisonExpression(_PatternExpression): class _ComparisonExpression(_PatternExpression):
"""Pattern Comparison Expression
Args:
operator (str): operator of comparison expression
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, operator, lhs, rhs, negated=False): def __init__(self, operator, lhs, rhs, negated=False):
if operator == "=" and isinstance(rhs, (ListConstant, list)): if operator == "=" and isinstance(rhs, (ListConstant, list)):
self.operator = "IN" self.operator = "IN"
@ -259,56 +351,134 @@ class _ComparisonExpression(_PatternExpression):
class EqualityComparisonExpression(_ComparisonExpression): class EqualityComparisonExpression(_ComparisonExpression):
"""Pattern Equality Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(EqualityComparisonExpression, self).__init__("=", lhs, rhs, negated) super(EqualityComparisonExpression, self).__init__("=", lhs, rhs, negated)
class GreaterThanComparisonExpression(_ComparisonExpression): class GreaterThanComparisonExpression(_ComparisonExpression):
"""Pattern Greater-than Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(GreaterThanComparisonExpression, self).__init__(">", lhs, rhs, negated) super(GreaterThanComparisonExpression, self).__init__(">", lhs, rhs, negated)
class LessThanComparisonExpression(_ComparisonExpression): class LessThanComparisonExpression(_ComparisonExpression):
"""Pattern Less-than Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(LessThanComparisonExpression, self).__init__("<", lhs, rhs, negated) super(LessThanComparisonExpression, self).__init__("<", lhs, rhs, negated)
class GreaterThanEqualComparisonExpression(_ComparisonExpression): class GreaterThanEqualComparisonExpression(_ComparisonExpression):
"""Pattern Greater-Than-or-Equal-to Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(GreaterThanEqualComparisonExpression, self).__init__(">=", lhs, rhs, negated) super(GreaterThanEqualComparisonExpression, self).__init__(">=", lhs, rhs, negated)
class LessThanEqualComparisonExpression(_ComparisonExpression): class LessThanEqualComparisonExpression(_ComparisonExpression):
"""Pattern Less-Than-or-Equal-to Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(LessThanEqualComparisonExpression, self).__init__("<=", lhs, rhs, negated) super(LessThanEqualComparisonExpression, self).__init__("<=", lhs, rhs, negated)
class InComparisonExpression(_ComparisonExpression): class InComparisonExpression(_ComparisonExpression):
"""'in' Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(InComparisonExpression, self).__init__("IN", lhs, rhs, negated) super(InComparisonExpression, self).__init__("IN", lhs, rhs, negated)
class LikeComparisonExpression(_ComparisonExpression): class LikeComparisonExpression(_ComparisonExpression):
"""'like' Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(LikeComparisonExpression, self).__init__("LIKE", lhs, rhs, negated) super(LikeComparisonExpression, self).__init__("LIKE", lhs, rhs, negated)
class MatchesComparisonExpression(_ComparisonExpression): class MatchesComparisonExpression(_ComparisonExpression):
"""'Matches' Comparison Expression
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False): def __init__(self, lhs, rhs, negated=False):
super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated) super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated)
class IsSubsetComparisonExpression(_ComparisonExpression): class IsSubsetComparisonExpression(_ComparisonExpression):
def __init__(self, lhs, rhs, negated=False): """ 'is subset' Comparison Expression
super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False):
super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
class IsSupersetComparisonExpression(_ComparisonExpression): class IsSupersetComparisonExpression(_ComparisonExpression):
def __init__(self, lhs, rhs, negated=False): """ 'is super set' Comparison Expression
super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
Args:
lhs (ObjectPath OR str): object path of left-hand-side component of expression
rhs (ObjectPath OR str): object path of right-hand-side component of expression
negated (bool): comparison expression negated. Default: False
"""
def __init__(self, lhs, rhs, negated=False):
super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
class _BooleanExpression(_PatternExpression): class _BooleanExpression(_PatternExpression):
"""Boolean Pattern Expression
Args:
operator (str): boolean operator
operands (list): boolean operands
"""
def __init__(self, operator, operands): def __init__(self, operator, operands):
self.operator = operator self.operator = operator
self.operands = [] self.operands = []
@ -324,21 +494,37 @@ class _BooleanExpression(_PatternExpression):
def __str__(self): def __str__(self):
sub_exprs = [] sub_exprs = []
for o in self.operands: for o in self.operands:
sub_exprs.append("%s" % o) sub_exprs.append(str(o))
return (" " + self.operator + " ").join(sub_exprs) return (" " + self.operator + " ").join(sub_exprs)
class AndBooleanExpression(_BooleanExpression): class AndBooleanExpression(_BooleanExpression):
"""'AND' Boolean Pattern Expression. Only use if both operands are of
the same root object.
Args:
operands (list): AND operands
"""
def __init__(self, operands): def __init__(self, operands):
super(AndBooleanExpression, self).__init__("AND", operands) super(AndBooleanExpression, self).__init__("AND", operands)
class OrBooleanExpression(_BooleanExpression): class OrBooleanExpression(_BooleanExpression):
"""'OR' Boolean Pattern Expression. Only use if both operands are of the same root object
Args:
operands (list): OR operands
"""
def __init__(self, operands): def __init__(self, operands):
super(OrBooleanExpression, self).__init__("OR", operands) super(OrBooleanExpression, self).__init__("OR", operands)
class ObservationExpression(_PatternExpression): class ObservationExpression(_PatternExpression):
"""Observation Expression
Args:
operand (str): observation expression operand
"""
def __init__(self, operand): def __init__(self, operand):
self.operand = operand self.operand = operand
@ -347,6 +533,12 @@ class ObservationExpression(_PatternExpression):
class _CompoundObservationExpression(_PatternExpression): class _CompoundObservationExpression(_PatternExpression):
"""Compound Observation Expression
Args:
operator (str): compound observation operator
operands (str): compound observation operands
"""
def __init__(self, operator, operands): def __init__(self, operator, operands):
self.operator = operator self.operator = operator
self.operands = operands self.operands = operands
@ -359,21 +551,41 @@ class _CompoundObservationExpression(_PatternExpression):
class AndObservationExpression(_CompoundObservationExpression): class AndObservationExpression(_CompoundObservationExpression):
"""'AND' Compound Observation Pattern Expression
Args:
operands (str): compound observation operands
"""
def __init__(self, operands): def __init__(self, operands):
super(AndObservationExpression, self).__init__("AND", operands) super(AndObservationExpression, self).__init__("AND", operands)
class OrObservationExpression(_CompoundObservationExpression): class OrObservationExpression(_CompoundObservationExpression):
"""Pattern 'OR' Compound Observation Expression
Args:
operands (str): compound observation operands
"""
def __init__(self, operands): def __init__(self, operands):
super(OrObservationExpression, self).__init__("OR", operands) super(OrObservationExpression, self).__init__("OR", operands)
class FollowedByObservationExpression(_CompoundObservationExpression): class FollowedByObservationExpression(_CompoundObservationExpression):
"""Pattern 'Followed by' Compound Observation Expression
Args:
operands (str): compound observation operands
"""
def __init__(self, operands): def __init__(self, operands):
super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands) super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands)
class ParentheticalExpression(_PatternExpression): class ParentheticalExpression(_PatternExpression):
"""Pattern Parenthetical Observation Expression
Args:
exp (str): observation expression
"""
def __init__(self, exp): def __init__(self, exp):
self.expression = exp self.expression = exp
if hasattr(exp, "root_type"): if hasattr(exp, "root_type"):
@ -388,6 +600,11 @@ class _ExpressionQualifier(_PatternExpression):
class RepeatQualifier(_ExpressionQualifier): class RepeatQualifier(_ExpressionQualifier):
"""Pattern Repeat Qualifier
Args:
times_to_repeat (int): times the qualifiers is repeated
"""
def __init__(self, times_to_repeat): def __init__(self, times_to_repeat):
if isinstance(times_to_repeat, IntegerConstant): if isinstance(times_to_repeat, IntegerConstant):
self.times_to_repeat = times_to_repeat self.times_to_repeat = times_to_repeat
@ -401,6 +618,11 @@ class RepeatQualifier(_ExpressionQualifier):
class WithinQualifier(_ExpressionQualifier): class WithinQualifier(_ExpressionQualifier):
"""Pattern 'Within' Qualifier
Args:
number_of_seconds (int): seconds value for 'within' qualifier
"""
def __init__(self, number_of_seconds): def __init__(self, number_of_seconds):
if isinstance(number_of_seconds, IntegerConstant): if isinstance(number_of_seconds, IntegerConstant):
self.number_of_seconds = number_of_seconds self.number_of_seconds = number_of_seconds
@ -414,6 +636,12 @@ class WithinQualifier(_ExpressionQualifier):
class StartStopQualifier(_ExpressionQualifier): class StartStopQualifier(_ExpressionQualifier):
"""Pattern Start/Stop Qualifier
Args:
start_time (TimestampConstant OR datetime.date): start timestamp for qualifier
stop_time (TimestampConstant OR datetime.date): stop timestamp for qualifier
"""
def __init__(self, start_time, stop_time): def __init__(self, start_time, stop_time):
if isinstance(start_time, TimestampConstant): if isinstance(start_time, TimestampConstant):
self.start_time = start_time self.start_time = start_time
@ -433,6 +661,12 @@ class StartStopQualifier(_ExpressionQualifier):
class QualifiedObservationExpression(_PatternExpression): class QualifiedObservationExpression(_PatternExpression):
"""Pattern Qualified Observation Expression
Args:
observation_expression (PatternExpression OR _CompoundObservationExpression OR ): pattern expression
qualifier (_ExpressionQualifier): pattern expression qualifier
"""
def __init__(self, observation_expression, qualifier): def __init__(self, observation_expression, qualifier):
self.observation_expression = observation_expression self.observation_expression = observation_expression
self.qualifier = qualifier self.qualifier = qualifier

1
stix2/v20/sdo.py
View File

@ -368,5 +368,4 @@ def CustomObject(type='x-custom-type', properties=None):
sorted([x for x in properties if x[0].startswith('x_')], key=lambda x: x[0]), sorted([x for x in properties if x[0].startswith('x_')], key=lambda x: x[0]),
])) ]))
return _custom_object_builder(cls, type, _properties, '2.0') return _custom_object_builder(cls, type, _properties, '2.0')
return wrapper return wrapper

3
tox.ini
View File

@ -1,5 +1,5 @@
[tox] [tox]
envlist = py27,py34,py35,py36,style,isort-check,packaging envlist = py27,py34,py35,py36,py37,style,isort-check,packaging
[testenv] [testenv]
deps = deps =
@ -44,3 +44,4 @@ python =
3.4: py34, style 3.4: py34, style
3.5: py35, style 3.5: py35, style
3.6: py36, style, packaging 3.6: py36, style, packaging
3.7: py37, style