diff --git a/README.rst b/README.rst
index 1e35909..afdf6c3 100644
--- a/README.rst
+++ b/README.rst
@@ -3,32 +3,40 @@
cti-python-stix2
================
-This is an `OASIS TC Open Repository `__.
+This is an `OASIS TC Open
+Repository `__.
See the `Governance <#governance>`__ section for more information.
-This repository provides Python APIs for serializing and de-serializing STIX2
-JSON content, along with higher-level APIs for common tasks, including data
-markings, versioning, and for resolving STIX IDs across multiple data sources.
+This repository provides Python APIs for serializing and de-
+serializing
+STIX 2 JSON content, along with higher-level APIs for common tasks,
+including data markings, versioning, and for resolving STIX IDs across
+multiple data sources.
-For more information, see `the documentation `__ on ReadTheDocs.
+For more information, see `the
+documentation `__ on
+ReadTheDocs.
Installation
------------
Install with `pip `__:
-.. code-block:: bash
+::
- $ pip install stix2
+ pip install stix2
Usage
-----
-To create a STIX object, provide keyword arguments to the type's constructor.
-Certain required attributes of all objects, such as ``type`` or ``id``, will
-be set automatically if not provided as keyword arguments.
+To create a STIX object, provide keyword arguments to the type's
+constructor. Certain required attributes of all objects, such as
+``type`` or
+``id``, will be set automatically if not provided as keyword
+arguments.
-.. code-block:: python
+.. code:: python
from stix2 import Indicator
@@ -36,9 +44,10 @@ be set automatically if not provided as keyword arguments.
indicator_types=["malicious-activity"],
pattern="[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']")
-To parse a STIX JSON string into a Python STIX object, use ``parse()``:
+To parse a STIX JSON string into a Python STIX object, use
+``parse()``:
-.. code-block:: python
+.. code:: python
from stix2 import parse
@@ -58,95 +67,145 @@ To parse a STIX JSON string into a Python STIX object, use ``parse()``:
print(indicator)
-For more in-depth documentation, please see `https://stix2.readthedocs.io/ `__.
+For more in-depth documentation, please see
+`https://stix2.readthedocs.io/ `__.
STIX 2.X Technical Specification Support
----------------------------------------
-This version of python-stix2 supports STIX 2.1 by default. Although, the
-`stix2` Python library is built to support multiple versions of the STIX
-Technical Specification. With every major release of stix2 the ``import stix2``
-statement will automatically load the SDO/SROs equivalent to the most recent
-supported 2.X Technical Specification. Please see the library documentation
+This version of python-stix2 supports STIX 2.0 by default. Although,
+the
+`stix2` Python library is built to support multiple versions of the
+STIX
+Technical Specification. With every major release of stix2 the
+``import stix2``
+statement will automatically load the SDO/SROs equivalent to the most
+recent
+supported 2.X Technical Specification. Please see the library
+documentation
for more details.
Governance
----------
-This GitHub public repository (**https://github.com/oasis-open/cti-python-stix2**) was
-`proposed `__ and
-`approved `__
+This GitHub public repository (
+**https://github.com/oasis-open/cti-python-stix2** ) was
+`proposed `__
+and
+`approved `__
[`bis `__] by the
-`OASIS Cyber Threat Intelligence (CTI) TC `__
-as an `OASIS TC Open Repository `__
-to support development of open source resources related to Technical Committee work.
+`OASIS Cyber Threat Intelligence (CTI)
+TC `__ as an `OASIS TC
+Open
+Repository `__
+to support development of open source resources related to Technical
+Committee work.
-While this TC Open Repository remains associated with the sponsor TC, its
-development priorities, leadership, intellectual property terms, participation
-rules, and other matters of governance are `separate and distinct
-`__
+While this TC Open Repository remains associated with the sponsor TC,
+its
+development priorities, leadership, intellectual property terms,
+participation rules, and other matters of governance are `separate and
+distinct `__
from the OASIS TC Process and related policies.
All contributions made to this TC Open Repository are subject to open
-source license terms expressed in the `BSD-3-Clause License `__.
-That license was selected as the declared `"Applicable License" `__
+source license terms expressed in the `BSD-3-Clause
+License `__.
+That license was selected as the declared `"Applicable
+License" `__
when the TC Open Repository was created.
-As documented in `"Public Participation Invited
-`__",
-contributions to this OASIS TC Open Repository are invited from all parties,
-whether affiliated with OASIS or not. Participants must have a GitHub account,
-but no fees or OASIS membership obligations are required. Participation is
-expected to be consistent with the `OASIS TC Open Repository Guidelines and Procedures
-`__,
-the open source `LICENSE `__
+As documented in `"Public Participation
+Invited `__",
+contributions to this OASIS TC Open Repository are invited from all
+parties, whether affiliated with OASIS or not. Participants must have
+a
+GitHub account, but no fees or OASIS membership obligations are
+required. Participation is expected to be consistent with the `OASIS
+TC Open Repository Guidelines and
+Procedures `__,
+the open source
+`LICENSE `__
designated for this particular repository, and the requirement for an
-`Individual Contributor License Agreement `__
+`Individual Contributor License
+Agreement `__
that governs intellectual property.
Maintainers
~~~~~~~~~~~
-TC Open Repository `Maintainers `__
+TC Open Repository
+`Maintainers `__
are responsible for oversight of this project's community development
-activities, including evaluation of GitHub
-`pull requests `__
-and `preserving `__
-open source principles of openness and fairness. Maintainers are recognized
-and trusted experts who serve to implement community goals and consensus design
-preferences.
+activities, including evaluation of GitHub `pull
+requests `__
+and
+`preserving `__
+open source principles of openness and fairness. Maintainers are
+recognized and trusted experts who serve to implement community goals
+and consensus design preferences.
-Initially, the associated TC members have designated one or more persons to
-serve as Maintainer(s); subsequently, participating community members may
-select additional or substitute Maintainers, per `consensus agreements
-`__.
+Initially, the associated TC members have designated one or more
+persons
+to serve as Maintainer(s); subsequently, participating community
+members
+may select additional or substitute Maintainers, per `consensus
+agreements `__.
-.. _currentmaintainers:
+.. _currentMaintainers:
**Current Maintainers of this TC Open Repository**
-- `Greg Back `__; GitHub ID:
- https://github.com/gtback/; WWW: `MITRE Corporation `__
- `Chris Lenk `__; GitHub ID:
- https://github.com/clenk/; WWW: `MITRE Corporation `__
+ https://github.com/clenk/; WWW: `MITRE
+ Corporation `__
About OASIS TC Open Repositories
--------------------------------
-- `TC Open Repositories: Overview and Resources `__
-- `Frequently Asked Questions `__
-- `Open Source Licenses `__
-- `Contributor License Agreements (CLAs) `__
-- `Maintainers' Guidelines and Agreement `__
+- `TC Open Repositories: Overview and
+ Resources `__
+- `Frequently Asked
+ Questions `__
+- `Open Source
+ Licenses `__
+- `Contributor License Agreements
+ (CLAs) `__
+- `Maintainers' Guidelines and
+ Agreement `__
Feedback
--------
-Questions or comments about this TC Open Repository's activities should be
-composed as GitHub issues or comments. If use of an issue/comment is not
+Questions or comments about this TC Open Repository's activities
+should be
+composed as GitHub issues or comments. If use of an issue/comment is
+not
possible or appropriate, questions may be directed by email to the
-Maintainer(s) `listed above <#currentmaintainers>`__. Please send general
-questions about TC Open Repository participation to OASIS Staff at
+Maintainer(s) `listed above <#currentmaintainers>`__. Please send
+general questions about TC Open Repository participation to OASIS
+Staff at
repository-admin@oasis-open.org and any specific CLA-related questions
to repository-cla@oasis-open.org.
diff --git a/docs/guide/creating.ipynb b/docs/guide/creating.ipynb
index 61bbe15..058aae3 100644
--- a/docs/guide/creating.ipynb
+++ b/docs/guide/creating.ipynb
@@ -881,7 +881,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.6.3"
+ "version": "3.6.5"
}
},
"nbformat": 4,
diff --git a/docs/guide/patterns.ipynb b/docs/guide/patterns.ipynb
new file mode 100644
index 0000000..ee06675
--- /dev/null
+++ b/docs/guide/patterns.ipynb
@@ -0,0 +1,509 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# STIX2 Patterns\n",
+ "\n",
+ "The Python ``stix2`` library supports STIX 2 patterning insofar that patterns may be used for the pattern property of Indicators, identical to the STIX 2 specification. ``stix2`` does not evaluate patterns against STIX 2 content; for that functionality see [cti-pattern-matcher](https://github.com/oasis-open/cti-pattern-matcher).\n",
+ "\n",
+ "Patterns in the ``stix2`` library are built compositely from the bottom up, creating subcomponent expressions first before those at higher levels.\n",
+ "\n",
+ "## API Tips\n",
+ "\n",
+ "### ObservationExpression\n",
+ "\n",
+ "Within the STIX 2 Patterning specification, Observation Expressions denote a complete expression to be evaluated against a discrete observation. In other words, an Observation Expression must be created to apply to a single Observation instance. This is further made clear by the visual brackets(```[]```) that encapsulate an Observation Expression. Thus, whatever sub expressions that are within the Observation Expression are meant to be matched against the same Observable instance.\n",
+ "\n",
+ "This requirement manifests itself within the ``stix2`` library via ```ObservationExpression```. When creating STIX 2 observation expressions, whenever the current expression is complete, wrap it with ```ObservationExpression()```. This allows the complete pattern expression - no matter its complexity - to be rendered as a proper specification-adhering string. __*Note: When pattern expressions are added to Indicator objects, the expression objects are implicitly converted to string representations*__. While the extra step may seem tedious in the construction of simple pattern expressions, this explicit marking of observation expressions becomes vital when converting the pattern expressions to strings. \n",
+ "\n",
+ "In all the examples, you can observe how in the process of building pattern expressions, when an Observation Expression is completed, it is wrapped with ```ObservationExpression()```.\n",
+ "\n",
+ "### ParentheticalExpression\n",
+ "\n",
+ "Do not be confused by the ```ParentheticalExpression``` object. It is not a distinct expression type but is also used to properly craft pattern expressions by denoting order priority and grouping of expression components. Use it in a similar manner as ```ObservationExpression```, wrapping completed subcomponent expressions with ```ParentheticalExpression()``` if explicit ordering is required. For usage examples with ```ParentheticalExpression```'s, see [here](#Compound-Observation-Expressions).\n",
+ "\n",
+ "### BooleanExpressions vs CompoundObservationExpressions\n",
+ "\n",
+ "Be careful to note the difference between these two very similar pattern components. \n",
+ "\n",
+ "__BooleanExpressions__\n",
+ "\n",
+ " - [AndBooleanExpression](../api/stix2.patterns.rst#stix2.patterns.AndBooleanExpression)\n",
+ " - [OrbooleanExpression](../api/stix2.patterns.rst#stix2.patterns.OrBooleanExpression)\n",
+ " \n",
+ " __Usage__: When the boolean sub-expressions refer to the *same* root object \n",
+ "\n",
+ " __Example__:\n",
+ " ```[domain-name:value = \"www.5z8.info\" AND domain-name:resolvess_to_refs[*].value = \"'198.51.100.1/32'\"]```\n",
+ " \n",
+ " __Rendering__: when pattern is rendered, brackets or parenthesis will encapsulate boolean expression\n",
+ " \n",
+ "__CompoundObservationExpressions__\n",
+ "\n",
+ " - [AndObservationExpression](../api/stix2.patterns.rst#stix2.patterns.AndObservationExpression)\n",
+ " - [OrObservationExpression](../api/stix2.patterns.rst#stix2.patterns.OrObservationExpression)\n",
+ " \n",
+ " __Usage__: When the boolean sub-expressions refer to *different* root objects\n",
+ "\n",
+ " __Example__:\n",
+ " ```[file:name=\"foo.dll\"] AND [process:name = \"procfoo\"]```\n",
+ " \n",
+ " __Rendering__: when pattern is rendered, brackets will encapsulate each boolean sub-expression\n",
+ "\n",
+ "\n",
+ "\n",
+ "## Examples\n",
+ "\n",
+ "### Comparison Expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from stix2 import DomainName, File, IPv4Address\n",
+ "from stix2 import (ObjectPath, EqualityComparisonExpression, ObservationExpression,\n",
+ " GreaterThanComparisonExpression, IsSubsetComparisonExpression,\n",
+ " FloatConstant, StringConstant)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Equality Comparison expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\t[domain-name:value = 'site.of.interest.zaz']\n",
+ "\n",
+ "\t[file:parent_directory_ref.path = 'C:\\\\Windows\\\\System32']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "lhs = ObjectPath(\"domain-name\", [\"value\"])\n",
+ "ece_1 = ObservationExpression(EqualityComparisonExpression(lhs, \"site.of.interest.zaz\"))\n",
+ "print(\"\\t{}\\n\".format(ece_1))\n",
+ "\n",
+ "lhs = ObjectPath(\"file\", [\"parent_directory_ref\",\"path\"])\n",
+ "ece_2 = ObservationExpression(EqualityComparisonExpression(lhs, \"C:\\\\Windows\\\\System32\"))\n",
+ "print(\"\\t{}\\n\".format(ece_2))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### Greater-than Comparison expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\t[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "lhs = ObjectPath(\"file\", [\"extensions\", \"windows-pebinary-ext\", \"sections[*]\", \"entropy\"])\n",
+ "gte = ObservationExpression(GreaterThanComparisonExpression(lhs, FloatConstant(\"7.0\")))\n",
+ "print(\"\\t{}\\n\".format(gte))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### IsSubset Comparison expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "\t[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "lhs = ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"])\n",
+ "iss = ObservationExpression(IsSubsetComparisonExpression(lhs, StringConstant(\"2001:0db8:dead:beef:0000:0000:0000:0000/64\")))\n",
+ "print(\"\\t{}\\n\".format(iss))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Compound Observation Expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from stix2 import (IntegerConstant, HashConstant, ObjectPath,\n",
+ " EqualityComparisonExpression, AndBooleanExpression,\n",
+ " OrBooleanExpression, ParentheticalExpression,\n",
+ " AndObservationExpression, OrObservationExpression,\n",
+ " FollowedByObservationExpression, ObservationExpression)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### AND boolean"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(AND)\n",
+ "[email-message:sender_ref.value = 'stark@example.com' AND email-message:subject = 'Conference Info']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece3 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"sender_ref\", \"value\"]), \"stark@example.com\")\n",
+ "ece4 = EqualityComparisonExpression(ObjectPath(\"email-message\", [\"subject\"]), \"Conference Info\")\n",
+ "abe = ObservationExpression(AndBooleanExpression([ece3, ece4]))\n",
+ "print(\"(AND)\\n{}\\n\".format(abe))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### OR boolean"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(OR)\n",
+ "[url:value = 'http://example.com/foo' OR url:value = 'http://example.com/bar']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece5 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/foo\")\n",
+ "ece6 = EqualityComparisonExpression(ObjectPath(\"url\", [\"value\"]), \"http://example.com/bar\")\n",
+ "obe = ObservationExpression(OrBooleanExpression([ece5, ece6]))\n",
+ "print(\"(OR)\\n{}\\n\".format(obe))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### ( OR ) AND boolean"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(OR,AND)\n",
+ "[(file:name = 'pdf.exe' OR file:size = 371712) AND file:created = 2014-01-13 07:03:17+00:00]\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece7 = EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"pdf.exe\")\n",
+ "ece8 = EqualityComparisonExpression(ObjectPath(\"file\", [\"size\"]), IntegerConstant(\"371712\"))\n",
+ "ece9 = EqualityComparisonExpression(ObjectPath(\"file\", [\"created\"]), \"2014-01-13T07:03:17Z\")\n",
+ "obe1 = OrBooleanExpression([ece7, ece8])\n",
+ "pobe = ParentheticalExpression(obe1)\n",
+ "abe1 = ObservationExpression(AndBooleanExpression([pobe, ece9]))\n",
+ "print(\"(OR,AND)\\n{}\\n\".format(abe1))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### ( AND ) OR ( OR ) observation"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(AND,OR,OR)\n",
+ "([file:name = 'foo.dll'] AND [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) OR [process:name = 'fooproc' OR process:name = 'procfoo']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece20 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
+ "ece21 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
+ "ece22 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"fooproc\")\n",
+ "ece23 = EqualityComparisonExpression(ObjectPath(\"process\", [\"name\"]), \"procfoo\")\n",
+ "# NOTE: we need to use AND/OR observation expression instead of just boolean \n",
+ "# expressions as the operands are not on the same object-type\n",
+ "aoe = ParentheticalExpression(AndObservationExpression([ece20, ece21]))\n",
+ "obe2 = ObservationExpression(OrBooleanExpression([ece22, ece23]))\n",
+ "ooe = OrObservationExpression([aoe, obe2])\n",
+ "print(\"(AND,OR,OR)\\n{}\\n\".format(ooe))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### FOLLOWED-BY"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(FollowedBy)\n",
+ "[file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
+ "ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
+ "fbe = FollowedByObservationExpression([ece10, ece11])\n",
+ "print(\"(FollowedBy)\\n{}\\n\".format(fbe))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Qualified Observation Expressions"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from stix2 import (TimestampConstant, HashConstant, ObjectPath, EqualityComparisonExpression,\n",
+ " AndBooleanExpression, WithinQualifier, RepeatQualifier, StartStopQualifier,\n",
+ " QualifiedObservationExpression, FollowedByObservationExpression,\n",
+ " ParentheticalExpression, ObservationExpression)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### WITHIN"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(WITHIN)\n",
+ "([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [win-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece10 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"hashes\", \"MD5\"]), HashConstant(\"79054025255fb1a26e4bc422aef54eb4\", \"MD5\")))\n",
+ "ece11 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"win-registry-key\", [\"key\"]), \"HKEY_LOCAL_MACHINE\\\\foo\\\\bar\"))\n",
+ "fbe = FollowedByObservationExpression([ece10, ece11])\n",
+ "par = ParentheticalExpression(fbe)\n",
+ "qoe = QualifiedObservationExpression(par, WithinQualifier(300))\n",
+ "print(\"(WITHIN)\\n{}\\n\".format(qoe))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### REPEATS, WITHIN"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(REPEAT, WITHIN)\n",
+ "[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 180 SECONDS\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece12 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"type\"]), \"domain-name\")\n",
+ "ece13 = EqualityComparisonExpression(ObjectPath(\"network-traffic\", [\"dst_ref\", \"value\"]), \"example.com\")\n",
+ "abe2 = ObservationExpression(AndBooleanExpression([ece12, ece13]))\n",
+ "qoe1 = QualifiedObservationExpression(QualifiedObservationExpression(abe2, RepeatQualifier(5)), WithinQualifier(180))\n",
+ "print(\"(REPEAT, WITHIN)\\n{}\\n\".format(qoe1))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "#### START, STOP"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "(START-STOP)\n",
+ "[file:name = 'foo.dll'] START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"foo.dll\"))\n",
+ "ssq = StartStopQualifier(TimestampConstant('2016-06-01T00:00:00Z'), TimestampConstant('2016-07-01T00:00:00Z'))\n",
+ "qoe2 = QualifiedObservationExpression(ece14, ssq)\n",
+ "print(\"(START-STOP)\\n{}\\n\".format(qoe2))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Attaching patterns to STIX2 Domain objects\n",
+ "\n",
+ "\n",
+ "### Example"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "{\n",
+ " \"type\": \"indicator\",\n",
+ " \"id\": \"indicator--219bc5fc-fdbf-4b54-a2fc-921be7ab3acb\",\n",
+ " \"created\": \"2018-08-29T23:58:00.548Z\",\n",
+ " \"modified\": \"2018-08-29T23:58:00.548Z\",\n",
+ " \"name\": \"Cryptotorch\",\n",
+ " \"pattern\": \"[file:name = '$$t00rzch$$.elf']\",\n",
+ " \"valid_from\": \"2018-08-29T23:58:00.548391Z\",\n",
+ " \"labels\": [\n",
+ " \"malware\",\n",
+ " \"ransomware\"\n",
+ " ]\n",
+ "}\n"
+ ]
+ }
+ ],
+ "source": [
+ "from stix2 import Indicator, EqualityComparisonExpression, ObservationExpression\n",
+ "\n",
+ "ece14 = ObservationExpression(EqualityComparisonExpression(ObjectPath(\"file\", [\"name\"]), \"$$t00rzch$$.elf\"))\n",
+ "ind = Indicator(name=\"Cryptotorch\", labels=[\"malware\", \"ransomware\"], pattern=ece14)\n",
+ "print(ind)"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/stix2/patterns.py b/stix2/patterns.py
index 9668c2a..8bcaea3 100644
--- a/stix2/patterns.py
+++ b/stix2/patterns.py
@@ -18,6 +18,11 @@ class _Constant(object):
class StringConstant(_Constant):
+ """Pattern string constant
+
+ Args:
+ value (str): string value
+ """
def __init__(self, value):
self.value = value
@@ -26,17 +31,27 @@ class StringConstant(_Constant):
class TimestampConstant(_Constant):
+ """Pattern timestamp constant
+
+ Args:
+ value (datetime.datetime OR str): if string, must be a timestamp string
+ """
def __init__(self, value):
try:
self.value = parse_into_datetime(value)
except Exception:
- raise ValueError("must be a datetime object or timestamp string.")
+ raise ValueError("Must be a datetime object or timestamp string.")
def __str__(self):
return "t%s" % repr(self.value)
class IntegerConstant(_Constant):
+ """Pattern interger constant
+
+ Args:
+ value (int): integer value
+ """
def __init__(self, value):
try:
self.value = int(value)
@@ -59,6 +74,13 @@ class FloatConstant(_Constant):
class BooleanConstant(_Constant):
+ """Pattern boolean constant
+
+ Args:
+ value (str OR int):
+ (str) 'true', 't' for True; 'false', 'f' for False
+ (int) 1 for True; 0 for False
+ """
def __init__(self, value):
if isinstance(value, bool):
self.value = value
@@ -106,6 +128,15 @@ _HASH_REGEX = {
class HashConstant(StringConstant):
+ """Pattern hash constant
+
+ Args:
+ value (str): hash value
+ type (str): hash algorithm name. Supported hash algorithms:
+ "MD5", "MD6", RIPEMD160", "SHA1", "SHA224", "SHA256",
+ "SHA384", "SHA512", "SHA3224", "SHA3256", "SHA3384",
+ "SHA3512", "SSDEEP", "WHIRLPOOL"
+ """
def __init__(self, value, type):
key = type.upper().replace('-', '')
if key in _HASH_REGEX:
@@ -116,7 +147,11 @@ class HashConstant(StringConstant):
class BinaryConstant(_Constant):
+ """Pattern binary constant
+ Args:
+ value (str): base64 encoded string value
+ """
def __init__(self, value):
try:
base64.b64decode(value)
@@ -129,6 +164,11 @@ class BinaryConstant(_Constant):
class HexConstant(_Constant):
+ """Pattern hexadecimal constant
+
+ Args:
+ value (str): hexadecimal value
+ """
def __init__(self, value):
if not re.match('^([a-fA-F0-9]{2})+$', value):
raise ValueError("must contain an even number of hexadecimal characters")
@@ -139,6 +179,11 @@ class HexConstant(_Constant):
class ListConstant(_Constant):
+ """Pattern list constant
+
+ Args:
+ value (list): list of values
+ """
def __init__(self, values):
self.value = values
@@ -147,6 +192,12 @@ class ListConstant(_Constant):
def make_constant(value):
+ """Convert value to Pattern constant, best effort attempt
+ at determining root value type and corresponding conversion
+
+ Args:
+ value: value to convert to Pattern constant
+ """
if isinstance(value, _Constant):
return value
@@ -182,6 +233,16 @@ class _ObjectPathComponent(object):
class BasicObjectPathComponent(_ObjectPathComponent):
+ """Basic object path component (for an observation or expression)
+
+ By "Basic", implies that the object path component is not a
+ list, object reference or futher referenced property, i.e. terminal
+ component
+
+ Args:
+ property_name (str): object property name
+ is_key (bool): is dictionary key, default: False
+ """
def __init__(self, property_name, is_key=False):
self.property_name = property_name
# TODO: set is_key to True if this component is a dictionary key
@@ -192,6 +253,12 @@ class BasicObjectPathComponent(_ObjectPathComponent):
class ListObjectPathComponent(_ObjectPathComponent):
+ """List object path component (for an observation or expression)
+
+ Args:
+ property_name (str): list object property name
+ index (int): index of the list property's value that is specified
+ """
def __init__(self, property_name, index):
self.property_name = property_name
self.index = index
@@ -201,6 +268,11 @@ class ListObjectPathComponent(_ObjectPathComponent):
class ReferenceObjectPathComponent(_ObjectPathComponent):
+ """Reference object path component (for an observation or expression)
+
+ Args:
+ reference_property_name (str): reference object property name
+ """
def __init__(self, reference_property_name):
self.property_name = reference_property_name
@@ -209,6 +281,12 @@ class ReferenceObjectPathComponent(_ObjectPathComponent):
class ObjectPath(object):
+ """Pattern operand object (property) path
+
+ Args:
+ object_type_name (str): name of object type for corresponding object path component
+ property_path (_ObjectPathComponent OR str): object path
+ """
def __init__(self, object_type_name, property_path):
self.object_type_name = object_type_name
self.property_path = [
@@ -221,11 +299,17 @@ class ObjectPath(object):
return "%s:%s" % (self.object_type_name, ".".join(["%s" % x for x in self.property_path]))
def merge(self, other):
+ """Extend the object property with that of the supplied object property path"""
self.property_path.extend(other.property_path)
return self
@staticmethod
def make_object_path(lhs):
+ """Create ObjectPath from string encoded object path
+
+ Args:
+ lhs (str): object path of left-hand-side component of expression
+ """
path_as_parts = lhs.split(":")
return ObjectPath(path_as_parts[0], path_as_parts[1].split("."))
@@ -235,6 +319,14 @@ class _PatternExpression(object):
class _ComparisonExpression(_PatternExpression):
+ """Pattern Comparison Expression
+
+ Args:
+ operator (str): operator of comparison expression
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, operator, lhs, rhs, negated=False):
if operator == "=" and isinstance(rhs, (ListConstant, list)):
self.operator = "IN"
@@ -259,56 +351,134 @@ class _ComparisonExpression(_PatternExpression):
class EqualityComparisonExpression(_ComparisonExpression):
+ """Pattern Equality Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(EqualityComparisonExpression, self).__init__("=", lhs, rhs, negated)
class GreaterThanComparisonExpression(_ComparisonExpression):
+ """Pattern Greater-than Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(GreaterThanComparisonExpression, self).__init__(">", lhs, rhs, negated)
class LessThanComparisonExpression(_ComparisonExpression):
+ """Pattern Less-than Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(LessThanComparisonExpression, self).__init__("<", lhs, rhs, negated)
class GreaterThanEqualComparisonExpression(_ComparisonExpression):
+ """Pattern Greater-Than-or-Equal-to Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(GreaterThanEqualComparisonExpression, self).__init__(">=", lhs, rhs, negated)
class LessThanEqualComparisonExpression(_ComparisonExpression):
+ """Pattern Less-Than-or-Equal-to Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
+
def __init__(self, lhs, rhs, negated=False):
super(LessThanEqualComparisonExpression, self).__init__("<=", lhs, rhs, negated)
class InComparisonExpression(_ComparisonExpression):
+ """'in' Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(InComparisonExpression, self).__init__("IN", lhs, rhs, negated)
class LikeComparisonExpression(_ComparisonExpression):
+ """'like' Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
+
def __init__(self, lhs, rhs, negated=False):
super(LikeComparisonExpression, self).__init__("LIKE", lhs, rhs, negated)
class MatchesComparisonExpression(_ComparisonExpression):
+ """'Matches' Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
def __init__(self, lhs, rhs, negated=False):
super(MatchesComparisonExpression, self).__init__("MATCHES", lhs, rhs, negated)
class IsSubsetComparisonExpression(_ComparisonExpression):
- def __init__(self, lhs, rhs, negated=False):
- super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
+ """ 'is subset' Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
+ def __init__(self, lhs, rhs, negated=False):
+ super(IsSubsetComparisonExpression, self).__init__("ISSUBSET", lhs, rhs, negated)
class IsSupersetComparisonExpression(_ComparisonExpression):
- def __init__(self, lhs, rhs, negated=False):
- super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
+ """ 'is super set' Comparison Expression
+
+ Args:
+ lhs (ObjectPath OR str): object path of left-hand-side component of expression
+ rhs (ObjectPath OR str): object path of right-hand-side component of expression
+ negated (bool): comparison expression negated. Default: False
+ """
+ def __init__(self, lhs, rhs, negated=False):
+ super(IsSupersetComparisonExpression, self).__init__("ISSUPERSET", lhs, rhs, negated)
class _BooleanExpression(_PatternExpression):
+ """Boolean Pattern Expression
+
+ Args:
+ operator (str): boolean operator
+ operands (list): boolean operands
+ """
def __init__(self, operator, operands):
self.operator = operator
self.operands = []
@@ -324,21 +494,37 @@ class _BooleanExpression(_PatternExpression):
def __str__(self):
sub_exprs = []
for o in self.operands:
- sub_exprs.append("%s" % o)
+ sub_exprs.append(str(o))
return (" " + self.operator + " ").join(sub_exprs)
class AndBooleanExpression(_BooleanExpression):
+ """'AND' Boolean Pattern Expression. Only use if both operands are of
+ the same root object.
+
+ Args:
+ operands (list): AND operands
+ """
def __init__(self, operands):
super(AndBooleanExpression, self).__init__("AND", operands)
class OrBooleanExpression(_BooleanExpression):
+ """'OR' Boolean Pattern Expression. Only use if both operands are of the same root object
+
+ Args:
+ operands (list): OR operands
+ """
def __init__(self, operands):
super(OrBooleanExpression, self).__init__("OR", operands)
class ObservationExpression(_PatternExpression):
+ """Observation Expression
+
+ Args:
+ operand (str): observation expression operand
+ """
def __init__(self, operand):
self.operand = operand
@@ -347,6 +533,12 @@ class ObservationExpression(_PatternExpression):
class _CompoundObservationExpression(_PatternExpression):
+ """Compound Observation Expression
+
+ Args:
+ operator (str): compound observation operator
+ operands (str): compound observation operands
+ """
def __init__(self, operator, operands):
self.operator = operator
self.operands = operands
@@ -359,21 +551,41 @@ class _CompoundObservationExpression(_PatternExpression):
class AndObservationExpression(_CompoundObservationExpression):
+ """'AND' Compound Observation Pattern Expression
+
+ Args:
+ operands (str): compound observation operands
+ """
def __init__(self, operands):
super(AndObservationExpression, self).__init__("AND", operands)
class OrObservationExpression(_CompoundObservationExpression):
+ """Pattern 'OR' Compound Observation Expression
+
+ Args:
+ operands (str): compound observation operands
+ """
def __init__(self, operands):
super(OrObservationExpression, self).__init__("OR", operands)
class FollowedByObservationExpression(_CompoundObservationExpression):
+ """Pattern 'Followed by' Compound Observation Expression
+
+ Args:
+ operands (str): compound observation operands
+ """
def __init__(self, operands):
super(FollowedByObservationExpression, self).__init__("FOLLOWEDBY", operands)
class ParentheticalExpression(_PatternExpression):
+ """Pattern Parenthetical Observation Expression
+
+ Args:
+ exp (str): observation expression
+ """
def __init__(self, exp):
self.expression = exp
if hasattr(exp, "root_type"):
@@ -388,6 +600,11 @@ class _ExpressionQualifier(_PatternExpression):
class RepeatQualifier(_ExpressionQualifier):
+ """Pattern Repeat Qualifier
+
+ Args:
+ times_to_repeat (int): times the qualifiers is repeated
+ """
def __init__(self, times_to_repeat):
if isinstance(times_to_repeat, IntegerConstant):
self.times_to_repeat = times_to_repeat
@@ -401,6 +618,11 @@ class RepeatQualifier(_ExpressionQualifier):
class WithinQualifier(_ExpressionQualifier):
+ """Pattern 'Within' Qualifier
+
+ Args:
+ number_of_seconds (int): seconds value for 'within' qualifier
+ """
def __init__(self, number_of_seconds):
if isinstance(number_of_seconds, IntegerConstant):
self.number_of_seconds = number_of_seconds
@@ -414,6 +636,12 @@ class WithinQualifier(_ExpressionQualifier):
class StartStopQualifier(_ExpressionQualifier):
+ """Pattern Start/Stop Qualifier
+
+ Args:
+ start_time (TimestampConstant OR datetime.date): start timestamp for qualifier
+ stop_time (TimestampConstant OR datetime.date): stop timestamp for qualifier
+ """
def __init__(self, start_time, stop_time):
if isinstance(start_time, TimestampConstant):
self.start_time = start_time
@@ -433,6 +661,12 @@ class StartStopQualifier(_ExpressionQualifier):
class QualifiedObservationExpression(_PatternExpression):
+ """Pattern Qualified Observation Expression
+
+ Args:
+ observation_expression (PatternExpression OR _CompoundObservationExpression OR ): pattern expression
+ qualifier (_ExpressionQualifier): pattern expression qualifier
+ """
def __init__(self, observation_expression, qualifier):
self.observation_expression = observation_expression
self.qualifier = qualifier
diff --git a/stix2/test/test_pattern_expressions.py b/stix2/test/test_pattern_expressions.py
new file mode 100644
index 0000000..a4d0a5c
--- /dev/null
+++ b/stix2/test/test_pattern_expressions.py
@@ -0,0 +1,379 @@
+import datetime
+
+import pytest
+
+import stix2
+
+
+def test_create_comparison_expression():
+
+ exp = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
+ stix2.HashConstant("aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f", "SHA-256")) # noqa
+
+ assert str(exp) == "file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f'"
+
+
+def test_boolean_expression():
+ exp1 = stix2.MatchesComparisonExpression("email-message:from_ref.value",
+ stix2.StringConstant(".+\\@example\\.com$"))
+ exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name",
+ stix2.StringConstant("^Final Report.+\\.exe$"))
+ exp = stix2.AndBooleanExpression([exp1, exp2])
+
+ assert str(exp) == "email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$'" # noqa
+
+
+def test_boolean_expression_with_parentheses():
+ exp1 = stix2.MatchesComparisonExpression(stix2.ObjectPath("email-message",
+ [stix2.ReferenceObjectPathComponent("from_ref"),
+ stix2.BasicObjectPathComponent("value")]),
+ stix2.StringConstant(".+\\@example\\.com$"))
+ exp2 = stix2.MatchesComparisonExpression("email-message:body_multipart[*].body_raw_ref.name",
+ stix2.StringConstant("^Final Report.+\\.exe$"))
+ exp = stix2.ParentheticalExpression(stix2.AndBooleanExpression([exp1, exp2]))
+ assert str(exp) == "(email-message:from_ref.value MATCHES '.+\\\\@example\\\\.com$' AND email-message:body_multipart[*].body_raw_ref.name MATCHES '^Final Report.+\\\\.exe$')" # noqa
+
+
+def test_hash_followed_by_registryKey_expression_python_constant():
+ hash_exp = stix2.EqualityComparisonExpression("file:hashes.MD5",
+ stix2.HashConstant("79054025255fb1a26e4bc422aef54eb4", "MD5"))
+ o_exp1 = stix2.ObservationExpression(hash_exp)
+ reg_exp = stix2.EqualityComparisonExpression(stix2.ObjectPath("windows-registry-key", ["key"]),
+ stix2.StringConstant("HKEY_LOCAL_MACHINE\\foo\\bar"))
+ o_exp2 = stix2.ObservationExpression(reg_exp)
+ fb_exp = stix2.FollowedByObservationExpression([o_exp1, o_exp2])
+ para_exp = stix2.ParentheticalExpression(fb_exp)
+ qual_exp = stix2.WithinQualifier(300)
+ exp = stix2.QualifiedObservationExpression(para_exp, qual_exp)
+ assert str(exp) == "([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS" # noqa
+
+
+def test_hash_followed_by_registryKey_expression():
+ hash_exp = stix2.EqualityComparisonExpression("file:hashes.MD5",
+ stix2.HashConstant("79054025255fb1a26e4bc422aef54eb4", "MD5"))
+ o_exp1 = stix2.ObservationExpression(hash_exp)
+ reg_exp = stix2.EqualityComparisonExpression(stix2.ObjectPath("windows-registry-key", ["key"]),
+ stix2.StringConstant("HKEY_LOCAL_MACHINE\\foo\\bar"))
+ o_exp2 = stix2.ObservationExpression(reg_exp)
+ fb_exp = stix2.FollowedByObservationExpression([o_exp1, o_exp2])
+ para_exp = stix2.ParentheticalExpression(fb_exp)
+ qual_exp = stix2.WithinQualifier(stix2.IntegerConstant(300))
+ exp = stix2.QualifiedObservationExpression(para_exp, qual_exp)
+ assert str(exp) == "([file:hashes.MD5 = '79054025255fb1a26e4bc422aef54eb4'] FOLLOWEDBY [windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\\\foo\\\\bar']) WITHIN 300 SECONDS" # noqa
+
+
+def test_file_observable_expression():
+ exp1 = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
+ stix2.HashConstant(
+ "aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f",
+ 'SHA-256'))
+ exp2 = stix2.EqualityComparisonExpression("file:mime_type", stix2.StringConstant("application/x-pdf"))
+ bool_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
+ assert str(bool_exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
+
+
+@pytest.mark.parametrize("observation_class, op", [
+ (stix2.AndObservationExpression, 'AND'),
+ (stix2.OrObservationExpression, 'OR'),
+])
+def test_multiple_file_observable_expression(observation_class, op):
+ exp1 = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
+ stix2.HashConstant(
+ "bf07a7fbb825fc0aae7bf4a1177b2b31fcf8a3feeaf7092761e18c859ee52a9c",
+ 'SHA-256'))
+ exp2 = stix2.EqualityComparisonExpression("file:hashes.MD5",
+ stix2.HashConstant("cead3f77f6cda6ec00f57d76c9a6879f", "MD5"))
+ bool1_exp = stix2.OrBooleanExpression([exp1, exp2])
+ exp3 = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
+ stix2.HashConstant(
+ "aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f",
+ 'SHA-256'))
+ op1_exp = stix2.ObservationExpression(bool1_exp)
+ op2_exp = stix2.ObservationExpression(exp3)
+ exp = observation_class([op1_exp, op2_exp])
+ assert str(exp) == "[file:hashes.'SHA-256' = 'bf07a7fbb825fc0aae7bf4a1177b2b31fcf8a3feeaf7092761e18c859ee52a9c' OR file:hashes.MD5 = 'cead3f77f6cda6ec00f57d76c9a6879f'] {} [file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f']".format(op) # noqa
+
+
+def test_root_types():
+ ast = stix2.ObservationExpression(
+ stix2.AndBooleanExpression(
+ [stix2.ParentheticalExpression(
+ stix2.OrBooleanExpression([
+ stix2.EqualityComparisonExpression("a:b", stix2.StringConstant("1")),
+ stix2.EqualityComparisonExpression("b:c", stix2.StringConstant("2"))])),
+ stix2.EqualityComparisonExpression(u"b:d", stix2.StringConstant("3"))]))
+ assert str(ast) == "[(a:b = '1' OR b:c = '2') AND b:d = '3']"
+
+
+def test_artifact_payload():
+ exp1 = stix2.EqualityComparisonExpression("artifact:mime_type",
+ "application/vnd.tcpdump.pcap")
+ exp2 = stix2.MatchesComparisonExpression("artifact:payload_bin",
+ stix2.StringConstant("\\xd4\\xc3\\xb2\\xa1\\x02\\x00\\x04\\x00"))
+ and_exp = stix2.ObservationExpression(stix2.AndBooleanExpression([exp1, exp2]))
+ assert str(and_exp) == "[artifact:mime_type = 'application/vnd.tcpdump.pcap' AND artifact:payload_bin MATCHES '\\\\xd4\\\\xc3\\\\xb2\\\\xa1\\\\x02\\\\x00\\\\x04\\\\x00']" # noqa
+
+
+def test_greater_than_python_constant():
+ exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy", 7.0)
+ exp = stix2.ObservationExpression(exp1)
+ assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]"
+
+
+def test_greater_than():
+ exp1 = stix2.GreaterThanComparisonExpression("file:extensions.windows-pebinary-ext.sections[*].entropy",
+ stix2.FloatConstant(7.0))
+ exp = stix2.ObservationExpression(exp1)
+ assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]"
+
+
+def test_less_than():
+ exp = stix2.LessThanComparisonExpression("file:size", 1024)
+ assert str(exp) == "file:size < 1024"
+
+
+def test_greater_than_or_equal():
+ exp = stix2.GreaterThanEqualComparisonExpression("file:size",
+ 1024)
+
+ assert str(exp) == "file:size >= 1024"
+
+
+def test_less_than_or_equal():
+ exp = stix2.LessThanEqualComparisonExpression("file:size",
+ 1024)
+ assert str(exp) == "file:size <= 1024"
+
+
+def test_not():
+ exp = stix2.LessThanComparisonExpression("file:size",
+ 1024,
+ negated=True)
+ assert str(exp) == "file:size NOT < 1024"
+
+
+def test_and_observable_expression():
+ exp1 = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:account_type",
+ "unix"),
+ stix2.EqualityComparisonExpression("user-account:user_id",
+ stix2.StringConstant("1007")),
+ stix2.EqualityComparisonExpression("user-account:account_login",
+ "Peter")])
+ exp2 = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:account_type",
+ "unix"),
+ stix2.EqualityComparisonExpression("user-account:user_id",
+ stix2.StringConstant("1008")),
+ stix2.EqualityComparisonExpression("user-account:account_login",
+ "Paul")])
+ exp3 = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:account_type",
+ "unix"),
+ stix2.EqualityComparisonExpression("user-account:user_id",
+ stix2.StringConstant("1009")),
+ stix2.EqualityComparisonExpression("user-account:account_login",
+ "Mary")])
+ exp = stix2.AndObservationExpression([stix2.ObservationExpression(exp1),
+ stix2.ObservationExpression(exp2),
+ stix2.ObservationExpression(exp3)])
+ assert str(exp) == "[user-account:account_type = 'unix' AND user-account:user_id = '1007' AND user-account:account_login = 'Peter'] AND [user-account:account_type = 'unix' AND user-account:user_id = '1008' AND user-account:account_login = 'Paul'] AND [user-account:account_type = 'unix' AND user-account:user_id = '1009' AND user-account:account_login = 'Mary']" # noqa
+
+
+def test_invalid_and_observable_expression():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:display_name",
+ "admin"),
+ stix2.EqualityComparisonExpression("email-addr:display_name",
+ stix2.StringConstant("admin"))])
+ assert "All operands to an 'AND' expression must have the same object type" in str(excinfo)
+
+
+def test_hex():
+ exp_and = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("file:mime_type",
+ "image/bmp"),
+ stix2.EqualityComparisonExpression("file:magic_number_hex",
+ stix2.HexConstant("ffd8"))])
+ exp = stix2.ObservationExpression(exp_and)
+ assert str(exp) == "[file:mime_type = 'image/bmp' AND file:magic_number_hex = h'ffd8']"
+
+
+def test_multiple_qualifiers():
+ exp_and = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("network-traffic:dst_ref.type",
+ "domain-name"),
+ stix2.EqualityComparisonExpression("network-traffic:dst_ref.value",
+ "example.com")])
+ exp_ob = stix2.ObservationExpression(exp_and)
+ qual_rep = stix2.RepeatQualifier(5)
+ qual_within = stix2.WithinQualifier(stix2.IntegerConstant(1800))
+ exp = stix2.QualifiedObservationExpression(stix2.QualifiedObservationExpression(exp_ob, qual_rep), qual_within)
+ assert str(exp) == "[network-traffic:dst_ref.type = 'domain-name' AND network-traffic:dst_ref.value = 'example.com'] REPEATS 5 TIMES WITHIN 1800 SECONDS" # noqa
+
+
+def test_set_op():
+ exp = stix2.ObservationExpression(stix2.IsSubsetComparisonExpression("network-traffic:dst_ref.value",
+ "2001:0db8:dead:beef:0000:0000:0000:0000/64"))
+ assert str(exp) == "[network-traffic:dst_ref.value ISSUBSET '2001:0db8:dead:beef:0000:0000:0000:0000/64']"
+
+
+def test_timestamp():
+ ts = stix2.TimestampConstant('2014-01-13T07:03:17Z')
+ assert str(ts) == "t'2014-01-13T07:03:17Z'"
+
+
+def test_boolean():
+ exp = stix2.EqualityComparisonExpression("email-message:is_multipart",
+ True)
+ assert str(exp) == "email-message:is_multipart = true"
+
+
+def test_binary():
+ const = stix2.BinaryConstant("dGhpcyBpcyBhIHRlc3Q=")
+ exp = stix2.EqualityComparisonExpression("artifact:payload_bin",
+ const)
+ assert str(exp) == "artifact:payload_bin = b'dGhpcyBpcyBhIHRlc3Q='"
+
+
+def test_list():
+ exp = stix2.InComparisonExpression("process:name",
+ ['proccy', 'proximus', 'badproc'])
+ assert str(exp) == "process:name IN ('proccy', 'proximus', 'badproc')"
+
+
+def test_list2():
+ # alternate way to construct an "IN" Comparison Expression
+ exp = stix2.EqualityComparisonExpression("process:name",
+ ['proccy', 'proximus', 'badproc'])
+ assert str(exp) == "process:name IN ('proccy', 'proximus', 'badproc')"
+
+
+def test_invalid_constant_type():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.EqualityComparisonExpression("artifact:payload_bin",
+ {'foo': 'bar'})
+ assert 'Unable to create a constant' in str(excinfo)
+
+
+def test_invalid_integer_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.IntegerConstant('foo')
+ assert 'must be an integer' in str(excinfo)
+
+
+def test_invalid_timestamp_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.TimestampConstant('foo')
+ assert 'Must be a datetime object or timestamp string' in str(excinfo)
+
+
+def test_invalid_float_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.FloatConstant('foo')
+ assert 'must be a float' in str(excinfo)
+
+
+@pytest.mark.parametrize("data, result", [
+ (True, True),
+ (False, False),
+ ('True', True),
+ ('False', False),
+ ('true', True),
+ ('false', False),
+ ('t', True),
+ ('f', False),
+ ('T', True),
+ ('F', False),
+ (1, True),
+ (0, False),
+])
+def test_boolean_constant(data, result):
+ boolean = stix2.BooleanConstant(data)
+ assert boolean.value == result
+
+
+def test_invalid_boolean_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.BooleanConstant('foo')
+ assert 'must be a boolean' in str(excinfo)
+
+
+@pytest.mark.parametrize("hashtype, data", [
+ ('MD5', 'zzz'),
+ ('ssdeep', 'zzz=='),
+])
+def test_invalid_hash_constant(hashtype, data):
+ with pytest.raises(ValueError) as excinfo:
+ stix2.HashConstant(data, hashtype)
+ assert 'is not a valid {} hash'.format(hashtype) in str(excinfo)
+
+
+def test_invalid_hex_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.HexConstant('mm')
+ assert "must contain an even number of hexadecimal characters" in str(excinfo)
+
+
+def test_invalid_binary_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.BinaryConstant('foo')
+ assert 'must contain a base64' in str(excinfo)
+
+
+def test_escape_quotes_and_backslashes():
+ exp = stix2.MatchesComparisonExpression("file:name",
+ "^Final Report.+\\.exe$")
+ assert str(exp) == "file:name MATCHES '^Final Report.+\\\\.exe$'"
+
+
+def test_like():
+ exp = stix2.LikeComparisonExpression("directory:path",
+ "C:\\Windows\\%\\foo")
+ assert str(exp) == "directory:path LIKE 'C:\\\\Windows\\\\%\\\\foo'"
+
+
+def test_issuperset():
+ exp = stix2.IsSupersetComparisonExpression("ipv4-addr:value",
+ "198.51.100.0/24")
+ assert str(exp) == "ipv4-addr:value ISSUPERSET '198.51.100.0/24'"
+
+
+def test_repeat_qualifier():
+ qual = stix2.RepeatQualifier(stix2.IntegerConstant(5))
+ assert str(qual) == 'REPEATS 5 TIMES'
+
+
+def test_invalid_repeat_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.RepeatQualifier('foo')
+ assert 'is not a valid argument for a Repeat Qualifier' in str(excinfo)
+
+
+def test_invalid_within_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.WithinQualifier('foo')
+ assert 'is not a valid argument for a Within Qualifier' in str(excinfo)
+
+
+def test_startstop_qualifier():
+ qual = stix2.StartStopQualifier(stix2.TimestampConstant('2016-06-01T00:00:00Z'),
+ datetime.datetime(2017, 3, 12, 8, 30, 0))
+ assert str(qual) == "START t'2016-06-01T00:00:00Z' STOP t'2017-03-12T08:30:00Z'"
+
+ qual2 = stix2.StartStopQualifier(datetime.date(2016, 6, 1),
+ stix2.TimestampConstant('2016-07-01T00:00:00Z'))
+ assert str(qual2) == "START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'"
+
+
+def test_invalid_startstop_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.StartStopQualifier('foo',
+ stix2.TimestampConstant('2016-06-01T00:00:00Z'))
+ assert 'is not a valid argument for a Start/Stop Qualifier' in str(excinfo)
+
+ with pytest.raises(ValueError) as excinfo:
+ stix2.StartStopQualifier(datetime.date(2016, 6, 1),
+ 'foo')
+ assert 'is not a valid argument for a Start/Stop Qualifier' in str(excinfo)
+
+
+def test_make_constant_already_a_constant():
+ str_const = stix2.StringConstant('Foo')
+ result = stix2.patterns.make_constant(str_const)
+ assert result is str_const