This OASIS Open Repository ( github.com/oasis-open/cti-python-stix2 ) is a community public repository that supports participation by anyone, whether affiliated with OASIS or not. Substantive contributions (repository "code") and related feedback is invited from all parties, following the common conventions for participation in GitHub public repository projects. Participation is expected to be consistent with the OASIS Open Repository Guidelines and Procedures, the LICENSE designated for this particular repository (BSD-3-Clause License), and the requirement for an Individual Contributor License Agreement. Please see the repository README document for other details.
+This OASIS TC Open Repository ( github.com/oasis-open/cti-python-stix2 ) is a community public repository that supports participation by anyone, whether affiliated with OASIS or not. Substantive contributions (repository "code") and related feedback is invited from all parties, following the common conventions for participation in GitHub public repository projects. Participation is expected to be consistent with the OASIS TC Open Repository Guidelines and Procedures, the LICENSE designated for this particular repository (BSD-3-Clause License), and the requirement for an Individual Contributor License Agreement. Please see the repository README document for other details.
Content accepted as "contributions" to this Open Repository, as defined below, are distinct from any Contributions made to the associated OASIS Cyber Threat Intelligence (CTI) TC itself. Participation in the associated Technical Committee is governed by the OASIS Bylaws, OASIS TC Process, IPR Policy, and related policies. This Open Repository is not subject to the OASIS TC-related policies. Open Repository governance is defined by separate participation and contribution guidelines as referenced in the OASIS Open Repositories Overview.
+Content accepted as "contributions" to this TC Open Repository, as defined below, are distinct from any Contributions made to the associated OASIS Cyber Threat Intelligence (CTI) TC itself. Participation in the associated Technical Committee is governed by the OASIS Bylaws, OASIS TC Process, IPR Policy, and related policies. This TC Open Repository is not subject to the OASIS TC-related policies. TC Open Repository governance is defined by separate participation and contribution guidelines as referenced in the OASIS TC Open Repositories Overview.
Because different licenses apply to the OASIS TC's specification work, and this Open Repository, there is no guarantee that the licensure of specific repository material will be compatible with licensing requirements of an implementation of a TC's specification. Please refer to the LICENSE file for the terms of this material, and to the OASIS IPR Policy for the terms applicable to the TC's specifications, including any applicable declarations.
+Because different licenses apply to the OASIS TC's specification work, and this TC Open Repository, there is no guarantee that the licensure of specific repository material will be compatible with licensing requirements of an implementation of a TC's specification. Please refer to the LICENSE file for the terms of this material, and to the OASIS IPR Policy for the terms applicable to the TC's specifications, including any applicable declarations.
Formally, "contribution" to this Open Repository refers to content merged into the "Code" repository (repository changes represented by code commits), following the GitHub definition of contributor: "someone who has contributed to a project by having a pull request merged but does not have collaborator [i.e., direct write] access." Anyone who signs the Open Repository Individual Contributor License Agreement (CLA), signifying agreement with the licensing requirement, may contribute substantive content — subject to evaluation of a GitHub pull request. The main web page for this repository, as with any GitHub public repository, displays a link to a document listing contributions to the repository's default branch (filtered by Commits, Additions, and Deletions).
+Formally, "contribution" to this TC Open Repository refers to content merged into the "Code" repository (repository changes represented by code commits), following the GitHub definition of contributor: "someone who has contributed to a project by having a pull request merged but does not have collaborator [i.e., direct write] access." Anyone who signs the TC Open Repository Individual Contributor License Agreement (CLA), signifying agreement with the licensing requirement, may contribute substantive content — subject to evaluation of a GitHub pull request. The main web page for this repository, as with any GitHub public repository, displays a link to a document listing contributions to the repository's default branch (filtered by Commits, Additions, and Deletions).
-This Open Repository, as with GitHub public repositories generally, also accepts public feedback from any GitHub user. Public feedback includes opening issues, authoring and editing comments, participating in conversations, making wiki edits, creating repository stars, and making suggestions via pull requests. Such feedback does not constitute an OASIS Open Repository contribution. Some details are presented under "Read permissions" in the table of permission levels for a GitHub organization. Technical content intended as a substantive contribution (repository "Code") to an Open Repository is subject to evaluation, and requires a signed Individual CLA.
+This TC Open Repository, as with GitHub public repositories generally, also accepts public feedback from any GitHub user. Public feedback includes opening issues, authoring and editing comments, participating in conversations, making wiki edits, creating repository stars, and making suggestions via pull requests. Such feedback does not constitute an OASIS TC Open Repository contribution. Some details are presented under "Read permissions" in the table of permission levels for a GitHub organization. Technical content intended as a substantive contribution (repository "Code") to an TC Open Repository is subject to evaluation, and requires a signed Individual CLA.
OASIS Open Repositories use the familiar fork-and-pull collaboration model supported by GitHub and other distributed version-control systems. Any GitHub user wishing to contribute should fork the repository, make additions or other modifications, and then submit a pull request. GitHub pull requests should be accompanied by supporting comments and/or issues. Community conversations about pull requests, supported by GitHub notifications, will provide the basis for a consensus determination to merge, modify, close, or take other action, as communicated by the repository Maintainers.
+OASIS TC Open Repositories use the familiar fork-and-pull collaboration model supported by GitHub and other distributed version-control systems. Any GitHub user wishing to contribute should fork the repository, make additions or other modifications, and then submit a pull request. GitHub pull requests should be accompanied by supporting comments and/or issues. Community conversations about pull requests, supported by GitHub notifications, will provide the basis for a consensus determination to merge, modify, close, or take other action, as communicated by the repository Maintainers.
Questions or comments about this Open Repository's activities should be composed as GitHub issues or comments. If use of an issue/comment is not possible or appropriate, questions may be directed by email to the repository Maintainer(s). Please send general questions about Open Repository participation to OASIS Staff at repository-admin@oasis-open.org and any specific CLA-related questions to repository-cla@oasis-open.org.
+Questions or comments about this TC Open Repository's activities should be composed as GitHub issues or comments. If use of an issue/comment is not possible or appropriate, questions may be directed by email to the repository Maintainer(s). Please send general questions about TC Open Repository participation to OASIS Staff at repository-admin@oasis-open.org and any specific CLA-related questions to repository-cla@oasis-open.org.
{\n",
" "type": "indicator",\n",
- " "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",\n",
- " "created": "2017-09-26T23:33:39.829Z",\n",
- " "modified": "2017-09-26T23:33:39.829Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
+ " "id": "indicator--548af3be-39d7-4a3e-93c2-1a63cccf8951",\n",
+ " "created": "2018-04-05T18:32:24.193Z",\n",
+ " "modified": "2018-04-05T18:32:24.193Z",\n",
" "name": "File hash for malware variant",\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-09-26T23:33:39.829952Z"\n",
+ " "valid_from": "2018-04-05T18:32:24.193659Z",\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ]\n",
"}\n",
"{\n",
" "type": "malware",\n",
- " "id": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2",\n",
- " "created": "2017-09-26T23:33:56.908Z",\n",
- " "modified": "2017-09-26T23:33:56.908Z",\n",
+ " "id": "malware--3d7f0c1c-616a-4868-aa7b-150821d2a429",\n",
+ " "created": "2018-04-05T18:32:46.584Z",\n",
+ " "modified": "2018-04-05T18:32:46.584Z",\n",
" "name": "Poison Ivy",\n",
" "labels": [\n",
" "remote-access-trojan"\n",
@@ -592,12 +588,12 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "relationship",\n",
- " "id": "relationship--637aa3b1-d4b8-4bc4-85e7-77cc82b198a3",\n",
- " "created": "2017-09-26T23:34:01.765Z",\n",
- " "modified": "2017-09-26T23:34:01.765Z",\n",
+ " "id": "relationship--34ddc7b4-4965-4615-b286-1c8bbaa1e7db",\n",
+ " "created": "2018-04-05T18:32:49.474Z",\n",
+ " "modified": "2018-04-05T18:32:49.474Z",\n",
" "relationship_type": "indicates",\n",
- " "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",\n",
- " "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"\n",
+ " "source_ref": "indicator--548af3be-39d7-4a3e-93c2-1a63cccf8951",\n",
+ " "target_ref": "malware--3d7f0c1c-616a-4868-aa7b-150821d2a429"\n",
"}\n",
"
\n"
],
@@ -704,12 +700,12 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "relationship",\n",
- " "id": "relationship--70fe77c2-ab00-4181-a2dc-fe5567d971ca",\n",
- " "created": "2017-09-26T23:34:03.923Z",\n",
- " "modified": "2017-09-26T23:34:03.923Z",\n",
+ " "id": "relationship--0a646403-f7e7-4cfd-b945-cab5cde05857",\n",
+ " "created": "2018-04-05T18:32:51.417Z",\n",
+ " "modified": "2018-04-05T18:32:51.417Z",\n",
" "relationship_type": "indicates",\n",
- " "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",\n",
- " "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"\n",
+ " "source_ref": "indicator--548af3be-39d7-4a3e-93c2-1a63cccf8951",\n",
+ " "target_ref": "malware--3d7f0c1c-616a-4868-aa7b-150821d2a429"\n",
"}\n",
"
\n"
],
@@ -814,26 +810,26 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "bundle",\n",
- " "id": "bundle--2536c43d-c874-418e-886c-20a22120d8cb",\n",
+ " "id": "bundle--f83477e5-f853-47e1-a267-43f3aa1bd5b0",\n",
" "spec_version": "2.0",\n",
" "objects": [\n",
" {\n",
" "type": "indicator",\n",
- " "id": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",\n",
- " "created": "2017-09-26T23:33:39.829Z",\n",
- " "modified": "2017-09-26T23:33:39.829Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
+ " "id": "indicator--548af3be-39d7-4a3e-93c2-1a63cccf8951",\n",
+ " "created": "2018-04-05T18:32:24.193Z",\n",
+ " "modified": "2018-04-05T18:32:24.193Z",\n",
" "name": "File hash for malware variant",\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-09-26T23:33:39.829952Z"\n",
+ " "valid_from": "2018-04-05T18:32:24.193659Z",\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ]\n",
" },\n",
" {\n",
" "type": "malware",\n",
- " "id": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2",\n",
- " "created": "2017-09-26T23:33:56.908Z",\n",
- " "modified": "2017-09-26T23:33:56.908Z",\n",
+ " "id": "malware--3d7f0c1c-616a-4868-aa7b-150821d2a429",\n",
+ " "created": "2018-04-05T18:32:46.584Z",\n",
+ " "modified": "2018-04-05T18:32:46.584Z",\n",
" "name": "Poison Ivy",\n",
" "labels": [\n",
" "remote-access-trojan"\n",
@@ -841,12 +837,12 @@
" },\n",
" {\n",
" "type": "relationship",\n",
- " "id": "relationship--637aa3b1-d4b8-4bc4-85e7-77cc82b198a3",\n",
- " "created": "2017-09-26T23:34:01.765Z",\n",
- " "modified": "2017-09-26T23:34:01.765Z",\n",
+ " "id": "relationship--34ddc7b4-4965-4615-b286-1c8bbaa1e7db",\n",
+ " "created": "2018-04-05T18:32:49.474Z",\n",
+ " "modified": "2018-04-05T18:32:49.474Z",\n",
" "relationship_type": "indicates",\n",
- " "source_ref": "indicator--dbcbd659-c927-4f9a-994f-0a2632274394",\n",
- " "target_ref": "malware--d7fd675d-94eb-4d95-b0bc-b3c5e28e8ed2"\n",
+ " "source_ref": "indicator--548af3be-39d7-4a3e-93c2-1a63cccf8951",\n",
+ " "target_ref": "malware--3d7f0c1c-616a-4868-aa7b-150821d2a429"\n",
" }\n",
" ]\n",
"}\n",
@@ -871,21 +867,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/custom.ipynb b/docs/guide/custom.ipynb
index 7f30401..e651084 100644
--- a/docs/guide/custom.ipynb
+++ b/docs/guide/custom.ipynb
@@ -4,7 +4,6 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -23,9 +22,8 @@
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -70,7 +70,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 3,
"metadata": {},
"outputs": [
{
@@ -99,7 +99,7 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -175,9 +175,9 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "identity",\n",
- " "id": "identity--00c5743f-2d5e-4d66-88f1-1842584f4519",\n",
- " "created": "2017-11-09T16:17:44.596Z",\n",
- " "modified": "2017-11-09T16:17:44.596Z",\n",
+ " "id": "identity--87aac643-341b-413a-b702-ea5820416155",\n",
+ " "created": "2018-04-05T18:38:10.269Z",\n",
+ " "modified": "2018-04-05T18:38:10.269Z",\n",
" "name": "John Smith",\n",
" "identity_class": "individual",\n",
" "x_foo": "bar"\n",
@@ -188,7 +188,7 @@
""
]
},
- "execution_count": 2,
+ "execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@@ -211,6 +211,117 @@
"Alternatively, setting ``allow_custom`` to ``True`` will allow custom properties without requiring a ``custom_properties`` dictionary."
]
},
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "identity",\n",
+ " "id": "identity--a1ad0a6f-39ab-4642-9a72-aaa198b1eee2",\n",
+ " "created": "2018-04-05T18:38:12.270Z",\n",
+ " "modified": "2018-04-05T18:38:12.270Z",\n",
+ " "name": "John Smith",\n",
+ " "identity_class": "individual",\n",
+ " "x_foo": "bar"\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "identity2 = Identity(name=\"John Smith\",\n",
+ " identity_class=\"individual\",\n",
+ " x_foo=\"bar\",\n",
+ " allow_custom=True)\n",
+ "print(identity2)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Likewise, when parsing STIX content with custom properties, pass ``allow_custom=True`` to [parse()](../api/stix2.core.rst#stix2.core.parse):"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 6,
@@ -287,15 +398,7 @@
".highlight .vg { color: #19177C } /* Name.Variable.Global */\n",
".highlight .vi { color: #19177C } /* Name.Variable.Instance */\n",
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
- ".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
- " "x_foo": "bar",\n",
- " "type": "identity",\n",
- " "id": "identity--1e8188eb-245f-400b-839d-7f612169c514",\n",
- " "created": "2017-09-26T21:02:22.708Z",\n",
- " "modified": "2017-09-26T21:02:22.708Z",\n",
- " "name": "John Smith",\n",
- " "identity_class": "individual"\n",
- "}\n",
+ ".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */bar\n",
"\n"
],
"text/plain": [
@@ -307,34 +410,6 @@
"output_type": "execute_result"
}
],
- "source": [
- "identity2 = Identity(name=\"John Smith\",\n",
- " identity_class=\"individual\",\n",
- " x_foo=\"bar\",\n",
- " allow_custom=True)\n",
- "print(identity2)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Likewise, when parsing STIX content with custom properties, pass ``allow_custom=True`` to [parse()](../api/stix2.core.rst#stix2.core.parse):"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {},
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "bar\n"
- ]
- }
- ],
"source": [
"from stix2 import parse\n",
"\n",
@@ -364,10 +439,8 @@
},
{
"cell_type": "code",
- "execution_count": 8,
- "metadata": {
- "collapsed": true
- },
+ "execution_count": 7,
+ "metadata": {},
"outputs": [],
"source": [
"from stix2 import CustomObject, properties\n",
@@ -391,7 +464,7 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 8,
"metadata": {},
"outputs": [
{
@@ -467,9 +540,9 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "x-animal",\n",
- " "id": "x-animal--caebdf17-9d2a-4c84-8864-7406326618f0",\n",
- " "created": "2017-09-26T21:02:34.724Z",\n",
- " "modified": "2017-09-26T21:02:34.724Z",\n",
+ " "id": "x-animal--b1e4fe7f-7985-451d-855c-6ba5c265b22a",\n",
+ " "created": "2018-04-05T18:38:19.790Z",\n",
+ " "modified": "2018-04-05T18:38:19.790Z",\n",
" "species": "lion",\n",
" "animal_class": "mammal"\n",
"}\n",
@@ -479,7 +552,7 @@
""
]
},
- "execution_count": 9,
+ "execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
@@ -499,7 +572,7 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 9,
"metadata": {},
"outputs": [
{
@@ -525,15 +598,90 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": 10,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "shark\n"
- ]
+ "data": {
+ "text/html": [
+ "shark\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 10,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -558,7 +706,7 @@
},
{
"cell_type": "code",
- "execution_count": 12,
+ "execution_count": 11,
"metadata": {},
"outputs": [
{
@@ -593,7 +741,7 @@
},
{
"cell_type": "code",
- "execution_count": 13,
+ "execution_count": 12,
"metadata": {},
"outputs": [
{
@@ -678,7 +826,7 @@
""
]
},
- "execution_count": 13,
+ "execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
@@ -709,16 +857,172 @@
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 13,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "foobaz\n",
- "5\n"
- ]
+ "data": {
+ "text/html": [
+ "foobaz\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "5\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 13,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -759,7 +1063,7 @@
},
{
"cell_type": "code",
- "execution_count": 15,
+ "execution_count": 16,
"metadata": {},
"outputs": [
{
@@ -843,7 +1147,7 @@
""
]
},
- "execution_count": 15,
+ "execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
@@ -872,16 +1176,172 @@
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 17,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "bla\n",
- "50\n"
- ]
+ "data": {
+ "text/html": [
+ "bla\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "50\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 17,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -918,21 +1378,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/datastore.ipynb b/docs/guide/datastore.ipynb
index ac27a82..c3980a0 100644
--- a/docs/guide/datastore.ipynb
+++ b/docs/guide/datastore.ipynb
@@ -23,7 +23,7 @@
},
{
"cell_type": "code",
- "execution_count": 40,
+ "execution_count": 2,
"metadata": {
"collapsed": true,
"nbsphinx": "hidden"
@@ -33,36 +33,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "# without this configuration, only last print() call is outputted in cells\n",
- "from IPython.core.interactiveshell import InteractiveShell\n",
- "InteractiveShell.ast_node_interactivity = \"all\""
+ "globals()['print'] = json_print"
]
},
{
@@ -71,9 +60,9 @@
"source": [
"# DataStore API\n",
"\n",
- "CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
+ "The ``stix2`` library features an interface for pulling and pushing STIX 2 content. This interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX 2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX 2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
"\n",
- "The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. CTI Python STIX2 provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
+ "The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. The ``stix2`` library provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
]
},
{
@@ -328,10 +317,10 @@
"from stix2 import CompositeDataSource, FileSystemSource, TAXIICollectionSource\n",
"\n",
"# create FileSystemStore\n",
- "fs = FileSystemSource(\"/home/michael/cti-python-stix2/stix2/test/stix2_data/\")\n",
+ "fs = FileSystemSource(\"/tmp/stix2_source\")\n",
"\n",
"# create TAXIICollectionSource\n",
- "colxn = Collection('https://test.freetaxii.com:8000/osint/collections/a9c22eaf-0f3e-482c-8bb4-45ae09e75d9b/')\n",
+ "colxn = Collection('http://127.0.0.1:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/')\n",
"ts = TAXIICollectionSource(colxn)\n",
"\n",
"# add them both to the CompositeDataSource\n",
@@ -344,7 +333,7 @@
"\n",
"# get an object that is only in the TAXII collection\n",
"ind = cs.get('indicator--02b90f02-a96a-43ee-88f1-1e87297941f2')\n",
- "print(ind)\n"
+ "print(ind)"
]
},
{
@@ -353,9 +342,16 @@
"source": [
"## Filters\n",
"\n",
- "The CTI Python STIX2 DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
+ "The ``stix2`` DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. Filters can be used to explicitly include or exclude results with certain criteria. For example:\n",
"\n",
- "Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX2 objects can be filtered on. In addition, TAXII2 Filtering parameters for fields can also be used in filters.\n",
+ "* only trust content from a set of object creators\n",
+ "* exclude content from certain (untrusted) object creators\n",
+ "* only include content with a confidence above a certain threshold (once confidence is added to STIX 2)\n",
+ "* only return content that can be shared with external parties (e.g. only content that has TLP:GREEN markings)\n",
+ "\n",
+ "Filters can be created and supplied with every call to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
+ "\n",
+ "Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX 2 objects can be filtered on. In addition, TAXII 2 Filtering parameters for fields can also be used in filters.\n",
"\n",
"TAXII2 filter fields:\n",
"\n",
@@ -389,7 +385,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 3,
"metadata": {
"collapsed": true
},
@@ -424,25 +420,23 @@
{
"cell_type": "code",
"execution_count": 6,
- "metadata": {
- "collapsed": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"from stix2 import MemoryStore, FileSystemStore, FileSystemSource\n",
"\n",
- "fs = FileSystemStore(\"/home/michael/Desktop/sample_stix2_data\")\n",
- "fs_source = FileSystemSource(\"/home/michael/Desktop/sample_stix2_data\")\n",
+ "fs = FileSystemStore(\"/tmp/stix2_store\")\n",
+ "fs_source = FileSystemSource(\"/tmp/stix2_source\")\n",
"\n",
"# attach filter to FileSystemStore\n",
"fs.source.filters.add(f)\n",
"\n",
"# attach multiple filters to FileSystemStore\n",
- "fs.source.filters.update([f1,f2])\n",
+ "fs.source.filters.add([f1,f2])\n",
"\n",
"# can also attach filters to a Source\n",
"# attach multiple filters to FileSystemSource\n",
- "fs_source.filters.update([f3, f4])\n",
+ "fs_source.filters.add([f3, f4])\n",
"\n",
"\n",
"mem = MemoryStore()\n",
@@ -452,7 +446,7 @@
"mem.source.filters.add(f)\n",
"\n",
"# attach multiple filters to a MemoryStore\n",
- "mem.source.filters.update([f1,f2])"
+ "mem.source.filters.add([f1,f2])"
]
},
{
@@ -466,8 +460,10 @@
},
{
"cell_type": "code",
- "execution_count": 13,
- "metadata": {},
+ "execution_count": 10,
+ "metadata": {
+ "collapsed": true
+ },
"outputs": [],
"source": [
"from stix2 import Campaign, Identity, Indicator, Malware, Relationship\n",
@@ -492,7 +488,7 @@
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 11,
"metadata": {},
"outputs": [
{
@@ -568,9 +564,9 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "identity",\n",
- " "id": "identity--be3baac0-9aba-48a8-81e4-4408b1c379a8",\n",
- " "created": "2017-11-21T22:14:45.213Z",\n",
- " "modified": "2017-11-21T22:14:45.213Z",\n",
+ " "id": "identity--b67cf8d4-cc1a-4bb7-9402-fffcff17c9a9",\n",
+ " "created": "2018-04-05T20:43:54.117Z",\n",
+ " "modified": "2018-04-05T20:43:54.117Z",\n",
" "name": "John Doe",\n",
" "identity_class": "individual"\n",
"}\n",
@@ -580,7 +576,7 @@
""
]
},
- "execution_count": 14,
+ "execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
@@ -598,7 +594,7 @@
},
{
"cell_type": "code",
- "execution_count": 15,
+ "execution_count": 12,
"metadata": {},
"outputs": [
{
@@ -607,7 +603,7 @@
"3"
]
},
- "execution_count": 15,
+ "execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
@@ -626,16 +622,16 @@
},
{
"cell_type": "code",
- "execution_count": 27,
+ "execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
- "[Relationship(type='relationship', id='relationship--bd6fd399-c907-4feb-b1da-b90f15942f1d', created='2017-11-21T22:14:45.214Z', modified='2017-11-21T22:14:45.214Z', relationship_type=u'indicates', source_ref='indicator--5ee33ff0-c50d-456b-a8dd-b5d1b69a66e8', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4')]"
+ "[Relationship(type='relationship', id='relationship--3b9cb248-5c2c-425d-85d0-680bfef6e69d', created='2018-04-05T20:43:54.134Z', modified='2018-04-05T20:43:54.134Z', relationship_type='indicates', source_ref='indicator--61deb2a5-305a-490e-83b3-9839a9677368', target_ref='malware--9fe343d8-edf7-4f4a-bb6c-a221fb75142d')]"
]
},
- "execution_count": 27,
+ "execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
@@ -653,16 +649,16 @@
},
{
"cell_type": "code",
- "execution_count": 28,
+ "execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
- "[Relationship(type='relationship', id='relationship--7eb7f5cd-8bf2-4f7c-8756-84c0b5693b9a', created='2017-11-21T22:14:45.215Z', modified='2017-11-21T22:14:45.215Z', relationship_type=u'targets', source_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4', target_ref='identity--be3baac0-9aba-48a8-81e4-4408b1c379a8')]"
+ "[Relationship(type='relationship', id='relationship--8d322508-423b-4d51-be85-a95ad083f8af', created='2018-04-05T20:43:54.134Z', modified='2018-04-05T20:43:54.134Z', relationship_type='targets', source_ref='malware--9fe343d8-edf7-4f4a-bb6c-a221fb75142d', target_ref='identity--b67cf8d4-cc1a-4bb7-9402-fffcff17c9a9')]"
]
},
- "execution_count": 28,
+ "execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
@@ -680,17 +676,17 @@
},
{
"cell_type": "code",
- "execution_count": 30,
+ "execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
- "[Relationship(type='relationship', id='relationship--bd6fd399-c907-4feb-b1da-b90f15942f1d', created='2017-11-21T22:14:45.214Z', modified='2017-11-21T22:14:45.214Z', relationship_type=u'indicates', source_ref='indicator--5ee33ff0-c50d-456b-a8dd-b5d1b69a66e8', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4'),\n",
- " Relationship(type='relationship', id='relationship--3c759d40-c92a-430e-aab6-77d5c5763302', created='2017-11-21T22:14:45.215Z', modified='2017-11-21T22:14:45.215Z', relationship_type=u'uses', source_ref='campaign--82ab7aa4-d13b-4e99-8a09-ebcba30668a7', target_ref='malware--66c0bc78-4e27-4d80-a565-a07e6eb6fba4')]"
+ "[Relationship(type='relationship', id='relationship--3b9cb248-5c2c-425d-85d0-680bfef6e69d', created='2018-04-05T20:43:54.134Z', modified='2018-04-05T20:43:54.134Z', relationship_type='indicates', source_ref='indicator--61deb2a5-305a-490e-83b3-9839a9677368', target_ref='malware--9fe343d8-edf7-4f4a-bb6c-a221fb75142d'),\n",
+ " Relationship(type='relationship', id='relationship--93e5afe0-d1fb-4315-8d08-10951f7a99b6', created='2018-04-05T20:43:54.134Z', modified='2018-04-05T20:43:54.134Z', relationship_type='uses', source_ref='campaign--edfd885c-bc31-4051-9bc2-08e057542d56', target_ref='malware--9fe343d8-edf7-4f4a-bb6c-a221fb75142d')]"
]
},
- "execution_count": 30,
+ "execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
@@ -708,16 +704,16 @@
},
{
"cell_type": "code",
- "execution_count": 42,
+ "execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
- "[Campaign(type='campaign', id='campaign--82ab7aa4-d13b-4e99-8a09-ebcba30668a7', created='2017-11-21T22:14:45.213Z', modified='2017-11-21T22:14:45.213Z', name=u'Charge', description=u'Attack!')]"
+ "[Campaign(type='campaign', id='campaign--edfd885c-bc31-4051-9bc2-08e057542d56', created='2018-04-05T20:43:54.117Z', modified='2018-04-05T20:43:54.117Z', name='Charge', description='Attack!')]"
]
},
- "execution_count": 42,
+ "execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
diff --git a/docs/guide/environment.ipynb b/docs/guide/environment.ipynb
index 3ece7c4..fdc57c8 100644
--- a/docs/guide/environment.ipynb
+++ b/docs/guide/environment.ipynb
@@ -4,7 +4,6 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -25,7 +24,6 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -68,9 +68,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "metadata": {
- "collapsed": true
- },
+ "metadata": {},
"outputs": [],
"source": [
"from stix2 import Environment, MemoryStore\n",
@@ -87,18 +85,16 @@
},
{
"cell_type": "code",
- "execution_count": 4,
- "metadata": {
- "collapsed": true
- },
+ "execution_count": 6,
+ "metadata": {},
"outputs": [],
"source": [
"from stix2 import CompositeDataSource, FileSystemSink, FileSystemSource, MemorySource\n",
"\n",
"src = CompositeDataSource()\n",
- "src.add_data_sources([MemorySource(), FileSystemSource(\"/tmp/stix_source\")])\n",
+ "src.add_data_sources([MemorySource(), FileSystemSource(\"/tmp/stix2_source\")])\n",
"env2 = Environment(source=src,\n",
- " sink=FileSystemSink(\"/tmp/stix_sink\"))"
+ " sink=FileSystemSink(\"/tmp/stix2_sink\"))"
]
},
{
@@ -110,10 +106,8 @@
},
{
"cell_type": "code",
- "execution_count": 5,
- "metadata": {
- "collapsed": true
- },
+ "execution_count": 7,
+ "metadata": {},
"outputs": [],
"source": [
"from stix2 import Indicator\n",
@@ -131,139 +125,6 @@
"You can retrieve STIX objects from the [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource) in the [Environment](../api/stix2.environment.rst#stix2.environment.Environment) with [get()](../api/stix2.environment.rst#stix2.environment.Environment.get), [query()](../api/stix2.environment.rst#stix2.environment.Environment.query), [all_versions()](../api/stix2.environment.rst#stix2.environment.Environment.all_versions), [creator_of()](../api/stix2.datastore.rst#stix2.datastore.DataSource.creator_of), [related_to()](../api/stix2.datastore.rst#stix2.datastore.DataSource.related_to), and [relationships()](../api/stix2.datastore.rst#stix2.datastore.DataSource.relationships) just as you would for a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource)."
]
},
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "{\n",
- " "type": "indicator",\n",
- " "id": "indicator--01234567-89ab-cdef-0123-456789abcdef",\n",
- " "created": "2017-10-02T13:20:39.373Z",\n",
- " "modified": "2017-10-02T13:20:39.373Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-02T13:20:39.3737Z"\n",
- "}\n",
- "
\n"
- ],
- "text/plain": [
- ""
- ]
- },
- "execution_count": 6,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "print(env.get(\"indicator--01234567-89ab-cdef-0123-456789abcdef\"))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Creating STIX Objects With Defaults\n",
- "\n",
- "To create STIX objects with default values for certain properties, use an [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory). For instance, say we want all objects we create to have a ``created_by_ref`` property pointing to the ``Identity`` object representing our organization."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": 7,
- "metadata": {
- "collapsed": true
- },
- "outputs": [],
- "source": [
- "from stix2 import Indicator, ObjectFactory\n",
- "\n",
- "factory = ObjectFactory(created_by_ref=\"identity--311b2d2d-f010-5473-83ec-1edf84858f4c\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {
- "collapsed": true
- },
- "source": [
- "Once you've set up the [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory), use its [create()](../api/stix2.environment.rst#stix2.environment.ObjectFactory.create) method, passing in the class for the type of object you wish to create, followed by the other properties and their values for the object."
- ]
- },
{
"cell_type": "code",
"execution_count": 8,
@@ -342,15 +203,14 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--c92ad60d-449d-4adf-86b3-4e5951a8f480",\n",
- " "created_by_ref": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",\n",
- " "created": "2017-10-02T13:23:00.607Z",\n",
- " "modified": "2017-10-02T13:23:00.607Z",\n",
+ " "id": "indicator--01234567-89ab-cdef-0123-456789abcdef",\n",
+ " "created": "2018-04-05T19:27:53.923Z",\n",
+ " "modified": "2018-04-05T19:27:53.923Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:27:53.923548Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-02T13:23:00.607216Z"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -363,6 +223,138 @@
"output_type": "execute_result"
}
],
+ "source": [
+ "print(env.get(\"indicator--01234567-89ab-cdef-0123-456789abcdef\"))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Creating STIX Objects With Defaults\n",
+ "\n",
+ "To create STIX objects with default values for certain properties, use an [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory). For instance, say we want all objects we create to have a ``created_by_ref`` property pointing to the ``Identity`` object representing our organization."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 13,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from stix2 import Indicator, ObjectFactory\n",
+ "\n",
+ "factory = ObjectFactory(created_by_ref=\"identity--311b2d2d-f010-5473-83ec-1edf84858f4c\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "Once you've set up the [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory), use its [create()](../api/stix2.environment.rst#stix2.environment.ObjectFactory.create) method, passing in the class for the type of object you wish to create, followed by the other properties and their values for the object."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 14,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "indicator",\n",
+ " "id": "indicator--c1b421c0-9c6b-4276-9b73-1b8684a5a0d2",\n",
+ " "created_by_ref": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",\n",
+ " "created": "2018-04-05T19:28:48.776Z",\n",
+ " "modified": "2018-04-05T19:28:48.776Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:28:48.776442Z",\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 14,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
"source": [
"ind = factory.create(Indicator,\n",
" labels=[\"malicious-activity\"],\n",
@@ -388,7 +380,7 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 15,
"metadata": {},
"outputs": [
{
@@ -464,14 +456,14 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--ae206b9f-8723-4fcf-beb7-8b1b9a2570ab",\n",
+ " "id": "indicator--30a3b39c-5f57-4e7f-9eaf-e1abcb643da4",\n",
" "created": "2017-09-25T18:07:46.255Z",\n",
" "modified": "2017-09-25T18:07:46.255Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:28:53.268567Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-02T13:23:05.790562Z"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -479,7 +471,7 @@
""
]
},
- "execution_count": 9,
+ "execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
@@ -498,7 +490,7 @@
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 16,
"metadata": {},
"outputs": [
{
@@ -574,15 +566,15 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--a8e2be68-b496-463f-9ff4-f620046e7cf2",\n",
+ " "id": "indicator--6c5bbaaf-6dac-44b0-a0df-86c27b3f6ecb",\n",
" "created_by_ref": "identity--962cabe5-f7f3-438a-9169-585a8c971d12",\n",
" "created": "2017-09-25T18:07:46.255Z",\n",
" "modified": "2017-09-25T18:07:46.255Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:29:56.55129Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-02T13:23:08.32424Z"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -590,7 +582,7 @@
""
]
},
- "execution_count": 10,
+ "execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
@@ -614,7 +606,7 @@
},
{
"cell_type": "code",
- "execution_count": 11,
+ "execution_count": 17,
"metadata": {},
"outputs": [
{
@@ -690,15 +682,15 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--89ba04ea-cce9-47a3-acd3-b6379ce51581",\n",
+ " "id": "indicator--d1b8c3f6-1de1-44c1-b079-3df307224a0d",\n",
" "created_by_ref": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",\n",
- " "created": "2017-10-02T13:23:29.629Z",\n",
- " "modified": "2017-10-02T13:23:29.629Z",\n",
+ " "created": "2018-04-05T19:29:59.605Z",\n",
+ " "modified": "2018-04-05T19:29:59.605Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:29:59.605463Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-02T13:23:29.629857Z"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -706,7 +698,7 @@
""
]
},
- "execution_count": 11,
+ "execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
@@ -725,21 +717,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/filesystem.ipynb b/docs/guide/filesystem.ipynb
index cff73d6..b3aca88 100644
--- a/docs/guide/filesystem.ipynb
+++ b/docs/guide/filesystem.ipynb
@@ -4,7 +4,6 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -25,7 +24,6 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -58,9 +58,9 @@
"source": [
"## FileSystem \n",
"\n",
- "The FileSystem suite contains [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore), [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink). Under the hood, all FileSystem objects point to a file directory (on disk) that contains STIX2 content. \n",
+ "The FileSystem suite contains [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore), [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink). Under the hood, all FileSystem objects point to a file directory (on disk) that contains STIX 2 content. \n",
"\n",
- "The directory and file structure of the intended STIX2 content should be:\n",
+ "The directory and file structure of the intended STIX 2 content should be:\n",
"\n",
"```\n",
"stix2_content/\n",
@@ -82,7 +82,7 @@
" /STIX2 Domain Object type\n",
"```\n",
"\n",
- "The master STIX2 content directory contains subdirectories, each of which aligns to a STIX2 domain object type (i.e. \"attack-pattern\", \"campaign\", \"malware\", etc.). Within each STIX2 domain object subdirectory are JSON files that are STIX2 domain objects of the specified type. The name of the json files correspond to the ID of the STIX2 domain object found within that file. A real example of the FileSystem directory structure:\n",
+ "The master STIX 2 content directory contains subdirectories, each of which aligns to a STIX 2 domain object type (i.e. \"attack-pattern\", \"campaign\", \"malware\", etc.). Within each STIX 2 domain object subdirectory are JSON files that are STIX 2 domain objects of the specified type. The name of the json files correspond to the ID of the STIX 2 domain object found within that file. A real example of the FileSystem directory structure:\n",
"\n",
"```\n",
"stix2_content/\n",
@@ -107,70 +107,142 @@
" /vulnerability\n",
"```\n",
"\n",
- "[FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) is intended for use cases where STIX2 content is retrieved and pushed to the same file directory. As [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) is just a wrapper around a paired [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink) that point the same file directory.\n",
+ "[FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) is intended for use cases where STIX 2 content is retrieved and pushed to the same file directory. As [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) is just a wrapper around a paired [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink) that point the same file directory.\n",
"\n",
- "For use cases where STIX2 content will only be retrieved or pushed, then a [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink) can be used individually. They can also be used individually when STIX2 content will be retrieved from one distinct file directory and pushed to another.\n",
+ "For use cases where STIX 2 content will only be retrieved or pushed, then a [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource) and [FileSystemSink](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink) can be used individually. They can also be used individually when STIX 2 content will be retrieved from one distinct file directory and pushed to another.\n",
"\n",
"### FileSystem API\n",
"\n",
- "A note on [get()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.get), [all_versions()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.all_versions), and [query()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.query): The format of the STIX2 content targeted by the FileSystem suite is JSON files. When the [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) retrieves STIX2 content (in JSON) from disk, it will attempt to parse the content into full-featured python-stix2 objects and returned as such. \n",
+ "A note on [get()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.get), [all_versions()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.all_versions), and [query()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource.query): The format of the STIX2 content targeted by the FileSystem suite is JSON files. When the [FileSystemStore](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemStore) retrieves STIX 2 content (in JSON) from disk, it will attempt to parse the content into full-featured python-stix2 objects and returned as such. \n",
"\n",
"A note on [add()](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSink.add): When STIX content is added (pushed) to the file system, the STIX content can be supplied in the following forms: Python STIX objects, Python dictionaries (of valid STIX objects or Bundles), JSON-encoded strings (of valid STIX objects or Bundles), or a (Python) list of any of the previously listed types. Any of the previous STIX content forms will be converted to a STIX JSON object (in a STIX Bundle) and written to disk. \n",
"\n",
"### FileSystem Examples\n",
"\n",
"#### FileSystemStore\n",
- " "
+ "\n",
+ "Use the FileSystemStore when you want to both retrieve STIX content from the file system and push STIX content to it, too."
]
},
{
"cell_type": "code",
- "execution_count": 10,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"modified\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"name\": \"PowerDuke\",\n",
- " \"description\": \"PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0139\",\n",
- " \"external_id\": \"S0139\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Volexity PowerDuke November 2016\",\n",
- " \"description\": \"Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.\",\n",
- " \"url\": \"https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n"
- ]
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "malware",\n",
+ " "id": "malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a",\n",
+ " "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",\n",
+ " "created": "2017-05-31T21:33:19.746Z",\n",
+ " "modified": "2017-05-31T21:33:19.746Z",\n",
+ " "name": "PowerDuke",\n",
+ " "description": "PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]",\n",
+ " "labels": [\n",
+ " "malware"\n",
+ " ],\n",
+ " "external_references": [\n",
+ " {\n",
+ " "source_name": "mitre-attack",\n",
+ " "url": "https://attack.mitre.org/wiki/Software/S0139",\n",
+ " "external_id": "S0139"\n",
+ " },\n",
+ " {\n",
+ " "source_name": "Volexity PowerDuke November 2016",\n",
+ " "description": "Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.",\n",
+ " "url": "https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/"\n",
+ " }\n",
+ " ],\n",
+ " "object_marking_refs": [\n",
+ " "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
"from stix2 import FileSystemStore\n",
"\n",
- "\"\"\"\n",
- "Working with the FileSystemStore, where STIX content can be retrieved and pushed to a file system.\n",
- "\"\"\"\n",
- "\n",
"# create FileSystemStore\n",
- "fs = FileSystemStore(\"/home/michael/Desktop/sample_stix2_data\")\n",
+ "fs = FileSystemStore(\"/tmp/stix2_store\")\n",
"\n",
"# retrieve STIX2 content from FileSystemStore\n",
"ap = fs.get(\"attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6\")\n",
@@ -221,54 +293,128 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "#### FileSystemSource - (if STIX content is only to be retrieved from FileSystem)"
+ "#### FileSystemSource\n",
+ "\n",
+ "Use the FileSystemSource when you only want to retrieve STIX content from the file system."
]
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 6,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"type\": \"attack-pattern\",\n",
- " \"id\": \"attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:30:54.176Z\",\n",
- " \"modified\": \"2017-05-31T21:30:54.176Z\",\n",
- " \"name\": \"Indicator Removal from Tools\",\n",
- " \"description\": \"If a malicious...command-line parameters, Process monitoring\",\n",
- " \"kill_chain_phases\": [\n",
- " {\n",
- " \"kill_chain_name\": \"mitre-attack\",\n",
- " \"phase_name\": \"defense-evasion\"\n",
- " }\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Technique/T1066\",\n",
- " \"external_id\": \"T1066\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n"
- ]
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "attack-pattern",\n",
+ " "id": "attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6",\n",
+ " "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",\n",
+ " "created": "2017-05-31T21:30:54.176Z",\n",
+ " "modified": "2017-05-31T21:30:54.176Z",\n",
+ " "name": "Indicator Removal from Tools",\n",
+ " "description": "If a malicious...command-line parameters, Process monitoring",\n",
+ " "kill_chain_phases": [\n",
+ " {\n",
+ " "kill_chain_name": "mitre-attack",\n",
+ " "phase_name": "defense-evasion"\n",
+ " }\n",
+ " ],\n",
+ " "external_references": [\n",
+ " {\n",
+ " "source_name": "mitre-attack",\n",
+ " "url": "https://attack.mitre.org/wiki/Technique/T1066",\n",
+ " "external_id": "T1066"\n",
+ " }\n",
+ " ],\n",
+ " "object_marking_refs": [\n",
+ " "marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 6,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
"from stix2 import FileSystemSource\n",
- "\"\"\"\n",
- "Working with FileSystemSource for retrieving STIX content.\n",
- "\"\"\"\n",
"\n",
"# create FileSystemSource\n",
- "fs_source = FileSystemSource(\"/home/michael/Desktop/sample_stix2_data\")\n",
+ "fs_source = FileSystemSource(\"/tmp/stix2_source\")\n",
"\n",
"# retrieve STIX 2 objects\n",
"ap = fs_source.get(\"attack-pattern--00d0b012-8a03-410e-95de-5826bf542de6\")\n",
@@ -279,149 +425,336 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 7,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--0f862b01-99da-47cc-9bdb-db4a86a95bb1\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:32:54.772Z\",\n",
- " \"modified\": \"2017-05-31T21:32:54.772Z\",\n",
- " \"name\": \"Emissary\",\n",
- " \"description\": \"Emissary is a Trojan that has been used by Lotus Blossom. It shares code with Elise, with both Trojans being part of a malware group referred to as LStudio.[[Citation: Lotus Blossom Dec 2015]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0082\",\n",
- " \"external_id\": \"S0082\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Lotus Blossom Dec 2015\",\n",
- " \"description\": \"Falcone, R. and Miller-Osborn, J.. (2015, December 18). Attack on French Diplomat Linked to Operation Lotus Blossom. Retrieved February 15, 2016.\",\n",
- " \"url\": \"http://researchcenter.paloaltonetworks.com/2015/12/attack-on-french-diplomat-linked-to-operation-lotus-blossom/\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n",
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--2a6f4c7b-e690-4cc7-ab6b-1f821fb6b80b\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:32:33.348Z\",\n",
- " \"modified\": \"2017-05-31T21:32:33.348Z\",\n",
- " \"name\": \"LOWBALL\",\n",
- " \"description\": \"LOWBALL is malware used by admin@338. It was used in August 2015 in email messages targeting Hong Kong-based media organizations.[[Citation: FireEye admin@338]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0042\",\n",
- " \"external_id\": \"S0042\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"FireEye admin@338\",\n",
- " \"description\": \"FireEye Threat Intelligence. (2015, December 1). China-based Cyber Threat Group Uses Dropbox for Malware Communications and Targets Hong Kong Media Outlets. Retrieved December 4, 2015.\",\n",
- " \"url\": \"https://www.fireeye.com/blog/threat-research/2015/11/china-based-threat.html\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n",
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"modified\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"name\": \"PowerDuke\",\n",
- " \"description\": \"PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0139\",\n",
- " \"external_id\": \"S0139\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Volexity PowerDuke November 2016\",\n",
- " \"description\": \"Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.\",\n",
- " \"url\": \"https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n",
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--0db09158-6e48-4e7c-8ce7-2b10b9c0c039\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:32:55.126Z\",\n",
- " \"modified\": \"2017-05-31T21:32:55.126Z\",\n",
- " \"name\": \"Misdat\",\n",
- " \"description\": \"Misdat is a backdoor that was used by Dust Storm from 2010 to 2011.[[Citation: Cylance Dust Storm]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0083\",\n",
- " \"external_id\": \"S0083\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Cylance Dust Storm\",\n",
- " \"description\": \"Gross, J. (2016, February 23). Operation Dust Storm. Retrieved February 25, 2016.\",\n",
- " \"url\": \"https://www.cylance.com/hubfs/2015%20cylance%20website/assets/operation-dust-storm/Op%20Dust%20Storm%20Report.pdf?t=1456259131512\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n",
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--1d808f62-cf63-4063-9727-ff6132514c22\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:33:06.433Z\",\n",
- " \"modified\": \"2017-05-31T21:33:06.433Z\",\n",
- " \"name\": \"WEBC2\",\n",
- " \"description\": \"WEBC2 is a backdoor used by APT1 to retrieve a Web page from a predetermined C2 server.[[Citation: Mandiant APT1 Appendix]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0109\",\n",
- " \"external_id\": \"S0109\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Mandiant APT1 Appendix\",\n",
- " \"description\": \"Mandiant. (n.d.). Appendix C (Digital) - The Malware Arsenal. Retrieved July 18, 2016.\",\n",
- " \"url\": \"https://www.fireeye.com/content/dam/fireeye-www/services/pdfs/mandiant-apt1-report-appendix.zip\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n"
- ]
+ "data": {
+ "text/html": [
+ "malware--96b08451-b27a-4ff6-893f-790e26393a8e\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "malware--b42378e0-f147-496f-992a-26a49705395b\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "malware--6b616fc1-1505-48e3-8b2c-0d19337bff38\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "malware--92ec0cbd-2c30-44a2-b270-73f4ec949841\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -434,46 +767,95 @@
"mals = fs_source.query(query)\n",
"\n",
"for mal in mals:\n",
- " print(mal)"
+ " print(mal.id)"
]
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 8,
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "{\n",
- " \"type\": \"malware\",\n",
- " \"id\": \"malware--00c3bfcb-99bd-4767-8c03-b08f585f5c8a\",\n",
- " \"created_by_ref\": \"identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5\",\n",
- " \"created\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"modified\": \"2017-05-31T21:33:19.746Z\",\n",
- " \"name\": \"PowerDuke\",\n",
- " \"description\": \"PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.[[Citation: Volexity PowerDuke November 2016]]\",\n",
- " \"labels\": [\n",
- " \"malware\"\n",
- " ],\n",
- " \"external_references\": [\n",
- " {\n",
- " \"source_name\": \"mitre-attack\",\n",
- " \"url\": \"https://attack.mitre.org/wiki/Software/S0139\",\n",
- " \"external_id\": \"S0139\"\n",
- " },\n",
- " {\n",
- " \"source_name\": \"Volexity PowerDuke November 2016\",\n",
- " \"description\": \"Adair, S.. (2016, November 9). PowerDuke: Widespread Post-Election Spear Phishing Campaigns Targeting Think Tanks and NGOs. Retrieved January 11, 2017.\",\n",
- " \"url\": \"https://www.volexity.com/blog/2016/11/09/powerduke-post-election-spear-phishing-campaigns-targeting-think-tanks-and-ngos/\"\n",
- " }\n",
- " ],\n",
- " \"object_marking_refs\": [\n",
- " \"marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168\"\n",
- " ]\n",
- "}\n"
- ]
+ "data": {
+ "text/html": [
+ "malware--92ec0cbd-2c30-44a2-b270-73f4ec949841\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -484,30 +866,28 @@
"\n",
"# for visual purposes\n",
"for mal in mals:\n",
- " print(mal)"
+ " print(mal.id)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
- "#### FileSystemSink - (if STIX content is only to be pushed to FileSystem)"
+ "#### FileSystemSink\n",
+ "\n",
+ "Use the FileSystemSink when you only want to push STIX content to the file system."
]
},
{
"cell_type": "code",
- "execution_count": 7,
- "metadata": {
- "collapsed": true
- },
+ "execution_count": 10,
+ "metadata": {},
"outputs": [],
"source": [
- "from stix2 import FileSystemSink, Campaign\n",
- "\"\"\"\n",
- "Working with FileSystemSink for pushing STIX content.\n",
- "\"\"\"\n",
+ "from stix2 import FileSystemSink, Campaign, Indicator\n",
+ "\n",
"# create FileSystemSink\n",
- "fs_sink = FileSystemSink(\"/home/michael/Desktop/sample_stix2_data\")\n",
+ "fs_sink = FileSystemSink(\"/tmp/stix2_sink\")\n",
"\n",
"# create STIX objects and add to sink\n",
"camp = Campaign(name=\"The Crusades\",\n",
@@ -532,21 +912,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/markings.ipynb b/docs/guide/markings.ipynb
index fcacf47..8230daf 100644
--- a/docs/guide/markings.ipynb
+++ b/docs/guide/markings.ipynb
@@ -2,9 +2,8 @@
"cells": [
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 6,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -23,9 +22,8 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 5,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -68,127 +68,127 @@
"To create an object with a (predefined) TLP marking to an object, just provide it as a keyword argument to the constructor. The TLP markings can easily be imported from python-stix2."
]
},
- {
- "cell_type": "code",
- "execution_count": 3,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "{\n",
- " "type": "indicator",\n",
- " "id": "indicator--65ff0082-bb92-4812-9b74-b144b858297f",\n",
- " "created": "2017-11-13T14:42:14.641Z",\n",
- " "modified": "2017-11-13T14:42:14.641Z",\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-11-13T14:42:14.641818Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
- " "object_marking_refs": [\n",
- " "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"\n",
- " ]\n",
- "}\n",
- "
\n"
- ],
- "text/plain": [
- ""
- ]
- },
- "execution_count": 3,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from stix2 import Indicator, TLP_AMBER\n",
- "\n",
- "indicator = Indicator(labels=[\"malicious-activity\"],\n",
- " pattern=\"[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']\",\n",
- " object_marking_refs=TLP_AMBER)\n",
- "print(indicator)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "If you’re creating your own marking (for example, a ``Statement`` marking), first create the statement marking:"
- ]
- },
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "indicator",\n",
+ " "id": "indicator--95a71cff-fad0-4ffb-a641-8a6eaa642290",\n",
+ " "created": "2018-04-05T19:49:47.924Z",\n",
+ " "modified": "2018-04-05T19:49:47.924Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:47.924708Z",\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ],\n",
+ " "object_marking_refs": [\n",
+ " "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "from stix2 import Indicator, TLP_AMBER\n",
+ "\n",
+ "indicator = Indicator(labels=[\"malicious-activity\"],\n",
+ " pattern=\"[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']\",\n",
+ " object_marking_refs=TLP_AMBER)\n",
+ "print(indicator)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If you’re creating your own marking (for example, a ``Statement`` marking), first create the statement marking:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
"outputs": [
{
"data": {
@@ -263,8 +263,8 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "marking-definition",\n",
- " "id": "marking-definition--d16f0975-c5dd-4b25-a41d-af4afcc5da92",\n",
- " "created": "2017-11-13T14:43:30.558058Z",\n",
+ " "id": "marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368",\n",
+ " "created": "2018-04-05T19:49:53.98008Z",\n",
" "definition_type": "statement",\n",
" "definition": {\n",
" "statement": "Copyright 2017, Example Corp"\n",
@@ -276,7 +276,7 @@
""
]
},
- "execution_count": 7,
+ "execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
@@ -300,7 +300,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 9,
"metadata": {},
"outputs": [
{
@@ -376,16 +376,16 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--526cda4e-6745-4cd6-852f-0750c6a79784",\n",
- " "created": "2017-10-04T14:43:09.586Z",\n",
- " "modified": "2017-10-04T14:43:09.586Z",\n",
+ " "id": "indicator--7caeab49-2472-41bb-a988-2f990aea99bd",\n",
+ " "created": "2018-04-05T19:49:55.763Z",\n",
+ " "modified": "2018-04-05T19:49:55.763Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:55.763364Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:43:09.586133Z",\n",
" "object_marking_refs": [\n",
- " "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"\n",
+ " "marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368"\n",
" ]\n",
"}\n",
"
\n"
@@ -394,7 +394,7 @@
""
]
},
- "execution_count": 5,
+ "execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
@@ -408,7 +408,7 @@
},
{
"cell_type": "code",
- "execution_count": 6,
+ "execution_count": 10,
"metadata": {},
"outputs": [
{
@@ -484,14 +484,14 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--1505b789-fcd2-48ee-bea9-3b20627a4abd",\n",
- " "created": "2017-10-04T14:43:20.049Z",\n",
- " "modified": "2017-10-04T14:43:20.049Z",\n",
+ " "id": "indicator--4eb21bbe-b8a9-4348-86cf-1ed52f9abdd7",\n",
+ " "created": "2018-04-05T19:49:57.248Z",\n",
+ " "modified": "2018-04-05T19:49:57.248Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:57.248658Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:43:20.049166Z",\n",
" "object_marking_refs": [\n",
" "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"\n",
" ]\n",
@@ -502,7 +502,7 @@
""
]
},
- "execution_count": 6,
+ "execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
@@ -523,7 +523,7 @@
},
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": 11,
"metadata": {},
"outputs": [
{
@@ -599,9 +599,9 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "malware",\n",
- " "id": "malware--f7128008-f6ab-4d43-a8a2-a681651268f8",\n",
- " "created": "2017-11-13T14:43:34.857Z",\n",
- " "modified": "2017-11-13T14:43:34.857Z",\n",
+ " "id": "malware--ef1eddbb-b5a5-47e0-b607-75b9870d8d91",\n",
+ " "created": "2018-04-05T19:49:59.103Z",\n",
+ " "modified": "2018-04-05T19:49:59.103Z",\n",
" "name": "Poison Ivy",\n",
" "description": "A ransomware related to ...",\n",
" "labels": [\n",
@@ -609,7 +609,7 @@
" ],\n",
" "granular_markings": [\n",
" {\n",
- " "marking_ref": "marking-definition--d16f0975-c5dd-4b25-a41d-af4afcc5da92",\n",
+ " "marking_ref": "marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368",\n",
" "selectors": [\n",
" "description"\n",
" ]\n",
@@ -628,7 +628,7 @@
""
]
},
- "execution_count": 8,
+ "execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
@@ -661,7 +661,7 @@
},
{
"cell_type": "code",
- "execution_count": 8,
+ "execution_count": 12,
"metadata": {},
"outputs": [
{
@@ -705,7 +705,7 @@
},
{
"cell_type": "code",
- "execution_count": 21,
+ "execution_count": 13,
"metadata": {},
"outputs": [
{
@@ -781,17 +781,17 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",\n",
- " "created": "2017-10-04T14:42:54.685Z",\n",
- " "modified": "2017-10-04T15:03:46.599Z",\n",
+ " "id": "indicator--95a71cff-fad0-4ffb-a641-8a6eaa642290",\n",
+ " "created": "2018-04-05T19:49:47.924Z",\n",
+ " "modified": "2018-04-05T19:50:03.387Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:47.924708Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:42:54.685184Z",\n",
" "object_marking_refs": [\n",
- " "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82",\n",
- " "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"\n",
+ " "marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368",\n",
+ " "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"\n",
" ]\n",
"}\n",
"
\n"
@@ -800,7 +800,7 @@
""
]
},
- "execution_count": 21,
+ "execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
@@ -819,7 +819,7 @@
},
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": 14,
"metadata": {},
"outputs": [
{
@@ -895,14 +895,14 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",\n",
- " "created": "2017-10-04T14:42:54.685Z",\n",
- " "modified": "2017-10-04T15:03:54.290Z",\n",
+ " "id": "indicator--95a71cff-fad0-4ffb-a641-8a6eaa642290",\n",
+ " "created": "2018-04-05T19:49:47.924Z",\n",
+ " "modified": "2018-04-05T19:50:05.109Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:47.924708Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:42:54.685184Z",\n",
" "object_marking_refs": [\n",
" "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"\n",
" ]\n",
@@ -913,7 +913,7 @@
""
]
},
- "execution_count": 22,
+ "execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
@@ -932,7 +932,7 @@
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": 15,
"metadata": {},
"outputs": [
{
@@ -1008,17 +1008,17 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",\n",
- " "created": "2017-10-04T14:42:54.685Z",\n",
- " "modified": "2017-10-04T15:04:04.218Z",\n",
+ " "id": "indicator--95a71cff-fad0-4ffb-a641-8a6eaa642290",\n",
+ " "created": "2018-04-05T19:49:47.924Z",\n",
+ " "modified": "2018-04-05T19:50:06.773Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:47.924708Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:42:54.685184Z",\n",
" "object_marking_refs": [\n",
- " "marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da",\n",
- " "marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53"\n",
+ " "marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368",\n",
+ " "marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da"\n",
" ]\n",
"}\n",
"
\n"
@@ -1027,7 +1027,7 @@
""
]
},
- "execution_count": 23,
+ "execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
@@ -1048,7 +1048,7 @@
},
{
"cell_type": "code",
- "execution_count": 12,
+ "execution_count": 16,
"metadata": {},
"outputs": [
{
@@ -1124,14 +1124,14 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--409a0b15-1108-4251-8aee-a08995976561",\n",
- " "created": "2017-10-04T14:42:54.685Z",\n",
- " "modified": "2017-10-04T14:54:39.331Z",\n",
+ " "id": "indicator--95a71cff-fad0-4ffb-a641-8a6eaa642290",\n",
+ " "created": "2018-04-05T19:49:47.924Z",\n",
+ " "modified": "2018-04-05T19:50:08.616Z",\n",
+ " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
+ " "valid_from": "2018-04-05T19:49:47.924708Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
- " ],\n",
- " "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-10-04T14:42:54.685184Z"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -1139,7 +1139,7 @@
""
]
},
- "execution_count": 12,
+ "execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
@@ -1167,17 +1167,17 @@
},
{
"cell_type": "code",
- "execution_count": 19,
+ "execution_count": 17,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
- "['marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da',\n",
- " 'marking-definition--030bb5c6-c5eb-4e9c-8e7a-b9aab08ded53']"
+ "['marking-definition--13680b12-3d19-4b42-abe6-0d31effe5368',\n",
+ " 'marking-definition--34098fce-860f-48ae-8e50-ebd3cc5e41da']"
]
},
- "execution_count": 19,
+ "execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
@@ -1195,7 +1195,7 @@
},
{
"cell_type": "code",
- "execution_count": 9,
+ "execution_count": 18,
"metadata": {},
"outputs": [
{
@@ -1204,7 +1204,7 @@
"['marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9']"
]
},
- "execution_count": 9,
+ "execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
@@ -1224,7 +1224,7 @@
},
{
"cell_type": "code",
- "execution_count": 14,
+ "execution_count": 19,
"metadata": {},
"outputs": [
{
@@ -1233,7 +1233,7 @@
"['marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9']"
]
},
- "execution_count": 14,
+ "execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
@@ -1251,7 +1251,7 @@
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 20,
"metadata": {},
"outputs": [
{
@@ -1260,7 +1260,7 @@
"True"
]
},
- "execution_count": 16,
+ "execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
@@ -1271,7 +1271,7 @@
},
{
"cell_type": "code",
- "execution_count": 17,
+ "execution_count": 21,
"metadata": {},
"outputs": [
{
@@ -1280,7 +1280,7 @@
"True"
]
},
- "execution_count": 17,
+ "execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
@@ -1291,7 +1291,7 @@
},
{
"cell_type": "code",
- "execution_count": 18,
+ "execution_count": 22,
"metadata": {
"scrolled": true
},
@@ -1302,7 +1302,7 @@
"False"
]
},
- "execution_count": 18,
+ "execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
@@ -1314,21 +1314,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/memory.ipynb b/docs/guide/memory.ipynb
index 79a8f33..6b6d5cb 100644
--- a/docs/guide/memory.ipynb
+++ b/docs/guide/memory.ipynb
@@ -4,7 +4,6 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -25,7 +24,6 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -151,12 +151,12 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--2f61e4e7-0891-4e09-b79a-66f5e594fec0",\n",
- " "created": "2017-11-17T17:01:31.590Z",\n",
- " "modified": "2017-11-17T17:01:31.590Z",\n",
+ " "id": "indicator--41a960c7-a6d4-406d-9156-0069cb3bd40d",\n",
+ " "created": "2018-04-05T19:50:41.222Z",\n",
+ " "modified": "2018-04-05T19:50:41.222Z",\n",
" "description": "Crusades C2 implant",\n",
" "pattern": "[file:hashes.'SHA-256' = '54b7e05e39a59428743635242e4a867c932140a999f52a1e54fa7ee6a440c73b']",\n",
- " "valid_from": "2017-11-17T17:01:31.590939Z",\n",
+ " "valid_from": "2018-04-05T19:50:41.222522Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ]\n",
@@ -267,12 +267,12 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--ddb765ba-ff1e-4285-bf33-1f6d08f583d6",\n",
- " "created": "2017-11-17T17:01:31.799Z",\n",
- " "modified": "2017-11-17T17:01:31.799Z",\n",
+ " "id": "indicator--ba2a7acb-a3ac-420b-9288-09988aa99408",\n",
+ " "created": "2018-04-05T19:50:43.343Z",\n",
+ " "modified": "2018-04-05T19:50:43.343Z",\n",
" "description": "Crusades stage 2 implant variant",\n",
" "pattern": "[file:hashes.'SHA-256' = '31a45e777e4d58b97f4c43e38006f8cd6580ddabc4037905b2fad734712b582c']",\n",
- " "valid_from": "2017-11-17T17:01:31.799228Z",\n",
+ " "valid_from": "2018-04-05T19:50:43.343298Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ]\n",
@@ -386,9 +386,9 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "malware",\n",
- " "id": "malware--e8170e70-522f-4ec3-aa22-afb55bfad0b0",\n",
- " "created": "2017-11-17T17:01:31.806Z",\n",
- " "modified": "2017-11-17T17:01:31.806Z",\n",
+ " "id": "malware--9e9b87ce-2b2b-455a-8d5b-26384ccc8d52",\n",
+ " "created": "2018-04-05T19:50:43.346Z",\n",
+ " "modified": "2018-04-05T19:50:43.346Z",\n",
" "name": "Alexios",\n",
" "labels": [\n",
" "rootkit"\n",
@@ -412,120 +412,6 @@
"print(mal)"
]
},
- {
- "cell_type": "code",
- "execution_count": 6,
- "metadata": {},
- "outputs": [
- {
- "data": {
- "text/html": [
- "{\n",
- " "type": "report",\n",
- " "id": "report--2add14d6-bbf3-4308-bb8e-226d314a08e4",\n",
- " "created": "2017-05-08T18:34:08.042Z",\n",
- " "modified": "2017-05-08T18:34:08.042Z",\n",
- " "name": "The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.",\n",
- " "published": "2017-05-08T10:24:11.011Z",\n",
- " "object_refs": [\n",
- " "malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4"\n",
- " ],\n",
- " "labels": [\n",
- " "threat-report"\n",
- " ]\n",
- "}\n",
- "
\n"
- ],
- "text/plain": [
- ""
- ]
- },
- "execution_count": 6,
- "metadata": {},
- "output_type": "execute_result"
- }
- ],
- "source": [
- "from stix2 import Filter\n",
- "\n",
- "# add json formatted string to MemoryStore\n",
- "# Again, would NOT manually create json-formatted string\n",
- "# but taken as an output form from another source\n",
- "report = '{\"type\": \"report\",\"id\": \"report--2add14d6-bbf3-4308-bb8e-226d314a08e4\",\"labels\": [\"threat-report\"], \"name\": \"The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.\", \"published\": \"2017-05-08T10:24:11.011Z\", \"object_refs\":[\"malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4\"], \"created\": \"2017-05-08T18:34:08.042Z\", \"modified\": \"2017-05-08T18:34:08.042Z\"}'\n",
- "\n",
- "mem.add(report)\n",
- "\n",
- "print(mem.get(\"report--2add14d6-bbf3-4308-bb8e-226d314a08e4\"))"
- ]
- },
{
"cell_type": "markdown",
"metadata": {},
@@ -610,17 +496,13 @@
".highlight .vi { color: #19177C } /* Name.Variable.Instance */\n",
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
- " "type": "report",\n",
- " "id": "report--2add14d6-bbf3-4308-bb8e-226d314a08e4",\n",
- " "created": "2017-05-08T18:34:08.042Z",\n",
- " "modified": "2017-05-08T18:34:08.042Z",\n",
- " "name": "The Crusades: Looking into the relentless infiltration of Israels digital infrastructure.",\n",
- " "published": "2017-05-08T10:24:11.011Z",\n",
- " "object_refs": [\n",
- " "malware--2daa14d6-cbf3-4308-bb8e-226d324a08e4"\n",
- " ],\n",
+ " "type": "malware",\n",
+ " "id": "malware--9e9b87ce-2b2b-455a-8d5b-26384ccc8d52",\n",
+ " "created": "2018-04-05T19:50:43.346Z",\n",
+ " "modified": "2018-04-05T19:50:43.346Z",\n",
+ " "name": "Alexios",\n",
" "labels": [\n",
- " "threat-report"\n",
+ " "rootkit"\n",
" ]\n",
"}\n",
"
\n"
@@ -643,30 +525,30 @@
"# load(add) STIX content from json file into MemoryStore\n",
"mem_2.load_from_file(\"path_to_target_file.json\")\n",
"\n",
- "report = mem_2.get(\"report--2add14d6-bbf3-4308-bb8e-226d314a08e4\")\n",
+ "report = mem_2.get(\"malware--9e9b87ce-2b2b-455a-8d5b-26384ccc8d52\")\n",
"\n",
- "# for visualpurposes\n",
+ "# for visual purposes\n",
"print(report)"
]
}
],
"metadata": {
"kernelspec": {
- "display_name": "cti-python-stix2",
+ "display_name": "Python 3",
"language": "python",
- "name": "cti-python-stix2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/parsing.ipynb b/docs/guide/parsing.ipynb
index d24f994..4bd026f 100644
--- a/docs/guide/parsing.ipynb
+++ b/docs/guide/parsing.ipynb
@@ -4,7 +4,6 @@
"cell_type": "code",
"execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -25,7 +24,6 @@
"cell_type": "code",
"execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -63,7 +63,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Parsing STIX content is as easy as calling the [parse()](../api/stix2.core.rst#stix2.core.parse) function on a JSON string. It will automatically determine the type of the object. The STIX objects within `bundle` objects, and the cyber observables contained within `observed-data` objects will be parsed as well."
+ "Parsing STIX content is as easy as calling the [parse()](../api/stix2.core.rst#stix2.core.parse) function on a JSON string, dictionary, or file-like object. It will automatically determine the type of the object. The STIX objects within `bundle` objects, and the cyber observables contained within `observed-data` objects will be parsed as well.\n",
+ "\n",
+ "**Parsing a string**"
]
},
{
@@ -72,12 +74,184 @@
"metadata": {},
"outputs": [
{
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "observed-data\n",
- "0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038\n"
- ]
+ "data": {
+ "text/html": [
+ "<class 'stix2.v20.sdo.ObservedData'>\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "observed-data",\n",
+ " "id": "observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf",\n",
+ " "created": "2016-04-06T19:58:16.000Z",\n",
+ " "modified": "2016-04-06T19:58:16.000Z",\n",
+ " "first_observed": "2015-12-21T19:00:00Z",\n",
+ " "last_observed": "2015-12-21T19:00:00Z",\n",
+ " "number_observed": 50,\n",
+ " "objects": {\n",
+ " "0": {\n",
+ " "type": "file",\n",
+ " "hashes": {\n",
+ " "SHA-256": "0969de02ecf8a5f003e3f6d063d848c8a193aada092623f8ce408c15bcb5f038"\n",
+ " }\n",
+ " }\n",
+ " }\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 3,
+ "metadata": {},
+ "output_type": "execute_result"
}
],
"source": [
@@ -102,28 +276,462 @@
"}\"\"\"\n",
"\n",
"obj = parse(input_string)\n",
- "print(obj.type)\n",
- "print(obj.objects[\"0\"].hashes['SHA-256'])"
+ "print(type(obj))\n",
+ "print(obj)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Parsing a dictionary**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<class 'stix2.v20.sdo.Identity'>\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "identity",\n",
+ " "id": "identity--311b2d2d-f010-5473-83ec-1edf84858f4c",\n",
+ " "created": "2015-12-21T19:59:11.000Z",\n",
+ " "modified": "2015-12-21T19:59:11.000Z",\n",
+ " "name": "Cole Powers",\n",
+ " "identity_class": "individual"\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 4,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "input_dict = {\n",
+ " \"type\": \"identity\",\n",
+ " \"id\": \"identity--311b2d2d-f010-5473-83ec-1edf84858f4c\",\n",
+ " \"created\": \"2015-12-21T19:59:11Z\",\n",
+ " \"modified\": \"2015-12-21T19:59:11Z\",\n",
+ " \"name\": \"Cole Powers\",\n",
+ " \"identity_class\": \"individual\"\n",
+ "}\n",
+ "\n",
+ "obj = parse(input_dict)\n",
+ "print(type(obj))\n",
+ "print(obj)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**Parsing a file-like object**"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "<class 'stix2.v20.sdo.CourseOfAction'>\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "course-of-action",\n",
+ " "id": "course-of-action--d9727aee-48b8-4fdb-89e2-4c49746ba4dd",\n",
+ " "created_by_ref": "identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5",\n",
+ " "created": "2017-05-31T21:30:41.022Z",\n",
+ " "modified": "2017-05-31T21:30:41.022Z",\n",
+ " "name": "Data from Network Shared Drive Mitigation",\n",
+ " "description": "Identify unnecessary system utilities or potentially malicious software that may be used to collect data from a network share, and audit and/or block them by using whitelisting[[CiteRef::Beechey 2010]] tools, like AppLocker,[[CiteRef::Windows Commands JPCERT]][[CiteRef::NSA MS AppLocker]] or Software Restriction Policies[[CiteRef::Corio 2008]] where appropriate.[[CiteRef::TechNet Applocker vs SRP]]"\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 5,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "file_handle = open(\"/tmp/stix2_store/course-of-action/course-of-action--d9727aee-48b8-4fdb-89e2-4c49746ba4dd.json\")\n",
+ "\n",
+ "obj = parse(file_handle)\n",
+ "print(type(obj))\n",
+ "print(obj)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Parsing Custom STIX Content"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.core.rst#stix2.core.parse). Just supply the keyword argument ``allow_custom=True``. When ``allow_custom`` is specified, [parse()](../api/stix2.core.rst#stix2.core.parse) will attempt to convert the supplied STIX content to known STIX 2 domain objects and/or previously defined [custom STIX 2 objects](custom.ipynb). If the conversion cannot be completed (and ``allow_custom`` is specified), [parse()](../api/stix2.core.rst#stix2.core.parse) will treat the supplied STIX 2 content as valid STIX 2 objects and return them. **Warning: Specifying allow_custom may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom content occurs where the custom content supplied is not valid STIX 2**. This is an axiomatic possibility as the ``stix2`` library cannot guarantee proper processing of unknown custom STIX 2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n",
+ "\n",
+ "For examples of parsing STIX 2 objects with custom STIX properties, see [Custom STIX Content: Custom Properties](custom.ipynb#Custom-Properties)\n",
+ "\n",
+ "For examples of parsing defined custom STIX 2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n",
+ "\n",
+ "For retrieving STIX 2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX 2 content unknown to the user, the user can create a STIX 2 DataStore/Source with the flag ``allow_custom=True``. As mentioned, this will configure the DataStore/Source to allow for unknown STIX 2 content to be returned (albeit not converted to full STIX 2 domain objects and properties); the ``stix2`` library may preclude processing the unknown content, if the content is not valid or actual STIX 2 domain objects and properties."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "collapsed": true
+ },
+ "outputs": [],
+ "source": [
+ "from taxii2client import Collection\n",
+ "from stix2 import CompositeDataSource, FileSystemSource, TAXIICollectionSource\n",
+ "\n",
+ "# to allow for the retrieval of unknown custom STIX2 content,\n",
+ "# just create *Stores/*Sources with the 'allow_custom' flag\n",
+ "\n",
+ "# create FileSystemStore\n",
+ "fs = FileSystemSource(\"/path/to/stix2_data/\", allow_custom=True)\n",
+ "\n",
+ "# create TAXIICollectionSource\n",
+ "colxn = Collection('http://taxii_url')\n",
+ "ts = TAXIICollectionSource(colxn, allow_custom=True)"
]
}
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/serializing.ipynb b/docs/guide/serializing.ipynb
index 1046d9f..e58302e 100644
--- a/docs/guide/serializing.ipynb
+++ b/docs/guide/serializing.ipynb
@@ -31,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -142,12 +144,12 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--85773ceb-c768-45f6-bb04-b4d813809e48",\n",
- " "created": "2018-03-13T19:49:53.392Z",\n",
- " "modified": "2018-03-13T19:49:53.392Z",\n",
+ " "id": "indicator--4336ace8-d985-413a-8e32-f749ba268dc3",\n",
+ " "created": "2018-04-05T20:01:20.012Z",\n",
+ " "modified": "2018-04-05T20:01:20.012Z",\n",
" "name": "File hash for malware variant",\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2018-03-13T19:49:53.392627Z",\n",
+ " "valid_from": "2018-04-05T20:01:20.012209Z",\n",
" "labels": [\n",
" "malicious-activity"\n",
" ]\n",
@@ -256,7 +258,7 @@
".highlight .vg { color: #19177C } /* Name.Variable.Global */\n",
".highlight .vi { color: #19177C } /* Name.Variable.Instance */\n",
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
- ".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{"name": "File hash for malware variant", "labels": ["malicious-activity"], "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']", "type": "indicator", "id": "indicator--85773ceb-c768-45f6-bb04-b4d813809e48", "created": "2018-03-13T19:49:53.392Z", "modified": "2018-03-13T19:49:53.392Z", "valid_from": "2018-03-13T19:49:53.392627Z"}\n",
+ ".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{"name": "File hash for malware variant", "labels": ["malicious-activity"], "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']", "type": "indicator", "id": "indicator--4336ace8-d985-413a-8e32-f749ba268dc3", "created": "2018-04-05T20:01:20.012Z", "modified": "2018-04-05T20:01:20.012Z", "valid_from": "2018-04-05T20:01:20.012209Z"}\n",
"
\n"
],
"text/plain": [
@@ -362,10 +364,10 @@
" ],\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
" "type": "indicator",\n",
- " "id": "indicator--85773ceb-c768-45f6-bb04-b4d813809e48",\n",
- " "created": "2018-03-13T19:49:53.392Z",\n",
- " "modified": "2018-03-13T19:49:53.392Z",\n",
- " "valid_from": "2018-03-13T19:49:53.392627Z"\n",
+ " "id": "indicator--4336ace8-d985-413a-8e32-f749ba268dc3",\n",
+ " "created": "2018-04-05T20:01:20.012Z",\n",
+ " "modified": "2018-04-05T20:01:20.012Z",\n",
+ " "valid_from": "2018-04-05T20:01:20.012209Z"\n",
"}\n",
"
\n"
],
diff --git a/docs/guide/taxii.ipynb b/docs/guide/taxii.ipynb
index 4045a98..4e81388 100644
--- a/docs/guide/taxii.ipynb
+++ b/docs/guide/taxii.ipynb
@@ -33,23 +33,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
diff --git a/docs/guide/ts_support.ipynb b/docs/guide/ts_support.ipynb
index 6ceef99..445e263 100644
--- a/docs/guide/ts_support.ipynb
+++ b/docs/guide/ts_support.ipynb
@@ -23,9 +23,8 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 1,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +32,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -58,7 +59,7 @@
"source": [
"## Technical Specification Support\n",
"\n",
- "### How imports will work\n",
+ "### How imports work\n",
"\n",
"Imports can be used in different ways depending on the use case and support levels.\n",
"\n",
@@ -228,7 +229,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### How parsing will work\n",
+ "### How parsing works\n",
"If the ``version`` positional argument is not provided. The data will be parsed using the latest version of STIX 2.X supported by the `stix2` library.\n",
"\n",
"You can lock your [parse()](../api/stix2.core.rst#stix2.core.parse) method to a specific STIX version by:"
@@ -236,7 +237,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 2,
"metadata": {},
"outputs": [
{
@@ -328,7 +329,7 @@
""
]
},
- "execution_count": 3,
+ "execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
@@ -362,7 +363,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "### How will custom content work\n",
+ "### How custom content works\n",
"\n",
"[CustomObject](../api/stix2.v20.sdo.rst#stix2.v20.sdo.CustomObject), [CustomObservable](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomObservable), [CustomMarking](../api/stix2.v20.common.rst#stix2.v20.common.CustomMarking) and [CustomExtension](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomExtension) must be registered explicitly by STIX version. This is a design decision since properties or requirements may change as the STIX Technical Specification advances.\n",
"\n",
@@ -396,21 +397,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/versioning.ipynb b/docs/guide/versioning.ipynb
index fb3b866..6074d00 100644
--- a/docs/guide/versioning.ipynb
+++ b/docs/guide/versioning.ipynb
@@ -2,9 +2,8 @@
"cells": [
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 2,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -23,9 +22,8 @@
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 3,
"metadata": {
- "collapsed": true,
"nbsphinx": "hidden"
},
"outputs": [],
@@ -33,23 +31,25 @@
"# JSON output syntax highlighting\n",
"from __future__ import print_function\n",
"from pygments import highlight\n",
- "from pygments.lexers import JsonLexer\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
"from pygments.formatters import HtmlFormatter\n",
- "from IPython.display import HTML\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
"\n",
- "original_print = print\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
"\n",
"def json_print(inpt):\n",
" string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
" if string[0] == '{':\n",
- " formatter = HtmlFormatter()\n",
- " return HTML('{}'.format(\n",
- " formatter.get_style_defs('.highlight'),\n",
- " highlight(string, JsonLexer(), formatter)))\n",
+ " lexer = JsonLexer()\n",
" else:\n",
- " original_print(inpt)\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
"\n",
- "print = json_print"
+ "globals()['print'] = json_print"
]
},
{
@@ -68,7 +68,7 @@
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -144,15 +144,15 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--92bb1ae4-db9c-4d6e-8ded-ef7280b4439a",\n",
+ " "id": "indicator--dd052ff6-e404-444b-beb9-eae96d1e79ea",\n",
" "created": "2016-01-01T08:00:00.000Z",\n",
- " "modified": "2017-09-26T23:39:07.149Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
+ " "modified": "2018-04-05T20:02:51.161Z",\n",
" "name": "File hash for Foobar malware",\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-09-26T23:39:07.132129Z"\n",
+ " "valid_from": "2018-04-05T20:02:51.138312Z",\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -160,7 +160,7 @@
""
]
},
- "execution_count": 3,
+ "execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@@ -187,7 +187,7 @@
},
{
"cell_type": "code",
- "execution_count": 4,
+ "execution_count": 5,
"metadata": {
"scrolled": true
},
@@ -216,7 +216,7 @@
},
{
"cell_type": "code",
- "execution_count": 5,
+ "execution_count": 6,
"metadata": {},
"outputs": [
{
@@ -292,16 +292,16 @@
".highlight .vm { color: #19177C } /* Name.Variable.Magic */\n",
".highlight .il { color: #666666 } /* Literal.Number.Integer.Long */{\n",
" "type": "indicator",\n",
- " "id": "indicator--92bb1ae4-db9c-4d6e-8ded-ef7280b4439a",\n",
+ " "id": "indicator--dd052ff6-e404-444b-beb9-eae96d1e79ea",\n",
" "created": "2016-01-01T08:00:00.000Z",\n",
- " "modified": "2017-09-26T23:39:09.463Z",\n",
- " "labels": [\n",
- " "malicious-activity"\n",
- " ],\n",
+ " "modified": "2018-04-05T20:02:54.704Z",\n",
" "name": "File hash for Foobar malware",\n",
" "pattern": "[file:hashes.md5 = 'd41d8cd98f00b204e9800998ecf8427e']",\n",
- " "valid_from": "2017-09-26T23:39:07.132129Z",\n",
- " "revoked": true\n",
+ " "valid_from": "2018-04-05T20:02:51.138312Z",\n",
+ " "revoked": true,\n",
+ " "labels": [\n",
+ " "malicious-activity"\n",
+ " ]\n",
"}\n",
"
\n"
],
@@ -309,7 +309,7 @@
""
]
},
- "execution_count": 5,
+ "execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
@@ -322,21 +322,21 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 2",
+ "display_name": "Python 3",
"language": "python",
- "name": "python2"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
- "version": 2
+ "version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
- "pygments_lexer": "ipython2",
- "version": "2.7.12"
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
}
},
"nbformat": 4,
diff --git a/docs/guide/workbench.ipynb b/docs/guide/workbench.ipynb
new file mode 100644
index 0000000..6594841
--- /dev/null
+++ b/docs/guide/workbench.ipynb
@@ -0,0 +1,801 @@
+{
+ "cells": [
+ {
+ "cell_type": "code",
+ "execution_count": 1,
+ "metadata": {
+ "nbsphinx": "hidden"
+ },
+ "outputs": [],
+ "source": [
+ "# Delete this cell to re-enable tracebacks\n",
+ "import sys\n",
+ "ipython = get_ipython()\n",
+ "\n",
+ "def hide_traceback(exc_tuple=None, filename=None, tb_offset=None,\n",
+ " exception_only=False, running_compiled_code=False):\n",
+ " etype, value, tb = sys.exc_info()\n",
+ " return ipython._showtraceback(etype, value, ipython.InteractiveTB.get_exception_only(etype, value))\n",
+ "\n",
+ "ipython.showtraceback = hide_traceback"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 2,
+ "metadata": {
+ "nbsphinx": "hidden"
+ },
+ "outputs": [],
+ "source": [
+ "# JSON output syntax highlighting\n",
+ "from __future__ import print_function\n",
+ "from pygments import highlight\n",
+ "from pygments.lexers import JsonLexer, TextLexer\n",
+ "from pygments.formatters import HtmlFormatter\n",
+ "from IPython.display import display, HTML\n",
+ "from IPython.core.interactiveshell import InteractiveShell\n",
+ "\n",
+ "InteractiveShell.ast_node_interactivity = \"all\"\n",
+ "\n",
+ "def json_print(inpt):\n",
+ " string = str(inpt)\n",
+ " formatter = HtmlFormatter()\n",
+ " if string[0] == '{':\n",
+ " lexer = JsonLexer()\n",
+ " else:\n",
+ " lexer = TextLexer()\n",
+ " return HTML('{}'.format(\n",
+ " formatter.get_style_defs('.highlight'),\n",
+ " highlight(string, lexer, formatter)))\n",
+ "\n",
+ "globals()['print'] = json_print"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Using The Workbench"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The [Workbench API](../api/stix2.workbench.rst) hides most of the complexity of the rest of the library to make it easy to interact with STIX data. To use it, just import everything from ``stix2.workbench``:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 3,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "from stix2.workbench import *"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Retrieving STIX Data\n",
+ "\n",
+ "To get some STIX data to work with, let's set up a DataSource and add it to our workbench."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 4,
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "from taxii2client import Collection\n",
+ "\n",
+ "collection = Collection(\"http://127.0.0.1:5000/trustgroup1/collections/91a7b528-80eb-42ed-a74d-c6fbd5a26116/\", user=\"admin\", password=\"Password0\")\n",
+ "tc_source = TAXIICollectionSource(collection)\n",
+ "add_data_source(tc_source)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "collapsed": true
+ },
+ "source": [
+ "Now we can get all of the indicators from the data source."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "response = indicators()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Similar functions are available for the other STIX Object types. See the full list [here](../api/stix2.workbench.rst#stix2.workbench.attack_patterns).\n",
+ "\n",
+ "If you want to only retrieve *some* indicators, you can pass in one or more [Filters](../api/datastore/stix2.datastore.filters.rst). This example finds all the indicators created by a specific identity:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 6,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "response = indicators(filters=Filter('created_by_ref', '=', 'identity--adede3e8-bf44-4e6f-b3c9-1958cbc3b188'))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The objects returned let you easily traverse their relationships. Get all Relationship objects involving that object with ``.relationships()``, all other objects related to this object with ``.related()``, and the Identity object for the creator of the object (if one exists) with ``.created_by()``. For full details on these methods and their arguments, see the [Workbench API](../api/stix2.workbench.rst) documentation."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "indicates\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ },
+ {
+ "data": {
+ "text/html": [
+ "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 7,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "for i in indicators():\n",
+ " for rel in i.relationships():\n",
+ " print(rel.source_ref)\n",
+ " print(rel.relationship_type)\n",
+ " print(rel.target_ref)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "malware",\n",
+ " "id": "malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111",\n",
+ " "created": "2017-01-27T13:49:53.997Z",\n",
+ " "modified": "2017-01-27T13:49:53.997Z",\n",
+ " "name": "Poison Ivy",\n",
+ " "description": "Poison Ivy",\n",
+ " "labels": [\n",
+ " "remote-access-trojan"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 8,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "for i in indicators():\n",
+ " for obj in i.related():\n",
+ " print(obj)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "If there are a lot of related objects, you can narrow it down by passing in one or more [Filters](../api/datastore/stix2.datastore.filters.rst) just as before. For example, if we want to get only the indicators related to a specific piece of malware (and not any entities that use it or are targeted by it):"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "{\n",
+ " "type": "indicator",\n",
+ " "id": "indicator--a932fcc6-e032-176c-126f-cb970a5a1ade",\n",
+ " "created": "2014-05-08T09:00:00.000Z",\n",
+ " "modified": "2014-05-08T09:00:00.000Z",\n",
+ " "name": "File hash for Poison Ivy variant",\n",
+ " "pattern": "[file:hashes.'SHA-256' = 'ef537f25c895bfa782526529a9b63d97aa631564d5d789c2b765448c8635fb6c']",\n",
+ " "valid_from": "2014-05-08T09:00:00Z",\n",
+ " "labels": [\n",
+ " "file-hash-watchlist"\n",
+ " ]\n",
+ "}\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 9,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "malware = get('malware--fdd60b30-b67c-11e3-b0b9-f01faf20d111')\n",
+ "indicator = malware.related(filters=Filter('type', '=', 'indicator'))\n",
+ "print(indicator[0])"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### Creating STIX Data\n",
+ "\n",
+ "To create a STIX object, just use that object's class constructor. Once it's created, add it to the workbench with [save()](../api/datastore/stix2.workbench.rst#stix2.workbench.save)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "identity = Identity(name=\"ACME Threat Intel Co.\", identity_class=\"organization\")\n",
+ "save(identity)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "You can also set defaults for certain properties when creating objects. For example, let's set the default creator to be the identity object we just created:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 11,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "set_default_creator(identity)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now when we create an indicator (or any other STIX Domain Object), it will automatically have the right ``create_by_ref`` value."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 12,
+ "metadata": {},
+ "outputs": [
+ {
+ "data": {
+ "text/html": [
+ "ACME Threat Intel Co.\n",
+ "
\n"
+ ],
+ "text/plain": [
+ ""
+ ]
+ },
+ "execution_count": 12,
+ "metadata": {},
+ "output_type": "execute_result"
+ }
+ ],
+ "source": [
+ "indicator = Indicator(labels=[\"malicious-activity\"], pattern=\"[file:hashes.MD5 = 'd41d8cd98f00b204e9800998ecf8427e']\")\n",
+ "save(indicator)\n",
+ "\n",
+ "indicator_creator = get(indicator.created_by_ref)\n",
+ "print(indicator_creator.name)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Defaults can also be set for the [created timestamp](../api/datastore/stix2.workbench.rst#stix2.workbench.set_default_created), [external references](../api/datastore/stix2.workbench.rst#stix2.workbench.set_default_external_refs) and [object marking references](../api/datastore/stix2.workbench.rst#stix2.workbench.set_default_object_marking_refs)."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "\n",
+ "**Warning:**\n",
+ "\n",
+ "The workbench layer replaces STIX Object classes with special versions of them that use \"wrappers\" to provide extra functionality. Because of this, we recommend that you **either use the workbench layer or the rest of the library, but not both**. In other words, don't import from both ``stix2.workbench`` and any other submodules of ``stix2``.\n",
+ "\n",
+ ""
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.6.3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/docs/index.rst b/docs/index.rst
index 62d07ff..023400e 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -3,8 +3,17 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
-Welcome to stix2's documentation!
-=================================
+STIX 2 Python API Documentation
+===============================
+
+Welcome to the STIX 2 Python API's documentation. This library is designed to
+help you work with STIX 2 content. For more information about STIX 2, see the
+`website `_ of the OASIS Cyber Threat Intelligence
+Technical Committee.
+
+Get started with an `overview `_ of the library, then take a look
+at the `guides and tutorials `_ to see how to use it. For information
+about a specific class or function, see the `API reference `_.
.. toctree::
:maxdepth: 3
@@ -13,8 +22,6 @@ Welcome to stix2's documentation!
overview
guide
api_ref
- datastore_api
- roadmap
contributing
diff --git a/docs/overview.rst b/docs/overview.rst
index 0306a22..0396ee8 100644
--- a/docs/overview.rst
+++ b/docs/overview.rst
@@ -4,7 +4,7 @@ Overview
Goals
-----
-High level goals/principles of the python-stix2 library:
+High level goals/principles of the Python ``stix2`` library:
1. It should be as easy as possible (but no easier!) to perform common tasks of
producing, consuming, and processing STIX 2 content.
@@ -17,22 +17,22 @@ Design Decisions
----------------
To accomplish these goals, and to incorporate lessons learned while developing
-python-stix (for STIX 1.x), several decisions influenced the design of
-python-stix2:
+``python-stix`` (for STIX 1.x), several decisions influenced the design of the
+``stix2`` library:
1. All data structures are immutable by default. In contrast to python-stix,
where users would create an object and then assign attributes to it, in
- python-stix2 all properties must be provided when creating the object.
+ ``stix2`` all properties must be provided when creating the object.
2. Where necessary, library objects should act like ``dict``'s. When treated as
a ``str``, the JSON reprentation of the object should be used.
3. Core Python data types (including numeric types, ``datetime``) should be used
when appropriate, and serialized to the correct format in JSON as specified
- in the STIX 2.0 spec.
+ in the STIX 2 spec.
Architecture
------------
-The `stix2` library APIs are divided into three logical layers, representing
+The ``stix2`` library is divided into three logical layers, representing
different levels of abstraction useful in different types of scripts and larger
applications. It is possible to combine multiple layers in the same program,
and the higher levels build on the layers below.
@@ -41,7 +41,7 @@ and the higher levels build on the layers below.
Object Layer
^^^^^^^^^^^^
-The lowest layer, **Object Layer**, is where Python objects representing STIX 2
+The lowest layer, the **Object Layer**, is where Python objects representing STIX 2
data types (such as SDOs, SROs, and Cyber Observable Objects, as well as
non-top-level objects like External References, Kill Chain phases, and Cyber
Observable extensions) are created, and can be serialized and deserialized
@@ -57,8 +57,6 @@ not implemented as references between the Python objects themselves, but by
simply having the same values in ``id`` and reference properties. There is no
referential integrity maintained by the ``stix2`` library.
-*This layer is mostly complete.*
-
Environment Layer
^^^^^^^^^^^^^^^^^
@@ -79,8 +77,7 @@ intelligence ecosystem.
Each of these components can be used individually, or combined as part of an
``Environment``. These ``Environment`` objects allow different settings to be
used by different users of a multi-user application (such as a web application).
-
-*This layer is mostly complete.*
+For more information, check out `this Environment tutorial `_.
Workbench Layer
^^^^^^^^^^^^^^^
@@ -89,9 +86,7 @@ The highest layer of the ``stix2`` APIs is the **Workbench Layer**, designed for
a single user in a highly-interactive analytical environment (such as a `Jupyter
Notebook `_.
diff --git a/docs/roadmap.rst b/docs/roadmap.rst
deleted file mode 100644
index 7fb9d8b..0000000
--- a/docs/roadmap.rst
+++ /dev/null
@@ -1,17 +0,0 @@
-Development Roadmap
-===================
-
-.. warning::
-
- Prior to version 1.0, all APIs are considered unstable and subject to
- change.
-
-This is a list of (planned) features before version 1.0 is released.
-
-* Serialization of all STIX and Cyber Observable objects to JSON.
-* De-serialization (parsing) of all STIX and Cyber Observable objects.
-* APIs for versioning (revising and revoking) STIX objects.
-* APIs for marking STIX objects and interpreting markings of STIX objects.
-* :ref:`datastore_api`, providing a common interface for querying sources
- of STIX content (such as objects in memory, on a filesystem, in a database, or
- via a TAXII feed).
diff --git a/requirements.txt b/requirements.txt
index d6abb63..5ac0b37 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,10 +1,10 @@
bumpversion
ipython
-nbsphinx>=0.3.0
+nbsphinx==0.3.2
pre-commit
pytest
pytest-cov
-sphinx
+sphinx<1.6
sphinx-prompt
tox
diff --git a/setup.cfg b/setup.cfg
index b252746..ac3e5d5 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -1,5 +1,5 @@
[bumpversion]
-current_version = 0.5.1
+current_version = 1.0.0
commit = True
tag = True
diff --git a/setup.py b/setup.py
index fa68616..9700ec8 100644
--- a/setup.py
+++ b/setup.py
@@ -4,11 +4,12 @@ import os.path
from setuptools import find_packages, setup
-here = os.path.abspath(os.path.dirname(__file__))
+BASE_DIR = os.path.dirname(os.path.abspath(__file__))
+VERSION_FILE = os.path.join(BASE_DIR, 'stix2', 'version.py')
def get_version():
- with open('stix2/version.py', encoding="utf-8") as f:
+ with open(VERSION_FILE) as f:
for line in f.readlines():
if line.startswith("__version__"):
version = line.split()[-1].strip('"')
@@ -16,7 +17,7 @@ def get_version():
raise AttributeError("Package does not have a __version__")
-with open(os.path.join(here, 'README.rst'), encoding='utf-8') as f:
+with open('README.rst') as f:
long_description = f.read()
@@ -52,6 +53,8 @@ setup(
'simplejson',
'six',
'stix2-patterns',
- 'taxii2-client',
],
+ extras_require={
+ 'taxii': ['taxii2-client']
+ }
)
diff --git a/stix2/__init__.py b/stix2/__init__.py
index 401d44b..449be68 100644
--- a/stix2/__init__.py
+++ b/stix2/__init__.py
@@ -11,6 +11,7 @@
patterns
properties
utils
+ workbench
v20.common
v20.observables
v20.sdo
@@ -31,11 +32,12 @@ from .environment import Environment, ObjectFactory
from .markings import (add_markings, clear_markings, get_markings, is_marked,
remove_markings, set_markings)
from .patterns import (AndBooleanExpression, AndObservationExpression,
- BasicObjectPathComponent, EqualityComparisonExpression,
+ BasicObjectPathComponent, BinaryConstant,
+ BooleanConstant, EqualityComparisonExpression,
FloatConstant, FollowedByObservationExpression,
GreaterThanComparisonExpression,
GreaterThanEqualComparisonExpression, HashConstant,
- HexConstant, IntegerConstant,
+ HexConstant, InComparisonExpression, IntegerConstant,
IsSubsetComparisonExpression,
IsSupersetComparisonExpression,
LessThanComparisonExpression,
@@ -48,7 +50,7 @@ from .patterns import (AndBooleanExpression, AndObservationExpression,
ReferenceObjectPathComponent, RepeatQualifier,
StartStopQualifier, StringConstant, TimestampConstant,
WithinQualifier)
-from .utils import get_dict, new_version, revoke
+from .utils import new_version, revoke
from .v20 import * # This import will always be the latest STIX 2.X version
from .version import __version__
diff --git a/stix2/base.py b/stix2/base.py
index 898f489..e128d48 100644
--- a/stix2/base.py
+++ b/stix2/base.py
@@ -6,9 +6,9 @@ import datetime as dt
import simplejson as json
-from .exceptions import (AtLeastOnePropertyError, DependentPropertiesError,
- ExtraPropertiesError, ImmutableError,
- InvalidObjRefError, InvalidValueError,
+from .exceptions import (AtLeastOnePropertyError, CustomContentError,
+ DependentPropertiesError, ExtraPropertiesError,
+ ImmutableError, InvalidObjRefError, InvalidValueError,
MissingPropertiesError,
MutuallyExclusivePropertiesError)
from .markings.utils import validate
@@ -22,6 +22,33 @@ DEFAULT_ERROR = "{type} must have {property}='{expected}'."
class STIXJSONEncoder(json.JSONEncoder):
+ """Custom JSONEncoder subclass for serializing Python ``stix2`` objects.
+
+ If an optional property with a default value specified in the STIX 2 spec
+ is set to that default value, it will be left out of the serialized output.
+
+ An example of this type of property include the ``revoked`` common property.
+ """
+
+ def default(self, obj):
+ if isinstance(obj, (dt.date, dt.datetime)):
+ return format_datetime(obj)
+ elif isinstance(obj, _STIXBase):
+ tmp_obj = dict(copy.deepcopy(obj))
+ for prop_name in obj._defaulted_optional_properties:
+ del tmp_obj[prop_name]
+ return tmp_obj
+ else:
+ return super(STIXJSONEncoder, self).default(obj)
+
+
+class STIXJSONIncludeOptionalDefaultsEncoder(json.JSONEncoder):
+ """Custom JSONEncoder subclass for serializing Python ``stix2`` objects.
+
+ Differs from ``STIXJSONEncoder`` in that if an optional property with a default
+ value specified in the STIX 2 spec is set to that default value, it will be
+ included in the serialized output.
+ """
def default(self, obj):
if isinstance(obj, (dt.date, dt.datetime)):
@@ -61,6 +88,8 @@ class _STIXBase(collections.Mapping):
try:
kwargs[prop_name] = prop.clean(kwargs[prop_name])
except ValueError as exc:
+ if self.__allow_custom and isinstance(exc, CustomContentError):
+ return
raise InvalidValueError(self.__class__, prop_name, reason=str(exc))
# interproperty constraint methods
@@ -97,6 +126,7 @@ class _STIXBase(collections.Mapping):
def __init__(self, allow_custom=False, **kwargs):
cls = self.__class__
+ self.__allow_custom = allow_custom
# Use the same timestamp for any auto-generated datetimes
self.__now = get_timestamp()
@@ -119,14 +149,25 @@ class _STIXBase(collections.Mapping):
setting_kwargs[prop_name] = prop_value
# Detect any missing required properties
- required_properties = get_required_properties(cls._properties)
- missing_kwargs = set(required_properties) - set(setting_kwargs)
+ required_properties = set(get_required_properties(cls._properties))
+ missing_kwargs = required_properties - set(setting_kwargs)
if missing_kwargs:
raise MissingPropertiesError(cls, missing_kwargs)
for prop_name, prop_metadata in cls._properties.items():
self._check_property(prop_name, prop_metadata, setting_kwargs)
+ # Cache defaulted optional properties for serialization
+ defaulted = []
+ for name, prop in cls._properties.items():
+ try:
+ if (not prop.required and not hasattr(prop, '_fixed_value') and
+ prop.default() == setting_kwargs[name]):
+ defaulted.append(name)
+ except (AttributeError, KeyError):
+ continue
+ self._defaulted_optional_properties = defaulted
+
self._inner = setting_kwargs
self._check_object_constraints()
@@ -148,7 +189,7 @@ class _STIXBase(collections.Mapping):
(self.__class__.__name__, name))
def __setattr__(self, name, value):
- if name != '_inner' and not name.startswith("_STIXBase__"):
+ if not name.startswith("_"):
raise ImmutableError(self.__class__, name)
super(_STIXBase, self).__setattr__(name, value)
@@ -167,6 +208,7 @@ class _STIXBase(collections.Mapping):
if isinstance(self, _Observable):
# Assume: valid references in the original object are still valid in the new version
new_inner['_valid_refs'] = {'*': '*'}
+ new_inner['allow_custom'] = self.__allow_custom
return cls(**new_inner)
def properties_populated(self):
@@ -180,7 +222,7 @@ class _STIXBase(collections.Mapping):
def revoke(self):
return _revoke(self)
- def serialize(self, pretty=False, **kwargs):
+ def serialize(self, pretty=False, include_optional_defaults=False, **kwargs):
"""
Serialize a STIX object.
@@ -188,6 +230,8 @@ class _STIXBase(collections.Mapping):
pretty (bool): If True, output properties following the STIX specs
formatting. This includes indentation. Refer to notes for more
details. (Default: ``False``)
+ include_optional_defaults (bool): Determines whether to include
+ optional properties set to the default value defined in the spec.
**kwargs: The arguments for a json.dumps() call.
Returns:
@@ -210,7 +254,10 @@ class _STIXBase(collections.Mapping):
kwargs.update({'indent': 4, 'separators': (",", ": "), 'item_sort_key': sort_by})
- return json.dumps(self, cls=STIXJSONEncoder, **kwargs)
+ if include_optional_defaults:
+ return json.dumps(self, cls=STIXJSONIncludeOptionalDefaultsEncoder, **kwargs)
+ else:
+ return json.dumps(self, cls=STIXJSONEncoder, **kwargs)
class _Observable(_STIXBase):
diff --git a/stix2/core.py b/stix2/core.py
index b6d295d..0b222a9 100644
--- a/stix2/core.py
+++ b/stix2/core.py
@@ -9,7 +9,7 @@ import stix2
from . import exceptions
from .base import _STIXBase
from .properties import IDProperty, ListProperty, Property, TypeProperty
-from .utils import get_class_hierarchy_names, get_dict
+from .utils import _get_dict, get_class_hierarchy_names
class STIXObjectProperty(Property):
@@ -25,7 +25,7 @@ class STIXObjectProperty(Property):
for x in get_class_hierarchy_names(value)):
return value
try:
- dictified = get_dict(value)
+ dictified = _get_dict(value)
except ValueError:
raise ValueError("This property may only contain a dictionary or object")
if dictified == {}:
@@ -73,18 +73,57 @@ STIX2_OBJ_MAPS = {}
def parse(data, allow_custom=False, version=None):
- """Deserialize a string or file-like object into a STIX object.
+ """Convert a string, dict or file-like object into a STIX object.
Args:
data (str, dict, file-like object): The STIX 2 content to be parsed.
- allow_custom (bool): Whether to allow custom properties or not.
- Default: False.
+ allow_custom (bool): Whether to allow custom properties as well unknown
+ custom objects. Note that unknown custom objects cannot be parsed
+ into STIX objects, and will be returned as is. Default: False.
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
Returns:
An instantiated Python STIX object.
+ WARNING: 'allow_custom=True' will allow for the return of any supplied STIX
+ dict(s) that cannot be found to map to any known STIX object types (both STIX2
+ domain objects or defined custom STIX2 objects); NO validation is done. This is
+ done to allow the processing of possibly unknown custom STIX objects (example
+ scenario: I need to query a third-party TAXII endpoint that could provide custom
+ STIX objects that I dont know about ahead of time)
+
+ """
+ # convert STIX object to dict, if not already
+ obj = _get_dict(data)
+
+ # convert dict to full python-stix2 obj
+ obj = dict_to_stix2(obj, allow_custom, version)
+
+ return obj
+
+
+def dict_to_stix2(stix_dict, allow_custom=False, version=None):
+ """convert dictionary to full python-stix2 object
+
+ Args:
+ stix_dict (dict): a python dictionary of a STIX object
+ that (presumably) is semantically correct to be parsed
+ into a full python-stix2 obj
+ allow_custom (bool): Whether to allow custom properties as well unknown
+ custom objects. Note that unknown custom objects cannot be parsed
+ into STIX objects, and will be returned as is. Default: False.
+
+ Returns:
+ An instantiated Python STIX object
+
+ WARNING: 'allow_custom=True' will allow for the return of any supplied STIX
+ dict(s) that cannot be found to map to any known STIX object types (both STIX2
+ domain objects or defined custom STIX2 objects); NO validation is done. This is
+ done to allow the processing of possibly unknown custom STIX objects (example
+ scenario: I need to query a third-party TAXII endpoint that could provide custom
+ STIX objects that I dont know about ahead of time)
+
"""
if not version:
# Use latest version
@@ -93,16 +132,20 @@ def parse(data, allow_custom=False, version=None):
v = 'v' + version.replace('.', '')
OBJ_MAP = STIX2_OBJ_MAPS[v]
- obj = get_dict(data)
- if 'type' not in obj:
- raise exceptions.ParseError("Can't parse object with no 'type' property: %s" % str(obj))
+ if 'type' not in stix_dict:
+ raise exceptions.ParseError("Can't parse object with no 'type' property: %s" % str(stix_dict))
try:
- obj_class = OBJ_MAP[obj['type']]
+ obj_class = OBJ_MAP[stix_dict['type']]
except KeyError:
- raise exceptions.ParseError("Can't parse unknown object type '%s'! For custom types, use the CustomObject decorator." % obj['type'])
- return obj_class(allow_custom=allow_custom, **obj)
+ if allow_custom:
+ # flag allows for unknown custom objects too, but will not
+ # be parsed into STIX object, returned as is
+ return stix_dict
+ raise exceptions.ParseError("Can't parse unknown object type '%s'! For custom types, use the CustomObject decorator." % stix_dict['type'])
+
+ return obj_class(allow_custom=allow_custom, **stix_dict)
def _register_type(new_type, version=None):
diff --git a/stix2/datastore/__init__.py b/stix2/datastore/__init__.py
index 78f7555..7fdf515 100644
--- a/stix2/datastore/__init__.py
+++ b/stix2/datastore/__init__.py
@@ -1,4 +1,4 @@
-"""Python STIX 2.0 DataStore API
+"""Python STIX 2.0 DataStore API.
.. autosummary::
:toctree: datastore
@@ -16,7 +16,7 @@ import uuid
from six import with_metaclass
-from stix2.datastore.filters import Filter
+from stix2.datastore.filters import Filter, FilterSet
from stix2.utils import deduplicate
@@ -73,7 +73,7 @@ class DataStoreMixin(object):
stix_id (str): the id of the STIX object to retrieve.
Returns:
- stix_objs (list): a list of STIX objects
+ list: All versions of the specified STIX object.
"""
try:
@@ -91,7 +91,7 @@ class DataStoreMixin(object):
to conduct search on.
Returns:
- stix_objs (list): a list of STIX objects
+ list: The STIX objects matching the query.
"""
try:
@@ -136,7 +136,7 @@ class DataStoreMixin(object):
object is the target_ref. Default: False.
Returns:
- (list): List of Relationship objects involving the given STIX object.
+ list: The Relationship objects involving the given STIX object.
"""
try:
@@ -162,9 +162,11 @@ class DataStoreMixin(object):
object is the source_ref. Default: False.
target_only (bool): Only examine Relationships for which this
object is the target_ref. Default: False.
+ filters (list): list of additional filters the related objects must
+ match.
Returns:
- (list): List of STIX objects related to the given STIX object.
+ list: The STIX objects related to the given STIX object.
"""
try:
@@ -175,8 +177,8 @@ class DataStoreMixin(object):
def add(self, *args, **kwargs):
"""Method for storing STIX objects.
- Define custom behavior before storing STIX objects using the associated
- DataSink. Translates add() to the appropriate DataSink call.
+ Defines custom behavior before storing STIX objects using the
+ appropriate method call on the associated DataSink.
Args:
stix_objs (list): a list of STIX objects
@@ -220,13 +222,13 @@ class DataSource(with_metaclass(ABCMeta)):
Attributes:
id (str): A unique UUIDv4 to identify this DataSource.
- filters (set): A collection of filters attached to this DataSource.
+ filters (FilterSet): A collection of filters attached to this DataSource.
"""
def __init__(self):
super(DataSource, self).__init__()
self.id = make_id()
- self.filters = set()
+ self.filters = FilterSet()
@abstractmethod
def get(self, stix_id):
@@ -240,7 +242,7 @@ class DataSource(with_metaclass(ABCMeta)):
specified by the "id".
Returns:
- stix_obj: the STIX object
+ stix_obj: The STIX object.
"""
@@ -258,7 +260,7 @@ class DataSource(with_metaclass(ABCMeta)):
specified by the "id".
Returns:
- stix_objs (list): a list of STIX objects
+ list: All versions of the specified STIX object.
"""
@@ -273,7 +275,7 @@ class DataSource(with_metaclass(ABCMeta)):
to conduct search on.
Returns:
- stix_objs (list): a list of STIX objects
+ list: The STIX objects that matched the query.
"""
@@ -311,7 +313,7 @@ class DataSource(with_metaclass(ABCMeta)):
object is the target_ref. Default: False.
Returns:
- (list): List of Relationship objects involving the given STIX object.
+ list: The Relationship objects involving the given STIX object.
"""
results = []
@@ -338,7 +340,7 @@ class DataSource(with_metaclass(ABCMeta)):
return results
- def related_to(self, obj, relationship_type=None, source_only=False, target_only=False):
+ def related_to(self, obj, relationship_type=None, source_only=False, target_only=False, filters=None):
"""Retrieve STIX Objects that have a Relationship involving the given
STIX object.
@@ -354,9 +356,11 @@ class DataSource(with_metaclass(ABCMeta)):
object is the source_ref. Default: False.
target_only (bool): Only examine Relationships for which this
object is the target_ref. Default: False.
+ filters (list): list of additional filters the related objects must
+ match.
Returns:
- (list): List of STIX objects related to the given STIX object.
+ list: The STIX objects related to the given STIX object.
"""
results = []
@@ -372,10 +376,13 @@ class DataSource(with_metaclass(ABCMeta)):
ids = set()
for r in rels:
ids.update((r.source_ref, r.target_ref))
- ids.remove(obj_id)
+ ids.discard(obj_id)
+
+ # Assemble filters
+ filter_list = FilterSet(filters)
for i in ids:
- results.append(self.get(i))
+ results.extend(self.query([f for f in filter_list] + [Filter('id', '=', i)]))
return results
@@ -420,23 +427,24 @@ class CompositeDataSource(DataSource):
Args:
stix_id (str): the id of the STIX object to retrieve.
- _composite_filters (list): a list of filters passed from a
+ _composite_filters (FilterSet): a collection of filters passed from a
CompositeDataSource (i.e. if this CompositeDataSource is attached
to another parent CompositeDataSource), not user supplied.
Returns:
- stix_obj: the STIX object to be returned.
+ stix_obj: The STIX object to be returned.
"""
if not self.has_data_sources():
raise AttributeError('CompositeDataSource has no data sources')
all_data = []
- all_filters = set()
- all_filters.update(self.filters)
+ all_filters = FilterSet()
+
+ all_filters.add(self.filters)
if _composite_filters:
- all_filters.update(_composite_filters)
+ all_filters.add(_composite_filters)
# for every configured Data Source, call its retrieve handler
for ds in self.data_sources:
@@ -466,24 +474,24 @@ class CompositeDataSource(DataSource):
Args:
stix_id (str): id of the STIX objects to retrieve.
- _composite_filters (list): a list of filters passed from a
+ _composite_filters (FilterSet): a collection of filters passed from a
CompositeDataSource (i.e. if this CompositeDataSource is
attached to a parent CompositeDataSource), not user supplied.
Returns:
- all_data (list): list of STIX objects that have the specified id
+ list: The STIX objects that have the specified id.
"""
if not self.has_data_sources():
raise AttributeError('CompositeDataSource has no data sources')
all_data = []
- all_filters = set()
+ all_filters = FilterSet()
- all_filters.update(self.filters)
+ all_filters.add(self.filters)
if _composite_filters:
- all_filters.update(_composite_filters)
+ all_filters.add(_composite_filters)
# retrieve STIX objects from all configured data sources
for ds in self.data_sources:
@@ -505,29 +513,29 @@ class CompositeDataSource(DataSource):
Args:
query (list): list of filters to search on.
- _composite_filters (list): a list of filters passed from a
+ _composite_filters (FilterSet): a collection of filters passed from a
CompositeDataSource (i.e. if this CompositeDataSource is
attached to a parent CompositeDataSource), not user supplied.
Returns:
- all_data (list): list of STIX objects to be returned
+ list: The STIX objects to be returned.
"""
if not self.has_data_sources():
raise AttributeError('CompositeDataSource has no data sources')
if not query:
- # don't mess with the query (i.e. convert to a set, as that's done
+ # don't mess with the query (i.e. deduplicate, as that's done
# within the specific DataSources that are called)
query = []
all_data = []
+ all_filters = FilterSet()
- all_filters = set()
- all_filters.update(self.filters)
+ all_filters.add(self.filters)
if _composite_filters:
- all_filters.update(_composite_filters)
+ all_filters.add(_composite_filters)
# federate query to all attached data sources,
# pass composite filters to id
@@ -561,7 +569,7 @@ class CompositeDataSource(DataSource):
object is the target_ref. Default: False.
Returns:
- (list): List of Relationship objects involving the given STIX object.
+ list: The Relationship objects involving the given STIX object.
"""
if not self.has_data_sources():
@@ -597,9 +605,11 @@ class CompositeDataSource(DataSource):
object is the source_ref. Default: False.
target_only (bool): Only examine Relationships for which this
object is the target_ref. Default: False.
+ filters (list): list of additional filters the related objects must
+ match.
Returns:
- (list): List of STIX objects related to the given STIX object.
+ list: The STIX objects related to the given STIX object.
"""
if not self.has_data_sources():
diff --git a/stix2/datastore/filesystem.py b/stix2/datastore/filesystem.py
index a6f31cf..c13b02c 100644
--- a/stix2/datastore/filesystem.py
+++ b/stix2/datastore/filesystem.py
@@ -8,7 +8,7 @@ import os
from stix2.core import Bundle, parse
from stix2.datastore import DataSink, DataSource, DataStoreMixin
-from stix2.datastore.filters import Filter, apply_common_filters
+from stix2.datastore.filters import Filter, FilterSet, apply_common_filters
from stix2.utils import deduplicate, get_class_hierarchy_names
@@ -165,7 +165,7 @@ class FileSystemSource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX object to be retrieved.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -195,7 +195,7 @@ class FileSystemSource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX objects to be retrieved.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -217,7 +217,7 @@ class FileSystemSource(DataSource):
Args:
query (list): list of filters to search on
- _composite_filters (set): set of filters passed from the
+ _composite_filters (FilterSet): collection of filters passed from the
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -231,20 +231,13 @@ class FileSystemSource(DataSource):
all_data = []
- if query is None:
- query = set()
- else:
- if not isinstance(query, list):
- # make sure dont make set from a Filter object,
- # need to make a set from a list of Filter objects (even if just one Filter)
- query = [query]
- query = set(query)
+ query = FilterSet(query)
# combine all query filters
if self.filters:
- query.update(self.filters)
+ query.add(self.filters)
if _composite_filters:
- query.update(_composite_filters)
+ query.add(_composite_filters)
# extract any filters that are for "type" or "id" , as we can then do
# filtering before reading in the STIX objects. A STIX 'type' filter
@@ -343,8 +336,8 @@ class FileSystemSource(DataSource):
search space of a FileSystemStore (or FileSystemSink).
"""
- file_filters = set()
+ file_filters = []
for filter_ in query:
if filter_.property == "id" or filter_.property == "type":
- file_filters.add(filter_)
+ file_filters.append(filter_)
return file_filters
diff --git a/stix2/datastore/filters.py b/stix2/datastore/filters.py
index 9065b61..a32b14a 100644
--- a/stix2/datastore/filters.py
+++ b/stix2/datastore/filters.py
@@ -4,6 +4,9 @@ Filters for Python STIX 2.0 DataSources, DataSinks, DataStores
"""
import collections
+from datetime import datetime
+
+from stix2.utils import format_datetime
"""Supported filter operations"""
FILTER_OPS = ['=', '!=', 'in', '>', '<', '>=', '<=']
@@ -66,6 +69,10 @@ class Filter(collections.namedtuple("Filter", ['property', 'op', 'value'])):
if isinstance(value, list):
value = tuple(value)
+ if isinstance(value, datetime):
+ # if value is a datetime obj, convert to str
+ value = format_datetime(value)
+
_check_filter_components(prop, op, value)
self = super(Filter, cls).__new__(cls, prop, op, value)
@@ -81,6 +88,12 @@ class Filter(collections.namedtuple("Filter", ['property', 'op', 'value'])):
True if property matches the filter,
False otherwise.
"""
+ if isinstance(stix_obj_property, datetime):
+ # if a datetime obj, convert to str format before comparison
+ # NOTE: this check seems like it should be done upstream
+ # but will put here for now
+ stix_obj_property = format_datetime(stix_obj_property)
+
if self.op == "=":
return stix_obj_property == self.value
elif self.op == "!=":
@@ -152,19 +165,94 @@ def _check_filter(filter_, stix_obj):
# Check embedded properties, from e.g. granular_markings or external_references
sub_property = filter_.property.split(".", 1)[1]
sub_filter = filter_._replace(property=sub_property)
+
if isinstance(stix_obj[prop], list):
for elem in stix_obj[prop]:
if _check_filter(sub_filter, elem) is True:
return True
return False
+
else:
return _check_filter(sub_filter, stix_obj[prop])
+
elif isinstance(stix_obj[prop], list):
# Check each item in list property to see if it matches
for elem in stix_obj[prop]:
if filter_._check_property(elem) is True:
return True
return False
+
else:
# Check if property matches
return filter_._check_property(stix_obj[prop])
+
+
+class FilterSet(object):
+ """Internal STIX2 class to facilitate the grouping of Filters
+ into sets. The primary motivation for this class came from the problem
+ that Filters that had a dict as a value could not be added to a Python
+ set as dicts are not hashable. Thus this class provides set functionality
+ but internally stores filters in a list.
+ """
+
+ def __init__(self, filters=None):
+ """
+ Args:
+ filters: see FilterSet.add()
+ """
+ self._filters = []
+ if filters:
+ self.add(filters)
+
+ def __iter__(self):
+ """Provide iteration functionality of FilterSet."""
+ for f in self._filters:
+ yield f
+
+ def __len__(self):
+ """Provide built-in len() utility of FilterSet."""
+ return len(self._filters)
+
+ def add(self, filters=None):
+ """Add a Filter, FilterSet, or list of Filters to the FilterSet.
+
+ Operates like set, only adding unique stix2.Filters to the FilterSet
+
+ NOTE: method designed to be very accomodating (i.e. even accepting filters=None)
+ as it allows for blind calls (very useful in DataStore)
+
+ Args:
+ filters: stix2.Filter OR list of stix2.Filter OR stix2.FilterSet
+
+ """
+ if not filters:
+ # so add() can be called blindly, useful for
+ # DataStore/Environment usage of filter operations
+ return
+
+ if not isinstance(filters, (FilterSet, list)):
+ filters = [filters]
+
+ for f in filters:
+ if f not in self._filters:
+ self._filters.append(f)
+
+ def remove(self, filters=None):
+ """Remove a Filter, list of Filters, or FilterSet from the FilterSet.
+
+ NOTE: method designed to be very accomodating (i.e. even accepting filters=None)
+ as it allows for blind calls (very useful in DataStore)
+
+ Args:
+ filters: stix2.Filter OR list of stix2.Filter or stix2.FilterSet
+ """
+ if not filters:
+ # so remove() can be called blindly, useful for
+ # DataStore/Environemnt usage of filter ops
+ return
+
+ if not isinstance(filters, (FilterSet, list)):
+ filters = [filters]
+
+ for f in filters:
+ self._filters.remove(f)
diff --git a/stix2/datastore/memory.py b/stix2/datastore/memory.py
index 1c4ff6c..c43da8b 100644
--- a/stix2/datastore/memory.py
+++ b/stix2/datastore/memory.py
@@ -18,7 +18,7 @@ import os
from stix2.base import _STIXBase
from stix2.core import Bundle, parse
from stix2.datastore import DataSink, DataSource, DataStoreMixin
-from stix2.datastore.filters import Filter, apply_common_filters
+from stix2.datastore.filters import Filter, FilterSet, apply_common_filters
def _add(store, stix_data=None, version=None):
@@ -197,7 +197,7 @@ class MemorySource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX object to be retrieved.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
Returns:
@@ -236,7 +236,7 @@ class MemorySource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX 2 object to retrieve.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
Returns:
@@ -258,7 +258,7 @@ class MemorySource(DataSource):
Args:
query (list): list of filters to search on
- _composite_filters (set): set of filters passed from the
+ _composite_filters (FilterSet): collection of filters passed from the
CompositeDataSource, not user supplied
Returns:
@@ -269,19 +269,15 @@ class MemorySource(DataSource):
"""
if query is None:
- query = set()
+ query = FilterSet()
else:
- if not isinstance(query, list):
- # make sure don't make set from a Filter object,
- # need to make a set from a list of Filter objects (even if just one Filter)
- query = [query]
- query = set(query)
+ query = FilterSet(query)
# combine all query filters
if self.filters:
- query.update(self.filters)
+ query.add(self.filters)
if _composite_filters:
- query.update(_composite_filters)
+ query.add(_composite_filters)
# Apply STIX common property filters.
all_data = list(apply_common_filters(self._data.values(), query))
diff --git a/stix2/datastore/taxii.py b/stix2/datastore/taxii.py
index 0a58763..faa2669 100644
--- a/stix2/datastore/taxii.py
+++ b/stix2/datastore/taxii.py
@@ -6,7 +6,7 @@ from requests.exceptions import HTTPError
from stix2.base import _STIXBase
from stix2.core import Bundle, parse
from stix2.datastore import DataSink, DataSource, DataStoreMixin
-from stix2.datastore.filters import Filter, apply_common_filters
+from stix2.datastore.filters import Filter, FilterSet, apply_common_filters
from stix2.utils import deduplicate
TAXII_FILTERS = ['added_after', 'id', 'type', 'version']
@@ -120,7 +120,7 @@ class TAXIICollectionSource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX object to be retrieved.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -132,11 +132,12 @@ class TAXIICollectionSource(DataSource):
"""
# combine all query filters
- query = set()
+ query = FilterSet()
+
if self.filters:
- query.update(self.filters)
+ query.add(self.filters)
if _composite_filters:
- query.update(_composite_filters)
+ query.add(_composite_filters)
# dont extract TAXII filters from query (to send to TAXII endpoint)
# as directly retrieveing a STIX object by ID
@@ -164,7 +165,7 @@ class TAXIICollectionSource(DataSource):
Args:
stix_id (str): The STIX ID of the STIX objects to be retrieved.
- _composite_filters (set): set of filters passed from the parent
+ _composite_filters (FilterSet): collection of filters passed from the parent
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -198,7 +199,7 @@ class TAXIICollectionSource(DataSource):
Args:
query (list): list of filters to search on
- _composite_filters (set): set of filters passed from the
+ _composite_filters (FilterSet): collection of filters passed from the
CompositeDataSource, not user supplied
version (str): Which STIX2 version to use. (e.g. "2.0", "2.1"). If
None, use latest version.
@@ -209,20 +210,13 @@ class TAXIICollectionSource(DataSource):
parsed into python STIX objects and then returned.
"""
- if query is None:
- query = set()
- else:
- if not isinstance(query, list):
- # make sure dont make set from a Filter object,
- # need to make a set from a list of Filter objects (even if just one Filter)
- query = [query]
- query = set(query)
+ query = FilterSet(query)
# combine all query filters
if self.filters:
- query.update(self.filters)
+ query.add(self.filters)
if _composite_filters:
- query.update(_composite_filters)
+ query.add(_composite_filters)
# parse taxii query params (that can be applied remotely)
taxii_filters = self._parse_taxii_filters(query)
@@ -268,17 +262,16 @@ class TAXIICollectionSource(DataSource):
Args:
- query (set): set of filters to extract which ones are TAXII
+ query (list): list of filters to extract which ones are TAXII
specific.
- Returns:
- taxii_filters (set): set of the TAXII filters
+ Returns: a list of the TAXII filters
"""
- taxii_filters = set()
+ taxii_filters = []
for filter_ in query:
if filter_.property in TAXII_FILTERS:
- taxii_filters.add(filter_)
+ taxii_filters.append(filter_)
return taxii_filters
diff --git a/stix2/environment.py b/stix2/environment.py
index eb5583e..cc589ae 100644
--- a/stix2/environment.py
+++ b/stix2/environment.py
@@ -1,3 +1,6 @@
+"""Python STIX 2.0 Environment API.
+"""
+
import copy
from .core import parse as _parse
@@ -30,19 +33,43 @@ class ObjectFactory(object):
self._defaults = {}
if created_by_ref:
- self._defaults['created_by_ref'] = created_by_ref
+ self.set_default_creator(created_by_ref)
if created:
- self._defaults['created'] = created
- # If the user provides a default "created" time, we also want to use
- # that as the modified time.
- self._defaults['modified'] = created
+ self.set_default_created(created)
if external_references:
- self._defaults['external_references'] = external_references
+ self.set_default_external_refs(external_references)
if object_marking_refs:
- self._defaults['object_marking_refs'] = object_marking_refs
+ self.set_default_object_marking_refs(object_marking_refs)
self._list_append = list_append
self._list_properties = ['external_references', 'object_marking_refs']
+ def set_default_creator(self, creator=None):
+ """Set default value for the `created_by_ref` property.
+
+ """
+ self._defaults['created_by_ref'] = creator
+
+ def set_default_created(self, created=None):
+ """Set default value for the `created` property.
+
+ """
+ self._defaults['created'] = created
+ # If the user provides a default "created" time, we also want to use
+ # that as the modified time.
+ self._defaults['modified'] = created
+
+ def set_default_external_refs(self, external_references=None):
+ """Set default external references.
+
+ """
+ self._defaults['external_references'] = external_references
+
+ def set_default_object_marking_refs(self, object_marking_refs=None):
+ """Set default object markings.
+
+ """
+ self._defaults['object_marking_refs'] = object_marking_refs
+
def create(self, cls, **kwargs):
"""Create a STIX object using object factory defaults.
@@ -94,6 +121,7 @@ class Environment(DataStoreMixin):
.. automethod:: relationships
.. automethod:: related_to
.. automethod:: add
+
"""
def __init__(self, factory=ObjectFactory(), store=None, source=None, sink=None):
@@ -113,17 +141,27 @@ class Environment(DataStoreMixin):
return self.factory.create(*args, **kwargs)
create.__doc__ = ObjectFactory.create.__doc__
+ def set_default_creator(self, *args, **kwargs):
+ return self.factory.set_default_creator(*args, **kwargs)
+ set_default_creator.__doc__ = ObjectFactory.set_default_creator.__doc__
+
+ def set_default_created(self, *args, **kwargs):
+ return self.factory.set_default_created(*args, **kwargs)
+ set_default_created.__doc__ = ObjectFactory.set_default_created.__doc__
+
+ def set_default_external_refs(self, *args, **kwargs):
+ return self.factory.set_default_external_refs(*args, **kwargs)
+ set_default_external_refs.__doc__ = ObjectFactory.set_default_external_refs.__doc__
+
+ def set_default_object_marking_refs(self, *args, **kwargs):
+ return self.factory.set_default_object_marking_refs(*args, **kwargs)
+ set_default_object_marking_refs.__doc__ = ObjectFactory.set_default_object_marking_refs.__doc__
+
def add_filters(self, *args, **kwargs):
- try:
- return self.source.filters.update(*args, **kwargs)
- except AttributeError:
- raise AttributeError('Environment has no data source')
+ return self.source.filters.add(*args, **kwargs)
def add_filter(self, *args, **kwargs):
- try:
- return self.source.filters.add(*args, **kwargs)
- except AttributeError:
- raise AttributeError('Environment has no data source')
+ return self.source.filters.add(*args, **kwargs)
def parse(self, *args, **kwargs):
return _parse(*args, **kwargs)
diff --git a/stix2/exceptions.py b/stix2/exceptions.py
index 841a8e9..79c5a81 100644
--- a/stix2/exceptions.py
+++ b/stix2/exceptions.py
@@ -163,6 +163,13 @@ class ParseError(STIXError, ValueError):
super(ParseError, self).__init__(msg)
+class CustomContentError(STIXError, ValueError):
+ """Custom STIX Content (SDO, Observable, Extension, etc.) detected."""
+
+ def __init__(self, msg):
+ super(CustomContentError, self).__init__(msg)
+
+
class InvalidSelectorError(STIXError, AssertionError):
"""Granular Marking selector violation. The selector must resolve into an existing STIX object property."""
diff --git a/stix2/patterns.py b/stix2/patterns.py
index 94ae7d2..23ce71b 100644
--- a/stix2/patterns.py
+++ b/stix2/patterns.py
@@ -3,8 +3,11 @@
import base64
import binascii
+import datetime
import re
+from .utils import parse_into_datetime
+
def escape_quotes_and_backslashes(s):
return s.replace(u'\\', u'\\\\').replace(u"'", u"\\'")
@@ -24,10 +27,13 @@ class StringConstant(_Constant):
class TimestampConstant(_Constant):
def __init__(self, value):
- self.value = value
+ try:
+ self.value = parse_into_datetime(value)
+ except Exception:
+ raise ValueError("must be a datetime object or timestamp string.")
def __str__(self):
- return "t'%s'" % escape_quotes_and_backslashes(self.value)
+ return "t%s" % repr(self.value)
class IntegerConstant(_Constant):
@@ -46,7 +52,7 @@ class FloatConstant(_Constant):
try:
self.value = float(value)
except Exception:
- raise ValueError("must be an float.")
+ raise ValueError("must be a float.")
def __str__(self):
return "%s" % self.value
@@ -56,24 +62,29 @@ class BooleanConstant(_Constant):
def __init__(self, value):
if isinstance(value, bool):
self.value = value
+ return
trues = ['true', 't']
falses = ['false', 'f']
try:
if value.lower() in trues:
self.value = True
- if value.lower() in falses:
+ return
+ elif value.lower() in falses:
self.value = False
+ return
except AttributeError:
if value == 1:
self.value = True
- if value == 0:
+ return
+ elif value == 0:
self.value = False
+ return
raise ValueError("must be a boolean value.")
def __str__(self):
- return "%s" % self.value
+ return str(self.value).lower()
_HASH_REGEX = {
@@ -132,20 +143,25 @@ class ListConstant(_Constant):
self.value = values
def __str__(self):
- return "(" + ", ".join([("%s" % x) for x in self.value]) + ")"
+ return "(" + ", ".join([("%s" % make_constant(x)) for x in self.value]) + ")"
def make_constant(value):
+ try:
+ return parse_into_datetime(value)
+ except ValueError:
+ pass
+
if isinstance(value, str):
return StringConstant(value)
+ elif isinstance(value, bool):
+ return BooleanConstant(value)
elif isinstance(value, int):
return IntegerConstant(value)
elif isinstance(value, float):
return FloatConstant(value)
elif isinstance(value, list):
return ListConstant(value)
- elif isinstance(value, bool):
- return BooleanConstant(value)
else:
raise ValueError("Unable to create a constant from %s" % value)
@@ -210,15 +226,12 @@ class ObjectPath(object):
class _PatternExpression(object):
-
- @staticmethod
- def escape_quotes_and_backslashes(s):
- return s.replace(u'\\', u'\\\\').replace(u"'", u"\\'")
+ pass
class _ComparisonExpression(_PatternExpression):
def __init__(self, operator, lhs, rhs, negated=False):
- if operator == "=" and isinstance(rhs, ListConstant):
+ if operator == "=" and isinstance(rhs, (ListConstant, list)):
self.operator = "IN"
else:
self.operator = operator
@@ -234,13 +247,6 @@ class _ComparisonExpression(_PatternExpression):
self.root_type = self.lhs.object_type_name
def __str__(self):
- # if isinstance(self.rhs, list):
- # final_rhs = []
- # for r in self.rhs:
- # final_rhs.append("'" + self.escape_quotes_and_backslashes("%s" % r) + "'")
- # rhs_string = "(" + ", ".join(final_rhs) + ")"
- # else:
- # rhs_string = self.rhs
if self.negated:
return "%s NOT %s %s" % (self.lhs, self.operator, self.rhs)
else:
@@ -383,7 +389,7 @@ class RepeatQualifier(_ExpressionQualifier):
elif isinstance(times_to_repeat, int):
self.times_to_repeat = IntegerConstant(times_to_repeat)
else:
- raise ValueError("%s is not a valid argument for a Within Qualifier" % times_to_repeat)
+ raise ValueError("%s is not a valid argument for a Repeat Qualifier" % times_to_repeat)
def __str__(self):
return "REPEATS %s TIMES" % self.times_to_repeat
@@ -404,18 +410,18 @@ class WithinQualifier(_ExpressionQualifier):
class StartStopQualifier(_ExpressionQualifier):
def __init__(self, start_time, stop_time):
- if isinstance(start_time, IntegerConstant):
+ if isinstance(start_time, TimestampConstant):
self.start_time = start_time
- elif isinstance(start_time, int):
- self.start_time = IntegerConstant(start_time)
+ elif isinstance(start_time, datetime.date):
+ self.start_time = TimestampConstant(start_time)
else:
- raise ValueError("%s is not a valid argument for a Within Qualifier" % start_time)
- if isinstance(stop_time, IntegerConstant):
+ raise ValueError("%s is not a valid argument for a Start/Stop Qualifier" % start_time)
+ if isinstance(stop_time, TimestampConstant):
self.stop_time = stop_time
- elif isinstance(stop_time, int):
- self.stop_time = IntegerConstant(stop_time)
+ elif isinstance(stop_time, datetime.date):
+ self.stop_time = TimestampConstant(stop_time)
else:
- raise ValueError("%s is not a valid argument for a Within Qualifier" % stop_time)
+ raise ValueError("%s is not a valid argument for a Start/Stop Qualifier" % stop_time)
def __str__(self):
return "START %s STOP %s" % (self.start_time, self.stop_time)
diff --git a/stix2/properties.py b/stix2/properties.py
index ca7f04c..39d383a 100644
--- a/stix2/properties.py
+++ b/stix2/properties.py
@@ -12,7 +12,7 @@ from stix2patterns.validator import run_validator
from .base import _STIXBase
from .exceptions import DictionaryKeyError
-from .utils import get_dict, parse_into_datetime
+from .utils import _get_dict, parse_into_datetime
class Property(object):
@@ -129,6 +129,8 @@ class ListProperty(Property):
# constructor again
result.append(valid)
continue
+ elif type(self.contained) is DictionaryProperty:
+ obj_type = dict
else:
obj_type = self.contained
@@ -232,7 +234,7 @@ class DictionaryProperty(Property):
def clean(self, value):
try:
- dictified = get_dict(value)
+ dictified = _get_dict(value)
except ValueError:
raise ValueError("The dictionary property must contain a dictionary")
if dictified == {}:
diff --git a/stix2/test/constants.py b/stix2/test/constants.py
index 3db39d6..ab7fcf3 100644
--- a/stix2/test/constants.py
+++ b/stix2/test/constants.py
@@ -34,14 +34,18 @@ RELATIONSHIP_IDS = [
'relationship--a0cbb21c-8daf-4a7f-96aa-7155a4ef8f70'
]
-# All required args for a Campaign instance
+# *_KWARGS contains all required arguments to create an instance of that STIX object
+# *_MORE_KWARGS contains all the required arguments, plus some optional ones
+
+ATTACK_PATTERN_KWARGS = dict(
+ name="Phishing",
+)
+
CAMPAIGN_KWARGS = dict(
name="Green Group Attacks Against Finance",
description="Campaign by Green Group against a series of targets in the financial services sector.",
)
-
-# All required args for a Campaign instance, plus some optional args
CAMPAIGN_MORE_KWARGS = dict(
type='campaign',
id=CAMPAIGN_ID,
@@ -52,25 +56,29 @@ CAMPAIGN_MORE_KWARGS = dict(
description="Campaign by Green Group against a series of targets in the financial services sector.",
)
-# Minimum required args for an Identity instance
+COURSE_OF_ACTION_KWARGS = dict(
+ name="Block",
+)
+
IDENTITY_KWARGS = dict(
name="John Smith",
identity_class="individual",
)
-# Minimum required args for an Indicator instance
INDICATOR_KWARGS = dict(
labels=['malicious-activity'],
pattern="[file:hashes.MD5 = 'd41d8cd98f00b204e9800998ecf8427e']",
)
-# Minimum required args for a Malware instance
+INTRUSION_SET_KWARGS = dict(
+ name="Bobcat Breakin",
+)
+
MALWARE_KWARGS = dict(
labels=['ransomware'],
name="Cryptolocker",
)
-# All required args for a Malware instance, plus some optional args
MALWARE_MORE_KWARGS = dict(
type='malware',
id=MALWARE_ID,
@@ -81,14 +89,45 @@ MALWARE_MORE_KWARGS = dict(
description="A ransomware related to ..."
)
-# Minimum required args for a Relationship instance
+OBSERVED_DATA_KWARGS = dict(
+ first_observed=FAKE_TIME,
+ last_observed=FAKE_TIME,
+ number_observed=1,
+ objects={
+ "0": {
+ "type": "windows-registry-key",
+ "key": "HKEY_LOCAL_MACHINE\\System\\Foo\\Bar",
+ }
+ }
+)
+
+REPORT_KWARGS = dict(
+ labels=["campaign"],
+ name="Bad Cybercrime",
+ published=FAKE_TIME,
+ object_refs=[INDICATOR_ID],
+)
+
RELATIONSHIP_KWARGS = dict(
relationship_type="indicates",
source_ref=INDICATOR_ID,
target_ref=MALWARE_ID,
)
-# Minimum required args for a Sighting instance
SIGHTING_KWARGS = dict(
sighting_of_ref=INDICATOR_ID,
)
+
+THREAT_ACTOR_KWARGS = dict(
+ labels=["crime-syndicate"],
+ name="Evil Org",
+)
+
+TOOL_KWARGS = dict(
+ labels=["remote-access"],
+ name="VNC",
+)
+
+VULNERABILITY_KWARGS = dict(
+ name="Heartbleed",
+)
diff --git a/stix2/test/test_bundle.py b/stix2/test/test_bundle.py
index 8b14172..2d14654 100644
--- a/stix2/test/test_bundle.py
+++ b/stix2/test/test_bundle.py
@@ -6,7 +6,7 @@ import stix2
EXPECTED_BUNDLE = """{
"type": "bundle",
- "id": "bundle--00000000-0000-0000-0000-000000000004",
+ "id": "bundle--00000000-0000-0000-0000-000000000007",
"spec_version": "2.0",
"objects": [
{
@@ -22,7 +22,7 @@ EXPECTED_BUNDLE = """{
},
{
"type": "malware",
- "id": "malware--00000000-0000-0000-0000-000000000002",
+ "id": "malware--00000000-0000-0000-0000-000000000003",
"created": "2017-01-01T12:34:56.000Z",
"modified": "2017-01-01T12:34:56.000Z",
"name": "Cryptolocker",
@@ -32,7 +32,7 @@ EXPECTED_BUNDLE = """{
},
{
"type": "relationship",
- "id": "relationship--00000000-0000-0000-0000-000000000003",
+ "id": "relationship--00000000-0000-0000-0000-000000000005",
"created": "2017-01-01T12:34:56.000Z",
"modified": "2017-01-01T12:34:56.000Z",
"relationship_type": "indicates",
@@ -44,7 +44,7 @@ EXPECTED_BUNDLE = """{
EXPECTED_BUNDLE_DICT = {
"type": "bundle",
- "id": "bundle--00000000-0000-0000-0000-000000000004",
+ "id": "bundle--00000000-0000-0000-0000-000000000007",
"spec_version": "2.0",
"objects": [
{
@@ -60,7 +60,7 @@ EXPECTED_BUNDLE_DICT = {
},
{
"type": "malware",
- "id": "malware--00000000-0000-0000-0000-000000000002",
+ "id": "malware--00000000-0000-0000-0000-000000000003",
"created": "2017-01-01T12:34:56.000Z",
"modified": "2017-01-01T12:34:56.000Z",
"name": "Cryptolocker",
@@ -70,7 +70,7 @@ EXPECTED_BUNDLE_DICT = {
},
{
"type": "relationship",
- "id": "relationship--00000000-0000-0000-0000-000000000003",
+ "id": "relationship--00000000-0000-0000-0000-000000000005",
"created": "2017-01-01T12:34:56.000Z",
"modified": "2017-01-01T12:34:56.000Z",
"relationship_type": "indicates",
diff --git a/stix2/test/test_custom.py b/stix2/test/test_custom.py
index 76ad61b..0e004a8 100644
--- a/stix2/test/test_custom.py
+++ b/stix2/test/test_custom.py
@@ -197,6 +197,24 @@ def test_custom_object_no_init_2():
assert no2.property1 == 'something'
+def test_custom_object_invalid_type_name():
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.sdo.CustomObject('x', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
+ class NewObj(object):
+ pass # pragma: no cover
+ assert "Invalid type name 'x': " in str(excinfo.value)
+
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.sdo.CustomObject('x_new_object', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
+ class NewObj2(object):
+ pass # pragma: no cover
+ assert "Invalid type name 'x_new_object':" in str(excinfo.value)
+
+
def test_parse_custom_object_type():
nt_string = """{
"type": "x-new-type",
@@ -221,6 +239,20 @@ def test_parse_unregistered_custom_object_type():
assert "use the CustomObject decorator." in str(excinfo.value)
+def test_parse_unregistered_custom_object_type_w_allow_custom():
+ """parse an unknown custom object, allowed by passing
+ 'allow_custom' flag
+ """
+ nt_string = """{
+ "type": "x-foobar-observable",
+ "created": "2015-12-21T19:59:11Z",
+ "property1": "something"
+ }"""
+
+ custom_obj = stix2.parse(nt_string, allow_custom=True)
+ assert custom_obj["type"] == "x-foobar-observable"
+
+
@stix2.observables.CustomObservable('x-new-observable', [
('property1', stix2.properties.StringProperty(required=True)),
('property2', stix2.properties.IntegerProperty()),
@@ -281,6 +313,24 @@ def test_custom_observable_object_no_init_2():
assert no2.property1 == 'something'
+def test_custom_observable_object_invalid_type_name():
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.observables.CustomObservable('x', [
+ ('property1', stix2.properties.StringProperty()),
+ ])
+ class NewObs(object):
+ pass # pragma: no cover
+ assert "Invalid observable type name 'x':" in str(excinfo.value)
+
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.observables.CustomObservable('x_new_obs', [
+ ('property1', stix2.properties.StringProperty()),
+ ])
+ class NewObs2(object):
+ pass # pragma: no cover
+ assert "Invalid observable type name 'x_new_obs':" in str(excinfo.value)
+
+
def test_custom_observable_object_invalid_ref_property():
with pytest.raises(ValueError) as excinfo:
@stix2.observables.CustomObservable('x-new-obs', [
@@ -349,6 +399,7 @@ def test_parse_custom_observable_object():
}"""
nt = stix2.parse_observable(nt_string, [])
+ assert isinstance(nt, stix2.core._STIXBase)
assert nt.property1 == 'something'
@@ -358,10 +409,46 @@ def test_parse_unregistered_custom_observable_object():
"property1": "something"
}"""
- with pytest.raises(stix2.exceptions.ParseError) as excinfo:
+ with pytest.raises(stix2.exceptions.CustomContentError) as excinfo:
stix2.parse_observable(nt_string)
assert "Can't parse unknown observable type" in str(excinfo.value)
+ parsed_custom = stix2.parse_observable(nt_string, allow_custom=True)
+ assert parsed_custom['property1'] == 'something'
+ with pytest.raises(AttributeError) as excinfo:
+ assert parsed_custom.property1 == 'something'
+ assert not isinstance(parsed_custom, stix2.core._STIXBase)
+
+
+def test_parse_unregistered_custom_observable_object_with_no_type():
+ nt_string = """{
+ "property1": "something"
+ }"""
+
+ with pytest.raises(stix2.exceptions.ParseError) as excinfo:
+ stix2.parse_observable(nt_string, allow_custom=True)
+ assert "Can't parse observable with no 'type' property" in str(excinfo.value)
+
+
+def test_parse_observed_data_with_custom_observable():
+ input_str = """{
+ "type": "observed-data",
+ "id": "observed-data--dc20c4ca-a2a3-4090-a5d5-9558c3af4758",
+ "created": "2016-04-06T19:58:16.000Z",
+ "modified": "2016-04-06T19:58:16.000Z",
+ "first_observed": "2015-12-21T19:00:00Z",
+ "last_observed": "2015-12-21T19:00:00Z",
+ "number_observed": 1,
+ "objects": {
+ "0": {
+ "type": "x-foobar-observable",
+ "property1": "something"
+ }
+ }
+ }"""
+ parsed = stix2.parse(input_str, allow_custom=True)
+ assert parsed.objects['0']['property1'] == 'something'
+
def test_parse_invalid_custom_observable_object():
nt_string = """{
@@ -421,10 +508,10 @@ def test_observed_data_with_custom_observable_object():
assert ob_data.objects['0'].property1 == 'something'
-@stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext', {
- 'property1': stix2.properties.StringProperty(required=True),
- 'property2': stix2.properties.IntegerProperty(),
-})
+@stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ('property2', stix2.properties.IntegerProperty()),
+])
class NewExtension():
def __init__(self, property2=None, **kwargs):
if property2 and property2 < 10:
@@ -465,15 +552,36 @@ def test_custom_extension_wrong_observable_type():
assert 'Cannot determine extension type' in excinfo.value.reason
+@pytest.mark.parametrize("data", [
+ """{
+ "keys": [
+ {
+ "test123": 123,
+ "test345": "aaaa"
+ }
+ ]
+}""",
+])
+def test_custom_extension_with_list_and_dict_properties_observable_type(data):
+ @stix2.observables.CustomExtension(stix2.UserAccount, 'some-extension', [
+ ('keys', stix2.properties.ListProperty(stix2.properties.DictionaryProperty, required=True))
+ ])
+ class SomeCustomExtension:
+ pass
+
+ example = SomeCustomExtension(keys=[{'test123': 123, 'test345': 'aaaa'}])
+ assert data == str(example)
+
+
def test_custom_extension_invalid_observable():
# These extensions are being applied to improperly-created Observables.
# The Observable classes should have been created with the CustomObservable decorator.
class Foo(object):
pass
with pytest.raises(ValueError) as excinfo:
- @stix2.observables.CustomExtension(Foo, 'x-new-ext', {
- 'property1': stix2.properties.StringProperty(required=True),
- })
+ @stix2.observables.CustomExtension(Foo, 'x-new-ext', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
class FooExtension():
pass # pragma: no cover
assert str(excinfo.value) == "'observable' must be a valid Observable class!"
@@ -481,9 +589,9 @@ def test_custom_extension_invalid_observable():
class Bar(stix2.observables._Observable):
pass
with pytest.raises(ValueError) as excinfo:
- @stix2.observables.CustomExtension(Bar, 'x-new-ext', {
- 'property1': stix2.properties.StringProperty(required=True),
- })
+ @stix2.observables.CustomExtension(Bar, 'x-new-ext', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
class BarExtension():
pass
assert "Unknown observable type" in str(excinfo.value)
@@ -492,35 +600,61 @@ def test_custom_extension_invalid_observable():
class Baz(stix2.observables._Observable):
_type = 'Baz'
with pytest.raises(ValueError) as excinfo:
- @stix2.observables.CustomExtension(Baz, 'x-new-ext', {
- 'property1': stix2.properties.StringProperty(required=True),
- })
+ @stix2.observables.CustomExtension(Baz, 'x-new-ext', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
class BazExtension():
pass
assert "Unknown observable type" in str(excinfo.value)
assert "Custom observables must be created with the @CustomObservable decorator." in str(excinfo.value)
+def test_custom_extension_invalid_type_name():
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.observables.CustomExtension(stix2.File, 'x', {
+ 'property1': stix2.properties.StringProperty(required=True),
+ })
+ class FooExtension():
+ pass # pragma: no cover
+ assert "Invalid extension type name 'x':" in str(excinfo.value)
+
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.observables.CustomExtension(stix2.File, 'x_new_ext', {
+ 'property1': stix2.properties.StringProperty(required=True),
+ })
+ class BlaExtension():
+ pass # pragma: no cover
+ assert "Invalid extension type name 'x_new_ext':" in str(excinfo.value)
+
+
def test_custom_extension_no_properties():
with pytest.raises(ValueError) as excinfo:
@stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext2', None)
class BarExtension():
pass
- assert "'properties' must be a dict!" in str(excinfo.value)
+ assert "Must supply a list, containing tuples." in str(excinfo.value)
def test_custom_extension_empty_properties():
+ with pytest.raises(ValueError) as excinfo:
+ @stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext2', [])
+ class BarExtension():
+ pass
+ assert "Must supply a list, containing tuples." in str(excinfo.value)
+
+
+def test_custom_extension_dict_properties():
with pytest.raises(ValueError) as excinfo:
@stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext2', {})
class BarExtension():
pass
- assert "'properties' must be a dict!" in str(excinfo.value)
+ assert "Must supply a list, containing tuples." in str(excinfo.value)
def test_custom_extension_no_init_1():
- @stix2.observables.CustomExtension(stix2.DomainName, 'x-new-extension', {
- 'property1': stix2.properties.StringProperty(required=True),
- })
+ @stix2.observables.CustomExtension(stix2.DomainName, 'x-new-extension', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
class NewExt():
pass
@@ -529,9 +663,9 @@ def test_custom_extension_no_init_1():
def test_custom_extension_no_init_2():
- @stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext2', {
- 'property1': stix2.properties.StringProperty(required=True),
- })
+ @stix2.observables.CustomExtension(stix2.DomainName, 'x-new-ext2', [
+ ('property1', stix2.properties.StringProperty(required=True)),
+ ])
class NewExt2(object):
pass
@@ -569,7 +703,11 @@ def test_parse_observable_with_unregistered_custom_extension():
with pytest.raises(ValueError) as excinfo:
stix2.parse_observable(input_str)
- assert "Can't parse Unknown extension type" in str(excinfo.value)
+ assert "Can't parse unknown extension type" in str(excinfo.value)
+
+ parsed_ob = stix2.parse_observable(input_str, allow_custom=True)
+ assert parsed_ob['extensions']['x-foobar-ext']['property1'] == 'foo'
+ assert not isinstance(parsed_ob['extensions']['x-foobar-ext'], stix2.core._STIXBase)
def test_register_custom_object():
@@ -580,3 +718,8 @@ def test_register_custom_object():
stix2._register_type(CustomObject2)
# Note that we will always check against newest OBJ_MAP.
assert (CustomObject2._type, CustomObject2) in stix2.OBJ_MAP.items()
+
+
+def test_extension_property_location():
+ assert 'extensions' in stix2.v20.observables.OBJ_MAP_OBSERVABLE['x-new-observable']._properties
+ assert 'extensions' not in stix2.v20.observables.EXT_MAP['domain-name']['x-new-ext']._properties
diff --git a/stix2/test/test_environment.py b/stix2/test/test_environment.py
index 84ca803..176d3f0 100644
--- a/stix2/test/test_environment.py
+++ b/stix2/test/test_environment.py
@@ -47,7 +47,7 @@ def test_object_factory_created():
assert ind.modified == FAKE_TIME
-def test_object_factory_external_resource():
+def test_object_factory_external_reference():
ext_ref = stix2.ExternalReference(source_name="ACME Threat Intel",
description="Threat report")
factory = stix2.ObjectFactory(external_references=ext_ref)
diff --git a/stix2/test/test_external_reference.py b/stix2/test/test_external_reference.py
index 2b79f01..9b90998 100644
--- a/stix2/test/test_external_reference.py
+++ b/stix2/test/test_external_reference.py
@@ -42,7 +42,7 @@ def test_external_reference_capec():
)
assert str(ref) == CAPEC
- assert re.match("ExternalReference\(source_name=u?'capec', external_id=u?'CAPEC-550'\)", repr(ref))
+ assert re.match("ExternalReference\\(source_name=u?'capec', external_id=u?'CAPEC-550'\\)", repr(ref))
CAPEC_URL = """{
@@ -109,7 +109,7 @@ def test_external_reference_offline():
)
assert str(ref) == OFFLINE
- assert re.match("ExternalReference\(source_name=u?'ACME Threat Intel', description=u?'Threat report'\)", repr(ref))
+ assert re.match("ExternalReference\\(source_name=u?'ACME Threat Intel', description=u?'Threat report'\\)", repr(ref))
# Yikes! This works
assert eval("stix2." + repr(ref)) == ref
diff --git a/stix2/test/test_granular_markings.py b/stix2/test/test_granular_markings.py
index f8fc803..9e024a1 100644
--- a/stix2/test/test_granular_markings.py
+++ b/stix2/test/test_granular_markings.py
@@ -2,6 +2,7 @@
import pytest
from stix2 import TLP_RED, Malware, markings
+from stix2.exceptions import MarkingNotFoundError
from .constants import MALWARE_MORE_KWARGS as MALWARE_KWARGS_CONST
from .constants import MARKING_IDS
@@ -546,6 +547,20 @@ def test_remove_marking_bad_selector():
markings.remove_markings(before, ["marking-definition--1", "marking-definition--2"], ["title"])
+def test_remove_marking_not_present():
+ before = Malware(
+ granular_markings=[
+ {
+ "selectors": ["description"],
+ "marking_ref": MARKING_IDS[0]
+ }
+ ],
+ **MALWARE_KWARGS
+ )
+ with pytest.raises(MarkingNotFoundError):
+ markings.remove_markings(before, [MARKING_IDS[1]], ["description"])
+
+
IS_MARKED_TEST_DATA = [
Malware(
granular_markings=[
@@ -1044,3 +1059,10 @@ def test_clear_marking_bad_selector(data, selector):
"""Test bad selector raises exception."""
with pytest.raises(AssertionError):
markings.clear_markings(data, selector)
+
+
+@pytest.mark.parametrize("data", CLEAR_MARKINGS_TEST_DATA)
+def test_clear_marking_not_present(data):
+ """Test clearing markings for a selector that has no associated markings."""
+ with pytest.raises(MarkingNotFoundError):
+ data = markings.clear_markings(data, ["labels"])
diff --git a/stix2/test/test_indicator.py b/stix2/test/test_indicator.py
index 78d1bf2..c9b6e56 100644
--- a/stix2/test/test_indicator.py
+++ b/stix2/test/test_indicator.py
@@ -45,6 +45,7 @@ def test_indicator_with_all_required_properties():
labels=['malicious-activity'],
)
+ assert ind.revoked is False
assert str(ind) == EXPECTED_INDICATOR
rep = re.sub(r"(\[|=| )u('|\"|\\\'|\\\")", r"\g<1>\g<2>", repr(ind))
assert rep == EXPECTED_INDICATOR_REPR
diff --git a/stix2/test/test_malware.py b/stix2/test/test_malware.py
index 8c565cd..2228885 100644
--- a/stix2/test/test_malware.py
+++ b/stix2/test/test_malware.py
@@ -126,7 +126,7 @@ def test_parse_malware(data):
def test_parse_malware_invalid_labels():
- data = re.compile('\[.+\]', re.DOTALL).sub('1', EXPECTED_MALWARE)
+ data = re.compile('\\[.+\\]', re.DOTALL).sub('1', EXPECTED_MALWARE)
with pytest.raises(ValueError) as excinfo:
stix2.parse(data)
assert "Invalid value for Malware 'labels'" in str(excinfo.value)
diff --git a/stix2/test/test_memory.py b/stix2/test/test_memory.py
index 2384848..284c43e 100644
--- a/stix2/test/test_memory.py
+++ b/stix2/test/test_memory.py
@@ -136,6 +136,19 @@ def rel_mem_store():
yield MemoryStore(stix_objs)
+@pytest.fixture
+def fs_mem_store(request, mem_store):
+ filename = 'memory_test/mem_store.json'
+ mem_store.save_to_file(filename)
+
+ def fin():
+ # teardown, excecuted regardless of exception
+ shutil.rmtree(os.path.dirname(filename))
+ request.addfinalizer(fin)
+
+ return filename
+
+
def test_memory_source_get(mem_source):
resp = mem_source.get("indicator--d81f86b8-975b-bc0b-775e-810c5ad45a4f")
assert resp["id"] == "indicator--d81f86b8-975b-bc0b-775e-810c5ad45a4f"
@@ -187,9 +200,11 @@ def test_memory_store_query_multiple_filters(mem_store):
assert len(resp) == 1
-def test_memory_store_save_load_file(mem_store):
- filename = 'memory_test/mem_store.json'
- mem_store.save_to_file(filename)
+def test_memory_store_save_load_file(mem_store, fs_mem_store):
+ filename = fs_mem_store # the fixture fs_mem_store yields filename where the memory store was written to
+
+ # STIX2 contents of mem_store have already been written to file
+ # (this is done in fixture 'fs_mem_store'), so can already read-in here
contents = open(os.path.abspath(filename)).read()
assert '"id": "indicator--d81f86b9-975b-bc0b-775e-810c5ad45a4f",' in contents
@@ -200,8 +215,6 @@ def test_memory_store_save_load_file(mem_store):
assert mem_store2.get("indicator--d81f86b8-975b-bc0b-775e-810c5ad45a4f")
assert mem_store2.get("indicator--d81f86b9-975b-bc0b-775e-810c5ad45a4f")
- shutil.rmtree(os.path.dirname(filename))
-
def test_memory_store_add_invalid_object(mem_store):
ind = ('indicator', IND1) # tuple isn't valid
diff --git a/stix2/test/test_pattern_expressions.py b/stix2/test/test_pattern_expressions.py
index 0db1083..74a7d0f 100644
--- a/stix2/test/test_pattern_expressions.py
+++ b/stix2/test/test_pattern_expressions.py
@@ -1,3 +1,7 @@
+import datetime
+
+import pytest
+
import stix2
@@ -67,7 +71,11 @@ def test_file_observable_expression():
assert str(exp) == "[file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f' AND file:mime_type = 'application/x-pdf']" # noqa
-def test_multiple_file_observable_expression():
+@pytest.mark.parametrize("observation_class, op", [
+ (stix2.AndObservationExpression, 'AND'),
+ (stix2.OrObservationExpression, 'OR'),
+])
+def test_multiple_file_observable_expression(observation_class, op):
exp1 = stix2.EqualityComparisonExpression("file:hashes.'SHA-256'",
stix2.HashConstant(
"bf07a7fbb825fc0aae7bf4a1177b2b31fcf8a3feeaf7092761e18c859ee52a9c",
@@ -81,8 +89,8 @@ def test_multiple_file_observable_expression():
'SHA-256'))
op1_exp = stix2.ObservationExpression(bool1_exp)
op2_exp = stix2.ObservationExpression(exp3)
- exp = stix2.AndObservationExpression([op1_exp, op2_exp])
- assert str(exp) == "[file:hashes.'SHA-256' = 'bf07a7fbb825fc0aae7bf4a1177b2b31fcf8a3feeaf7092761e18c859ee52a9c' OR file:hashes.MD5 = 'cead3f77f6cda6ec00f57d76c9a6879f'] AND [file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f']" # noqa
+ exp = observation_class([op1_exp, op2_exp])
+ assert str(exp) == "[file:hashes.'SHA-256' = 'bf07a7fbb825fc0aae7bf4a1177b2b31fcf8a3feeaf7092761e18c859ee52a9c' OR file:hashes.MD5 = 'cead3f77f6cda6ec00f57d76c9a6879f'] {} [file:hashes.'SHA-256' = 'aec070645fe53ee3b3763059376134f058cc337247c978add178b6ccdfb0019f']".format(op) # noqa
def test_root_types():
@@ -120,6 +128,31 @@ def test_greater_than():
assert str(exp) == "[file:extensions.windows-pebinary-ext.sections[*].entropy > 7.0]"
+def test_less_than():
+ exp = stix2.LessThanComparisonExpression("file:size",
+ 1024)
+ assert str(exp) == "file:size < 1024"
+
+
+def test_greater_than_or_equal():
+ exp = stix2.GreaterThanEqualComparisonExpression("file:size",
+ 1024)
+ assert str(exp) == "file:size >= 1024"
+
+
+def test_less_than_or_equal():
+ exp = stix2.LessThanEqualComparisonExpression("file:size",
+ 1024)
+ assert str(exp) == "file:size <= 1024"
+
+
+def test_not():
+ exp = stix2.LessThanComparisonExpression("file:size",
+ 1024,
+ negated=True)
+ assert str(exp) == "file:size NOT < 1024"
+
+
def test_and_observable_expression():
exp1 = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:account_type",
"unix"),
@@ -145,6 +178,15 @@ def test_and_observable_expression():
assert str(exp) == "[user-account:account_type = 'unix' AND user-account:user_id = '1007' AND user-account:account_login = 'Peter'] AND [user-account:account_type = 'unix' AND user-account:user_id = '1008' AND user-account:account_login = 'Paul'] AND [user-account:account_type = 'unix' AND user-account:user_id = '1009' AND user-account:account_login = 'Mary']" # noqa
+def test_invalid_and_observable_expression():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("user-account:display_name",
+ "admin"),
+ stix2.EqualityComparisonExpression("email-addr:display_name",
+ stix2.StringConstant("admin"))])
+ assert "All operands to an 'AND' expression must have the same object type" in str(excinfo)
+
+
def test_hex():
exp_and = stix2.AndBooleanExpression([stix2.EqualityComparisonExpression("file:mime_type",
"image/bmp"),
@@ -175,3 +217,158 @@ def test_set_op():
def test_timestamp():
ts = stix2.TimestampConstant('2014-01-13T07:03:17Z')
assert str(ts) == "t'2014-01-13T07:03:17Z'"
+
+
+def test_boolean():
+ exp = stix2.EqualityComparisonExpression("email-message:is_multipart",
+ True)
+ assert str(exp) == "email-message:is_multipart = true"
+
+
+def test_binary():
+ const = stix2.BinaryConstant("dGhpcyBpcyBhIHRlc3Q=")
+ exp = stix2.EqualityComparisonExpression("artifact:payload_bin",
+ const)
+ assert str(exp) == "artifact:payload_bin = b'dGhpcyBpcyBhIHRlc3Q='"
+
+
+def test_list():
+ exp = stix2.InComparisonExpression("process:name",
+ ['proccy', 'proximus', 'badproc'])
+ assert str(exp) == "process:name IN ('proccy', 'proximus', 'badproc')"
+
+
+def test_list2():
+ # alternate way to construct an "IN" Comparison Expression
+ exp = stix2.EqualityComparisonExpression("process:name",
+ ['proccy', 'proximus', 'badproc'])
+ assert str(exp) == "process:name IN ('proccy', 'proximus', 'badproc')"
+
+
+def test_invalid_constant_type():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.EqualityComparisonExpression("artifact:payload_bin",
+ {'foo': 'bar'})
+ assert 'Unable to create a constant' in str(excinfo)
+
+
+def test_invalid_integer_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.IntegerConstant('foo')
+ assert 'must be an integer' in str(excinfo)
+
+
+def test_invalid_timestamp_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.TimestampConstant('foo')
+ assert 'must be a datetime object or timestamp string' in str(excinfo)
+
+
+def test_invalid_float_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.FloatConstant('foo')
+ assert 'must be a float' in str(excinfo)
+
+
+@pytest.mark.parametrize("data, result", [
+ (True, True),
+ (False, False),
+ ('True', True),
+ ('False', False),
+ ('true', True),
+ ('false', False),
+ ('t', True),
+ ('f', False),
+ ('T', True),
+ ('F', False),
+ (1, True),
+ (0, False),
+])
+def test_boolean_constant(data, result):
+ boolean = stix2.BooleanConstant(data)
+ assert boolean.value == result
+
+
+def test_invalid_boolean_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.BooleanConstant('foo')
+ assert 'must be a boolean' in str(excinfo)
+
+
+@pytest.mark.parametrize("hashtype, data", [
+ ('MD5', 'zzz'),
+ ('ssdeep', 'zzz=='),
+])
+def test_invalid_hash_constant(hashtype, data):
+ with pytest.raises(ValueError) as excinfo:
+ stix2.HashConstant(data, hashtype)
+ assert 'is not a valid {} hash'.format(hashtype) in str(excinfo)
+
+
+def test_invalid_hex_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.HexConstant('mm')
+ assert "must contain an even number of hexadecimal characters" in str(excinfo)
+
+
+def test_invalid_binary_constant():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.BinaryConstant('foo')
+ assert 'must contain a base64' in str(excinfo)
+
+
+def test_escape_quotes_and_backslashes():
+ exp = stix2.MatchesComparisonExpression("file:name",
+ "^Final Report.+\\.exe$")
+ assert str(exp) == "file:name MATCHES '^Final Report.+\\\\.exe$'"
+
+
+def test_like():
+ exp = stix2.LikeComparisonExpression("directory:path",
+ "C:\\Windows\\%\\foo")
+ assert str(exp) == "directory:path LIKE 'C:\\\\Windows\\\\%\\\\foo'"
+
+
+def test_issuperset():
+ exp = stix2.IsSupersetComparisonExpression("ipv4-addr:value",
+ "198.51.100.0/24")
+ assert str(exp) == "ipv4-addr:value ISSUPERSET '198.51.100.0/24'"
+
+
+def test_repeat_qualifier():
+ qual = stix2.RepeatQualifier(stix2.IntegerConstant(5))
+ assert str(qual) == 'REPEATS 5 TIMES'
+
+
+def test_invalid_repeat_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.RepeatQualifier('foo')
+ assert 'is not a valid argument for a Repeat Qualifier' in str(excinfo)
+
+
+def test_invalid_within_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.WithinQualifier('foo')
+ assert 'is not a valid argument for a Within Qualifier' in str(excinfo)
+
+
+def test_startstop_qualifier():
+ qual = stix2.StartStopQualifier(stix2.TimestampConstant('2016-06-01T00:00:00Z'),
+ datetime.datetime(2017, 3, 12, 8, 30, 0))
+ assert str(qual) == "START t'2016-06-01T00:00:00Z' STOP t'2017-03-12T08:30:00Z'"
+
+ qual2 = stix2.StartStopQualifier(datetime.date(2016, 6, 1),
+ stix2.TimestampConstant('2016-07-01T00:00:00Z'))
+ assert str(qual2) == "START t'2016-06-01T00:00:00Z' STOP t'2016-07-01T00:00:00Z'"
+
+
+def test_invalid_startstop_qualifier():
+ with pytest.raises(ValueError) as excinfo:
+ stix2.StartStopQualifier('foo',
+ stix2.TimestampConstant('2016-06-01T00:00:00Z'))
+ assert 'is not a valid argument for a Start/Stop Qualifier' in str(excinfo)
+
+ with pytest.raises(ValueError) as excinfo:
+ stix2.StartStopQualifier(datetime.date(2016, 6, 1),
+ 'foo')
+ assert 'is not a valid argument for a Start/Stop Qualifier' in str(excinfo)
diff --git a/stix2/test/test_properties.py b/stix2/test/test_properties.py
index 6bd1888..16ff06a 100644
--- a/stix2/test/test_properties.py
+++ b/stix2/test/test_properties.py
@@ -1,6 +1,6 @@
import pytest
-from stix2 import EmailMIMEComponent, ExtensionsProperty, TCPExt
+from stix2 import CustomObject, EmailMIMEComponent, ExtensionsProperty, TCPExt
from stix2.exceptions import AtLeastOnePropertyError, DictionaryKeyError
from stix2.properties import (BinaryProperty, BooleanProperty,
DictionaryProperty, EmbeddedObjectProperty,
@@ -16,6 +16,8 @@ def test_property():
p = Property()
assert p.required is False
+ assert p.clean('foo') == 'foo'
+ assert p.clean(3) == 3
def test_basic_clean():
@@ -264,6 +266,17 @@ def test_dictionary_property_invalid(d):
assert str(excinfo.value) == d[1]
+def test_property_list_of_dictionary():
+ @CustomObject('x-new-obj', [
+ ('property1', ListProperty(DictionaryProperty(), required=True)),
+ ])
+ class NewObj():
+ pass
+
+ test_obj = NewObj(property1=[{'foo': 'bar'}])
+ assert test_obj.property1[0]['foo'] == 'bar'
+
+
@pytest.mark.parametrize("value", [
{"sha256": "6db12788c37247f2316052e142f42f4b259d6561751e5f401a1ae2a6df9c674b"},
[('MD5', '2dfb1bcc980200c6706feee399d41b3f'), ('RIPEMD-160', 'b3a8cd8a27c90af79b3c81754f267780f443dfef')],
diff --git a/stix2/test/test_relationship.py b/stix2/test/test_relationship.py
index c6e2d6f..32ddf91 100644
--- a/stix2/test/test_relationship.py
+++ b/stix2/test/test_relationship.py
@@ -123,8 +123,8 @@ def test_create_relationship_from_objects_rather_than_ids(indicator, malware):
assert rel.relationship_type == 'indicates'
assert rel.source_ref == 'indicator--00000000-0000-0000-0000-000000000001'
- assert rel.target_ref == 'malware--00000000-0000-0000-0000-000000000002'
- assert rel.id == 'relationship--00000000-0000-0000-0000-000000000003'
+ assert rel.target_ref == 'malware--00000000-0000-0000-0000-000000000003'
+ assert rel.id == 'relationship--00000000-0000-0000-0000-000000000005'
def test_create_relationship_with_positional_args(indicator, malware):
@@ -132,8 +132,8 @@ def test_create_relationship_with_positional_args(indicator, malware):
assert rel.relationship_type == 'indicates'
assert rel.source_ref == 'indicator--00000000-0000-0000-0000-000000000001'
- assert rel.target_ref == 'malware--00000000-0000-0000-0000-000000000002'
- assert rel.id == 'relationship--00000000-0000-0000-0000-000000000003'
+ assert rel.target_ref == 'malware--00000000-0000-0000-0000-000000000003'
+ assert rel.id == 'relationship--00000000-0000-0000-0000-000000000005'
@pytest.mark.parametrize("data", [
diff --git a/stix2/test/test_sighting.py b/stix2/test/test_sighting.py
index ce1fab9..06f96b8 100644
--- a/stix2/test/test_sighting.py
+++ b/stix2/test/test_sighting.py
@@ -86,7 +86,7 @@ def test_create_sighting_from_objects_rather_than_ids(malware): # noqa: F811
rel = stix2.Sighting(sighting_of_ref=malware)
assert rel.sighting_of_ref == 'malware--00000000-0000-0000-0000-000000000001'
- assert rel.id == 'sighting--00000000-0000-0000-0000-000000000002'
+ assert rel.id == 'sighting--00000000-0000-0000-0000-000000000003'
@pytest.mark.parametrize("data", [
diff --git a/stix2/test/test_tool.py b/stix2/test/test_tool.py
index 21ece24..9fc2c22 100644
--- a/stix2/test/test_tool.py
+++ b/stix2/test/test_tool.py
@@ -19,6 +19,19 @@ EXPECTED = """{
]
}"""
+EXPECTED_WITH_REVOKED = """{
+ "type": "tool",
+ "id": "tool--8e2e2d2b-17d4-4cbf-938f-98ee46b3cd3f",
+ "created_by_ref": "identity--f431f809-377b-45e0-aa1c-6a4751cae5ff",
+ "created": "2016-04-06T20:03:48.000Z",
+ "modified": "2016-04-06T20:03:48.000Z",
+ "name": "VNC",
+ "revoked": false,
+ "labels": [
+ "remote-access"
+ ]
+}"""
+
def test_tool_example():
tool = stix2.Tool(
@@ -58,4 +71,24 @@ def test_parse_tool(data):
assert tool.labels == ["remote-access"]
assert tool.name == "VNC"
+
+def test_tool_no_workbench_wrappers():
+ tool = stix2.Tool(name='VNC', labels=['remote-access'])
+ with pytest.raises(AttributeError):
+ tool.created_by()
+
+
+def test_tool_serialize_with_defaults():
+ tool = stix2.Tool(
+ id="tool--8e2e2d2b-17d4-4cbf-938f-98ee46b3cd3f",
+ created_by_ref="identity--f431f809-377b-45e0-aa1c-6a4751cae5ff",
+ created="2016-04-06T20:03:48.000Z",
+ modified="2016-04-06T20:03:48.000Z",
+ name="VNC",
+ labels=["remote-access"],
+ )
+
+ assert tool.serialize(pretty=True, include_optional_defaults=True) == EXPECTED_WITH_REVOKED
+
+
# TODO: Add other examples
diff --git a/stix2/test/test_utils.py b/stix2/test/test_utils.py
index 9d25a5e..655cd61 100644
--- a/stix2/test/test_utils.py
+++ b/stix2/test/test_utils.py
@@ -62,7 +62,7 @@ def test_parse_datetime_invalid(ts):
[("a", 1,)],
])
def test_get_dict(data):
- assert stix2.utils.get_dict(data)
+ assert stix2.utils._get_dict(data)
@pytest.mark.parametrize('data', [
@@ -73,7 +73,7 @@ def test_get_dict(data):
])
def test_get_dict_invalid(data):
with pytest.raises(ValueError):
- stix2.utils.get_dict(data)
+ stix2.utils._get_dict(data)
@pytest.mark.parametrize('stix_id, typ', [
diff --git a/stix2/test/test_versioning.py b/stix2/test/test_versioning.py
index 233587e..fa3bddb 100644
--- a/stix2/test/test_versioning.py
+++ b/stix2/test/test_versioning.py
@@ -233,12 +233,19 @@ def test_remove_custom_stix_object():
("animal_class", stix2.properties.StringProperty()),
])
class Animal(object):
- def __init__(self, animal_class=None, **kwargs):
- if animal_class and animal_class not in ["mammal", "bird"]:
- raise ValueError("Not a recognized class of animal")
+ pass
animal = Animal(species="lion", animal_class="mammal")
nc = stix2.utils.remove_custom_stix(animal)
assert nc is None
+
+
+def test_remove_custom_stix_no_custom():
+ campaign_v1 = stix2.Campaign(**CAMPAIGN_MORE_KWARGS)
+ campaign_v2 = stix2.utils.remove_custom_stix(campaign_v1)
+
+ assert len(campaign_v1.keys()) == len(campaign_v2.keys())
+ assert campaign_v1.id == campaign_v2.id
+ assert campaign_v1.description == campaign_v2.description
diff --git a/stix2/test/test_workbench.py b/stix2/test/test_workbench.py
new file mode 100644
index 0000000..7857eb2
--- /dev/null
+++ b/stix2/test/test_workbench.py
@@ -0,0 +1,262 @@
+import os
+
+import stix2
+from stix2.workbench import (AttackPattern, Campaign, CourseOfAction,
+ ExternalReference, FileSystemSource, Filter,
+ Identity, Indicator, IntrusionSet, Malware,
+ MarkingDefinition, ObservedData, Relationship,
+ Report, StatementMarking, ThreatActor, Tool,
+ Vulnerability, add_data_source, all_versions,
+ attack_patterns, campaigns, courses_of_action,
+ create, get, identities, indicators,
+ intrusion_sets, malware, observed_data, query,
+ reports, save, set_default_created,
+ set_default_creator, set_default_external_refs,
+ set_default_object_marking_refs, threat_actors,
+ tools, vulnerabilities)
+
+from .constants import (ATTACK_PATTERN_ID, ATTACK_PATTERN_KWARGS, CAMPAIGN_ID,
+ CAMPAIGN_KWARGS, COURSE_OF_ACTION_ID,
+ COURSE_OF_ACTION_KWARGS, IDENTITY_ID, IDENTITY_KWARGS,
+ INDICATOR_ID, INDICATOR_KWARGS, INTRUSION_SET_ID,
+ INTRUSION_SET_KWARGS, MALWARE_ID, MALWARE_KWARGS,
+ OBSERVED_DATA_ID, OBSERVED_DATA_KWARGS, REPORT_ID,
+ REPORT_KWARGS, THREAT_ACTOR_ID, THREAT_ACTOR_KWARGS,
+ TOOL_ID, TOOL_KWARGS, VULNERABILITY_ID,
+ VULNERABILITY_KWARGS)
+
+
+def test_workbench_environment():
+
+ # Create a STIX object
+ ind = create(Indicator, id=INDICATOR_ID, **INDICATOR_KWARGS)
+ save(ind)
+
+ resp = get(INDICATOR_ID)
+ assert resp['labels'][0] == 'malicious-activity'
+
+ resp = all_versions(INDICATOR_ID)
+ assert len(resp) == 1
+
+ # Search on something other than id
+ q = [Filter('type', '=', 'vulnerability')]
+ resp = query(q)
+ assert len(resp) == 0
+
+
+def test_workbench_get_all_attack_patterns():
+ mal = AttackPattern(id=ATTACK_PATTERN_ID, **ATTACK_PATTERN_KWARGS)
+ save(mal)
+
+ resp = attack_patterns()
+ assert len(resp) == 1
+ assert resp[0].id == ATTACK_PATTERN_ID
+
+
+def test_workbench_get_all_campaigns():
+ cam = Campaign(id=CAMPAIGN_ID, **CAMPAIGN_KWARGS)
+ save(cam)
+
+ resp = campaigns()
+ assert len(resp) == 1
+ assert resp[0].id == CAMPAIGN_ID
+
+
+def test_workbench_get_all_courses_of_action():
+ coa = CourseOfAction(id=COURSE_OF_ACTION_ID, **COURSE_OF_ACTION_KWARGS)
+ save(coa)
+
+ resp = courses_of_action()
+ assert len(resp) == 1
+ assert resp[0].id == COURSE_OF_ACTION_ID
+
+
+def test_workbench_get_all_identities():
+ idty = Identity(id=IDENTITY_ID, **IDENTITY_KWARGS)
+ save(idty)
+
+ resp = identities()
+ assert len(resp) == 1
+ assert resp[0].id == IDENTITY_ID
+
+
+def test_workbench_get_all_indicators():
+ resp = indicators()
+ assert len(resp) == 1
+ assert resp[0].id == INDICATOR_ID
+
+
+def test_workbench_get_all_intrusion_sets():
+ ins = IntrusionSet(id=INTRUSION_SET_ID, **INTRUSION_SET_KWARGS)
+ save(ins)
+
+ resp = intrusion_sets()
+ assert len(resp) == 1
+ assert resp[0].id == INTRUSION_SET_ID
+
+
+def test_workbench_get_all_malware():
+ mal = Malware(id=MALWARE_ID, **MALWARE_KWARGS)
+ save(mal)
+
+ resp = malware()
+ assert len(resp) == 1
+ assert resp[0].id == MALWARE_ID
+
+
+def test_workbench_get_all_observed_data():
+ od = ObservedData(id=OBSERVED_DATA_ID, **OBSERVED_DATA_KWARGS)
+ save(od)
+
+ resp = observed_data()
+ assert len(resp) == 1
+ assert resp[0].id == OBSERVED_DATA_ID
+
+
+def test_workbench_get_all_reports():
+ rep = Report(id=REPORT_ID, **REPORT_KWARGS)
+ save(rep)
+
+ resp = reports()
+ assert len(resp) == 1
+ assert resp[0].id == REPORT_ID
+
+
+def test_workbench_get_all_threat_actors():
+ thr = ThreatActor(id=THREAT_ACTOR_ID, **THREAT_ACTOR_KWARGS)
+ save(thr)
+
+ resp = threat_actors()
+ assert len(resp) == 1
+ assert resp[0].id == THREAT_ACTOR_ID
+
+
+def test_workbench_get_all_tools():
+ tool = Tool(id=TOOL_ID, **TOOL_KWARGS)
+ save(tool)
+
+ resp = tools()
+ assert len(resp) == 1
+ assert resp[0].id == TOOL_ID
+
+
+def test_workbench_get_all_vulnerabilities():
+ vuln = Vulnerability(id=VULNERABILITY_ID, **VULNERABILITY_KWARGS)
+ save(vuln)
+
+ resp = vulnerabilities()
+ assert len(resp) == 1
+ assert resp[0].id == VULNERABILITY_ID
+
+
+def test_workbench_relationships():
+ rel = Relationship(INDICATOR_ID, 'indicates', MALWARE_ID)
+ save(rel)
+
+ ind = get(INDICATOR_ID)
+ resp = ind.relationships()
+ assert len(resp) == 1
+ assert resp[0].relationship_type == 'indicates'
+ assert resp[0].source_ref == INDICATOR_ID
+ assert resp[0].target_ref == MALWARE_ID
+
+
+def test_workbench_created_by():
+ intset = IntrusionSet(name="Breach 123", created_by_ref=IDENTITY_ID)
+ save(intset)
+ creator = intset.created_by()
+ assert creator.id == IDENTITY_ID
+
+
+def test_workbench_related():
+ rel1 = Relationship(MALWARE_ID, 'targets', IDENTITY_ID)
+ rel2 = Relationship(CAMPAIGN_ID, 'uses', MALWARE_ID)
+ save([rel1, rel2])
+
+ resp = get(MALWARE_ID).related()
+ assert len(resp) == 3
+ assert any(x['id'] == CAMPAIGN_ID for x in resp)
+ assert any(x['id'] == INDICATOR_ID for x in resp)
+ assert any(x['id'] == IDENTITY_ID for x in resp)
+
+ resp = get(MALWARE_ID).related(relationship_type='indicates')
+ assert len(resp) == 1
+
+
+def test_workbench_related_with_filters():
+ malware = Malware(labels=["ransomware"], name="CryptorBit", created_by_ref=IDENTITY_ID)
+ rel = Relationship(malware.id, 'variant-of', MALWARE_ID)
+ save([malware, rel])
+
+ filters = [Filter('created_by_ref', '=', IDENTITY_ID)]
+ resp = get(MALWARE_ID).related(filters=filters)
+
+ assert len(resp) == 1
+ assert resp[0].name == malware.name
+ assert resp[0].created_by_ref == IDENTITY_ID
+
+ # filters arg can also be single filter
+ resp = get(MALWARE_ID).related(filters=filters[0])
+ assert len(resp) == 1
+
+
+def test_add_data_source():
+ fs_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "stix2_data")
+ fs = FileSystemSource(fs_path)
+ add_data_source(fs)
+
+ resp = tools()
+ assert len(resp) == 3
+ resp_ids = [tool.id for tool in resp]
+ assert TOOL_ID in resp_ids
+ assert 'tool--03342581-f790-4f03-ba41-e82e67392e23' in resp_ids
+ assert 'tool--242f3da3-4425-4d11-8f5c-b842886da966' in resp_ids
+
+
+def test_additional_filter():
+ resp = tools(Filter('created_by_ref', '=', 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5'))
+ assert len(resp) == 2
+
+
+def test_additional_filters_list():
+ resp = tools([Filter('created_by_ref', '=', 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5'),
+ Filter('name', '=', 'Windows Credential Editor')])
+ assert len(resp) == 1
+
+
+def test_default_creator():
+ set_default_creator(IDENTITY_ID)
+ campaign = Campaign(**CAMPAIGN_KWARGS)
+
+ assert 'created_by_ref' not in CAMPAIGN_KWARGS
+ assert campaign.created_by_ref == IDENTITY_ID
+
+
+def test_default_created_timestamp():
+ timestamp = "2018-03-19T01:02:03.000Z"
+ set_default_created(timestamp)
+ campaign = Campaign(**CAMPAIGN_KWARGS)
+
+ assert 'created' not in CAMPAIGN_KWARGS
+ assert stix2.utils.format_datetime(campaign.created) == timestamp
+ assert stix2.utils.format_datetime(campaign.modified) == timestamp
+
+
+def test_default_external_refs():
+ ext_ref = ExternalReference(source_name="ACME Threat Intel",
+ description="Threat report")
+ set_default_external_refs(ext_ref)
+ campaign = Campaign(**CAMPAIGN_KWARGS)
+
+ assert campaign.external_references[0].source_name == "ACME Threat Intel"
+ assert campaign.external_references[0].description == "Threat report"
+
+
+def test_default_object_marking_refs():
+ stmt_marking = StatementMarking("Copyright 2016, Example Corp")
+ mark_def = MarkingDefinition(definition_type="statement",
+ definition=stmt_marking)
+ set_default_object_marking_refs(mark_def)
+ campaign = Campaign(**CAMPAIGN_KWARGS)
+
+ assert campaign.object_marking_refs[0] == mark_def.id
diff --git a/stix2/utils.py b/stix2/utils.py
index 37ff166..619652a 100644
--- a/stix2/utils.py
+++ b/stix2/utils.py
@@ -18,6 +18,8 @@ NOW = object()
# STIX object properties that cannot be modified
STIX_UNMOD_PROPERTIES = ["created", "created_by_ref", "id", "type"]
+TYPE_REGEX = r'^\-?[a-z0-9]+(-[a-z0-9]+)*\-?$'
+
class STIXdatetime(dt.datetime):
def __new__(cls, *args, **kwargs):
@@ -140,7 +142,7 @@ def parse_into_datetime(value, precision=None):
return STIXdatetime(ts, precision=precision)
-def get_dict(data):
+def _get_dict(data):
"""Return data as a dictionary.
Input can be a dictionary, string, or file-like object.
@@ -166,7 +168,7 @@ def get_dict(data):
def find_property_index(obj, properties, tuple_to_find):
"""Recursively find the property in the object model, return the index
according to the _properties OrderedDict. If it's a list look for
- individual objects.
+ individual objects. Returns and integer indicating its location
"""
from .base import _STIXBase
try:
@@ -183,6 +185,11 @@ def find_property_index(obj, properties, tuple_to_find):
tuple_to_find)
if val is not None:
return val
+ elif isinstance(item, dict):
+ for idx, val in enumerate(sorted(item)):
+ if (tuple_to_find[0] == val and
+ item.get(val) == tuple_to_find[1]):
+ return idx
elif isinstance(pv, dict):
if pv.get(tuple_to_find[0]) is not None:
try:
@@ -263,11 +270,11 @@ def get_class_hierarchy_names(obj):
def remove_custom_stix(stix_obj):
- """remove any custom STIX objects or properties
+ """Remove any custom STIX objects or properties.
Warning: This function is a best effort utility, in that
it will remove custom objects and properties based on the
- type names; i.e. if "x-" prefixes object types, and "x_"
+ type names; i.e. if "x-" prefixes object types, and "x\\_"
prefixes property types. According to the STIX2 spec,
those naming conventions are a SHOULDs not MUSTs, meaning
that valid custom STIX content may ignore those conventions
diff --git a/stix2/v20/common.py b/stix2/v20/common.py
index e915df6..0bba3b1 100644
--- a/stix2/v20/common.py
+++ b/stix2/v20/common.py
@@ -7,7 +7,7 @@ from ..markings import _MarkingsMixin
from ..properties import (HashesProperty, IDProperty, ListProperty, Property,
ReferenceProperty, SelectorProperty, StringProperty,
TimestampProperty, TypeProperty)
-from ..utils import NOW, get_dict
+from ..utils import NOW, _get_dict
class ExternalReference(_STIXBase):
@@ -125,7 +125,7 @@ class MarkingDefinition(_STIXBase, _MarkingsMixin):
raise ValueError("definition_type must be a valid marking type")
if not isinstance(kwargs['definition'], marking_type):
- defn = get_dict(kwargs['definition'])
+ defn = _get_dict(kwargs['definition'])
kwargs['definition'] = marking_type(**defn)
super(MarkingDefinition, self).__init__(**kwargs)
diff --git a/stix2/v20/observables.py b/stix2/v20/observables.py
index 83600b0..6c65e8e 100644
--- a/stix2/v20/observables.py
+++ b/stix2/v20/observables.py
@@ -6,16 +6,18 @@ Observable and do not have a ``_type`` attribute.
"""
from collections import OrderedDict
+import copy
+import re
from ..base import _Extension, _Observable, _STIXBase
-from ..exceptions import (AtLeastOnePropertyError, DependentPropertiesError,
- ParseError)
+from ..exceptions import (AtLeastOnePropertyError, CustomContentError,
+ DependentPropertiesError, ParseError)
from ..properties import (BinaryProperty, BooleanProperty, DictionaryProperty,
EmbeddedObjectProperty, EnumProperty, FloatProperty,
HashesProperty, HexProperty, IntegerProperty,
ListProperty, ObjectReferenceProperty, Property,
StringProperty, TimestampProperty, TypeProperty)
-from ..utils import get_dict
+from ..utils import TYPE_REGEX, _get_dict
class ObservableProperty(Property):
@@ -24,7 +26,11 @@ class ObservableProperty(Property):
def clean(self, value):
try:
- dictified = get_dict(value)
+ dictified = _get_dict(value)
+ # get deep copy since we are going modify the dict and might
+ # modify the original dict as _get_dict() does not return new
+ # dict when passed a dict
+ dictified = copy.deepcopy(dictified)
except ValueError:
raise ValueError("The observable property must contain a dictionary")
if dictified == {}:
@@ -49,7 +55,11 @@ class ExtensionsProperty(DictionaryProperty):
def clean(self, value):
try:
- dictified = get_dict(value)
+ dictified = _get_dict(value)
+ # get deep copy since we are going modify the dict and might
+ # modify the original dict as _get_dict() does not return new
+ # dict when passed a dict
+ dictified = copy.deepcopy(dictified)
except ValueError:
raise ValueError("The extensions property must contain a dictionary")
if dictified == {}:
@@ -67,7 +77,7 @@ class ExtensionsProperty(DictionaryProperty):
else:
raise ValueError("Cannot determine extension type.")
else:
- raise ValueError("The key used in the extensions dictionary is not an extension type name")
+ raise CustomContentError("Can't parse unknown extension type: {}".format(key))
else:
raise ValueError("The enclosing type '%s' has no extensions defined" % self.enclosing_type)
return dictified
@@ -915,7 +925,12 @@ def parse_observable(data, _valid_refs=None, allow_custom=False):
An instantiated Python STIX Cyber Observable object.
"""
- obj = get_dict(data)
+ obj = _get_dict(data)
+ # get deep copy since we are going modify the dict and might
+ # modify the original dict as _get_dict() does not return new
+ # dict when passed a dict
+ obj = copy.deepcopy(obj)
+
obj['_valid_refs'] = _valid_refs or []
if 'type' not in obj:
@@ -923,15 +938,23 @@ def parse_observable(data, _valid_refs=None, allow_custom=False):
try:
obj_class = OBJ_MAP_OBSERVABLE[obj['type']]
except KeyError:
- raise ParseError("Can't parse unknown observable type '%s'! For custom observables, "
- "use the CustomObservable decorator." % obj['type'])
+ if allow_custom:
+ # flag allows for unknown custom objects too, but will not
+ # be parsed into STIX observable object, just returned as is
+ return obj
+ raise CustomContentError("Can't parse unknown observable type '%s'! For custom observables, "
+ "use the CustomObservable decorator." % obj['type'])
if 'extensions' in obj and obj['type'] in EXT_MAP:
for name, ext in obj['extensions'].items():
- if name not in EXT_MAP[obj['type']]:
- raise ParseError("Can't parse Unknown extension type '%s' for observable type '%s'!" % (name, obj['type']))
- ext_class = EXT_MAP[obj['type']][name]
- obj['extensions'][name] = ext_class(allow_custom=allow_custom, **obj['extensions'][name])
+ try:
+ ext_class = EXT_MAP[obj['type']][name]
+ except KeyError:
+ if not allow_custom:
+ raise CustomContentError("Can't parse unknown extension type '%s'"
+ "for observable type '%s'!" % (name, obj['type']))
+ else: # extension was found
+ obj['extensions'][name] = ext_class(allow_custom=allow_custom, **obj['extensions'][name])
return obj_class(allow_custom=allow_custom, **obj)
@@ -959,6 +982,12 @@ def CustomObservable(type='x-custom-observable', properties=None):
class _Custom(cls, _Observable):
+ if not re.match(TYPE_REGEX, type):
+ raise ValueError("Invalid observable type name '%s': must only contain the "
+ "characters a-z (lowercase ASCII), 0-9, and hyphen (-)." % type)
+ elif len(type) < 3 or len(type) > 250:
+ raise ValueError("Invalid observable type name '%s': must be between 3 and 250 characters." % type)
+
_type = type
_properties = OrderedDict()
_properties.update([
@@ -979,6 +1008,9 @@ def CustomObservable(type='x-custom-observable', properties=None):
"is not a ListProperty containing ObjectReferenceProperty." % prop_name)
_properties.update(properties)
+ _properties.update([
+ ('extensions', ExtensionsProperty(enclosing_type=_type)),
+ ])
def __init__(self, **kwargs):
_Observable.__init__(self, **kwargs)
@@ -1029,13 +1061,17 @@ def CustomExtension(observable=None, type='x-custom-observable', properties=None
class _Custom(cls, _Extension):
- _type = type
- _properties = {
- 'extensions': ExtensionsProperty(enclosing_type=_type),
- }
+ if not re.match(TYPE_REGEX, type):
+ raise ValueError("Invalid extension type name '%s': must only contain the "
+ "characters a-z (lowercase ASCII), 0-9, and hyphen (-)." % type)
+ elif len(type) < 3 or len(type) > 250:
+ raise ValueError("Invalid extension type name '%s': must be between 3 and 250 characters." % type)
- if not isinstance(properties, dict) or not properties:
- raise ValueError("'properties' must be a dict!")
+ _type = type
+ _properties = OrderedDict()
+
+ if not properties or not isinstance(properties, list):
+ raise ValueError("Must supply a list, containing tuples. For example, [('property1', IntegerProperty())]")
_properties.update(properties)
diff --git a/stix2/v20/sdo.py b/stix2/v20/sdo.py
index 7ccc3e3..d7e9954 100644
--- a/stix2/v20/sdo.py
+++ b/stix2/v20/sdo.py
@@ -1,6 +1,8 @@
-"""STIX 2.0 Domain Objects"""
+"""STIX 2.0 Domain Objects.
+"""
from collections import OrderedDict
+import re
import stix2
@@ -9,7 +11,7 @@ from ..markings import _MarkingsMixin
from ..properties import (BooleanProperty, IDProperty, IntegerProperty,
ListProperty, PatternProperty, ReferenceProperty,
StringProperty, TimestampProperty, TypeProperty)
-from ..utils import NOW
+from ..utils import NOW, TYPE_REGEX
from .common import ExternalReference, GranularMarking, KillChainPhase
from .observables import ObservableProperty
@@ -34,7 +36,7 @@ class AttackPattern(STIXDomainObject):
('name', StringProperty(required=True)),
('description', StringProperty()),
('kill_chain_phases', ListProperty(KillChainPhase)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -61,7 +63,7 @@ class Campaign(STIXDomainObject):
('first_seen', TimestampProperty()),
('last_seen', TimestampProperty()),
('objective', StringProperty()),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -84,7 +86,7 @@ class CourseOfAction(STIXDomainObject):
('modified', TimestampProperty(default=lambda: NOW, precision='millisecond')),
('name', StringProperty(required=True)),
('description', StringProperty()),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -110,7 +112,7 @@ class Identity(STIXDomainObject):
('identity_class', StringProperty(required=True)),
('sectors', ListProperty(StringProperty)),
('contact_information', StringProperty()),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -137,7 +139,7 @@ class Indicator(STIXDomainObject):
('valid_from', TimestampProperty(default=lambda: NOW)),
('valid_until', TimestampProperty()),
('kill_chain_phases', ListProperty(KillChainPhase)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty, required=True)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -167,7 +169,7 @@ class IntrusionSet(STIXDomainObject):
('resource_level', StringProperty()),
('primary_motivation', StringProperty()),
('secondary_motivations', ListProperty(StringProperty)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -191,7 +193,7 @@ class Malware(STIXDomainObject):
('name', StringProperty(required=True)),
('description', StringProperty()),
('kill_chain_phases', ListProperty(KillChainPhase)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty, required=True)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -216,7 +218,7 @@ class ObservedData(STIXDomainObject):
('last_observed', TimestampProperty(required=True)),
('number_observed', IntegerProperty(required=True)),
('objects', ObservableProperty(required=True)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -241,7 +243,7 @@ class Report(STIXDomainObject):
('description', StringProperty()),
('published', TimestampProperty(required=True)),
('object_refs', ListProperty(ReferenceProperty, required=True)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty, required=True)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -272,7 +274,7 @@ class ThreatActor(STIXDomainObject):
('primary_motivation', StringProperty()),
('secondary_motivations', ListProperty(StringProperty)),
('personal_motivations', ListProperty(StringProperty)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty, required=True)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -297,7 +299,7 @@ class Tool(STIXDomainObject):
('description', StringProperty()),
('kill_chain_phases', ListProperty(KillChainPhase)),
('tool_version', StringProperty()),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty, required=True)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -320,7 +322,7 @@ class Vulnerability(STIXDomainObject):
('modified', TimestampProperty(default=lambda: NOW, precision='millisecond')),
('name', StringProperty(required=True)),
('description', StringProperty()),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -356,6 +358,13 @@ def CustomObject(type='x-custom-type', properties=None):
def custom_builder(cls):
class _Custom(cls, STIXDomainObject):
+
+ if not re.match(TYPE_REGEX, type):
+ raise ValueError("Invalid type name '%s': must only contain the "
+ "characters a-z (lowercase ASCII), 0-9, and hyphen (-)." % type)
+ elif len(type) < 3 or len(type) > 250:
+ raise ValueError("Invalid type name '%s': must be between 3 and 250 characters." % type)
+
_type = type
_properties = OrderedDict()
_properties.update([
@@ -373,7 +382,7 @@ def CustomObject(type='x-custom-type', properties=None):
# This is to follow the general properties structure.
_properties.update([
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
diff --git a/stix2/v20/sro.py b/stix2/v20/sro.py
index 7d7d3ae..e488229 100644
--- a/stix2/v20/sro.py
+++ b/stix2/v20/sro.py
@@ -32,7 +32,7 @@ class Relationship(STIXRelationshipObject):
('description', StringProperty()),
('source_ref', ReferenceProperty(required=True)),
('target_ref', ReferenceProperty(required=True)),
- ('revoked', BooleanProperty()),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
@@ -72,8 +72,8 @@ class Sighting(STIXRelationshipObject):
('sighting_of_ref', ReferenceProperty(required=True)),
('observed_data_refs', ListProperty(ReferenceProperty(type="observed-data"))),
('where_sighted_refs', ListProperty(ReferenceProperty(type="identity"))),
- ('summary', BooleanProperty()),
- ('revoked', BooleanProperty()),
+ ('summary', BooleanProperty(default=lambda: False)),
+ ('revoked', BooleanProperty(default=lambda: False)),
('labels', ListProperty(StringProperty)),
('external_references', ListProperty(ExternalReference)),
('object_marking_refs', ListProperty(ReferenceProperty(type="marking-definition"))),
diff --git a/stix2/version.py b/stix2/version.py
index dd9b22c..5becc17 100644
--- a/stix2/version.py
+++ b/stix2/version.py
@@ -1 +1 @@
-__version__ = "0.5.1"
+__version__ = "1.0.0"
diff --git a/stix2/workbench.py b/stix2/workbench.py
new file mode 100644
index 0000000..3697c63
--- /dev/null
+++ b/stix2/workbench.py
@@ -0,0 +1,304 @@
+"""Functions and class wrappers for interacting with STIX data at a high level.
+
+.. autofunction:: create
+.. autofunction:: set_default_creator
+.. autofunction:: set_default_created
+.. autofunction:: set_default_external_refs
+.. autofunction:: set_default_object_marking_refs
+.. autofunction:: get
+.. autofunction:: all_versions
+.. autofunction:: query
+.. autofunction:: creator_of
+.. autofunction:: relationships
+.. autofunction:: related_to
+.. autofunction:: save
+.. autofunction:: add_filters
+.. autofunction:: add_filter
+.. autofunction:: parse
+.. autofunction:: add_data_source
+.. autofunction:: add_data_sources
+
+"""
+
+import stix2
+from . import AttackPattern as _AttackPattern
+from . import Campaign as _Campaign
+from . import CourseOfAction as _CourseOfAction
+from . import Identity as _Identity
+from . import Indicator as _Indicator
+from . import IntrusionSet as _IntrusionSet
+from . import Malware as _Malware
+from . import ObservedData as _ObservedData
+from . import Report as _Report
+from . import ThreatActor as _ThreatActor
+from . import Tool as _Tool
+from . import Vulnerability as _Vulnerability
+from . import (AlternateDataStream, ArchiveExt, Artifact, AutonomousSystem, # noqa: F401
+ Bundle, CustomExtension, CustomMarking, CustomObservable,
+ Directory, DomainName, EmailAddress, EmailMessage,
+ EmailMIMEComponent, Environment, ExtensionsProperty,
+ ExternalReference, File, FileSystemSource, Filter,
+ GranularMarking, HTTPRequestExt, ICMPExt, IPv4Address,
+ IPv6Address, KillChainPhase, MACAddress, MarkingDefinition,
+ MemoryStore, Mutex, NetworkTraffic, NTFSExt, parse_observable,
+ PDFExt, Process, RasterImageExt, Relationship, Sighting,
+ SocketExt, Software, StatementMarking, TAXIICollectionSource,
+ TCPExt, TLP_AMBER, TLP_GREEN, TLP_RED, TLP_WHITE, TLPMarking,
+ UNIXAccountExt, URL, UserAccount, WindowsPEBinaryExt,
+ WindowsPEOptionalHeaderType, WindowsPESection,
+ WindowsProcessExt, WindowsRegistryKey, WindowsRegistryValueType,
+ WindowsServiceExt, X509Certificate, X509V3ExtenstionsType)
+from .datastore.filters import FilterSet
+
+# Use an implicit MemoryStore
+_environ = Environment(store=MemoryStore())
+
+create = _environ.create
+set_default_creator = _environ.set_default_creator
+set_default_created = _environ.set_default_created
+set_default_external_refs = _environ.set_default_external_refs
+set_default_object_marking_refs = _environ.set_default_object_marking_refs
+get = _environ.get
+all_versions = _environ.all_versions
+query = _environ.query
+creator_of = _environ.creator_of
+relationships = _environ.relationships
+related_to = _environ.related_to
+save = _environ.add
+add_filters = _environ.add_filters
+add_filter = _environ.add_filter
+parse = _environ.parse
+add_data_source = _environ.source.add_data_source
+add_data_sources = _environ.source.add_data_sources
+
+
+# Wrap SDOs with helper functions
+
+
+STIX_OBJS = [_AttackPattern, _Campaign, _CourseOfAction, _Identity,
+ _Indicator, _IntrusionSet, _Malware, _ObservedData, _Report,
+ _ThreatActor, _Tool, _Vulnerability]
+
+STIX_OBJ_DOCS = """
+
+.. method:: created_by(*args, **kwargs)
+
+ {}
+
+.. method:: relationships(*args, **kwargs)
+
+ {}
+
+.. method:: related(*args, **kwargs)
+
+ {}
+
+""".format(_environ.creator_of.__doc__,
+ _environ.relationships.__doc__,
+ _environ.related_to.__doc__)
+
+
+def _created_by_wrapper(self, *args, **kwargs):
+ return _environ.creator_of(self, *args, **kwargs)
+
+
+def _relationships_wrapper(self, *args, **kwargs):
+ return _environ.relationships(self, *args, **kwargs)
+
+
+def _related_wrapper(self, *args, **kwargs):
+ return _environ.related_to(self, *args, **kwargs)
+
+
+def _constructor_wrapper(obj_type):
+ # Use an intermediate wrapper class so the implicit environment will create objects that have our wrapper functions
+ wrapped_type = type(obj_type.__name__, obj_type.__bases__, dict(
+ created_by=_created_by_wrapper,
+ relationships=_relationships_wrapper,
+ related=_related_wrapper,
+ **obj_type.__dict__
+ ))
+
+ @staticmethod
+ def new_constructor(cls, *args, **kwargs):
+ x = _environ.create(wrapped_type, *args, **kwargs)
+ return x
+ return new_constructor
+
+
+def _setup_workbench():
+ # Create wrapper classes whose constructors call the implicit environment's create()
+ for obj_type in STIX_OBJS:
+ new_class_dict = {
+ '__new__': _constructor_wrapper(obj_type),
+ '__doc__': 'Workbench wrapper around the `{0} `__ object. {1}'.format(obj_type.__name__, STIX_OBJ_DOCS)
+ }
+ new_class = type(obj_type.__name__, (), new_class_dict)
+
+ # Add our new class to this module's globals and to the library-wide mapping.
+ # This allows parse() to use the wrapped classes.
+ globals()[obj_type.__name__] = new_class
+ stix2.OBJ_MAP[obj_type._type] = new_class
+ new_class = None
+
+
+_setup_workbench()
+
+
+# Functions to get all objects of a specific type
+
+
+def attack_patterns(filters=None):
+ """Retrieve all Attack Pattern objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'attack-pattern'))
+ return query(filter_list)
+
+
+def campaigns(filters=None):
+ """Retrieve all Campaign objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'campaign'))
+ return query(filter_list)
+
+
+def courses_of_action(filters=None):
+ """Retrieve all Course of Action objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'course-of-action'))
+ return query(filter_list)
+
+
+def identities(filters=None):
+ """Retrieve all Identity objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'identity'))
+ return query(filter_list)
+
+
+def indicators(filters=None):
+ """Retrieve all Indicator objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'indicator'))
+ return query(filter_list)
+
+
+def intrusion_sets(filters=None):
+ """Retrieve all Intrusion Set objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'intrusion-set'))
+ return query(filter_list)
+
+
+def malware(filters=None):
+ """Retrieve all Malware objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'malware'))
+ return query(filter_list)
+
+
+def observed_data(filters=None):
+ """Retrieve all Observed Data objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'observed-data'))
+ return query(filter_list)
+
+
+def reports(filters=None):
+ """Retrieve all Report objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'report'))
+ return query(filter_list)
+
+
+def threat_actors(filters=None):
+ """Retrieve all Threat Actor objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'threat-actor'))
+ return query(filter_list)
+
+
+def tools(filters=None):
+ """Retrieve all Tool objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'tool'))
+ return query(filter_list)
+
+
+def vulnerabilities(filters=None):
+ """Retrieve all Vulnerability objects.
+
+ Args:
+ filters (list, optional): A list of additional filters to apply to
+ the query.
+
+ """
+ filter_list = FilterSet(filters)
+ filter_list.add(Filter('type', '=', 'vulnerability'))
+ return query(filter_list)
diff --git a/tox.ini b/tox.ini
index 884288a..7c2e68c 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,5 +1,5 @@
[tox]
-envlist = py27,py34,py35,py36,pycodestyle,isort-check,taxii-datastore
+envlist = py27,py34,py35,py36,style,isort-check
[testenv]
deps =
@@ -8,22 +8,19 @@ deps =
pytest
pytest-cov
coverage
+ taxii2-client
commands =
- py.test --cov=stix2 stix2/test/ --cov-report term-missing
+ pytest --ignore=stix2/test/test_workbench.py --cov=stix2 stix2/test/ --cov-report term-missing
+ pytest stix2/test/test_workbench.py --cov=stix2 --cov-report term-missing --cov-append
passenv = CI TRAVIS TRAVIS_*
-[testenv:pycodestyle]
+[testenv:style]
deps =
flake8
- pycodestyle
commands =
- pycodestyle ./stix2
flake8
-[pycodestyle]
-max-line-length = 160
-
[flake8]
max-line-length = 160
@@ -33,17 +30,9 @@ commands =
isort -rc stix2 examples -df
isort -rc stix2 examples -c
-[testenv:taxii-datastore]
-deps =
- git+https://github.com/oasis-open/cti-taxii-client.git#egg=taxii2-client
- git+https://github.com/oasis-open/cti-taxii-server.git#egg=medallion
-
-commands =
- py.test --cov=stix2 stix2/test/test_datastore_taxii.py --cov-report term-missing
-
[travis]
python =
- 2.7: py27, pycodestyle, taxii-datastore
- 3.4: py34, pycodestyle
- 3.5: py35, pycodestyle
- 3.6: py36, pycodestyle, taxii-datastore
+ 2.7: py27, style
+ 3.4: py34, style
+ 3.5: py35, style
+ 3.6: py36, style