Clean up documentation

docs/api_ref.rst

@@ -7,6 +7,6 @@ functions in the ``stix2`` API, as given by the package's docstrings.
 .. note::
     All the classes and functions detailed in the pages below are importable
     directly from `stix2`. See also:
-    :ref:`How imports will work </guide/ts_support.ipynb#How-imports-will-work>`.
+    :ref:`How imports work </guide/ts_support.ipynb#How-imports-work>`.
 
 .. automodule:: stix2

docs/datastore_api.rst

@@ -1,39 +0,0 @@
-.. _datastore_api:
-
-DataStore API
-=============
-
-.. warning::
-
-    The DataStore API is still in the planning stages and may be subject to
-    major changes. We encourage anyone with feedback to contact the maintainers
-    to help ensure the API meets a large variety of use cases.
-
-One prominent feature of python-stix2 will be an interface for connecting
-different backend data stores containing STIX content. This will allow a uniform
-interface for querying and saving STIX content, and allow higher level code to
-be written without regard to the underlying data storage format. python-stix2
-will define the API and contain some default implementations of this API, but
-developers are encouraged to write their own implementations.
-
-Potential functions of the API include:
-
-* get a STIX Object by ID (returns the most recent version).
-* get all versions of a STIX object by ID.
-* get all relationships involving a given object, and all related objects.
-* save an object.
-* query for objects that match certain criteria (query syntax TBD).
-
-For all queries, the API will include a "filter" interface that can be used to
-either explicitly include or exclude results with certain criteria. For example,
-
-* only trust content from a set of object creators.
-* exclude content from certain (untrusted) object creators.
-* only include content with a confidence above a certain threshold (once
-  confidence is added to STIX).
-* only return content that can be shared with external parties (in other words,
-  that has TLP:GREEN markings).
-
-Additionally, the python-stix2 library will contain a "composite" data store,
-which implements the DataStore API while delegating functionality to one or more
-"child" data stores.

docs/guide.rst

@@ -1,6 +1,9 @@
 User's Guide
 ============
 
+This section of documentation contains guides and tutorials on how to use the
+``stix2`` library.
+
 .. toctree::
    :glob:
 

docs/guide/creating.ipynb

@@ -377,7 +377,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "To update the properties of an object, see the [Versioning](guide/versioning.ipynb) section."
+    "To update the properties of an object, see the [Versioning](versioning.ipynb) section."
    ]
   },
   {

docs/guide/datastore.ipynb

@@ -58,9 +58,9 @@
    "source": [
     "# DataStore API\n",
     "\n",
-    "CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
+    "The ``stix2`` library features an interface for pulling and pushing STIX 2 content. This interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX 2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX 2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
     "\n",
-    "The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. CTI Python STIX2 provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
+    "The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. The ``stix2`` library provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
    ]
   },
   {
@@ -340,7 +340,14 @@
    "source": [
     "## Filters\n",
     "\n",
-    "The CTI Python STIX2 DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
+    "The ``stix2`` DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. Filters can be used to explicitly include or exclude results with certain criteria. For example:\n",
+    "\n",
+    "* only trust content from a set of object creators\n",
+    "* exclude content from certain (untrusted) object creators\n",
+    "* only include content with a confidence above a certain threshold (once confidence is added to STIX 2)\n",
+    "* only return content that can be shared with external parties (e.g. only content that has TLP:GREEN markings)\n",
+    "\n",
+    "Filters can be created and supplied with every call to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
     "\n",
     "Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX 2 objects can be filtered on. In addition, TAXII 2 Filtering parameters for fields can also be used in filters.\n",
     "\n",

docs/guide/parsing.ipynb

@@ -676,20 +676,20 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Parsing Custom STIX Content"
+    "### Parsing Custom STIX Content"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.core.rst#stix2.core.parse). Just supply the keyword argument *allow_custom=True*. When *allow_custom* is specified, [parse()](../api/stix2.core.rst#stix2.core.parse) will attempt to convert the supplied STIX content to known STIX2 domain objects and/or previously defined custom defined STIX2 objects. If the conversion cannot be completed (and *allow_custom* is specified), [parse()](../api/stix2.core.rst#stix2.core.parse) will treat the supplied STIX2 content as valid STIX2 objects and return them. **Warning: Specifying *allow_custom* may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom STIX2 content occurs where the custom STIX2 content supplied is not valid STIX2**. This is an axiomatic possibility as the STIX2 library cannot guarantee proper processing of unknown custom STIX2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n",
+    "Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.core.rst#stix2.core.parse). Just supply the keyword argument ``allow_custom=True``. When ``allow_custom`` is specified, [parse()](../api/stix2.core.rst#stix2.core.parse) will attempt to convert the supplied STIX content to known STIX 2 domain objects and/or previously defined [custom STIX 2 objects](custom.ipynb). If the conversion cannot be completed (and ``allow_custom`` is specified), [parse()](../api/stix2.core.rst#stix2.core.parse) will treat the supplied STIX 2 content as valid STIX 2 objects and return them. **Warning: Specifying allow_custom may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom content occurs where the custom content supplied is not valid STIX 2**. This is an axiomatic possibility as the ``stix2`` library cannot guarantee proper processing of unknown custom STIX 2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n",
     "\n",
-    "For examples on parsing STIX2 objects with custom STIX properties, see [Custom STIX Content:Custom Properties](custom.ipynb#Custom-Properties)\n",
+    "For examples of parsing STIX 2 objects with custom STIX properties, see [Custom STIX Content: Custom Properties](custom.ipynb#Custom-Properties)\n",
     "\n",
-    "For examples on parsing defined custom STIX2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n",
+    "For examples of parsing defined custom STIX 2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n",
     "\n",
-    "For the case where it is desired to retrieve STIX2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX2 content unknown to the user, the user can create a STIX2 DataStore/Source with the flag *allow_custom=True*. As aforementioned this will configure the DataStore/Source to allow for unknown STIX2 content to be returned (albeit not converted to full STIX2 domain objects and properties); notable processing capabilites of the STIX2 library may be precluded by the unknown STIX2 content, if the content is not valid or actual STIX2 domain objects and properties."
+    "For retrieving STIX 2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX 2 content unknown to the user, the user can create a STIX 2 DataStore/Source with the flag ``allow_custom=True``. As mentioned, this will configure the DataStore/Source to allow for unknown STIX 2 content to be returned (albeit not converted to full STIX 2 domain objects and properties); the ``stix2`` library may preclude processing the unknown content, if the content is not valid or actual STIX 2 domain objects and properties."
    ]
   },
   {

docs/guide/ts_support.ipynb

@@ -59,7 +59,7 @@
    "source": [
     "## Technical Specification Support\n",
     "\n",
-    "### How imports will work\n",
+    "### How imports work\n",
     "\n",
     "Imports can be used in different ways depending on the use case and support levels.\n",
     "\n",
@@ -229,7 +229,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### How parsing will work\n",
+    "### How parsing works\n",
     "If the ``version`` positional argument is not provided. The data will be parsed using the latest version of STIX 2.X supported by the `stix2` library.\n",
     "\n",
     "You can lock your [parse()](../api/stix2.core.rst#stix2.core.parse) method to a specific STIX version by:"
@@ -363,7 +363,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### How will custom content work\n",
+    "### How custom content works\n",
     "\n",
     "[CustomObject](../api/stix2.v20.sdo.rst#stix2.v20.sdo.CustomObject), [CustomObservable](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomObservable), [CustomMarking](../api/stix2.v20.common.rst#stix2.v20.common.CustomMarking) and [CustomExtension](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomExtension) must be registered explicitly by STIX version. This is a design decision since properties or requirements may change as the STIX Technical Specification advances.\n",
     "\n",

docs/index.rst

@@ -3,8 +3,21 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-Welcome to stix2's documentation!
+STIX 2 Python API Documentation
-=================================
+===============================
+
+.. warning::
+
+    Prior to version 1.0, all APIs are considered unstable and subject to change.
+
+Welcome to the STIX 2 Python API's documentation. This library is designed to
+help you work with STIX 2 content. For more information about STIX 2, see the
+`website <http://cti-tc.github.io>`_ of the OASIS Cyber Threat Intelligence
+Technical Committee.
+
+Get started with an `overview <overview.rst>`_ of the library, then take a look
+at the `guides and tutorials <guide.rst>`_ to see how to use it. For information
+about a specific class or function, see the `API reference <api_ref.rst>`_.
 
 .. toctree::
    :maxdepth: 3
@@ -13,8 +26,6 @@ Welcome to stix2's documentation!
    overview
    guide
    api_ref
-   datastore_api
-   roadmap
    contributing
 
 

docs/overview.rst

@@ -4,7 +4,7 @@ Overview
 Goals
 -----
 
-High level goals/principles of the python-stix2 library:
+High level goals/principles of the Python ``stix2`` library:
 
 1. It should be as easy as possible (but no easier!) to perform common tasks of
    producing, consuming, and processing STIX 2 content.
@@ -17,22 +17,22 @@ Design Decisions
 ----------------
 
 To accomplish these goals, and to incorporate lessons learned while developing
-python-stix (for STIX 1.x), several decisions influenced the design of
+``python-stix`` (for STIX 1.x), several decisions influenced the design of the
-python-stix2:
+``stix2`` library:
 
 1. All data structures are immutable by default. In contrast to python-stix,
    where users would create an object and then assign attributes to it, in
-   python-stix2 all properties must be provided when creating the object.
+   ``stix2`` all properties must be provided when creating the object.
 2. Where necessary, library objects should act like ``dict``'s. When treated as
    a ``str``, the JSON reprentation of the object should be used.
 3. Core Python data types (including numeric types, ``datetime``) should be used
    when appropriate, and serialized to the correct format in JSON as specified
-   in the STIX 2.0 spec.
+   in the STIX 2 spec.
 
 Architecture
 ------------
 
-The `stix2` library APIs are divided into three logical layers, representing
+The ``stix2`` library is divided into three logical layers, representing
 different levels of abstraction useful in different types of scripts and larger
 applications. It is possible to combine multiple layers in the same program,
 and the higher levels build on the layers below.
@@ -41,7 +41,7 @@ and the higher levels build on the layers below.
 Object Layer
 ^^^^^^^^^^^^
 
-The lowest layer, **Object Layer**, is where Python objects representing STIX 2
+The lowest layer, the **Object Layer**, is where Python objects representing STIX 2
 data types (such as SDOs, SROs, and Cyber Observable Objects, as well as
 non-top-level objects like External References, Kill Chain phases, and Cyber
 Observable extensions) are created, and can be serialized and deserialized
@@ -57,8 +57,6 @@ not implemented as references between the Python objects themselves, but by
 simply having the same values in ``id`` and reference properties. There is no
 referential integrity maintained by the ``stix2`` library.
 
-*This layer is mostly complete.*
-
 Environment Layer
 ^^^^^^^^^^^^^^^^^
 
@@ -79,8 +77,7 @@ intelligence ecosystem.
 Each of these components can be used individually, or combined as part of an
 ``Environment``. These ``Environment`` objects allow different settings to be
 used by different users of a multi-user application (such as a web application).
-
+For more information, check out `this Environment tutorial <guide/environment.ipynb>`_.
-*This layer is mostly complete.*
 
 Workbench Layer
 ^^^^^^^^^^^^^^^
@@ -89,9 +86,7 @@ The highest layer of the ``stix2`` APIs is the **Workbench Layer**, designed for
 a single user in a highly-interactive analytical environment (such as a `Jupyter
 Notebook <https://jupyter.org/>`_). It builds on the lower layers of the API,
 while hiding most of their complexity. Unlike the other layers, this layer is
-designed to be used directly by end users. For users who are comfortable with,
+designed to be used directly by end users. For users who are comfortable with
 Python, the Workbench Layer makes it easy to quickly interact with STIX data
 from a variety of sources without needing to write and run one-off Python
-scripts.
+scripts. For more information, check out `this Workbench tutorial <guide/workbench.ipynb>`_.
-
-*This layer is currently being developed.*

docs/roadmap.rst

@@ -1,17 +0,0 @@
-Development Roadmap
-===================
-
-.. warning::
-
-    Prior to version 1.0, all APIs are considered unstable and subject to
-    change.
-
-This is a list of (planned) features before version 1.0 is released.
-
-* Serialization of all STIX and Cyber Observable objects to JSON.
-* De-serialization (parsing) of all STIX and Cyber Observable objects.
-* APIs for versioning (revising and revoking) STIX objects.
-* APIs for marking STIX objects and interpreting markings of STIX objects.
-* :ref:`datastore_api`, providing a common interface for querying sources
-  of STIX content (such as objects in memory, on a filesystem, in a database, or
-  via a TAXII feed).

stix2/datastore/__init__.py

@@ -1,4 +1,4 @@
-"""Python STIX 2.0 DataStore API
+"""Python STIX 2.0 DataStore API.
 
 .. autosummary::
    :toctree: datastore

stix2/environment.py

@@ -1,3 +1,6 @@
+"""Python STIX 2.0 Environment API.
+"""
+
 import copy
 
 from .core import parse as _parse

stix2/utils.py

@@ -263,11 +263,11 @@ def get_class_hierarchy_names(obj):
 
 
 def remove_custom_stix(stix_obj):
-    """remove any custom STIX objects or properties
+    """Remove any custom STIX objects or properties.
 
     Warning: This function is a best effort utility, in that
     it will remove custom objects and properties based on the
-    type names; i.e. if "x-" prefixes object types, and "x_"
+    type names; i.e. if "x-" prefixes object types, and "x\\_"
     prefixes property types. According to the STIX2 spec,
     those naming conventions are a SHOULDs not MUSTs, meaning
     that valid custom STIX content may ignore those conventions

stix2/v20/sdo.py

@@ -1,4 +1,5 @@
-"""STIX 2.0 Domain Objects"""
+"""STIX 2.0 Domain Objects.
+"""
 
 from collections import OrderedDict