Clean up documentation

stix2.0
Chris Lenk 2018-04-09 13:29:53 -04:00
parent 1d06e642a4
commit f83d9a56b5
15 changed files with 63 additions and 99 deletions

View File

@ -7,6 +7,6 @@ functions in the ``stix2`` API, as given by the package's docstrings.
.. note:: .. note::
All the classes and functions detailed in the pages below are importable All the classes and functions detailed in the pages below are importable
directly from `stix2`. See also: directly from `stix2`. See also:
:ref:`How imports will work </guide/ts_support.ipynb#How-imports-will-work>`. :ref:`How imports work </guide/ts_support.ipynb#How-imports-work>`.
.. automodule:: stix2 .. automodule:: stix2

View File

@ -1,39 +0,0 @@
.. _datastore_api:
DataStore API
=============
.. warning::
The DataStore API is still in the planning stages and may be subject to
major changes. We encourage anyone with feedback to contact the maintainers
to help ensure the API meets a large variety of use cases.
One prominent feature of python-stix2 will be an interface for connecting
different backend data stores containing STIX content. This will allow a uniform
interface for querying and saving STIX content, and allow higher level code to
be written without regard to the underlying data storage format. python-stix2
will define the API and contain some default implementations of this API, but
developers are encouraged to write their own implementations.
Potential functions of the API include:
* get a STIX Object by ID (returns the most recent version).
* get all versions of a STIX object by ID.
* get all relationships involving a given object, and all related objects.
* save an object.
* query for objects that match certain criteria (query syntax TBD).
For all queries, the API will include a "filter" interface that can be used to
either explicitly include or exclude results with certain criteria. For example,
* only trust content from a set of object creators.
* exclude content from certain (untrusted) object creators.
* only include content with a confidence above a certain threshold (once
confidence is added to STIX).
* only return content that can be shared with external parties (in other words,
that has TLP:GREEN markings).
Additionally, the python-stix2 library will contain a "composite" data store,
which implements the DataStore API while delegating functionality to one or more
"child" data stores.

View File

@ -1,6 +1,9 @@
User's Guide User's Guide
============ ============
This section of documentation contains guides and tutorials on how to use the
``stix2`` library.
.. toctree:: .. toctree::
:glob: :glob:

View File

@ -377,7 +377,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"To update the properties of an object, see the [Versioning](guide/versioning.ipynb) section." "To update the properties of an object, see the [Versioning](versioning.ipynb) section."
] ]
}, },
{ {

View File

@ -58,9 +58,9 @@
"source": [ "source": [
"# DataStore API\n", "# DataStore API\n",
"\n", "\n",
"CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n", "The ``stix2`` library features an interface for pulling and pushing STIX 2 content. This interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX 2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX 2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
"\n", "\n",
"The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. CTI Python STIX2 provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites." "The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. The ``stix2`` library provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
] ]
}, },
{ {
@ -340,7 +340,14 @@
"source": [ "source": [
"## Filters\n", "## Filters\n",
"\n", "\n",
"The CTI Python STIX2 DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n", "The ``stix2`` DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. Filters can be used to explicitly include or exclude results with certain criteria. For example:\n",
"\n",
"* only trust content from a set of object creators\n",
"* exclude content from certain (untrusted) object creators\n",
"* only include content with a confidence above a certain threshold (once confidence is added to STIX 2)\n",
"* only return content that can be shared with external parties (e.g. only content that has TLP:GREEN markings)\n",
"\n",
"Filters can be created and supplied with every call to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
"\n", "\n",
"Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX 2 objects can be filtered on. In addition, TAXII 2 Filtering parameters for fields can also be used in filters.\n", "Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX 2 objects can be filtered on. In addition, TAXII 2 Filtering parameters for fields can also be used in filters.\n",
"\n", "\n",

View File

@ -676,20 +676,20 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## Parsing Custom STIX Content" "### Parsing Custom STIX Content"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.core.rst#stix2.core.parse). Just supply the keyword argument *allow_custom=True*. When *allow_custom* is specified, [parse()](../api/stix2.core.rst#stix2.core.parse) will attempt to convert the supplied STIX content to known STIX2 domain objects and/or previously defined custom defined STIX2 objects. If the conversion cannot be completed (and *allow_custom* is specified), [parse()](../api/stix2.core.rst#stix2.core.parse) will treat the supplied STIX2 content as valid STIX2 objects and return them. **Warning: Specifying *allow_custom* may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom STIX2 content occurs where the custom STIX2 content supplied is not valid STIX2**. This is an axiomatic possibility as the STIX2 library cannot guarantee proper processing of unknown custom STIX2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n", "Parsing custom STIX objects and/or STIX objects with custom properties is also completed easily with [parse()](../api/stix2.core.rst#stix2.core.parse). Just supply the keyword argument ``allow_custom=True``. When ``allow_custom`` is specified, [parse()](../api/stix2.core.rst#stix2.core.parse) will attempt to convert the supplied STIX content to known STIX 2 domain objects and/or previously defined [custom STIX 2 objects](custom.ipynb). If the conversion cannot be completed (and ``allow_custom`` is specified), [parse()](../api/stix2.core.rst#stix2.core.parse) will treat the supplied STIX 2 content as valid STIX 2 objects and return them. **Warning: Specifying allow_custom may lead to critical errors if further processing (searching, filtering, modifying etc...) of the custom content occurs where the custom content supplied is not valid STIX 2**. This is an axiomatic possibility as the ``stix2`` library cannot guarantee proper processing of unknown custom STIX 2 objects that were explicitly flagged to be allowed, and thus may not be valid.\n",
"\n", "\n",
"For examples on parsing STIX2 objects with custom STIX properties, see [Custom STIX Content:Custom Properties](custom.ipynb#Custom-Properties)\n", "For examples of parsing STIX 2 objects with custom STIX properties, see [Custom STIX Content: Custom Properties](custom.ipynb#Custom-Properties)\n",
"\n", "\n",
"For examples on parsing defined custom STIX2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n", "For examples of parsing defined custom STIX 2 objects, see [Custom STIX Content: Custom STIX Object Types](custom.ipynb#Custom-STIX-Object-Types)\n",
"\n", "\n",
"For the case where it is desired to retrieve STIX2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX2 content unknown to the user, the user can create a STIX2 DataStore/Source with the flag *allow_custom=True*. As aforementioned this will configure the DataStore/Source to allow for unknown STIX2 content to be returned (albeit not converted to full STIX2 domain objects and properties); notable processing capabilites of the STIX2 library may be precluded by the unknown STIX2 content, if the content is not valid or actual STIX2 domain objects and properties." "For retrieving STIX 2 content from a source (e.g. file system, TAXII) that may possibly have custom STIX 2 content unknown to the user, the user can create a STIX 2 DataStore/Source with the flag ``allow_custom=True``. As mentioned, this will configure the DataStore/Source to allow for unknown STIX 2 content to be returned (albeit not converted to full STIX 2 domain objects and properties); the ``stix2`` library may preclude processing the unknown content, if the content is not valid or actual STIX 2 domain objects and properties."
] ]
}, },
{ {

View File

@ -59,7 +59,7 @@
"source": [ "source": [
"## Technical Specification Support\n", "## Technical Specification Support\n",
"\n", "\n",
"### How imports will work\n", "### How imports work\n",
"\n", "\n",
"Imports can be used in different ways depending on the use case and support levels.\n", "Imports can be used in different ways depending on the use case and support levels.\n",
"\n", "\n",
@ -229,7 +229,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### How parsing will work\n", "### How parsing works\n",
"If the ``version`` positional argument is not provided. The data will be parsed using the latest version of STIX 2.X supported by the `stix2` library.\n", "If the ``version`` positional argument is not provided. The data will be parsed using the latest version of STIX 2.X supported by the `stix2` library.\n",
"\n", "\n",
"You can lock your [parse()](../api/stix2.core.rst#stix2.core.parse) method to a specific STIX version by:" "You can lock your [parse()](../api/stix2.core.rst#stix2.core.parse) method to a specific STIX version by:"
@ -363,7 +363,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"### How will custom content work\n", "### How custom content works\n",
"\n", "\n",
"[CustomObject](../api/stix2.v20.sdo.rst#stix2.v20.sdo.CustomObject), [CustomObservable](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomObservable), [CustomMarking](../api/stix2.v20.common.rst#stix2.v20.common.CustomMarking) and [CustomExtension](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomExtension) must be registered explicitly by STIX version. This is a design decision since properties or requirements may change as the STIX Technical Specification advances.\n", "[CustomObject](../api/stix2.v20.sdo.rst#stix2.v20.sdo.CustomObject), [CustomObservable](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomObservable), [CustomMarking](../api/stix2.v20.common.rst#stix2.v20.common.CustomMarking) and [CustomExtension](../api/stix2.v20.observables.rst#stix2.v20.observables.CustomExtension) must be registered explicitly by STIX version. This is a design decision since properties or requirements may change as the STIX Technical Specification advances.\n",
"\n", "\n",

View File

@ -3,8 +3,21 @@
You can adapt this file completely to your liking, but it should at least You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive. contain the root `toctree` directive.
Welcome to stix2's documentation! STIX 2 Python API Documentation
================================= ===============================
.. warning::
Prior to version 1.0, all APIs are considered unstable and subject to change.
Welcome to the STIX 2 Python API's documentation. This library is designed to
help you work with STIX 2 content. For more information about STIX 2, see the
`website <http://cti-tc.github.io>`_ of the OASIS Cyber Threat Intelligence
Technical Committee.
Get started with an `overview <overview.rst>`_ of the library, then take a look
at the `guides and tutorials <guide.rst>`_ to see how to use it. For information
about a specific class or function, see the `API reference <api_ref.rst>`_.
.. toctree:: .. toctree::
:maxdepth: 3 :maxdepth: 3
@ -13,8 +26,6 @@ Welcome to stix2's documentation!
overview overview
guide guide
api_ref api_ref
datastore_api
roadmap
contributing contributing

View File

@ -4,7 +4,7 @@ Overview
Goals Goals
----- -----
High level goals/principles of the python-stix2 library: High level goals/principles of the Python ``stix2`` library:
1. It should be as easy as possible (but no easier!) to perform common tasks of 1. It should be as easy as possible (but no easier!) to perform common tasks of
producing, consuming, and processing STIX 2 content. producing, consuming, and processing STIX 2 content.
@ -17,22 +17,22 @@ Design Decisions
---------------- ----------------
To accomplish these goals, and to incorporate lessons learned while developing To accomplish these goals, and to incorporate lessons learned while developing
python-stix (for STIX 1.x), several decisions influenced the design of ``python-stix`` (for STIX 1.x), several decisions influenced the design of the
python-stix2: ``stix2`` library:
1. All data structures are immutable by default. In contrast to python-stix, 1. All data structures are immutable by default. In contrast to python-stix,
where users would create an object and then assign attributes to it, in where users would create an object and then assign attributes to it, in
python-stix2 all properties must be provided when creating the object. ``stix2`` all properties must be provided when creating the object.
2. Where necessary, library objects should act like ``dict``'s. When treated as 2. Where necessary, library objects should act like ``dict``'s. When treated as
a ``str``, the JSON reprentation of the object should be used. a ``str``, the JSON reprentation of the object should be used.
3. Core Python data types (including numeric types, ``datetime``) should be used 3. Core Python data types (including numeric types, ``datetime``) should be used
when appropriate, and serialized to the correct format in JSON as specified when appropriate, and serialized to the correct format in JSON as specified
in the STIX 2.0 spec. in the STIX 2 spec.
Architecture Architecture
------------ ------------
The `stix2` library APIs are divided into three logical layers, representing The ``stix2`` library is divided into three logical layers, representing
different levels of abstraction useful in different types of scripts and larger different levels of abstraction useful in different types of scripts and larger
applications. It is possible to combine multiple layers in the same program, applications. It is possible to combine multiple layers in the same program,
and the higher levels build on the layers below. and the higher levels build on the layers below.
@ -41,7 +41,7 @@ and the higher levels build on the layers below.
Object Layer Object Layer
^^^^^^^^^^^^ ^^^^^^^^^^^^
The lowest layer, **Object Layer**, is where Python objects representing STIX 2 The lowest layer, the **Object Layer**, is where Python objects representing STIX 2
data types (such as SDOs, SROs, and Cyber Observable Objects, as well as data types (such as SDOs, SROs, and Cyber Observable Objects, as well as
non-top-level objects like External References, Kill Chain phases, and Cyber non-top-level objects like External References, Kill Chain phases, and Cyber
Observable extensions) are created, and can be serialized and deserialized Observable extensions) are created, and can be serialized and deserialized
@ -57,8 +57,6 @@ not implemented as references between the Python objects themselves, but by
simply having the same values in ``id`` and reference properties. There is no simply having the same values in ``id`` and reference properties. There is no
referential integrity maintained by the ``stix2`` library. referential integrity maintained by the ``stix2`` library.
*This layer is mostly complete.*
Environment Layer Environment Layer
^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
@ -79,8 +77,7 @@ intelligence ecosystem.
Each of these components can be used individually, or combined as part of an Each of these components can be used individually, or combined as part of an
``Environment``. These ``Environment`` objects allow different settings to be ``Environment``. These ``Environment`` objects allow different settings to be
used by different users of a multi-user application (such as a web application). used by different users of a multi-user application (such as a web application).
For more information, check out `this Environment tutorial <guide/environment.ipynb>`_.
*This layer is mostly complete.*
Workbench Layer Workbench Layer
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
@ -89,9 +86,7 @@ The highest layer of the ``stix2`` APIs is the **Workbench Layer**, designed for
a single user in a highly-interactive analytical environment (such as a `Jupyter a single user in a highly-interactive analytical environment (such as a `Jupyter
Notebook <https://jupyter.org/>`_). It builds on the lower layers of the API, Notebook <https://jupyter.org/>`_). It builds on the lower layers of the API,
while hiding most of their complexity. Unlike the other layers, this layer is while hiding most of their complexity. Unlike the other layers, this layer is
designed to be used directly by end users. For users who are comfortable with, designed to be used directly by end users. For users who are comfortable with
Python, the Workbench Layer makes it easy to quickly interact with STIX data Python, the Workbench Layer makes it easy to quickly interact with STIX data
from a variety of sources without needing to write and run one-off Python from a variety of sources without needing to write and run one-off Python
scripts. scripts. For more information, check out `this Workbench tutorial <guide/workbench.ipynb>`_.
*This layer is currently being developed.*

View File

@ -1,17 +0,0 @@
Development Roadmap
===================
.. warning::
Prior to version 1.0, all APIs are considered unstable and subject to
change.
This is a list of (planned) features before version 1.0 is released.
* Serialization of all STIX and Cyber Observable objects to JSON.
* De-serialization (parsing) of all STIX and Cyber Observable objects.
* APIs for versioning (revising and revoking) STIX objects.
* APIs for marking STIX objects and interpreting markings of STIX objects.
* :ref:`datastore_api`, providing a common interface for querying sources
of STIX content (such as objects in memory, on a filesystem, in a database, or
via a TAXII feed).

View File

@ -1,4 +1,4 @@
"""Python STIX 2.0 DataStore API """Python STIX 2.0 DataStore API.
.. autosummary:: .. autosummary::
:toctree: datastore :toctree: datastore

View File

@ -1,3 +1,6 @@
"""Python STIX 2.0 Environment API.
"""
import copy import copy
from .core import parse as _parse from .core import parse as _parse

View File

@ -263,11 +263,11 @@ def get_class_hierarchy_names(obj):
def remove_custom_stix(stix_obj): def remove_custom_stix(stix_obj):
"""remove any custom STIX objects or properties """Remove any custom STIX objects or properties.
Warning: This function is a best effort utility, in that Warning: This function is a best effort utility, in that
it will remove custom objects and properties based on the it will remove custom objects and properties based on the
type names; i.e. if "x-" prefixes object types, and "x_" type names; i.e. if "x-" prefixes object types, and "x\\_"
prefixes property types. According to the STIX2 spec, prefixes property types. According to the STIX2 spec,
those naming conventions are a SHOULDs not MUSTs, meaning those naming conventions are a SHOULDs not MUSTs, meaning
that valid custom STIX content may ignore those conventions that valid custom STIX content may ignore those conventions

View File

@ -1,4 +1,5 @@
"""STIX 2.0 Domain Objects""" """STIX 2.0 Domain Objects.
"""
from collections import OrderedDict from collections import OrderedDict