Chris Lenk
parent
5f703509be
commit
eb2d87ce71
|
|
@ -71,7 +71,7 @@
|
|||
"source": [
|
||||
"# DataStore API\n",
|
||||
"\n",
|
||||
"CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) for both pulling and pushing.\n",
|
||||
"CTI Python STIX2 features a new interface for pulling and pushing STIX2 content. The new interface consists of [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) constructs: a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) for pulling STIX2 content, a [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) for pushing STIX2 content, and a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) for both pulling and pushing.\n",
|
||||
"\n",
|
||||
"The DataStore, [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource), [DataSink](../api/stix2.datastore.rst#stix2.datastore.DataSink) (collectively referred to as the \"DataStore suite\") APIs are not referenced directly by a user but are used as base classes, which are then subclassed by real DataStore suites. CTI Python STIX2 provides the DataStore suites of [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst). Users are also encouraged to subclass the base classes and create their own custom DataStore suites."
|
||||
]
|
||||
|
|
@ -86,7 +86,7 @@
|
|||
"\n",
|
||||
"[CompositeDataSource](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource) is just a wrapper around a set of defined [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource) (e.g. [FileSystemSource](../api/datastore/stix2.datastore.filesystem.rst#stix2.datastore.filesystem.FileSystemSource)) that federates `get()`/`all_versions()`/`query()` calls individually to each of the attached [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource) , collects the results from each [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) and returns them.\n",
|
||||
"\n",
|
||||
"Filters can be attached to [CompositeDataSources](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource) just as they can be done to [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStore) and [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource). When `get()`/`all_versions()`/`query()` calls are made to the [CompositeDataSource](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource), it will pass along any query filters from the call and any of its own filters to the attached [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource). In addition, those [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource) may have their own attached filters as well. The effect is that all the filters are eventually combined when the `get()`/`all_versions()`/`query()` call is actually executed within a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource). \n",
|
||||
"Filters can be attached to [CompositeDataSources](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource) just as they can be done to [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) and [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource). When `get()`/`all_versions()`/`query()` calls are made to the [CompositeDataSource](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource), it will pass along any query filters from the call and any of its own filters to the attached [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource). In addition, those [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource) may have their own attached filters as well. The effect is that all the filters are eventually combined when the `get()`/`all_versions()`/`query()` call is actually executed within a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource). \n",
|
||||
"\n",
|
||||
"A [CompositeDataSource](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource) can also be attached to a [CompositeDataSource](../api/stix2.datastore.rst#stix2.datastore.CompositeDataSource) for multiple layers of grouped [DataSources](../api/stix2.datastore.rst#stix2.datastore.DataSource).\n",
|
||||
"\n",
|
||||
|
|
@ -353,7 +353,7 @@
|
|||
"source": [
|
||||
"## Filters\n",
|
||||
"\n",
|
||||
"The CTI Python STIX2 DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStore).\n",
|
||||
"The CTI Python STIX2 DataStore suites - [FileSystem](../api/datastore/stix2.datastore.filesystem.rst), [Memory](../api/datastore/stix2.datastore.memory.rst), and [TAXII](../api/datastore/stix2.datastore.taxii.rst) - all use the [Filters](../api/datastore/stix2.datastore.filters.rst) module to allow for the querying of STIX content. The basic functionality is that filters can be created and supplied everytime to calls to `query()`, and/or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) so that every future query placed to that [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) is evaluated against the attached filters, supplemented with any further filters supplied with the query call. Attached filters can also be removed from [DataStores](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin).\n",
|
||||
"\n",
|
||||
"Filters are very simple, as they consist of a field name, comparison operator and an object property value (i.e. value to compare to). All properties of STIX2 objects can be filtered on. In addition, TAXII2 Filtering parameters for fields can also be used in filters.\n",
|
||||
"\n",
|
||||
|
|
@ -418,7 +418,7 @@
|
|||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"For Filters to be applied to a query, they must be either supplied with the query call or attached a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore), more specifically to a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) whether that [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) is a part of a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) or stands by itself. "
|
||||
"For Filters to be applied to a query, they must be either supplied with the query call or attached to a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin), more specifically to a [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) whether that [DataSource](../api/stix2.datastore.rst#stix2.datastore.DataSource) is a part of a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) or stands by itself. "
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -461,7 +461,7 @@
|
|||
"source": [
|
||||
"## De-Referencing Relationships\n",
|
||||
"\n",
|
||||
"Given a STIX object, there are several ways to find other STIX objects related to it. To illustrate this, let's first create a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) and add some objects and relationships."
|
||||
"Given a STIX object, there are several ways to find other STIX objects related to it. To illustrate this, let's first create a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) and add some objects and relationships."
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -62,7 +62,7 @@
|
|||
"\n",
|
||||
"### Storing and Retrieving STIX Content\n",
|
||||
"\n",
|
||||
"An [Environment](../api/stix2.environment.rst#stix2.environment.Environment) can be set up with a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore) if you want to store and retrieve STIX content from the same place. "
|
||||
"An [Environment](../api/stix2.environment.rst#stix2.environment.Environment) can be set up with a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin) if you want to store and retrieve STIX content from the same place. "
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
@ -609,7 +609,7 @@
|
|||
"collapsed": true
|
||||
},
|
||||
"source": [
|
||||
"For the full power of the Environment layer, create an [Environment](../api/stix2.environment.rst#stix2.environment.Environment) with both a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStore)/[Source](../api/stix2.datastore.rst#stix2.datastore.DataSource)/[Sink](../api/stix2.datastore.rst#stix2.datastore.DataSink) and an [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory):"
|
||||
"For the full power of the Environment layer, create an [Environment](../api/stix2.environment.rst#stix2.environment.Environment) with both a [DataStore](../api/stix2.datastore.rst#stix2.datastore.DataStoreMixin)/[Source](../api/stix2.datastore.rst#stix2.datastore.DataSource)/[Sink](../api/stix2.datastore.rst#stix2.datastore.DataSink) and an [ObjectFactory](../api/stix2.environment.rst#stix2.environment.ObjectFactory):"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ def make_id():
|
|||
return str(uuid.uuid4())
|
||||
|
||||
|
||||
class DataStore(object):
|
||||
"""An implementer can subclass to create custom behavior from
|
||||
this class for the specific DataStores.
|
||||
class DataStoreMixin(object):
|
||||
"""Provides mechanisms for storing and retrieving STIX data. The specific
|
||||
behavior can be customized by subclasses.
|
||||
|
||||
Args:
|
||||
source (DataSource): An existing DataSource to use
|
||||
|
|
@ -41,7 +41,7 @@ class DataStore(object):
|
|||
|
||||
"""
|
||||
def __init__(self, source=None, sink=None):
|
||||
super(DataStore, self).__init__()
|
||||
super(DataStoreMixin, self).__init__()
|
||||
self.id = make_id()
|
||||
self.source = source
|
||||
self.sink = sink
|
||||
|
|
|
|||
|
|
@ -7,12 +7,12 @@ import json
|
|||
import os
|
||||
|
||||
from stix2.core import Bundle, parse
|
||||
from stix2.datastore import DataSink, DataSource, DataStore
|
||||
from stix2.datastore import DataSink, DataSource, DataStoreMixin
|
||||
from stix2.datastore.filters import Filter, apply_common_filters
|
||||
from stix2.utils import deduplicate, get_class_hierarchy_names
|
||||
|
||||
|
||||
class FileSystemStore(DataStore):
|
||||
class FileSystemStore(DataStoreMixin):
|
||||
"""Interface to a file directory of STIX objects.
|
||||
|
||||
FileSystemStore is a wrapper around a paired FileSystemSink
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ import os
|
|||
|
||||
from stix2.base import _STIXBase
|
||||
from stix2.core import Bundle, parse
|
||||
from stix2.datastore import DataSink, DataSource, DataStore
|
||||
from stix2.datastore import DataSink, DataSource, DataStoreMixin
|
||||
from stix2.datastore.filters import Filter, apply_common_filters
|
||||
|
||||
|
||||
|
|
@ -56,7 +56,7 @@ def _add(store, stix_data=None, version=None):
|
|||
" or a JSON formatted STIX bundle. stix_data was of type: " + str(type(stix_data)))
|
||||
|
||||
|
||||
class MemoryStore(DataStore):
|
||||
class MemoryStore(DataStoreMixin):
|
||||
"""Interface to an in-memory dictionary of STIX objects.
|
||||
|
||||
MemoryStore is a wrapper around a paired MemorySink and MemorySource.
|
||||
|
|
@ -124,7 +124,7 @@ class MemorySink(DataSink):
|
|||
Args:
|
||||
stix_data (dict OR list): valid STIX 2.0 content in
|
||||
bundle or a list.
|
||||
_store (bool): whether the MemorySink is a part of a DataStore,
|
||||
_store (bool): whether the MemorySink is a part of a MemoryStore,
|
||||
in which case "stix_data" is a direct reference to
|
||||
shared memory with DataSource. Not user supplied
|
||||
allow_custom (bool): whether to allow custom objects/properties
|
||||
|
|
@ -170,7 +170,7 @@ class MemorySource(DataSource):
|
|||
Args:
|
||||
stix_data (dict OR list OR STIX object): valid STIX 2.0 content in
|
||||
bundle or list.
|
||||
_store (bool): if the MemorySource is a part of a DataStore,
|
||||
_store (bool): if the MemorySource is a part of a MemoryStore,
|
||||
in which case "stix_data" is a direct reference to shared
|
||||
memory with DataSink. Not user supplied
|
||||
allow_custom (bool): whether to allow custom objects/properties
|
||||
|
|
|
|||
|
|
@ -5,14 +5,14 @@ from requests.exceptions import HTTPError
|
|||
|
||||
from stix2.base import _STIXBase
|
||||
from stix2.core import Bundle, parse
|
||||
from stix2.datastore import DataSink, DataSource, DataStore
|
||||
from stix2.datastore import DataSink, DataSource, DataStoreMixin
|
||||
from stix2.datastore.filters import Filter, apply_common_filters
|
||||
from stix2.utils import deduplicate
|
||||
|
||||
TAXII_FILTERS = ['added_after', 'id', 'type', 'version']
|
||||
|
||||
|
||||
class TAXIICollectionStore(DataStore):
|
||||
class TAXIICollectionStore(DataStoreMixin):
|
||||
"""Provides an interface to a local/remote TAXII Collection
|
||||
of STIX data. TAXIICollectionStore is a wrapper
|
||||
around a paired TAXIICollectionSink and TAXIICollectionSource.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import copy
|
||||
|
||||
from .core import parse as _parse
|
||||
from .datastore import CompositeDataSource, DataStore
|
||||
from .datastore import CompositeDataSource, DataStoreMixin
|
||||
|
||||
|
||||
class ObjectFactory(object):
|
||||
|
|
@ -75,7 +75,7 @@ class ObjectFactory(object):
|
|||
return cls(**properties)
|
||||
|
||||
|
||||
class Environment(object):
|
||||
class Environment(DataStoreMixin):
|
||||
"""Abstract away some of the nasty details of working with STIX content.
|
||||
|
||||
Args:
|
||||
|
|
@ -86,6 +86,14 @@ class Environment(object):
|
|||
source (DataSource, optional): Source for retrieving STIX objects.
|
||||
sink (DataSink, optional): Destination for saving STIX objects.
|
||||
Invalid if `store` is also provided.
|
||||
|
||||
.. automethod:: get
|
||||
.. automethod:: all_versions
|
||||
.. automethod:: query
|
||||
.. automethod:: creator_of
|
||||
.. automethod:: relationships
|
||||
.. automethod:: related_to
|
||||
.. automethod:: add
|
||||
"""
|
||||
|
||||
def __init__(self, factory=ObjectFactory(), store=None, source=None, sink=None):
|
||||
|
|
@ -105,14 +113,6 @@ class Environment(object):
|
|||
return self.factory.create(*args, **kwargs)
|
||||
create.__doc__ = ObjectFactory.create.__doc__
|
||||
|
||||
get = DataStore.__dict__['get']
|
||||
all_versions = DataStore.__dict__['all_versions']
|
||||
query = DataStore.__dict__['query']
|
||||
creator_of = DataStore.__dict__['creator_of']
|
||||
relationships = DataStore.__dict__['relationships']
|
||||
related_to = DataStore.__dict__['related_to']
|
||||
add = DataStore.__dict__['add']
|
||||
|
||||
def add_filters(self, *args, **kwargs):
|
||||
try:
|
||||
return self.source.filters.update(*args, **kwargs)
|
||||
|
|
|
|||
Loading…
Reference in New Issue