Commit Graph

46 Commits (b2108e90c67637fdbefc8871e11fb7abb925b0dc)

Author SHA1 Message Date
Emmanuelle Vargas-Gonzalez 2ea9c0c63c use it on filesystem.py data sink 2021-03-17 15:15:26 -04:00
Chris Lenk b4a0a9ea10 Remove six dependency (backwards breaking)
We've already removed Python 2 from our test harness, but this removes
all python 2 compatibility code left in.
2021-02-18 12:38:35 -05:00
Emmanuelle Vargas-Gonzalez c760e04c9a rename module to serialization.py 2020-07-22 14:31:26 -04:00
Chris Lenk 55ea84ece2 Fix bug when adding custom obj to FileSystemSink
... if the object type hasn't been registered.

Related: #439.
2020-07-20 00:04:32 -04:00
Chris Lenk 50df6f1474 Rename core.py -> parsing.py 2020-03-27 05:53:39 -04:00
Michael Chisholm 4c67142b92 Fix the filesystem store to support the new top-level 2.1 SCOs. 2020-02-15 19:02:53 -05:00
Emmanuelle Vargas-Gonzalez 25cfb25ef3 add encoding and propagate accordingly for calls 2020-01-15 14:12:58 -05:00
Desai, Kartikey H 767a758b28 Fix styling issue around imports for issue 232 2019-01-09 11:32:51 -05:00
Desai, Kartikey H 77b2e0e3e3 Remove a few comments and Fixes #232 2019-01-09 10:22:33 -05:00
Desai, Kartikey H 7883614d2f Fixes #232 2019-01-09 08:36:10 -05:00
Emmanuelle Vargas-Gonzalez 6e50bf5123 Formatting problems... 2018-12-11 13:48:56 -05:00
Emmanuelle Vargas-Gonzalez c8c48a305a Add future import to resolve compatibility problems 2018-12-11 13:41:19 -05:00
Emmanuelle Vargas-Gonzalez ff098a19b1 update method _timestamp2filename() since it introduces timing precision problems 2018-12-11 13:06:51 -05:00
Emmanuelle Vargas-Gonzalez 96b81fc489 pre-commit formatting changes 2018-12-06 15:19:50 -05:00
Emmanuelle Vargas-Gonzalez 3d099bec91 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 1.1.0-release 2018-12-06 15:08:36 -05:00
Emmanuelle Vargas-Gonzalez e1f7cc4028 change "Notes" for "Note" to keep visual effect in documentation 2018-11-30 09:39:05 -05:00
Emmanuelle Vargas-Gonzalez 71a2aa2611 update project documentation. 2018-11-28 16:51:00 -05:00
Michael Chisholm 3adf7800a8 Changed how filters work, with respect to datetime objects.
Timestamp properties can now be checked against filter values
which are either strings or datetime objects, using datetime
semantics (previously, it reduced to a string compare).
If a stix object property is datetime-valued and the filter
value is a string, the string is parsed to a datetime object,
rather than the other way around.

Filtering in the filesystem store now parses JSON dicts to
_STIXBase objects before applying the filters.

Due to the parsing change, bad JSON content can produce a
different kind of error, so I had to change one of the tests.
2018-11-27 17:36:17 -05:00
Michael Chisholm 63166ab256 Add some backward-compatibility to filesystem store: versioned
objects are searched for as ID-named json files in the type
directories, in addition to timestamp-named files in ID
directories.

Made a bugfix: fixed improper exception re-raises

Made an efficiency improvement: don't stat() files in
_get_matching_dir_entries() if no st_mode_test callable is given.
2018-11-27 15:24:09 -05:00
Michael Chisholm da13882eec Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-27 15:24:08 -05:00
Michael Chisholm 2b983368e5 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-27 15:24:07 -05:00
Michael Chisholm 9693c16cd1 Adjust import order to satisfy tox import check 2018-11-27 15:24:07 -05:00
Michael Chisholm 428a18ded2 Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-27 15:24:07 -05:00
Michael Chisholm 461e8bd5cb Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-27 15:24:07 -05:00
Michael Chisholm 0096835cfc Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-27 15:24:07 -05:00
Chris Lenk c4668f5dc1
Revert "Multi version filesystem store" 2018-11-07 10:10:06 -05:00
Michael Chisholm 7bb3d1f6a6 Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-06 16:06:26 -05:00
Michael Chisholm e2f5d60b51 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-01 20:25:00 -04:00
Michael Chisholm e2d9325356 Adjust import order to satisfy tox import check 2018-11-01 20:25:00 -04:00
Michael Chisholm ee57596d6a Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-01 20:25:00 -04:00
Michael Chisholm 51668a9a04 Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-01 20:25:00 -04:00
Michael Chisholm 9486b46f77 Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-01 20:25:00 -04:00
Emmanuelle Vargas-Gonzalez eff5369670 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-11-01 07:57:09 -04:00
Emmanuelle Vargas-Gonzalez 51a499cb33 Formatting changes made by the new pre-commit hook 'add trailing commas'
closes #189
2018-07-13 11:10:05 -04:00
Emmanuelle Vargas-Gonzalez 8cf68054d4 Remove str() as a way to serialize objects. Add support for encodings and Bundle versions. 2018-07-10 15:51:20 -04:00
Emmanuelle Vargas-Gonzalez 7fd379d0b5 Minor style changes.
Removed OrderedDict and update()... Also a lot of single quoting except for errors
2018-06-29 18:38:04 -04:00
Michael Chisholm 486c588306 Fix silly isort check errors.. 2018-06-14 15:56:02 -04:00
Michael Chisholm 0c3f826c24 First cut at splitting the Bundle implementation into v20 and
v21 variants.  Also fixed up unit tests and got them passing
again.
2018-06-14 15:56:02 -04:00
= 27647091a5 WIP - just at stash point 2018-04-11 13:36:52 -04:00
Michael K 383bf5755e
remove debug line 2018-03-21 22:06:46 -04:00
= f4558c0958 pre-commit errors 2018-03-14 19:09:25 -04:00
= 6f762e7ea0 woops forgot file, recommit of: more type checking of filesystem json files; added corresponding tests 2018-03-14 16:32:31 -04:00
= af14cd4f88 more type checking of filesystem json files; added corresponding tests 2018-03-14 16:28:44 -04:00
Greg Back 5820fa0845 GH-138: ignore invalid JSON files. 2018-03-14 10:06:23 -05:00
Chris Lenk eb2d87ce71 Rename "DataStore" class to "DataStoreMixin"
Fix #116.
2018-03-01 11:27:37 -05:00
Chris Lenk 5f703509be Rename "sources" subpackage to "datastore"
Fix #114.
2018-03-01 09:04:42 -05:00