Commit Graph

111 Commits (f7ebd34c8c9c2331dea5fe80389e6bfbff9fe124)

Author SHA1 Message Date
Chris Lenk b4a0a9ea10 Remove six dependency (backwards breaking)
We've already removed Python 2 from our test harness, but this removes
all python 2 compatibility code left in.
2021-02-18 12:38:35 -05:00
Emmanuelle Vargas-Gonzalez 85c14d1502 all changes from add-trailing-commas v2.0.2 2021-01-13 17:52:15 -05:00
Chris Lenk d17d01d165
Merge pull request #444 from chisholm/comp_ds_unversioned_objs
Update CompositeDataSource and deduplicate() support for unversioned objects
2020-10-15 08:57:55 -04:00
Emmanuelle Vargas-Gonzalez 7b35b82996 TAXIICollectionSource custom object handling fix 2020-09-29 11:21:02 -04:00
Michael Chisholm 7fa3c78dea Update CompositeDataSource and deduplicate() to handle unversioned
objects, including 2.1 SCOs.  Updated some unit tests to test
this.  Fixed a typo in a 2.0 unit test (2.0 deduplicate() test).
2020-08-17 18:38:29 -04:00
Emmanuelle Vargas-Gonzalez c760e04c9a rename module to serialization.py 2020-07-22 14:31:26 -04:00
Chris Lenk 55ea84ece2 Fix bug when adding custom obj to FileSystemSink
... if the object type hasn't been registered.

Related: #439.
2020-07-20 00:04:32 -04:00
Chris Lenk 10bdfe9e25 Fix KeyError in TAXII DataStore 2020-06-26 16:43:19 -04:00
Chris Lenk df92770d25
Merge pull request #384 from oasis-open/365-versioned-classes
Validate custom type/property name formats
2020-04-03 17:30:24 -04:00
Emmanuelle Vargas-Gonzalez 2b0d63c4b1 update test_datastore_taxii.py conftest.py for latest changes in medallion. add extra data used by filter 2020-04-03 17:19:36 -04:00
Chris Lenk 50df6f1474 Rename core.py -> parsing.py 2020-03-27 05:53:39 -04:00
Michael Chisholm 4c67142b92 Fix the filesystem store to support the new top-level 2.1 SCOs. 2020-02-15 19:02:53 -05:00
Michael Chisholm 98a654884d Fix the memory store to support the new top-level 2.1 SCOs. 2020-02-13 17:11:58 -05:00
Emmanuelle Vargas-Gonzalez c467f198c8 add encoding to MemorySource load_from_file() 2020-01-15 14:15:08 -05:00
Emmanuelle Vargas-Gonzalez 25cfb25ef3 add encoding and propagate accordingly for calls 2020-01-15 14:12:58 -05:00
John-Mark Gurney 1c03b4a1f0
minor grammar fix 2019-02-04 13:58:33 -08:00
Desai, Kartikey H 767a758b28 Fix styling issue around imports for issue 232 2019-01-09 11:32:51 -05:00
Desai, Kartikey H 77b2e0e3e3 Remove a few comments and Fixes #232 2019-01-09 10:22:33 -05:00
Desai, Kartikey H 7883614d2f Fixes #232 2019-01-09 08:36:10 -05:00
Emmanuelle Vargas-Gonzalez 6e50bf5123 Formatting problems... 2018-12-11 13:48:56 -05:00
Emmanuelle Vargas-Gonzalez c8c48a305a Add future import to resolve compatibility problems 2018-12-11 13:41:19 -05:00
Emmanuelle Vargas-Gonzalez ff098a19b1 update method _timestamp2filename() since it introduces timing precision problems 2018-12-11 13:06:51 -05:00
Emmanuelle Vargas-Gonzalez 96b81fc489 pre-commit formatting changes 2018-12-06 15:19:50 -05:00
Emmanuelle Vargas-Gonzalez bfa49ce37a Removed everything in test/v21/stix2_data/
Bring back optional version parameter to datastores. Update
documentation. Update v21 test suite
2018-12-06 15:11:30 -05:00
Emmanuelle Vargas-Gonzalez 3d099bec91 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 1.1.0-release 2018-12-06 15:08:36 -05:00
Emmanuelle Vargas-Gonzalez e1f7cc4028 change "Notes" for "Note" to keep visual effect in documentation 2018-11-30 09:39:05 -05:00
Emmanuelle Vargas-Gonzalez f1490a98c8 remove full path from `constants` and fix directory resolution 2018-11-29 13:49:06 -05:00
Emmanuelle Vargas-Gonzalez 79c9d85072 make Memory datastore return path where data was saved to 2018-11-29 10:25:15 -05:00
Emmanuelle Vargas-Gonzalez 71a2aa2611 update project documentation. 2018-11-28 16:51:00 -05:00
Emmanuelle Vargas-Gonzalez aaddeb8b97 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 1.1.0-release 2018-11-28 11:21:27 -05:00
Michael Chisholm 3a2f247f68 Fixed my own brainfart with converting string filter values to
datetimes: I'd converted the object property instead of the
filter value! :-P

Also, I fixed filter validation: it was checking for exact types
of the filter values and disallowing subtypes.  This library
includes a datetime subtype named STIXdatetime, and this type
should be usable as a filter value too.  So we need to allow
subtypes.
2018-11-27 18:38:55 -05:00
Michael Chisholm 18ff6f6094 Import cleanup to satisfy tox checks 2018-11-27 17:52:38 -05:00
Michael Chisholm 3adf7800a8 Changed how filters work, with respect to datetime objects.
Timestamp properties can now be checked against filter values
which are either strings or datetime objects, using datetime
semantics (previously, it reduced to a string compare).
If a stix object property is datetime-valued and the filter
value is a string, the string is parsed to a datetime object,
rather than the other way around.

Filtering in the filesystem store now parses JSON dicts to
_STIXBase objects before applying the filters.

Due to the parsing change, bad JSON content can produce a
different kind of error, so I had to change one of the tests.
2018-11-27 17:36:17 -05:00
Michael Chisholm 63166ab256 Add some backward-compatibility to filesystem store: versioned
objects are searched for as ID-named json files in the type
directories, in addition to timestamp-named files in ID
directories.

Made a bugfix: fixed improper exception re-raises

Made an efficiency improvement: don't stat() files in
_get_matching_dir_entries() if no st_mode_test callable is given.
2018-11-27 15:24:09 -05:00
Michael Chisholm da13882eec Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-27 15:24:08 -05:00
Michael Chisholm 2b983368e5 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-27 15:24:07 -05:00
Michael Chisholm 9693c16cd1 Adjust import order to satisfy tox import check 2018-11-27 15:24:07 -05:00
Michael Chisholm 428a18ded2 Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-27 15:24:07 -05:00
Michael Chisholm 461e8bd5cb Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-27 15:24:07 -05:00
Michael Chisholm 0096835cfc Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-27 15:24:07 -05:00
Emmanuelle Vargas-Gonzalez 51df054f33
Update memory.py 2018-11-14 15:16:49 -05:00
Emmanuelle Vargas-Gonzalez d6435a18fa Missing changes to key/value approach 2018-11-14 15:03:57 -05:00
Emmanuelle Vargas-Gonzalez c80f39ceed Change approach to allow for custom objects 2018-11-14 14:35:22 -05:00
Chris Lenk c4668f5dc1
Revert "Multi version filesystem store" 2018-11-07 10:10:06 -05:00
Michael Chisholm 7bb3d1f6a6 Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-06 16:06:26 -05:00
Michael Chisholm e2f5d60b51 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-01 20:25:00 -04:00
Michael Chisholm e2d9325356 Adjust import order to satisfy tox import check 2018-11-01 20:25:00 -04:00
Michael Chisholm ee57596d6a Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-01 20:25:00 -04:00
Michael Chisholm 51668a9a04 Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-01 20:25:00 -04:00
Michael Chisholm 9486b46f77 Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-01 20:25:00 -04:00