Commit Graph

794 Commits (e1356437fc12ecd6491c16c1e5c8bae08a3f1f6f)

Author SHA1 Message Date
Emmanuelle Vargas-Gonzalez 63c22aba99 fix path issues related to memory datastore 2018-11-29 12:17:26 -05:00
Emmanuelle Vargas-Gonzalez 6e9312efb7 fix test memory datastore teardown 2018-11-29 11:48:14 -05:00
Emmanuelle Vargas-Gonzalez 1b0fa0129f pre-commit changes 2018-11-29 11:06:27 -05:00
Emmanuelle Vargas-Gonzalez 7f3a8b6c80 more tests to improve coverage 2018-11-29 10:27:13 -05:00
Emmanuelle Vargas-Gonzalez 6f897bc91d small enhancements, fix property for TLPMarking 2018-11-29 10:26:20 -05:00
Emmanuelle Vargas-Gonzalez 79c9d85072 make Memory datastore return path where data was saved to 2018-11-29 10:25:15 -05:00
Emmanuelle Vargas-Gonzalez 682e90ccaa expose the confidence methods via `stix2.scales.<method>` 2018-11-28 17:17:05 -05:00
Emmanuelle Vargas-Gonzalez e896812754 minor code changes 2018-11-28 16:51:35 -05:00
Emmanuelle Vargas-Gonzalez 71a2aa2611 update project documentation. 2018-11-28 16:51:00 -05:00
Emmanuelle Vargas-Gonzalez 97a21c3064 my precious tables gone :( 2018-11-28 15:34:48 -05:00
Emmanuelle Vargas-Gonzalez c3031a0282 fix typo on DNI scale 2018-11-28 11:28:26 -05:00
Emmanuelle Vargas-Gonzalez aaddeb8b97 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 1.1.0-release 2018-11-28 11:21:27 -05:00
Michael Chisholm 17970a3faa Fixed a couple filter tests.
- a length check should come before the access, so you can verify
whether the access will succeed.

- Also removed some tests which can't work, due to the filter
changes.  In fact, a lot of these tests should probably be
removed or changed if we want to disallow running
apply_common_filters() on plain dicts.  They will often
coincidentally still succeed though, so I left them in.
2018-11-27 18:42:51 -05:00
Michael Chisholm 3a2f247f68 Fixed my own brainfart with converting string filter values to
datetimes: I'd converted the object property instead of the
filter value! :-P

Also, I fixed filter validation: it was checking for exact types
of the filter values and disallowing subtypes.  This library
includes a datetime subtype named STIXdatetime, and this type
should be usable as a filter value too.  So we need to allow
subtypes.
2018-11-27 18:38:55 -05:00
Michael Chisholm f57b9c34ef Add a newline to the end of a file 2018-11-27 17:58:01 -05:00
Michael Chisholm 18ff6f6094 Import cleanup to satisfy tox checks 2018-11-27 17:52:38 -05:00
Michael Chisholm 3adf7800a8 Changed how filters work, with respect to datetime objects.
Timestamp properties can now be checked against filter values
which are either strings or datetime objects, using datetime
semantics (previously, it reduced to a string compare).
If a stix object property is datetime-valued and the filter
value is a string, the string is parsed to a datetime object,
rather than the other way around.

Filtering in the filesystem store now parses JSON dicts to
_STIXBase objects before applying the filters.

Due to the parsing change, bad JSON content can produce a
different kind of error, so I had to change one of the tests.
2018-11-27 17:36:17 -05:00
Michael Chisholm d8a775c60d Fix some more improper exception re-raises in the filesystem
datastore test suite.  Add a new test corpus file, located so
as to test the backward compatibility functionality of
FileSystemSource.  Add a test to the suite which ensures that
this new file is found.
2018-11-27 15:24:09 -05:00
Michael Chisholm 63166ab256 Add some backward-compatibility to filesystem store: versioned
objects are searched for as ID-named json files in the type
directories, in addition to timestamp-named files in ID
directories.

Made a bugfix: fixed improper exception re-raises

Made an efficiency improvement: don't stat() files in
_get_matching_dir_entries() if no st_mode_test callable is given.
2018-11-27 15:24:09 -05:00
Michael Chisholm f615161110 Added some tests for adding markings to sinks and stores. 2018-11-27 15:24:08 -05:00
Michael Chisholm da13882eec Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-27 15:24:08 -05:00
Michael Chisholm 0cecbeb9d8 Ran trailing-whitespace pre-commit hook. It changed a bunch of
files, in ways we don't completely understand...
2018-11-27 15:24:08 -05:00
Michael Chisholm 0a8ff2ab2e Add some newer versions of a couple of object IDs in the stix2
test data corpus.  Updated filesystem store tests accordingly:
- Remove comments from all_versions tests stating that multiple
  versions are not supported.  Improve the tests to ensure that
  all versions are in fact retrieved.
- Update the get() test to assure that it gets only the latest
  version, when there is more than one version.
- Update some count checks, since there are more objects now
- Fix some typos
2018-11-27 15:24:07 -05:00
Michael Chisholm 2b983368e5 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-27 15:24:07 -05:00
Michael Chisholm 9693c16cd1 Adjust import order to satisfy tox import check 2018-11-27 15:24:07 -05:00
Michael Chisholm 428a18ded2 Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-27 15:24:07 -05:00
Michael Chisholm 461e8bd5cb Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-27 15:24:07 -05:00
Michael Chisholm 0096835cfc Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-27 15:24:07 -05:00
Emmanuelle Vargas-Gonzalez d01e6b47af Bump version: 1.0.3 → 1.0.4 2018-11-15 11:10:50 -05:00
Emmanuelle Vargas-Gonzalez 51df054f33
Update memory.py 2018-11-14 15:16:49 -05:00
Emmanuelle Vargas-Gonzalez d6435a18fa Missing changes to key/value approach 2018-11-14 15:03:57 -05:00
Emmanuelle Vargas-Gonzalez c80f39ceed Change approach to allow for custom objects 2018-11-14 14:35:22 -05:00
Chris Lenk c4668f5dc1
Revert "Multi version filesystem store" 2018-11-07 10:10:06 -05:00
Chris Lenk 150457c1bb
Merge pull request #222 from chisholm/multi_version_filesystem_store
Multi version filesystem store
2018-11-06 16:32:07 -05:00
Michael Chisholm b235e5773c Added some tests for adding markings to sinks and stores. 2018-11-06 16:15:33 -05:00
Michael Chisholm 7bb3d1f6a6 Fix FileSystemSource.get() to not look for the latest version of
an object when markings are queried, since markings are not
versioned.
2018-11-06 16:06:26 -05:00
Michael Chisholm 9f83f2140b Ran trailing-whitespace pre-commit hook. It changed a bunch of
files, in ways we don't completely understand...
2018-11-06 15:10:40 -05:00
Michael Chisholm a8d9aef673 Add some newer versions of a couple of object IDs in the stix2
test data corpus.  Updated filesystem store tests accordingly:
- Remove comments from all_versions tests stating that multiple
  versions are not supported.  Improve the tests to ensure that
  all versions are in fact retrieved.
- Update the get() test to assure that it gets only the latest
  version, when there is more than one version.
- Update some count checks, since there are more objects now
- Fix some typos
2018-11-01 20:25:00 -04:00
Michael Chisholm e2f5d60b51 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-01 20:25:00 -04:00
Michael Chisholm e2d9325356 Adjust import order to satisfy tox import check 2018-11-01 20:25:00 -04:00
Michael Chisholm ee57596d6a Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-01 20:25:00 -04:00
Michael Chisholm 51668a9a04 Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-01 20:25:00 -04:00
Michael Chisholm 9486b46f77 Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-01 20:25:00 -04:00
Chris Lenk 3b297c17b5 Use consistent errors for observable extensions
Whether or not the Observable type is in the EXT_MAP already, using a
custom extension without also using allow_custom=True should result in
the same behavior/error message.
2018-11-01 17:23:55 -04:00
Emmanuelle Vargas-Gonzalez 8d24015186 Update Memory datastore to allow for mapping objects 2018-11-01 10:54:58 -04:00
Emmanuelle Vargas-Gonzalez 5abe518b8a Bump version: 1.0.3 → 1.1.0 2018-11-01 09:55:37 -04:00
Emmanuelle Vargas-Gonzalez 493bd65ead Update README and refactor code to make 2.0 default. Update some tests 2018-11-01 09:21:02 -04:00
Emmanuelle Vargas-Gonzalez 5e5d10e7aa Finish alignment of 2.1 components 2018-11-01 08:17:34 -04:00
Emmanuelle Vargas-Gonzalez eff5369670 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-11-01 07:57:09 -04:00
Emmanuelle Vargas-Gonzalez f85f4e566b Bump version: 1.0.2 → 1.0.3 2018-10-31 13:46:47 -04:00
Chris Lenk 3542604b02
Merge pull request #209 from chisholm/taxii_api_updates
Fixed a test fixture to call the cti-taxii-client Collection constructor correctly
2018-10-31 12:04:34 -04:00
Chris Lenk db300d1f21 Fix `created` millisecond precision in TLPs
A marking definition's `created` property doesn't require millisecond
preprecision, but for TLP markings the TLP instances provided in the
spec must be used and they all use millisecond precision.
2018-10-25 13:58:21 -04:00
Chris Lenk 6613b55a43 Update MemoryStore documentation
...now that MemorySource normalizes its data to python-stix2 objects.
2018-10-23 14:29:56 -04:00
Michael Chisholm cbe8d22d0a Added support to multi-version memory stores for markings. Also
added some more unit tests which test storing/retrieving markings
from the stores.
2018-10-17 20:54:53 -04:00
Michael Chisholm 2d89cfb0cf Remove outdated TODO's from the memory datastore module-level
docstring.
2018-10-17 15:49:58 -04:00
Emmanuelle Vargas-Gonzalez d614343910 Rename tests with duplicate name. 2018-10-17 07:56:10 -04:00
Emmanuelle Vargas-Gonzalez 352749edb0 Add constrains to ObservedData and Sighting, tests updated. 2018-10-17 07:47:25 -04:00
Emmanuelle Vargas-Gonzalez f8a72b0937 Custom builder code updated for 3.7 support.
Updated properties to support more constrains.
Make all regexes literal strings.
Update tests to align to new constrains.
Workbench problem. _check_object_constraints() uses instance class to
perform proper class resolution calls.
2018-10-17 07:34:15 -04:00
Emmanuelle Vargas-Gonzalez b2ef77b322 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-10-17 07:30:23 -04:00
Michael Chisholm 864ba05b71 Fix import order to satisfy isort-check 2018-10-15 19:23:28 -04:00
Michael Chisholm d9f6a213c1 Fixed Memory source/sink/store so that it supports multiple versions
of objects.  Fixed several bugs too.
2018-10-15 18:02:49 -04:00
Michael Chisholm 5a0e102959 Modify docstring for apply_common_filters() to change what it
says are the required types for its parameters.  It gave
specific types (list and set), when really it worked with more
than just that (iterables).  And I certainly didn't only call
it with just lists and sets.
2018-10-15 18:02:48 -04:00
Emmanuelle Vargas-Gonzalez dec75082df Add new constrains parameters to IntegerProperty and FloatProperty
New constraints on timestamps, integer and floats for many objects
2018-10-15 15:02:59 -04:00
Emmanuelle Vargas-Gonzalez acd86c80dd Update tests to new object constraints 2018-10-15 14:48:52 -04:00
Maciej Urbanski e80fb4f59a
python 3.7 support 2018-10-03 18:00:38 +02:00
Michael Chisholm 541c682bf6 Fixed a test fixture to call the cti-taxii-client Collection
constructor correctly.  It had been recently changed to address
issue #39 in that library.
2018-09-07 17:47:24 -04:00
mbastian1135 5648d2bc45 added to pattern guide; reverting pattern str conversion (as was already proper) 2018-08-30 11:57:20 -04:00
mbastian1135 cd81d97beb added 'to_pattern()' utility for pattern expressions; more information on patterning guide 2018-08-27 16:24:23 -04:00
mbastian1135 10149b9345 added 'to_pattern()' utility for pattern expressions; more information on patterning guide 2018-08-27 16:21:14 -04:00
mbastian1135 a9de13107f pattern guide 2018-08-24 10:40:37 -04:00
Emmanuelle Vargas-Gonzalez 211b8d8cee Add core tests and minor change to parse_observable() 2018-07-25 20:53:53 -04:00
Emmanuelle Vargas-Gonzalez 5e71f9225b Add regex approach to load mappings 2018-07-25 14:06:18 -04:00
Emmanuelle Vargas-Gonzalez ad76e7155c MALWARE RESTORE POINT - Reverted changes to Malware based on STIX 2.1 CSD01
Use this commit to restore Malware changes.
2018-07-25 13:34:56 -04:00
Emmanuelle Vargas-Gonzalez 303159a818 pre-commit hooks changes 2018-07-25 13:32:22 -04:00
Emmanuelle Vargas-Gonzalez 21c84acc8f Add missing properties to Relationship object and update tests 2018-07-25 12:44:46 -04:00
Emmanuelle Vargas-Gonzalez af2a5605ce Add constraints to Location object 2018-07-25 12:43:57 -04:00
Emmanuelle Vargas-Gonzalez 40d656c94c Minor changes to tests. 2018-07-25 12:43:08 -04:00
Emmanuelle Vargas-Gonzalez 51a499cb33 Formatting changes made by the new pre-commit hook 'add trailing commas'
closes #189
2018-07-13 11:10:05 -04:00
Emmanuelle Vargas-Gonzalez 965d7fa788 Update v20 and v21 tests
In v20, only minor stuff that was addressing wrong spec. In v21, align tests with new/changed properties in the specs
2018-07-12 14:33:00 -04:00
Emmanuelle Vargas-Gonzalez bdfc221cb0 Update v21 properties to latest spec changes 2018-07-12 14:31:14 -04:00
Emmanuelle Vargas-Gonzalez 281dbfb0f4 Align tests with news additions from 'master' branch. 2018-07-11 09:43:37 -04:00
Emmanuelle Vargas-Gonzalez a042970a1f Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-07-11 09:41:42 -04:00
Emmanuelle Vargas-Gonzalez ee260b7574 Sort import check 2018-07-11 08:38:06 -04:00
Emmanuelle Vargas-Gonzalez e513c8d638 Hide builder methods in 'custom.py' and update imports accordingly 2018-07-11 08:11:47 -04:00
Emmanuelle Vargas-Gonzalez 6bd797e258 Fix 'test_memory_store_object_with_custom_property_in_bundle'
Since Bundles objects are now added instead of the Bundle itself, it now works as intended.
2018-07-10 16:19:40 -04:00
Emmanuelle Vargas-Gonzalez 834ef2c847 Fix check to collections.Mapping 2018-07-10 16:13:29 -04:00
Emmanuelle Vargas-Gonzalez c91bcd43f6 Fix location for test_object_property 2018-07-10 16:11:07 -04:00
Emmanuelle Vargas-Gonzalez 48e0442439 Fix tests in 'test_properties.py' 2018-07-10 16:10:01 -04:00
Emmanuelle Vargas-Gonzalez d24cddb547 Temporarily skip failing tests in workbench for v20, but the approach needs to be fixed 2018-07-10 16:08:36 -04:00
Emmanuelle Vargas-Gonzalez 4583da3ef2 Sort imports 2018-07-10 16:05:30 -04:00
Emmanuelle Vargas-Gonzalez ce42c02cee Fix tests that use property objects to call in the right path 2018-07-10 15:56:22 -04:00
Emmanuelle Vargas-Gonzalez 8cf68054d4 Remove str() as a way to serialize objects. Add support for encodings and Bundle versions. 2018-07-10 15:51:20 -04:00
Emmanuelle Vargas-Gonzalez 012eba4e9b Add new Bundle support for add_objects() request. Add encoding support.
Removed the version from all methods since it is no longer necessary.
2018-07-10 15:49:36 -04:00
Emmanuelle Vargas-Gonzalez 9b8cb09b1a Remove 'version' from calls to parse since it is no longer necessary
Also, fixed adding STIX2 Bundles to MemorySource. Enhancements to 'save_to_file'. Fix docstrings and encoding support when writing to file. closes #202
2018-07-10 15:43:58 -04:00
Emmanuelle Vargas-Gonzalez b6fefc52d9 Fix call to collect STIX2 mappings, make parse_observable available 2018-07-10 15:31:22 -04:00
Emmanuelle Vargas-Gonzalez 8d378fcf81 Remove STIXRelationshipObject from 'sro.py' 2018-07-10 15:27:05 -04:00
Emmanuelle Vargas-Gonzalez 023603d86f Remove duplicate code from 'sdo.py', removed STIXDomainObject
Apply proper 'spec_version' constraints to v21 objects
2018-07-10 15:22:21 -04:00
Emmanuelle Vargas-Gonzalez 54268ae7dd Remove observables and extension mappings, custom code and apply property constrain in v21 2018-07-10 15:20:16 -04:00
Emmanuelle Vargas-Gonzalez 1177694739 Adding docstrings to 2.1 objects 2018-07-10 15:15:33 -04:00
Emmanuelle Vargas-Gonzalez b722fdc0ed Remove duplicate register methods and CustomMarking code 2018-07-10 15:07:08 -04:00