Commit Graph

807 Commits (0ca2bc087a0a7440cf7f5e1666e07769809d3f9d)

Author SHA1 Message Date
Michael Chisholm a8d9aef673 Add some newer versions of a couple of object IDs in the stix2
test data corpus.  Updated filesystem store tests accordingly:
- Remove comments from all_versions tests stating that multiple
  versions are not supported.  Improve the tests to ensure that
  all versions are in fact retrieved.
- Update the get() test to assure that it gets only the latest
  version, when there is more than one version.
- Update some count checks, since there are more objects now
- Fix some typos
2018-11-01 20:25:00 -04:00
Michael Chisholm e2f5d60b51 Fix an indexing error which caused FileSystemSource.get() to return
the oldest object instead of the newest.
2018-11-01 20:25:00 -04:00
Michael Chisholm e2d9325356 Adjust import order to satisfy tox import check 2018-11-01 20:25:00 -04:00
Michael Chisholm ee57596d6a Implemented clenk's suggested changes in multi-version filesystem
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
  was already there)
- Use dict-style instead of attribute-style access to get stix
  object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
  to ensure that different times always result in different
  filenames.

Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
2018-11-01 20:25:00 -04:00
Michael Chisholm 51668a9a04 Removed the old FileSystemSource.query method. I'd renamed it
"query2" and forgot about it and left it there...
2018-11-01 20:25:00 -04:00
Michael Chisholm 9486b46f77 Add multi-version support to the filesystem datastore.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
2018-11-01 20:25:00 -04:00
Chris Lenk 3b297c17b5 Use consistent errors for observable extensions
Whether or not the Observable type is in the EXT_MAP already, using a
custom extension without also using allow_custom=True should result in
the same behavior/error message.
2018-11-01 17:23:55 -04:00
Emmanuelle Vargas-Gonzalez 8d24015186 Update Memory datastore to allow for mapping objects 2018-11-01 10:54:58 -04:00
Emmanuelle Vargas-Gonzalez 5abe518b8a Bump version: 1.0.3 → 1.1.0 2018-11-01 09:55:37 -04:00
Emmanuelle Vargas-Gonzalez 493bd65ead Update README and refactor code to make 2.0 default. Update some tests 2018-11-01 09:21:02 -04:00
Emmanuelle Vargas-Gonzalez 5e5d10e7aa Finish alignment of 2.1 components 2018-11-01 08:17:34 -04:00
Emmanuelle Vargas-Gonzalez eff5369670 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-11-01 07:57:09 -04:00
Emmanuelle Vargas-Gonzalez f85f4e566b Bump version: 1.0.2 → 1.0.3 2018-10-31 13:46:47 -04:00
Chris Lenk 3542604b02
Merge pull request #209 from chisholm/taxii_api_updates
Fixed a test fixture to call the cti-taxii-client Collection constructor correctly
2018-10-31 12:04:34 -04:00
Chris Lenk db300d1f21 Fix `created` millisecond precision in TLPs
A marking definition's `created` property doesn't require millisecond
preprecision, but for TLP markings the TLP instances provided in the
spec must be used and they all use millisecond precision.
2018-10-25 13:58:21 -04:00
Chris Lenk 6613b55a43 Update MemoryStore documentation
...now that MemorySource normalizes its data to python-stix2 objects.
2018-10-23 14:29:56 -04:00
Michael Chisholm cbe8d22d0a Added support to multi-version memory stores for markings. Also
added some more unit tests which test storing/retrieving markings
from the stores.
2018-10-17 20:54:53 -04:00
Michael Chisholm 2d89cfb0cf Remove outdated TODO's from the memory datastore module-level
docstring.
2018-10-17 15:49:58 -04:00
Emmanuelle Vargas-Gonzalez d614343910 Rename tests with duplicate name. 2018-10-17 07:56:10 -04:00
Emmanuelle Vargas-Gonzalez 352749edb0 Add constrains to ObservedData and Sighting, tests updated. 2018-10-17 07:47:25 -04:00
Emmanuelle Vargas-Gonzalez f8a72b0937 Custom builder code updated for 3.7 support.
Updated properties to support more constrains.
Make all regexes literal strings.
Update tests to align to new constrains.
Workbench problem. _check_object_constraints() uses instance class to
perform proper class resolution calls.
2018-10-17 07:34:15 -04:00
Emmanuelle Vargas-Gonzalez b2ef77b322 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-10-17 07:30:23 -04:00
Michael Chisholm 864ba05b71 Fix import order to satisfy isort-check 2018-10-15 19:23:28 -04:00
Michael Chisholm d9f6a213c1 Fixed Memory source/sink/store so that it supports multiple versions
of objects.  Fixed several bugs too.
2018-10-15 18:02:49 -04:00
Michael Chisholm 5a0e102959 Modify docstring for apply_common_filters() to change what it
says are the required types for its parameters.  It gave
specific types (list and set), when really it worked with more
than just that (iterables).  And I certainly didn't only call
it with just lists and sets.
2018-10-15 18:02:48 -04:00
Emmanuelle Vargas-Gonzalez dec75082df Add new constrains parameters to IntegerProperty and FloatProperty
New constraints on timestamps, integer and floats for many objects
2018-10-15 15:02:59 -04:00
Emmanuelle Vargas-Gonzalez acd86c80dd Update tests to new object constraints 2018-10-15 14:48:52 -04:00
Maciej Urbanski e80fb4f59a
python 3.7 support 2018-10-03 18:00:38 +02:00
Michael Chisholm 541c682bf6 Fixed a test fixture to call the cti-taxii-client Collection
constructor correctly.  It had been recently changed to address
issue #39 in that library.
2018-09-07 17:47:24 -04:00
mbastian1135 5648d2bc45 added to pattern guide; reverting pattern str conversion (as was already proper) 2018-08-30 11:57:20 -04:00
mbastian1135 cd81d97beb added 'to_pattern()' utility for pattern expressions; more information on patterning guide 2018-08-27 16:24:23 -04:00
mbastian1135 10149b9345 added 'to_pattern()' utility for pattern expressions; more information on patterning guide 2018-08-27 16:21:14 -04:00
mbastian1135 a9de13107f pattern guide 2018-08-24 10:40:37 -04:00
Emmanuelle Vargas-Gonzalez 211b8d8cee Add core tests and minor change to parse_observable() 2018-07-25 20:53:53 -04:00
Emmanuelle Vargas-Gonzalez 5e71f9225b Add regex approach to load mappings 2018-07-25 14:06:18 -04:00
Emmanuelle Vargas-Gonzalez ad76e7155c MALWARE RESTORE POINT - Reverted changes to Malware based on STIX 2.1 CSD01
Use this commit to restore Malware changes.
2018-07-25 13:34:56 -04:00
Emmanuelle Vargas-Gonzalez 303159a818 pre-commit hooks changes 2018-07-25 13:32:22 -04:00
Emmanuelle Vargas-Gonzalez 21c84acc8f Add missing properties to Relationship object and update tests 2018-07-25 12:44:46 -04:00
Emmanuelle Vargas-Gonzalez af2a5605ce Add constraints to Location object 2018-07-25 12:43:57 -04:00
Emmanuelle Vargas-Gonzalez 40d656c94c Minor changes to tests. 2018-07-25 12:43:08 -04:00
Emmanuelle Vargas-Gonzalez 51a499cb33 Formatting changes made by the new pre-commit hook 'add trailing commas'
closes #189
2018-07-13 11:10:05 -04:00
Emmanuelle Vargas-Gonzalez 965d7fa788 Update v20 and v21 tests
In v20, only minor stuff that was addressing wrong spec. In v21, align tests with new/changed properties in the specs
2018-07-12 14:33:00 -04:00
Emmanuelle Vargas-Gonzalez bdfc221cb0 Update v21 properties to latest spec changes 2018-07-12 14:31:14 -04:00
Emmanuelle Vargas-Gonzalez 281dbfb0f4 Align tests with news additions from 'master' branch. 2018-07-11 09:43:37 -04:00
Emmanuelle Vargas-Gonzalez a042970a1f Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-07-11 09:41:42 -04:00
Emmanuelle Vargas-Gonzalez ee260b7574 Sort import check 2018-07-11 08:38:06 -04:00
Emmanuelle Vargas-Gonzalez e513c8d638 Hide builder methods in 'custom.py' and update imports accordingly 2018-07-11 08:11:47 -04:00
Emmanuelle Vargas-Gonzalez 6bd797e258 Fix 'test_memory_store_object_with_custom_property_in_bundle'
Since Bundles objects are now added instead of the Bundle itself, it now works as intended.
2018-07-10 16:19:40 -04:00
Emmanuelle Vargas-Gonzalez 834ef2c847 Fix check to collections.Mapping 2018-07-10 16:13:29 -04:00
Emmanuelle Vargas-Gonzalez c91bcd43f6 Fix location for test_object_property 2018-07-10 16:11:07 -04:00
Emmanuelle Vargas-Gonzalez 48e0442439 Fix tests in 'test_properties.py' 2018-07-10 16:10:01 -04:00
Emmanuelle Vargas-Gonzalez d24cddb547 Temporarily skip failing tests in workbench for v20, but the approach needs to be fixed 2018-07-10 16:08:36 -04:00
Emmanuelle Vargas-Gonzalez 4583da3ef2 Sort imports 2018-07-10 16:05:30 -04:00
Emmanuelle Vargas-Gonzalez ce42c02cee Fix tests that use property objects to call in the right path 2018-07-10 15:56:22 -04:00
Emmanuelle Vargas-Gonzalez 8cf68054d4 Remove str() as a way to serialize objects. Add support for encodings and Bundle versions. 2018-07-10 15:51:20 -04:00
Emmanuelle Vargas-Gonzalez 012eba4e9b Add new Bundle support for add_objects() request. Add encoding support.
Removed the version from all methods since it is no longer necessary.
2018-07-10 15:49:36 -04:00
Emmanuelle Vargas-Gonzalez 9b8cb09b1a Remove 'version' from calls to parse since it is no longer necessary
Also, fixed adding STIX2 Bundles to MemorySource. Enhancements to 'save_to_file'. Fix docstrings and encoding support when writing to file. closes #202
2018-07-10 15:43:58 -04:00
Emmanuelle Vargas-Gonzalez b6fefc52d9 Fix call to collect STIX2 mappings, make parse_observable available 2018-07-10 15:31:22 -04:00
Emmanuelle Vargas-Gonzalez 8d378fcf81 Remove STIXRelationshipObject from 'sro.py' 2018-07-10 15:27:05 -04:00
Emmanuelle Vargas-Gonzalez 023603d86f Remove duplicate code from 'sdo.py', removed STIXDomainObject
Apply proper 'spec_version' constraints to v21 objects
2018-07-10 15:22:21 -04:00
Emmanuelle Vargas-Gonzalez 54268ae7dd Remove observables and extension mappings, custom code and apply property constrain in v21 2018-07-10 15:20:16 -04:00
Emmanuelle Vargas-Gonzalez 1177694739 Adding docstrings to 2.1 objects 2018-07-10 15:15:33 -04:00
Emmanuelle Vargas-Gonzalez b722fdc0ed Remove duplicate register methods and CustomMarking code 2018-07-10 15:07:08 -04:00
Emmanuelle Vargas-Gonzalez 5332d54383 Refactor Bundle
Removed redundant STIXObjectProperty, for 2.1 use validation specific to that version
2018-07-10 15:02:55 -04:00
Emmanuelle Vargas-Gonzalez 78d77254ae Add object mappings in the top of each version package 2018-07-10 14:59:43 -04:00
Emmanuelle Vargas-Gonzalez 03e19f197c Moved STIXDomainObject and STIXRelationshipObject here, the observable
parsing code, minor changes to the mappings collections and all _register methods
2018-07-10 14:56:31 -04:00
Emmanuelle Vargas-Gonzalez b76888c682 Add a new 'custom.py' module to store and consolidate all custom code 2018-07-10 14:54:17 -04:00
Emmanuelle Vargas-Gonzalez f669656a4d Removed 'ExtensionsProperty' from workbench. 2018-07-10 14:52:10 -04:00
Emmanuelle Vargas-Gonzalez 7da6f1ed88 Add a package-wide 'properties.py'
Consolidated STIXObjectProperty, ExtensionsProperty, ObservableProperty code. Also added a 'spec_version' to allow validation changes per spec.
2018-07-10 14:50:03 -04:00
Emmanuelle Vargas-Gonzalez fe64fb044f Removed per version 'properties.py' 2018-07-10 14:47:30 -04:00
Emmanuelle Vargas-Gonzalez 645a258c62 Fix file indentation 2018-07-10 14:46:46 -04:00
Emmanuelle Vargas-Gonzalez edd7148e3c It appears we did not support the case when the Bundle contains 'utf-8' 2018-07-09 15:26:57 -04:00
Emmanuelle Vargas-Gonzalez 70a1e9522b Add 'stix2_data' test directory for v20 tests 2018-07-09 15:22:08 -04:00
Emmanuelle Vargas-Gonzalez 0197f9fd17 Minor fixes to tests. Some datastore had strange parameters in calls.
Fix error values for CustomMarking and fix incorrect test data
2018-07-09 15:20:04 -04:00
Emmanuelle Vargas-Gonzalez 646d941032 Removed 'test_memory' file and moved all tests into 'test_datastore_memory'
The tests under 'test_memory' were moved into a new file called 'test_datastore_composite' to make clear what we are testing.
2018-07-09 15:15:29 -04:00
Emmanuelle Vargas-Gonzalez 546216c396 Remove unnecessary 'True' in Memory datastore tests. 2018-07-09 15:07:05 -04:00
Emmanuelle Vargas-Gonzalez 8aeac369f4 Returning double quotes here... went overboard. 2018-07-09 14:59:19 -04:00
Emmanuelle Vargas-Gonzalez 21d5451d1c Small changes to tests 2018-07-06 14:11:59 -04:00
Emmanuelle Vargas-Gonzalez 52c1850655 Small addition to patterns.py 2018-07-06 14:08:49 -04:00
Greg Back 68aac583c9
Merge pull request #204 from oasis-open/refactor-properties
Refactor Properties
2018-07-05 17:16:25 -05:00
Greg Back 49c1931b4b
Merge pull request #203 from oasis-open/200-filter-contains
Add "contains" filter
2018-07-05 15:47:09 -05:00
Emmanuelle Vargas-Gonzalez 5be1636b10 Update v20 tests to ensure right methods and classes are used 2018-07-05 15:23:25 -04:00
Emmanuelle Vargas-Gonzalez 2c5ddc14af Update v21 tests for some missing methods, ensure we are calling and
using the right classes.
2018-07-05 15:21:09 -04:00
Greg Back 7c8113a4d9
Merge pull request #201 from oasis-open/188-uuidv4
Ensure all UUIDs are V4
2018-07-05 11:46:44 -05:00
Greg Back 12a22c0939 Remove unnecessary code related to defining properties. 2018-07-05 11:39:44 -05:00
Greg Back 25409b2099 GH-188: Update error message for invalid IDs. 2018-07-05 11:25:48 -05:00
Emmanuelle Vargas-Gonzalez bfa86bf87e Format objects in observed_data 2018-07-03 10:32:04 -04:00
Emmanuelle Vargas-Gonzalez 04680d8a3d First pass at making sure everything uses v21 classes and representations 2018-07-03 09:40:51 -04:00
Emmanuelle Vargas-Gonzalez 3100fa1fb8 Move v20 tests to their own package 2018-07-03 07:02:57 -04:00
Emmanuelle Vargas-Gonzalez da5b16dc2f Create v21 test package with new spec changes 2018-07-03 07:00:18 -04:00
Chris Lenk f47809eaaf Add "contains" filter
This allows filtering on, for example, observed-data objects that
contain a specific cyber observable.
2018-07-02 18:28:27 -04:00
Emmanuelle Vargas-Gonzalez c2f5a40986 Create new test subpackages 2018-06-30 19:36:54 -04:00
Emmanuelle Vargas-Gonzalez 2e6bb74be8 Add spec version 2.1 to object missing the property. 2018-06-29 18:48:41 -04:00
Emmanuelle Vargas-Gonzalez 7fd379d0b5 Minor style changes.
Removed OrderedDict and update()... Also a lot of single quoting except for errors
2018-06-29 18:38:04 -04:00
Greg Back a2e2407025 GH-188: Fixes for old Python syntax. 2018-06-27 12:49:00 -05:00
Greg Back d2f703b9d0 Reuse reference regex to match IDs as well.
Note that this means that UUIDs will no longer be converted, even when
there's an unambiguous mapping.
2018-06-27 12:27:44 -05:00
Greg Back e749218f9b GH-188: Test that references are always valid UUID v4. 2018-06-27 12:02:13 -05:00
Greg Back e5903f1832 Sort constants. 2018-06-27 11:36:17 -05:00
Greg Back 845625c174 GH-188: WIP: Converting all IDs to be valid UUID v4. 2018-06-27 11:35:55 -05:00
Emmanuelle Vargas-Gonzalez 59fdd3082e Update tests. 2018-06-26 12:29:20 -04:00
Emmanuelle Vargas-Gonzalez 9baaad6e08 Sort imports. 2018-06-26 12:23:53 -04:00
Emmanuelle Vargas-Gonzalez bee537137b
Update base.py
Fix STIXJSONIncludeOptionalDefaultsEncoder call to super()
2018-06-26 11:10:36 -04:00
Trey Darley 5cbe886cdb split properties out by spec version 2018-06-26 09:32:24 +00:00
Trey Darley d44c2abd0f 2.1 spec (somewhat inexplicably) limits dict keys to 250 chars 2018-06-26 09:23:52 +00:00
Trey Darley cc58a3a4f4 2.1 removes 3 char limit on dict keys 2018-06-26 09:22:57 +00:00
Trey Darley 6b1da856dd split properties out by spec version 2018-06-26 09:22:04 +00:00
Emmanuelle Vargas-Gonzalez b852b91652 Reformat methods documentation 2018-06-25 10:06:07 -04:00
Emmanuelle Vargas-Gonzalez 0ddb7b3807 Update observables.RasterImageExt 'image_weight' property to 'image_width' 2018-06-25 08:55:12 -04:00
Emmanuelle Vargas-Gonzalez abd172eb3f Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-06-25 08:25:57 -04:00
Greg Back b2c6acfbf6 GH-188: Restrict valid UUID values. 2018-06-22 14:40:30 -05:00
Greg Back fdf5937a71 GH-194: Remove unnecessary allow_custom kwarg. 2018-06-21 12:15:28 -05:00
Greg Back 7baeb153fa Correct "image_weight" to "image_width".
Fix #194.
2018-06-20 15:24:27 -05:00
Greg Back 53a1a0329a Merge branch 'master' into stix2.1 2018-06-15 10:04:00 -05:00
Michael Chisholm f09960d7ff Revamp code related to "pretty" JSON serialization, fix lurking
crash bugs.
2018-06-14 21:29:50 -04:00
Michael Chisholm 240a75861e Updated stix2.parse()-related docstrings. Its description of how
the "version" parameter was used, was out of date.
2018-06-14 15:56:02 -04:00
Michael Chisholm 486c588306 Fix silly isort check errors.. 2018-06-14 15:56:02 -04:00
Michael Chisholm 3101584b3d Fix test_bundle to compare against stix2.1 relationships. The
fixture those particular tests use creates 2.1 relationships.
2018-06-14 15:56:02 -04:00
Michael Chisholm 0c3f826c24 First cut at splitting the Bundle implementation into v20 and
v21 variants.  Also fixed up unit tests and got them passing
again.
2018-06-14 15:56:02 -04:00
Michael Chisholm ef8d45723f Update many unit tests to work with the malware2.1 API changes
I made.  The bundle tests and Bundle itself have not been fixed
yet in this commit.
2018-06-14 15:56:01 -04:00
Michael Chisholm f211649529 Made some minimal changes to support the STIX 2.1 Malware SDO,
and the maec2stix tool.
2018-06-14 15:56:01 -04:00
Greg Back 2e0dfc6592 Merge remote-tracking branch 'origin/master' into stix2.1 2018-06-14 12:42:06 -05:00
Greg Back 858a9752df
Merge pull request #185 from LiamSennitt/master
fix custom STIX objects with nested dictionary error #184
2018-06-12 14:40:55 -05:00
Emmanuelle Vargas-Gonzalez c12336b55a Revert changes to iterables. 2018-06-12 15:03:25 -04:00
Emmanuelle Vargas-Gonzalez 78a480aa08 Change str.isdecimal to isdigit. On lists keep iterating if unless index is found 2018-06-12 14:45:15 -04:00
Emmanuelle Vargas-Gonzalez fcffb165ad Add tests for `find_property_index` and `iterate_over_values` 2018-06-12 14:38:35 -04:00
Emmanuelle Vargas-Gonzalez 94d6610ca1 Merge branch 'master' of github.com:LiamSennitt/cti-python-stix2 2018-06-12 12:58:01 -04:00
Emmanuelle Vargas-Gonzalez 9be819ea6a Minor tweaks to return immediately after recursive call. Ensure dicts are matched alphabetically 2018-06-12 12:57:25 -04:00
Liam Sennitt bdec14937d fix flake8 errors in new test 2018-06-12 17:45:48 +01:00
Liam Sennitt 02894b5be6 add test for nested dictionary case 2018-06-12 17:32:46 +01:00
Greg Back 75e478312a Additional changes to match updates in v20 code. 2018-06-11 13:37:45 -05:00
Emmanuelle Vargas-Gonzalez 91cae0b5b7 Split `find_property_index` and support multiple levels of nesting. 2018-06-08 15:43:21 -04:00
Emmanuelle Vargas-Gonzalez 60da259805 Add an example on how to call the serialize() method. 2018-06-08 15:42:59 -04:00
Michael Chisholm 5a9f627669 Pickle-proof stix objects 2018-06-06 15:30:45 -04:00
Liam Sennitt dbe9c3f087 fix custom STIX objects with nested dictionary error #184 2018-06-01 13:49:32 +01:00
Greg Back 3e159abd4d WIP: Merge branch 'master' into stix2.1 2018-05-23 10:43:52 -05:00
Greg Back d67f2da0ea
Merge pull request #181 from oasis-open/http_error
Http error
2018-05-22 12:53:29 -05:00
= 3fb1840289 safer error class 2018-05-22 10:03:06 -04:00
Chris Lenk 636630a5aa Bump version: 1.0.1 → 1.0.2 2018-05-18 13:06:32 -04:00
= e484b7c25f formatting 2018-05-18 11:45:40 -04:00
= 6004ec597c solution to get 404 mocking problem 2018-05-18 11:19:05 -04:00
Chris Lenk ddc09f70c7 Set allow_custom if using a custom_properties dict
Fixes #179.
2018-05-17 09:04:44 -04:00
Chris Lenk cf972479ed Pass allow_custom to object dicts in a Bundle 2018-05-16 15:37:30 -04:00
= 8fc421e7d4 think the mock client was erroneous for get_object(), couldnt get object by id, just returned random object 2018-05-16 15:17:34 -04:00
= a3313bc08f creating native DataSourceError exception class to wrap taxii client and server errors 2018-05-16 13:23:50 -04:00
Chris Lenk 69c31ca3fc Pass allow_custom to ExtensionsProperty
Also fix bug caused by _properties being a class variable rather than an
instance variable. If you created an object with allow_custom,
allow_custom would be set for all future instances.
2018-05-16 12:14:33 -04:00
= 2b4c5bf264 handling and modifying exception messages in a manner acceptable by all python versions 2018-05-15 17:42:19 -04:00
= 0d3f80f2fe removing taxii 503 error test as not reproducible in Travis environment 2018-05-15 15:41:46 -04:00
= 2392912533 handle TAXII client/server errors according to decided policy 2018-05-15 11:28:34 -04:00
Chris Lenk 06e5a33639 Pass allow_custom when adding to ObservedData 2018-05-11 17:28:55 -04:00
Chris Lenk 102623b29a Bump version: 1.0.0 → 1.0.1 2018-04-27 16:51:47 -04:00