Commit Graph

159 Commits (bd90e219adcd13b9ca562478549cdd6432b7c9ec)

Author SHA1 Message Date
Desai, Kartikey H 49077352d7 Updates and corrections for SCO WD 05 updates. Temp backup; testing and more fixes coming soon 2019-08-27 17:36:45 -04:00
Michael Chisholm c212c7c678 Fix handling of custom extensions: make sure when
allow_custom=True that you never get a half-cleaned property
value.
2019-08-26 17:10:54 -04:00
Desai, Kartikey H 5e9d6a6a14 Fix small indentation error 2019-08-21 08:49:33 -04:00
Desai, Kartikey H bf1b8b567d Updates to allow existing tests to pass 2019-08-21 02:00:41 -04:00
Desai, Kartikey H 46359ead69 Modify a few things 2019-08-19 13:35:17 -04:00
Desai, Kartikey H ec55463398 Update SCO stuff to WD 05 2019-08-19 09:39:13 -04:00
Michael Chisholm 5589480980 Improved the exception class hierarchy:
- Removed all plain python base classes (e.g. ValueError, TypeError)
- Renamed InvalidPropertyConfigurationError -> PropertyPresenceError,
  since incorrect values could be considered a property config error, and
  I really just wanted this class to apply to presence (co-)constraint
  violations.
- Added ObjectConfigurationError as a superclass of InvalidValueError,
  PropertyPresenceError, and any other exception that could be raised
  during _STIXBase object init, which is when the spec compliance
  checks happen.  This class is intended to represent general spec
  violations.
- Did some class reordering in exceptions.py, so all the
  ObjectConfigurationError subclasses were together.

Changed how property "cleaning" errors were handled:
- Previous docs said they should all be ValueErrors, but that would require
  extra exception check-and-replace complexity in the property
  implementations, so that requirement is removed.  Doc is changed to just
  say that cleaning problems should cause exceptions to be raised.
  _STIXBase._check_property() now handles most exception types, not just
  ValueError.
- Decided to try chaining the original clean error to the InvalidValueError,
  in case the extra diagnostics would be helpful in the future.  This is
  done via 'six' adapter function and only works on python3.
- A small amount of testing was removed, since it was looking at custom
  exception properties which became unavailable once the exception was
  replaced with InvalidValueError.

Did another pass through unit tests to fix breakage caused by the changed
exception class hierarchy.

Removed unnecessary observable extension handling code from
parse_observable(), since it was all duplicated in ExtensionsProperty.
The redundant code in parse_observable() had different exception behavior
than ExtensionsProperty, which makes the API inconsistent and unit tests
more complicated.  (Problems in ExtensionsProperty get replaced with
InvalidValueError, but extensions problems handled directly in
parse_observable() don't get the same replacement, and so the exception
type is different.)

Redid the workbench monkeypatching.  The old way was impossible to make
work, and had caused ugly ripple effect hackage in other parts of the
codebase.  Now, it replaces the global object maps with factory functions
which behave the same way when called, as real classes.  Had to fix up a
few unit tests to get them all passing with this monkeypatching in place.
Also remove all the xfail markings in the workbench test suite, since all
tests now pass.

Since workbench monkeypatching isn't currently affecting any unit tests,
tox.ini was simplified to remove the special-casing for running the
workbench tests.

Removed the v20 workbench test suite, since the workbench currently only
works with the latest stix object version.
2019-07-19 14:50:11 -04:00
Desai, Kartikey H f79b3c9876 Add functionality to _valid_refs to accept actual cyber observable objects instead of just strings with their types 2019-05-10 10:22:45 -04:00
Desai, Kartikey H e748923f19 Fixes #248 2019-04-17 10:08:34 -04:00
Emmanuelle Vargas-Gonzalez e1f7cc4028 change "Notes" for "Note" to keep visual effect in documentation 2018-11-30 09:39:05 -05:00
Emmanuelle Vargas-Gonzalez 71a2aa2611 update project documentation. 2018-11-28 16:51:00 -05:00
Emmanuelle Vargas-Gonzalez b2ef77b322 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2018-10-17 07:30:23 -04:00
Maciej Urbanski e80fb4f59a
python 3.7 support 2018-10-03 18:00:38 +02:00
Emmanuelle Vargas-Gonzalez 51a499cb33 Formatting changes made by the new pre-commit hook 'add trailing commas'
closes #189
2018-07-13 11:10:05 -04:00
Emmanuelle Vargas-Gonzalez 21d5451d1c Small changes to tests 2018-07-06 14:11:59 -04:00
Emmanuelle Vargas-Gonzalez 7fd379d0b5 Minor style changes.
Removed OrderedDict and update()... Also a lot of single quoting except for errors
2018-06-29 18:38:04 -04:00
Emmanuelle Vargas-Gonzalez 9baaad6e08 Sort imports. 2018-06-26 12:23:53 -04:00
Emmanuelle Vargas-Gonzalez bee537137b
Update base.py
Fix STIXJSONIncludeOptionalDefaultsEncoder call to super()
2018-06-26 11:10:36 -04:00
Michael Chisholm f09960d7ff Revamp code related to "pretty" JSON serialization, fix lurking
crash bugs.
2018-06-14 21:29:50 -04:00
Greg Back 858a9752df
Merge pull request #185 from LiamSennitt/master
fix custom STIX objects with nested dictionary error #184
2018-06-12 14:40:55 -05:00
Emmanuelle Vargas-Gonzalez 60da259805 Add an example on how to call the serialize() method. 2018-06-08 15:42:59 -04:00
Michael Chisholm 5a9f627669 Pickle-proof stix objects 2018-06-06 15:30:45 -04:00
Chris Lenk ddc09f70c7 Set allow_custom if using a custom_properties dict
Fixes #179.
2018-05-17 09:04:44 -04:00
Chris Lenk cf972479ed Pass allow_custom to object dicts in a Bundle 2018-05-16 15:37:30 -04:00
Chris Lenk 69c31ca3fc Pass allow_custom to ExtensionsProperty
Also fix bug caused by _properties being a class variable rather than an
instance variable. If you created an object with allow_custom,
allow_custom would be set for all future instances.
2018-05-16 12:14:33 -04:00
Chris Lenk 14dce03616 Provide default for revoked, sighting:summary.
This allows filter on un-revoked objects. Changes default JSONEncoder to
drop optional properties with default values in the spec if set to the
default value. They can be included by passing
include_optional_defaults=True to serialize().
2018-04-16 14:37:07 -04:00
Chris Lenk fc6a33b23e Disallow missing 'type' property with allow_custom
There was a bug where if you allowed custom content the library would parse an
object without the required 'type' property.
2018-04-13 11:18:56 -04:00
Chris Lenk 91376586d4 Simplify allowing custom observables/extensions 2018-04-12 16:33:08 -04:00
Chris Lenk 9ef5b395a8 Fix allowing custom observables and extensions 2018-04-12 14:20:24 -04:00
Emmanuelle Vargas-Gonzalez c98abb18f5 Fix docs 2018-03-16 14:26:41 -04:00
Chris Lenk 955cad8385 Clarify serialization in documentation 2018-03-13 16:01:42 -04:00
Chris Lenk 845934033e Avoid checking valid refs when deepcopying
Using `deepcopy` on an Observable object that referenced another (e.g.
domain name -> resolves to -> IPv4 address) would fail because no
`_valid_refs` were available. Assuming that the references have
already been checked and are valid when using `deepcopy`, we can use a
special value ('*') to skip the valid_refs check.

This will also allow creating new versions of ObservedData objects that
contain related objects, since `new_version` calls `deepcopy`.
2018-02-19 14:44:28 -05:00
Emmanuelle Vargas-Gonzalez 79475586d8 Revert object_properties() to #85 fix. Update tests accordingly 2017-11-03 14:17:36 -04:00
Emmanuelle Vargas-Gonzalez d6c14139f3 Extend object serialization options for _STIXBase 2017-11-03 08:02:32 -04:00
Emmanuelle Vargas-Gonzalez 8c2af813b2 Define CustomProperty. Make sure to update _properties dict when allow_custom=True 2017-11-02 21:56:18 -04:00
Emmanuelle Vargas-Gonzalez 6deaddc04e Make sure custom properties are returned by object. closes #85 2017-10-12 14:08:32 -04:00
Chris Lenk d7efd1f752 Set up Sphinx autodoc
This will automatically generate API documentation from docstrings.
2017-09-21 17:27:13 -04:00
Greg Back 124da846c3 Merge branch 'master' into markings 2017-08-31 20:36:59 +00:00
Greg Back b1054e780a Merge remote-tracking branch 'origin/master' into datastores 2017-08-31 19:52:48 +00:00
Chris Lenk 15287959a4 Modify versioning API to work on dictionaries
This includes new_version() and revoke().
2017-08-31 12:28:07 -04:00
clenk 0e658255a8 Move check to when custom observable is defined
Catch properties named "_ref" or "_refs" that aren't
ObjectReferenceProperty when the custom observable is defined, not when
it is instantiated.
2017-08-30 15:33:28 -04:00
Emmanuelle Vargas-Gonzalez b1ac24d46e Minor changes. 2017-08-29 15:08:26 -04:00
Emmanuelle Vargas-Gonzalez dd17e88ae0 Code refactor/clean-up, changed some exceptions. Update docstrings. 2017-08-24 12:47:14 -04:00
clenk 14b922ba92 Merge remote-tracking branch 'emmanvg/marking-support' into markings 2017-08-17 12:26:46 -04:00
Emmanuelle Vargas-Gonzalez 1b48ad9778 Changes to object serialization. 2017-08-15 13:40:47 -04:00
Emmanuelle Vargas-Gonzalez 5172f86a7b Change _STIXBase to output properties in spec order. closes #39 2017-08-11 15:10:44 -04:00
clenk e01ce132db Add timestamp precision for `created` and `modified`
Fix #24
2017-06-22 18:47:35 -04:00
clenk 8f1ae4e6d3 Add custom properties via 'allow_custom'
Custom properties can be specified by passing them to a STIX object
constructor in the 'custom_properties' argument, or with the
'allow_custom' argument set to True, which will add any unrecognized
keyword arguments as properties on the object. The 'allow_custom'
argument can also be used with the parse() and parse_observable()
functions.
An error is now raised when attempting to parse objects without a 'type'
property, such as external references, kill chain phases, and granular
markings. The object which contains them is what should be parsed, not
these objects themselves.
2017-06-12 13:04:20 -04:00
Emmanuelle Vargas-Gonzalez bf740b21eb Initial selector and marking_ref validation. 2017-06-12 08:06:37 -04:00
clenk 9036c7f7b8 Add custom properties 2017-06-09 12:30:19 -04:00
Emmanuelle Vargas-Gonzalez b9f63e8de6 Minor Code Changes. 2017-06-08 08:42:32 -04:00
clenk 860efcc230 Make object attribute access act like dictionary access, raising an
error for any property (including custom or optional) not set on the object.
2017-06-07 11:06:20 -04:00
Emmanuelle Vargas-Gonzalez a38b912d19 Change _check_property_dependency() to also check for values for the required properties. 2017-06-02 13:48:44 -04:00
Emmanuelle Vargas-Gonzalez e11b3adb3f Style change. 2017-06-02 10:22:31 -04:00
Emmanuelle Vargas-Gonzalez 51e28f64da Change __getattr__() to use __getitem__() instead. Other minor changes. 2017-06-02 10:10:50 -04:00
Emmanuelle Vargas-Gonzalez 0cf0f97143 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 2017-06-02 07:15:01 -04:00
clenk f321447ecc Make Travis happy with cosmetic fixes 2017-06-01 16:47:56 -04:00
Emmanuelle Vargas-Gonzalez 4d58fc3dbb Explicitly check for None, when intention is to set empty values or False. fixes #17 2017-06-01 15:25:03 -04:00
Emmanuelle Vargas-Gonzalez 5dc049d65a getattr() checks for attribute membership, returns the value if not None and raises AttributeError when is not present. fixes #19 2017-06-01 15:23:44 -04:00
Emmanuelle Vargas-Gonzalez 49f58ff513 Make Immutable error more descriptive. Also fixes #13 2017-06-01 12:43:06 -04:00
Greg Back 41f2ceb8e5 Change remaining 'fields' to 'properties' 2017-05-19 12:51:59 -05:00
clenk 9bc3242736 Refactor extension properties
to inherit from a base class so all extension classes will include the
check for at least one property
2017-05-18 14:04:28 -04:00
clenk 931de31a10 Merge branch 'parse-cyber-observables' of https://github.com/oasis-open/cti-python-stix2 into parse-cyber-observables 2017-05-18 11:08:12 -04:00
Chris Lenk b6e22bcbdf Merge pull request #12 from rpiazza/cyber-observables
Cyber observables
2017-05-18 11:02:10 -04:00
Richard Piazza f8242cffe4 correct misspelling on _check_object_constraints 2017-05-18 09:48:01 -04:00
Richard Piazza 8827149ef0 avoid {} set notation. It doesn't seem to be in 2.6 2017-05-17 15:51:07 -04:00
Richard Piazza f56ea8e951 added tests for process and file extensions
if no properties are passed to _check_at_least_one_property assume its all properties
when extensions are involved always check them in _check_at_least_one_property (caused by raising the exception on the enclosing type)
added _check_object_constaints to extensions, where appropriate
2017-05-17 15:33:28 -04:00
clenk 3e0e80141b For object reference properties, check the type of the object
referenced, not only that it is included in the local scope.
2017-05-17 15:21:02 -04:00
clenk 9761c37f20 Replace 'field' with 'property' to be consistent
with the specification
2017-05-16 12:27:30 -04:00
Richard Piazza a95e91005c too Chris' suggestions to improve interproperty constraint methods
added interproperty constraint check to ExternalReferences
2017-05-12 13:18:02 -04:00
Rich Piazza ae5fb51564 Merge branch 'parse-cyber-observables' into cyber-observables 2017-05-11 15:29:15 -04:00
Richard Piazza 94f4f48329 Changes so File object creation doesn't violate on of the MUSTs
Added three new exceptions: DependentPropertiestError, AtLeastOnePropertyError, MutuallyExclusivePropertiesError
Added tests for NetworkTraffic, Process, URL, WindowsRegistryKey and X509Certificate
Added error tests for EmailMessage, NetworkTraffic, Artifact,
Added interproperty checker methods to the base class:  _check_mutually_exclusive_properties, _check_at_least_one_property and _check_properties_dependency
Added interproperty checkers to Artifact, EmailMIMEComponent, EmailMessage, NetworkTraffic
Made NetworkTraffic.protocols required
Added X509V3ExtenstionsType class
Use EmbeddedObjectProperty for X509Certificate.x509_v3_extensions
2017-05-11 15:22:46 -04:00
clenk b18259af77 Merge branch 'parse-cyber-observables' of https://github.com/oasis-open/cti-python-stix2 into parse-cyber-observables 2017-05-10 09:20:09 -04:00
clenk 67935d08fe Fix isort settings, rename Observable to clarify
it's an internal class
2017-05-09 18:03:46 -04:00
Rich Piazza 7c1fd1e7f0 Merge branch 'parse-cyber-observables' into cyber-observables 2017-05-09 15:36:59 -04:00
Richard Piazza c3477b83bf encryption_algorithm was misspelled
uncomment test_file_example_encryption_error
added _check_object_constrains and properties_populated to base class
added ObjectConstraintError
added _check_object_constrains for File
2017-05-09 15:28:32 -04:00
clenk bdd18be6c3 Switch to isort for checking import order
because it has a pre-commit hook
2017-05-09 15:10:53 -04:00
clenk f4c813d84b Merge branch 'parse-cyber-observables' 2017-05-09 12:39:17 -04:00
clenk 555c81d30f Add EmailMessage and EmbeddedObjectProperty (for embedded object types
like EmailMIMEComponent)
2017-05-09 11:03:19 -04:00
Richard Piazza 125f57e297 added basic cyber observables
added some test cases
in Observable constructor permit no _valid_refs in kwargs
in Observable._check_property ensure that the prop_name is in the kwargs
2017-05-08 21:03:15 -04:00
clenk d26662776c Merge branch 'master' into parse-cyber-observables 2017-05-08 11:14:54 -04:00
clenk 04e3a72a7d Add EmailAddress and ObjectReferenceProperty 2017-05-05 12:32:02 -04:00
Richard Piazza 200bb8556f added unsetting capability
cleaned up MissingFieldsError tests
error when new modified property is earlier than current modified property
2017-05-04 16:34:08 -04:00
clenk c63ba8e447 Add ObservableProperty, DictionaryProperty, HashesProperty,
BinaryProperty, and HexProperty
2017-05-03 14:10:10 -04:00
Richard Piazza 5b8585b392 added versioning test for embedded_object
replaced VersioningError with RevokeError and UnmodifiablePropertyError
added __deepcopy__ to base class to handle embedded_objects
2017-05-03 12:14:09 -04:00
Richard Piazza cda2b47691 import style errors 2 2017-05-02 14:29:59 -04:00
Richard Piazza 411c087fc1 import style errors 2017-05-02 14:25:01 -04:00
Richard Piazza 4efe5357b1 Added versioning api, with tests 2017-05-02 14:06:42 -04:00
clenk 76acd8c0c2 Merge branch 'master' into parsing 2017-04-19 15:22:36 -04:00
Greg Back 6bf3584616 Create custom exception class for modifying an immutable object. 2017-04-18 15:06:41 -05:00
Greg Back 91cecb7b0c Add exception for extra/invalid custom properties. 2017-04-18 14:56:16 -05:00
Greg Back 32ff00559e Rename exception class. 2017-04-18 14:42:59 -05:00
Greg Back a7805c4ac0 Add Exception for missing values. 2017-04-18 14:41:18 -05:00
Greg Back 2aa1f5cedd Add exception for invalid Property values. 2017-04-18 14:19:16 -05:00
clenk 635a3ec389 Combine clean() and validate() 2017-04-17 15:13:11 -04:00
clenk bc8bdccece Merge branch 'master' into parsing 2017-04-11 15:05:22 -04:00
clenk bab8d187c9 Rework kill chain phases 2017-04-07 17:34:06 -04:00
Richard Piazza 3c17c9259c Add Sighting object and data markings
- Update ReferenceProperty to allow specifying a particular object type
- Update ListProperty and add SelectorProperty
- Add description to Relationship
2017-04-07 16:07:17 -05:00
Greg Back a2f5981dfb Improve test coverage 2017-03-22 08:26:13 -05:00
Greg Back a0600b5ba4 Fix or ignore Flake8 warnings. 2017-03-22 08:05:59 -05:00