autogenerated docstrings: v20/v21 is automatically referenced as
appropriate, based on stix2.DEFAULT_VERSION. To avoid
duplication, I also moved _STIX_VID from test_workbench.py to
workbench.py; the former now imports it from the latter.
that turns them on. These have the potential to affect subsequent
tests. The side effects include automatically setting
property values, and automatically appending additional values
to list-valued properties.
it doesn't make sense to have a test per STIX version. The
workbench only uses the latest supported STIX version. In
order to make this work, the test suite was modified to
dynamically compute some settings like where to get demo data,
based on the value of stix2.DEFAULT_VERSION.
Switched stix2.DEFAULT_VERSION back to "2.0", since I figure it
should be sync'd up with the 'from .vxx import *' import
statement from the top level package.
additionally required:
- Removing the v21 workbench test suite and reinstating the v20
test suite
- Fixing up a few v20 unit tests to work with the workbench
monkeypatching.
- I didn't revert the analogous changes I'd previously made to
the v21 unit tests, because I think they make sense even when
the workbench monkeypatching isn't happening.
- Removed all plain python base classes (e.g. ValueError, TypeError)
- Renamed InvalidPropertyConfigurationError -> PropertyPresenceError,
since incorrect values could be considered a property config error, and
I really just wanted this class to apply to presence (co-)constraint
violations.
- Added ObjectConfigurationError as a superclass of InvalidValueError,
PropertyPresenceError, and any other exception that could be raised
during _STIXBase object init, which is when the spec compliance
checks happen. This class is intended to represent general spec
violations.
- Did some class reordering in exceptions.py, so all the
ObjectConfigurationError subclasses were together.
Changed how property "cleaning" errors were handled:
- Previous docs said they should all be ValueErrors, but that would require
extra exception check-and-replace complexity in the property
implementations, so that requirement is removed. Doc is changed to just
say that cleaning problems should cause exceptions to be raised.
_STIXBase._check_property() now handles most exception types, not just
ValueError.
- Decided to try chaining the original clean error to the InvalidValueError,
in case the extra diagnostics would be helpful in the future. This is
done via 'six' adapter function and only works on python3.
- A small amount of testing was removed, since it was looking at custom
exception properties which became unavailable once the exception was
replaced with InvalidValueError.
Did another pass through unit tests to fix breakage caused by the changed
exception class hierarchy.
Removed unnecessary observable extension handling code from
parse_observable(), since it was all duplicated in ExtensionsProperty.
The redundant code in parse_observable() had different exception behavior
than ExtensionsProperty, which makes the API inconsistent and unit tests
more complicated. (Problems in ExtensionsProperty get replaced with
InvalidValueError, but extensions problems handled directly in
parse_observable() don't get the same replacement, and so the exception
type is different.)
Redid the workbench monkeypatching. The old way was impossible to make
work, and had caused ugly ripple effect hackage in other parts of the
codebase. Now, it replaces the global object maps with factory functions
which behave the same way when called, as real classes. Had to fix up a
few unit tests to get them all passing with this monkeypatching in place.
Also remove all the xfail markings in the workbench test suite, since all
tests now pass.
Since workbench monkeypatching isn't currently affecting any unit tests,
tox.ini was simplified to remove the special-casing for running the
workbench tests.
Removed the v20 workbench test suite, since the workbench currently only
works with the latest stix object version.
partly inspired by PR #263. This resulted in some error message
format changes (an improvement, I think), which caused some
unit test breakage. Removed those asserts from the unit tests,
since tests shouldn't be testing human-targeted error messages.
- a length check should come before the access, so you can verify
whether the access will succeed.
- Also removed some tests which can't work, due to the filter
changes. In fact, a lot of these tests should probably be
removed or changed if we want to disallow running
apply_common_filters() on plain dicts. They will often
coincidentally still succeed though, so I left them in.
datetimes: I'd converted the object property instead of the
filter value! :-P
Also, I fixed filter validation: it was checking for exact types
of the filter values and disallowing subtypes. This library
includes a datetime subtype named STIXdatetime, and this type
should be usable as a filter value too. So we need to allow
subtypes.
Timestamp properties can now be checked against filter values
which are either strings or datetime objects, using datetime
semantics (previously, it reduced to a string compare).
If a stix object property is datetime-valued and the filter
value is a string, the string is parsed to a datetime object,
rather than the other way around.
Filtering in the filesystem store now parses JSON dicts to
_STIXBase objects before applying the filters.
Due to the parsing change, bad JSON content can produce a
different kind of error, so I had to change one of the tests.
datastore test suite. Add a new test corpus file, located so
as to test the backward compatibility functionality of
FileSystemSource. Add a test to the suite which ensures that
this new file is found.
objects are searched for as ID-named json files in the type
directories, in addition to timestamp-named files in ID
directories.
Made a bugfix: fixed improper exception re-raises
Made an efficiency improvement: don't stat() files in
_get_matching_dir_entries() if no st_mode_test callable is given.
test data corpus. Updated filesystem store tests accordingly:
- Remove comments from all_versions tests stating that multiple
versions are not supported. Improve the tests to ensure that
all versions are in fact retrieved.
- Update the get() test to assure that it gets only the latest
version, when there is more than one version.
- Update some count checks, since there are more objects now
- Fix some typos
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
was already there)
- Use dict-style instead of attribute-style access to get stix
object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
to ensure that different times always result in different
filenames.
Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
test data corpus. Updated filesystem store tests accordingly:
- Remove comments from all_versions tests stating that multiple
versions are not supported. Improve the tests to ensure that
all versions are in fact retrieved.
- Update the get() test to assure that it gets only the latest
version, when there is more than one version.
- Update some count checks, since there are more objects now
- Fix some typos
store:
- Use utils.get_type_from_id() instead of my own (I didn't know it
was already there)
- Use dict-style instead of attribute-style access to get stix
object properties
- Convert timezone-aware timestamps to UTC in _timestamp2filename()
to ensure that different times always result in different
filenames.
Also added a couple new tests for _timestamp2filename(), which
exercises the timezone conversion code.
Factored out the _is_marking() function from the memory datastore
module to utils so it can be reused, and changed both filesystem
and memory datastore modules to import and use it.
Whether or not the Observable type is in the EXT_MAP already, using a
custom extension without also using allow_custom=True should result in
the same behavior/error message.
A marking definition's `created` property doesn't require millisecond
preprecision, but for TLP markings the TLP instances provided in the
spec must be used and they all use millisecond precision.
Updated properties to support more constrains.
Make all regexes literal strings.
Update tests to align to new constrains.
Workbench problem. _check_object_constraints() uses instance class to
perform proper class resolution calls.
says are the required types for its parameters. It gave
specific types (list and set), when really it worked with more
than just that (iterables). And I certainly didn't only call
it with just lists and sets.
Michael Chisholm