Previously the requirement that properties ending in `_ref(s)` were
instances of an appropriate type capable of containing a STIX
`identifier` (the current interpretation of section 3.1 of the STIX 2.1
spec) was only enforced for custom observables. This change refactors
the property checks for custom objects to enforce this and the STIX 2.1
property name requirement (also from section 3.1) in a common helper,
therefore extending the enforcement of both requirements to all custom
object types created by downstream projects.
There is a special case carved out for STIX 2.0 observables which are
required to contain "object references" rather than identifiers. This
special logic is encoded in the reference property validation helper and
enabled by a kwarg which is passed exclusively by the custom observable
registration helper.
and process properties in that order. This establishes iteration
order on object properties, making the object_properties() method
unnecessary. So the latter method has been deleted. All uses
of that method have been removed.
Removed unnecessary deepcopy() in STIXJSONEncoder, to improve
efficiency. This uncovered a bug which had been affecting
STIXdatetime instances. Not deepcopying doesn't trip the bug,
which can change serialization format. This caused a unit
test to fail, which was checking serialization format. I fixed
the unit test.
Fixed a bug in _STIXBase.__repr__ which caused it to omit all
properties with falsey values. This caused several unit tests
to break, since they were written against the old buggy repr
format. Notably, 'revoked=False' was never included in reprs
before, but it is now.
fixes.
- Fixed bugged logic in _STIXBase._check_at_least_one_property(),
and revamped the code to be simpler and clearer.
- Changed custom extension registration to auto-create an
"extension_type" property based on the attribute of that
name on the custom class, if present.
- The custom extension registration change above uncovered
what seemed like a bug in a unit test: a custom extension
was registered, but it was not given an extension type. The
test used the extension as extension_type="property-extension";
this now causes a standard error about an extra property. I
fixed the test to assign the custom extension the proper type.
- Changed an error message "Cannot determine extension type."
At that point in the code, we in fact have a registered
extension type and the class for it, so it didn't seem to make
any sense. I changed it to say that an extension of that type
couldn't be created from type of value given. Because this is
really about typing (in both the old and new code), I also
changed the exception class to TypeError.
- Changed customization behavior: unregistered
"extension-definition--" style extensions are no longer
considered custom. Their mere presence can no longer be
considered a customization, since it doesn't fit the
criteria, and in fact the extension mechanism is supposed to
supplant the old customization system anyway. Unregistered
extensions of other types are still considered custom.
- Fixed up unit tests according to the exception message
change.
- Added a unit test for unregistered "extension-definition--"
style extensions.
The "loophole" occurs when an object contains an unregistered top-level
extension. Since it's unregistered we can't tell if any custom
properties on the object were defined in that extension so we assume
that they are.
constraints (i.e. both generic type categories and specific
types). Also:
- more expansion/refinement of reference property unit tests
- bugfix: SROs are in OBJ_MAP too, it's not just SDOs! Oops...
- pre-commit stylistic fixes
assert prop.clean(...)
doesn't test well enough since clean() methods on this branch
produce 2-tuples, and you should test what's in the tuple, not
just that it returned something non-empty. So I fixed it in
several places to test the tuple contents.
work when a whitelist of generic category types is used.
Disallow hybrid constraints (both generic and specific at the
same time). Add more unit tests.
thought it should, but forgot to fix Report to use
ReferenceProperty in the way I thought it should! Oops.
Added some tests to ensure Report is working property with
custom ID types in object_refs.
and dicts, and add better raised exception types if versioning
couldn't be done. I changed workbench monkeypatching a bit, to
copy some class attributes over to the workbench wrapper
class-like callables, since some code expected those attributes
to be there (e.g. the versioning code).
dict/mapping values: do a simple verification of the value's
STIX version, not just its type. Added a lot more unit tests to
test behavior on dicts. To make the implementation work, I had
to move the detect_spec_version() function out of the parsing
module and into utils. So that required small changes at all
its previous call sites.
level with STIX versions in the same format as is used everywhere
else in the API: "X.Y", as opposed to the "vXY" format used by
the version-specific python packages. This eliminates all of
the awkward conversion from public API format to "vXX" format.
Also a little bit of code rearranging in the registration module
to ensure that some STIX 2.1-specific checks are done whether
version 2.1 is given explicitly or is defaulted to.
In the same module I also added a missing import of
stix2.properties, since my IDE was claiming it could not find a
function from that module.
- stix2.registry, which contains the class mapping structure
and code for scanning stix2 modules for its initial population
- stix2.registration, which contains code used to register custom
STIX types with the registry
- stix2.parsing, which contains code for creating instances of
registered stix2 classes from raw dicts.
This is intended to reduce circular import problems, by giving
dependent code the ability to import a module which has exactly
the functionality it needs, without pulling a lot of other stuff
it doesn't need. Fewer imports means less chance of an import
cycle.
* new packages for graph and object-based semantic equivalence
* new method graphically_equivalent for Environment, move equivalence methods out
* object equivalence function, methods used for object-based moved here.
* new graph_equivalence methods
* add notes
* add support for versioning checks (default disabled)
* new tests to cover graph equivalence and new methods
* added more imports to environment.py to prevent breaking changes
* variable changes, new fields for checks, reset depth check per call
* flexibility when object is not available on graph.
* refactor debug logging message
* new file stix2.equivalence.graph_equivalence.rst and stix2.equivalence.object_equivalence.rst for docs
* API documentation for new modules
* additional text required to build docs
* add more test methods for list_semantic_check an graphically_equivalent/versioning
* add logging debug messages, code clean-up
* include individual scoring on results dict, fix issue on list_semantic_check not keeping highest score
* include results as summary in prop_scores, minor tweaks
* Update __init__.py
doctrings update
* apply feedback from pull request
- rename semantic_check to reference_check
- rename modules to graph and object respectively to eliminate redundancy
- remove created_by_ref and object_marking_refs from graph WEIGHTS and rebalance
* update docs/ entries
* add more checks, make max score based on actual objects checked instead of the full list, only create entry when type is present in WEIGHTS dictionary
update tests to reflect changes
* rename package patterns -> pattern
* documentation, moving weights around
* more documentation moving
* rename WEIGHTS variable for graph_equivalence
maybe-sybr