Commit Graph

1352 Commits (2cda97cf5e51403c43a96ceb9073afab7706638a)

Author SHA1 Message Date
Michael Chisholm 2cda97cf5e Changed STIX object initialization to formulate a property order
and process properties in that order.  This establishes iteration
order on object properties, making the object_properties() method
unnecessary.  So the latter method has been deleted.  All uses
of that method have been removed.

Removed unnecessary deepcopy() in STIXJSONEncoder, to improve
efficiency.  This uncovered a bug which had been affecting
STIXdatetime instances.  Not deepcopying doesn't trip the bug,
which can change serialization format.  This caused a unit
test to fail, which was checking serialization format.  I fixed
the unit test.

Fixed a bug in _STIXBase.__repr__ which caused it to omit all
properties with falsey values.  This caused several unit tests
to break, since they were written against the old buggy repr
format.  Notably, 'revoked=False' was never included in reprs
before, but it is now.
2021-07-06 20:32:58 -04:00
Michael Chisholm 8bbf5fa461 Make extension instances work the same as other objects, with
respect to properties.  Before, properties were declared on
toplevel-property-extension extensions as if they were going
to be used in the normal way (as actual properties on instances
of the extension), but they were not used that way, and there
was some ugly hackage to make it work.  Despite the fact that
property instances were given during extension registration,
they were not used to typecheck, set defaults, etc on toplevel
property extension properties.

I changed how registration and object initialization works with
respect to properties associated with extensions.  Now,
extensions work the same as any other object and code is
cleaner.  Property instances associated with registered toplevel
extensions are used to enforce requirements like any other
object.

Added some unit tests specifically for property cleaning for
extensions.

Property order (for those contexts where it matters) is updated
to be spec-defined, toplevel extension, custom.
2021-07-06 14:27:40 -04:00
Michael Chisholm 93d2524d45 Remove excessive nested lists from CusomObservable decorator.
Remove iterable chaining from CustomObject decorator.  If all
values are guaranteed lists now, it no longer makes sense to use
it.  Simpler and clearer to use plain old list concatenation.
2021-07-02 21:10:52 -04:00
Michael Chisholm b9eba77008 Move the CustomExtension decorator from the v21.observables module
to v21.common.  Custom extensions are not specific to SCOs, so I
don't know why it was in that module.  Now, ExtensionDefinition
and CustomExtension are together in the same module, just like
MarkingDefinition and CustomMarking are together.  Made sense to
me.
2021-07-02 20:54:54 -04:00
Michael Chisholm 6fc39a70ea pre-commit stylistic fixes 2021-07-01 20:08:09 -04:00
Michael Chisholm ac8e46f491 Improve customization detection in the face of
toplevel-property-extension style extensions.  If an unregistered
extension of that type is encountered, all unrecognized toplevel
props will now be considered extension properties (not custom).
It will no longer turn on the allow_custom flag, which would
allow customizations everywhere.

Also, if all extensions of the aforementioned type are registered,
their properties are now used to properly distinguish between
extension and custom properties.  There need not be any ambiguity
in that case.
2021-06-30 20:20:29 -04:00
Michael Chisholm d87718c15c Bug fixes, hackage removal, and some pre-commit stylistic
fixes.

- Fixed bugged logic in _STIXBase._check_at_least_one_property(),
and revamped the code to be simpler and clearer.

- Changed custom extension registration to auto-create an
"extension_type" property based on the attribute of that
name on the custom class, if present.

- The custom extension registration change above uncovered
what seemed like a bug in a unit test: a custom extension
was registered, but it was not given an extension type.  The
test used the extension as extension_type="property-extension";
this now causes a standard error about an extra property.  I
fixed the test to assign the custom extension the proper type.
2021-06-30 17:50:00 -04:00
Michael Chisholm c7b4840232 Code style and behavioral improvements in ExtensionsProperty:
- Changed an error message "Cannot determine extension type."
At that point in the code, we in fact have a registered
extension type and the class for it, so it didn't seem to make
any sense.  I changed it to say that an extension of that type
couldn't be created from type of value given.  Because this is
really about typing (in both the old and new code), I also
changed the exception class to TypeError.

- Changed customization behavior: unregistered
"extension-definition--" style extensions are no longer
considered custom.  Their mere presence can no longer be
considered a customization, since it doesn't fit the
criteria, and in fact the extension mechanism is supposed to
supplant the old customization system anyway.  Unregistered
extensions of other types are still considered custom.

- Fixed up unit tests according to the exception message
change.

- Added a unit test for unregistered "extension-definition--"
style extensions.
2021-06-25 22:20:02 -04:00
Michael Chisholm 70718063d3 Remove relics of the old way customization was handled and
enforced, from _STIXBase20 and _STIXBase21.  That awkwardness
is no longer necessary.
2021-06-25 22:10:31 -04:00
Michael Chisholm 47b864cb4d Fix v21 ExtensionDefinition class to use the enum constant from
the vocab module for extension-type-enum, instead of replicating
that list.
2021-06-25 22:06:52 -04:00
Michael Chisholm e17f3e6c1c Add constants to the v21 vocab module for extension-type-enum 2021-06-25 22:04:59 -04:00
Chris Lenk 94859a0daf Fix tests, add a loophole to allow custom content
The "loophole" occurs when an object contains an unregistered top-level
extension. Since it's unregistered we can't tell if any custom
properties on the object were defined in that extension so we assume
that they are.
2021-06-14 17:34:06 -04:00
Chris Lenk 2528077eb0 Merge branch 'master' into dev-extensions-proposal 2021-06-13 01:48:33 -04:00
Chris Lenk 5cce864304 Test invalid_types ReferenceProperty with custom 2021-06-11 14:00:46 -04:00
Chris Lenk 7209346f0b Improve error message for invalid reference types 2021-06-11 13:59:48 -04:00
Michael Chisholm 2b064872a4 Add variables for each individual enum value in the vocab modules. 2021-04-15 22:06:57 -04:00
Emmanuelle Vargas-Gonzalez 5d83d44d0e adding documentation on specific lines, typos, and a bit of extra code 2021-04-15 12:28:58 -04:00
Chris Lenk a002c1736d Update guide docs, remove outdated comments 2021-04-14 16:39:56 -04:00
Michael Chisholm 4dc8e49d3c pre-commit stylistic fix 2021-04-14 12:58:09 -04:00
Michael Chisholm f301bc144f Disable custom vocab enforcement. Custom values for open-vocab
properties are now always accepted and not considered
customizations (has_custom will be False).  Just commented out
the enforcement code and xfail'd the unit tests, which makes it
easy to reinstate these behaviors later.  It was decided this
is too likely to break user code, and we don't currently have
a way to disallow customizations in other places while still
allowing custom open vocab values.  Safest to always allow
custom open vocab values, for now.
2021-04-14 12:48:32 -04:00
Michael Chisholm 33128908c7 Another pre-commit stylistic fix. Necessary since apparently
the git action uses a more recent version of pre-commit than
travis did, and the newer pre-commit requires different things
than the old one did.
2021-03-31 21:31:59 -04:00
Michael Chisholm 82148aa1ac Pre-commit stylistic fix 2021-03-31 21:27:06 -04:00
Michael Chisholm e735b537c7 Remove a workaround which was used in the 2.0 and 2.1 Report
objects, to structure object_refs type requirements as an empty
blacklist, instead of a whitelist.  I think it was originally
necessary due to the older implementation of ReferenceProperty
which was in place at the time.  With the implementation change
to invert whitelists with generics to blacklists when
allow_custom=True, a "full" whitelist is internally converted to
an empty blacklist anyway, so it winds up being the same thing.
But I think the full whitelist looks better in the code, so I
prefer that to the empty blacklist.
2021-03-31 20:56:31 -04:00
Michael Chisholm 0ddf0d2ca8 Clean up ReferenceProperty.clean(), to use new APIs which are
available after the rebase.  This simplifies the implementation.
Also made utils.to_enum() a module public function, since I
needed to use it outside that module.  Misc pre-commit stylistic
fixes.
2021-03-31 20:01:27 -04:00
Emmanuelle Vargas-Gonzalez 40023ec987 forgot to add this in the rebase... 2021-03-31 18:55:55 -04:00
Emmanuelle Vargas-Gonzalez bc053da3c4 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into dev-extensions-proposal 2021-03-31 18:53:02 -04:00
Emmanuelle Vargas-Gonzalez af046beeb5 add new-sdo or new-sro option in CustomObject decorator, add a test, and example in guide 2021-03-31 18:46:37 -04:00
Emmanuelle Vargas-Gonzalez 08d5781f54 add entry on extensions.ipynb, typos, and a couple of fixes for registry module and Marking decorator 2021-03-31 18:11:38 -04:00
Michael Chisholm c8b9eab53d Fixed some more silliness in properties.py... was it a rebase
glitch?  Something else I overlooked?  Not sure.
2021-03-31 16:49:14 -04:00
Michael Chisholm 6f60bed235 Make a minor post-rebase fix 2021-03-31 16:21:35 -04:00
Michael Chisholm c8c4e89415 Add back usage of spec_version where HashesProperty is used for
various STIX objects.  Also pre-commit stylistic fixes...
2021-03-31 16:20:33 -04:00
Michael Chisholm a8a65599bf Add back spec_version kwarg to HashesProperty, since it was used
to check key lengths.  Added some unit tests for hash keys.
Also added a library hash support test module I'd forgotten to
add before.
2021-03-31 16:20:33 -04:00
Michael Chisholm c590de8ea5 Add proper customization enforcement for open vocabs. This adds
a new OpenVocabProperty class.  It also requires a redesign of
HashesProperty and redoes general library support for hash
algorithms.
2021-03-31 16:20:28 -04:00
Michael Chisholm 62ee2b2b0a Pre-commit changes. Also remove a dupe test function which
slipped through due to a rather complex rebase... not too
surprising I missed it.
2021-03-31 15:32:54 -04:00
Michael Chisholm 414694d8fa Add support in ReferenceProperty for hybrid object type
constraints (i.e. both generic type categories and specific
types).  Also:
- more expansion/refinement of reference property unit tests
- bugfix: SROs are in OBJ_MAP too, it's not just SDOs!  Oops...
- pre-commit stylistic fixes
2021-03-31 15:32:51 -04:00
Michael Chisholm 986404b7b7 In the test_properties.py test suites, I thought code like
assert prop.clean(...)

doesn't test well enough since clean() methods on this branch
produce 2-tuples, and you should test what's in the tuple, not
just that it returned something non-empty.  So I fixed it in
several places to test the tuple contents.
2021-03-31 15:26:11 -04:00
Michael Chisholm 03c265c3a3 Add a check in ReferenceProperty constructor for an impossible
to satisfy type constraint: empty whitelist.  It would be silly
for anyone to do that, but I should check just in case I guess.
2021-03-31 15:26:11 -04:00
Michael Chisholm c7dd58ed89 Further ReferenceProperty refinements: make allow_custom=True
work when a whitelist of generic category types is used.
Disallow hybrid constraints (both generic and specific at the
same time).  Add more unit tests.
2021-03-31 15:26:09 -04:00
Michael Chisholm 387ce7e7cb Fix Report SDO: I'd fixed ReferenceProperty to work the way I
thought it should, but forgot to fix Report to use
ReferenceProperty in the way I thought it should!  Oops.
Added some tests to ensure Report is working property with
custom ID types in object_refs.
2021-03-31 15:22:00 -04:00
Michael Chisholm d2f960f2fc Initial revamp of customization enforcement and detection. 2021-03-31 15:21:55 -04:00
Emmanuelle Vargas-Gonzalez 203a2746e0 add new file test_extension_definition.py and assigned new UUIDs to the constants 2021-03-31 15:00:54 -04:00
Emmanuelle Vargas-Gonzalez 35ef718b70 add extension-definition ids (fail to validate) but keeping in Git history 2021-03-31 14:58:28 -04:00
Emmanuelle Vargas-Gonzalez dd6b847b10 small tweaks and pre-commit changes 2021-03-31 12:44:15 -04:00
Emmanuelle Vargas-Gonzalez 6d285c47ae update test suite for use of .serialize(pretty=True) 2021-03-31 12:39:14 -04:00
Emmanuelle Vargas-Gonzalez 9cc2e5bd3a remove backwards compatible imports, remove pretty=True from _STIXBase.__str()__, and simplify _STIXBase.__repr()__ (breaking) 2021-03-31 11:02:05 -04:00
Emmanuelle Vargas-Gonzalez bde9aaa13e update some TODOs and clear out some backwards kept imports (breaking) 2021-03-30 22:40:41 -04:00
Chris Lenk 2743b90fc0
Merge pull request #500 from emmanvg/add-fp-serialize
fp write for STIX Objects
2021-03-19 21:13:30 -04:00
Emmanuelle Vargas-Gonzalez 19196654c5
Apply suggestions from code review
Co-authored-by: Chris Lenk <clenk@users.noreply.github.com>
2021-03-19 15:31:01 -04:00
Emmanuelle Vargas-Gonzalez c2d360d223 apply fp_serialize() changes on main serialize() method 2021-03-18 18:08:31 -04:00
Emmanuelle Vargas-Gonzalez 922de111ed minor tweaks to docstrings 2021-03-18 10:14:36 -04:00