Commit Graph

733 Commits (c9bafaf5ae1c3d58e62a36ebb74327574e6bfc38)

Author SHA1 Message Date
Michael Chisholm 169ef87bc2 pre-commit stylistic fixes 2020-11-16 15:10:06 -05:00
Michael Chisholm f5bbbe7a3b Fix bug with reference type enforcement on property
where_sighted_refs on Sighting.
2020-11-16 15:04:24 -05:00
Chris Lenk a751df32c6
Fix Location semantic equivalence check for Location objects without the latitude and longitude properties (#467)
* Fix Location semantic equivalence check for Location objects without the latitude and longitude properties.

Uses contribution from @zrush-mitre (#464).

Fixes #462.

* Remove a line
2020-11-10 12:55:17 -05:00
Emmanuelle Vargas-Gonzalez fb705c4885
Graph Equivalence (#449)
* new packages for graph and object-based semantic equivalence

* new method graphically_equivalent for Environment, move equivalence methods out

* object equivalence function, methods used for object-based moved here.

* new graph_equivalence methods

* add notes

* add support for versioning checks (default disabled)

* new tests to cover graph equivalence and new methods

* added more imports to environment.py to prevent breaking changes

* variable changes, new fields for checks, reset depth check per call

* flexibility when object is not available on graph.

* refactor debug logging message

* new file stix2.equivalence.graph_equivalence.rst and stix2.equivalence.object_equivalence.rst for docs

* API documentation for new modules

* additional text required to build docs

* add more test methods for list_semantic_check an graphically_equivalent/versioning

* add logging debug messages, code clean-up

* include individual scoring on results dict, fix issue on list_semantic_check not keeping highest score

* include results as summary in prop_scores, minor tweaks

* Update __init__.py

doctrings update

* apply feedback from pull request

- rename semantic_check to reference_check
- rename modules to graph and object respectively to eliminate redundancy
- remove created_by_ref and object_marking_refs from graph WEIGHTS and rebalance

* update docs/ entries

* add more checks, make max score based on actual objects checked instead of the full list, only create entry when type is present in WEIGHTS dictionary

update tests to reflect changes

* rename package patterns -> pattern

* documentation, moving weights around

* more documentation moving

* rename WEIGHTS variable for graph_equivalence
2020-10-16 11:35:26 -04:00
Chris Lenk d17d01d165
Merge pull request #444 from chisholm/comp_ds_unversioned_objs
Update CompositeDataSource and deduplicate() support for unversioned objects
2020-10-15 08:57:55 -04:00
Chris Lenk ddb25c544a
Merge pull request #456 from chisholm/fix_comparison_expression_root_type
Fix object type tracking for AST comparison expression 'AND'
2020-09-18 09:59:53 -04:00
Michael Chisholm 7d64764ae3 Fix object type tracking for comparison expressions in the pattern
AST module.
2020-09-14 15:42:36 -04:00
Michael Chisholm bad42e5b78 pre-commit stylistic junk 2020-09-12 19:33:56 -04:00
Michael Chisholm ff35e8a01b Add some unit tests for the AST make_constant() function. 2020-09-12 19:14:29 -04:00
Michael Chisholm b5015b74ba pre-commit stylistic fixes 2020-08-19 12:10:51 -04:00
Michael Chisholm 3c25410a9d pre-commit stylistic fix 2020-08-19 11:51:47 -04:00
Michael Chisholm 7fa3c78dea Update CompositeDataSource and deduplicate() to handle unversioned
objects, including 2.1 SCOs.  Updated some unit tests to test
this.  Fixed a typo in a 2.0 unit test (2.0 deduplicate() test).
2020-08-17 18:38:29 -04:00
Michael Chisholm b6c2206491 Add some unit test suites for pattern equivalence which use some
STIX version-specific pattern features.
2020-08-14 19:56:49 -04:00
Michael Chisholm 320129e26c Add another unit test to help a bit with lack of coverage of
compare/comparison.py.  This one tests patterns with more
constant types.
2020-08-13 18:45:52 -04:00
Michael Chisholm c21b230edb pre-commit hook stylistic changes 2020-08-13 17:44:42 -04:00
Michael Chisholm 16a8c544ac Add a find_equivalent_patterns() function and unit tests, in case
a user wants a more efficient search capability.  (It is more
efficient than calling equivalent_patterns() over and over in a
loop, because it doesn't repeatedly re-canonicalize the search
pattern.)
2020-08-13 17:09:04 -04:00
Michael Chisholm bd5635f5be Add some unit tests for pattern equivalence. 2020-08-13 16:46:25 -04:00
Chris Lenk 1948b38eec
Merge pull request #438 from oasis-open/issue_437
hack for issue_435

Fixes #435.
2020-08-07 09:55:41 -04:00
Chris Lenk 1f9a844941
Merge pull request #439 from maybe-sybr/fix/customs-class-name
fix: Respect name of `@Custom*` decorated defs
2020-08-04 07:32:03 -04:00
Rich Piazza 8f76a84bbf handle quoted path components 2020-07-30 15:32:06 -04:00
maybe-sybr 15344527aa fix: Respect name of `@Custom*` decorated defs 2020-07-28 11:13:59 +10:00
Emmanuelle Vargas-Gonzalez 8cdbfed5e4
Merge pull request #431 from oasis-open/filesys-write-custom
Fix bug when adding custom object to FileSystemSink if the object type hasn't been registered
2020-07-27 09:43:38 -04:00
Rich Piazza b7a30befdc add tests and fix introduced bug 2020-07-25 14:47:40 -04:00
Rich Piazza 084941dd41 handle mixed boolean expressions 2020-07-24 11:40:21 -04:00
Emmanuelle Vargas-Gonzalez 8093898a3d move serialization-related methods to serialization.py
update tests that call specific methods from this area
2020-07-22 15:36:48 -04:00
Chris Lenk 55ea84ece2 Fix bug when adding custom obj to FileSystemSink
... if the object type hasn't been registered.

Related: #439.
2020-07-20 00:04:32 -04:00
Michael Chisholm d888eb60e2 Remove protocol_family and its uses from the v21 network
socket SCO extension.  It's a 2.0 property that got copied into
the 2.1 code by mistake.
2020-07-08 14:26:02 -04:00
Emmanuelle Vargas-Gonzalez bf66744db3 remove round from `fuzz.token_sort_ratio(str1, str2)` call 2020-07-08 11:02:56 -04:00
Chris Lenk 53dbc5835e Add test for coverage
Tests creating a property with both required and default.
2020-07-01 09:54:50 -04:00
Chris Lenk 3397805367 Consolidate version-independent property tests 2020-07-01 09:41:10 -04:00
Chris Lenk e25734d2b6 Fix trailing commas 2020-06-26 16:43:19 -04:00
Chris Lenk 3ef63d5e17 Update Workbench for 2.1 2020-06-26 16:43:19 -04:00
Chris Lenk d62f5ee141 Remove 'encoding' param from json.loads
It's ignored since Python 3.1 and we've dropped 2.7.
2020-06-26 16:43:19 -04:00
Chris Lenk f78b396648 Fix parsing tests 2020-06-26 16:43:19 -04:00
Chris Lenk b42b1cafa7 Move a couple tests for organization 2020-06-26 16:43:19 -04:00
Chris Lenk b2ee33208f Set STIX 2.1 as default, fix indicator.valid_from 2020-06-26 16:43:19 -04:00
Michael Chisholm bf5176f370 Add some new unit tests corresponding to ListProperty revamp. 2020-06-17 16:31:03 -04:00
Chris Lenk 9d05c9d3e2
Merge pull request #401 from chisholm/fix_versionable_detection
Fix versionability detection
2020-06-10 10:22:59 -04:00
Michael Chisholm 00b78e7a47 Stylistic pre-commit junk 2020-06-08 17:51:13 -04:00
Michael Chisholm 99bf4215a3 Change all uses of ObjectReferenceProperty in 2.1 SCO classes
to ReferenceProperty.  Add a unit test to ensure that when a
stix2 object is passed as a value of a ReferenceProperty, that
the ID is properly pulled out and used as the actual property
value, and that it doesn't affect the deterministic ID
calculation.
2020-06-08 17:24:33 -04:00
Chris Lenk 8bde22fa1f
Merge pull request #407 from emmanvg/flexibility-selector-syntax
Update SELECTOR_REGEX for GranularMarkings
2020-06-08 13:54:39 -04:00
Emmanuelle Vargas-Gonzalez aa18575c83 add `id` as special-case option in regex. closes #406 2020-06-08 09:27:12 -04:00
Michael Chisholm 9c5f950d5b Add a new_version() restriction preventing SCO ID contributing
properties from being changed, if a UUIDv5 is in use.  Changing
one of those properties would imply an ID change, which is not
allowed across versions.  Also:

- add a trailing comma
- change unchangable_properties to a set instead of a list,
  in case there are dupe props between STIX_UNMOD_PROPERTIES and
  sco_locked_props
2020-06-05 14:24:16 -04:00
Michael Chisholm 78d3b2c4c5 Add stix2.utils import to v20/test_versioning.py since the module
was used but not imported there.  Also import sorting from the
pre-commit hook.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5522050899 Improve versionability detection when creating new versions
of objects.  Move versioning-related stuff out of stix2.utils
to its own module.  Some misc versioning-related fixes.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5a5484d464 Move/merge some deterministic ID tests from the observed-data SDO
test suite to the test suite specific to testing deterministic
IDs.  This keeps the tests for that specific system in one place.
2020-06-05 14:14:44 -04:00
Michael Chisholm f17f782134 Bugfixes to the json un-escaping function. Added some unit tests
for that.
2020-06-05 14:14:44 -04:00
Michael Chisholm 2179028c91 Fix deterministic ID behavior when an empty "hashes" property is
given.  Now, an exception is raised.  Added a unit test for this.
2020-06-05 14:14:44 -04:00
Michael Chisholm c94feae623 Add a deterministic ID unit test with an embedded object 2020-06-05 14:14:43 -04:00
Michael Chisholm fa981affa9 stupid python 3.8 pre-commit still not adding the right trailing
commas....
2020-06-05 14:14:43 -04:00