Commit Graph

37 Commits (aab27b6559c449c85cfc3e422dcfcf7e8a7edf6c)

Author SHA1 Message Date
Michael Chisholm 3086b7ab5b Fix pattern semantic equivalence bug: comparison expression
DNF transform could trigger an error from the AST code which
does the SCO type checks on comparison expression ANDs.  The
transform had been ignoring SCO types.  Now, it will catch
the exception and drop the AND from the pattern.  Added a
couple new unit tests for this.
2021-12-23 00:59:41 -05:00
Emmanuelle Vargas-Gonzalez f7ebd34c8c
Update __init__.py
fix incident entry
2021-03-12 16:49:23 -05:00
Emmanuelle Vargas-Gonzalez c9e66def60 rename test, update the rest of the docstrings for object_similarity() and object_equivalence() 2021-03-10 13:32:02 -05:00
Emmanuelle Vargas-Gonzalez bd996b8750
Update __init__.py 2021-03-01 14:40:05 -05:00
Emmanuelle Vargas-Gonzalez 72a12e96ba update guide example, conf.py and remove some unused imports 2021-03-01 13:14:03 -05:00
Emmanuelle Vargas-Gonzalez 3efa4c1ce9 revert part changes 2021-03-01 12:44:35 -05:00
Emmanuelle Vargas-Gonzalez ff5014c606 expose configuration options, combine weight dictionary, update tests 2021-03-01 12:27:52 -05:00
Emmanuelle Vargas-Gonzalez f9a52eeed3 WIP: changes to both similarity functions, expose settings 2021-02-26 19:19:33 -05:00
Chris Lenk 64608e7bea
Merge pull request #491 from emmanvg/graph-eq-changes
Graph Equivalence Changes
2021-02-18 23:44:39 -05:00
Emmanuelle Vargas-Gonzalez 99453770cf doctring changes, _versioned_checks changes 2021-02-18 10:37:34 -05:00
Emmanuelle Vargas-Gonzalez 75574c9427
Update stix2/equivalence/object/__init__.py
docstrings

Co-authored-by: Chris Lenk <clenk@users.noreply.github.com>
2021-02-18 09:15:09 -05:00
Emmanuelle Vargas-Gonzalez 154fc4e236
Update stix2/equivalence/object/__init__.py
update variable names

Co-authored-by: Chris Lenk <clenk@users.noreply.github.com>
2021-02-18 09:14:34 -05:00
Emmanuelle Vargas-Gonzalez 52c5f3ad29
Update stix2/equivalence/object/__init__.py
Co-authored-by: Chris Lenk <clenk@users.noreply.github.com>
2021-02-18 09:14:10 -05:00
Emmanuelle Vargas-Gonzalez ee63e9faf4 resolve issue regarding reference_check or list_reference_check, remove redundant object_similarity call
update test suite
2021-02-17 21:30:14 -05:00
Emmanuelle Vargas-Gonzalez d2d85badb2 make some functions internal, add some docs for them 2021-02-16 11:01:07 -05:00
Emmanuelle Vargas-Gonzalez 09fd8c060b clear debug message 2021-02-16 10:23:02 -05:00
Emmanuelle Vargas-Gonzalez fbea229004 add styling changes 2021-02-16 09:44:03 -05:00
Emmanuelle Vargas-Gonzalez 02b076b3bb resolve issues with graph similarity
- new methods for graph equivalence and similarity
- remove sorting and len comparisons
- rename some variables
2021-02-16 00:57:26 -05:00
Michael Chisholm 745696cba1 Remove "canonical" from a couple more places in docstrings which
I missed.
2021-02-14 19:25:59 -05:00
Michael Chisholm ccaa8b62ae pre-commit stylistic fix 2021-02-11 19:35:28 -05:00
Michael Chisholm 631460f45f Rename various symbols and change various comments to refer to
normalization instead of canonicalization.
2021-02-11 19:33:57 -05:00
Michael Chisholm 9e9a61c71c Fix bug with observation expression DNF transformer, where it was
not preserving operand order when distributing FOLLOWEDBY.
2021-02-04 16:45:39 -05:00
Emmanuelle Vargas-Gonzalez 489970718f WIP: changes to graph_similarity
busted main loop, symmetrical properties not present
2021-02-01 22:35:37 -05:00
Chris Lenk a0d535336e Merge 'master' and fix imports for consistency 2021-01-15 12:34:10 -05:00
Emmanuelle Vargas-Gonzalez 85c14d1502 all changes from add-trailing-commas v2.0.2 2021-01-13 17:52:15 -05:00
Chris Lenk b55c3bb1df Fix pattern equivalence doc style for consistency 2020-11-20 16:34:11 -05:00
Chris Lenk 53f451b097 Use pattern equivalence in indicators 2020-11-20 09:33:56 -05:00
Chris Lenk a751df32c6
Fix Location semantic equivalence check for Location objects without the latitude and longitude properties (#467)
* Fix Location semantic equivalence check for Location objects without the latitude and longitude properties.

Uses contribution from @zrush-mitre (#464).

Fixes #462.

* Remove a line
2020-11-10 12:55:17 -05:00
Emmanuelle Vargas-Gonzalez 92ab1227ed docstrings, changes to equivalence.ipynb 2020-10-16 17:12:52 -04:00
Emmanuelle Vargas-Gonzalez fb705c4885
Graph Equivalence (#449)
* new packages for graph and object-based semantic equivalence

* new method graphically_equivalent for Environment, move equivalence methods out

* object equivalence function, methods used for object-based moved here.

* new graph_equivalence methods

* add notes

* add support for versioning checks (default disabled)

* new tests to cover graph equivalence and new methods

* added more imports to environment.py to prevent breaking changes

* variable changes, new fields for checks, reset depth check per call

* flexibility when object is not available on graph.

* refactor debug logging message

* new file stix2.equivalence.graph_equivalence.rst and stix2.equivalence.object_equivalence.rst for docs

* API documentation for new modules

* additional text required to build docs

* add more test methods for list_semantic_check an graphically_equivalent/versioning

* add logging debug messages, code clean-up

* include individual scoring on results dict, fix issue on list_semantic_check not keeping highest score

* include results as summary in prop_scores, minor tweaks

* Update __init__.py

doctrings update

* apply feedback from pull request

- rename semantic_check to reference_check
- rename modules to graph and object respectively to eliminate redundancy
- remove created_by_ref and object_marking_refs from graph WEIGHTS and rebalance

* update docs/ entries

* add more checks, make max score based on actual objects checked instead of the full list, only create entry when type is present in WEIGHTS dictionary

update tests to reflect changes

* rename package patterns -> pattern

* documentation, moving weights around

* more documentation moving

* rename WEIGHTS variable for graph_equivalence
2020-10-16 11:35:26 -04:00
Emmanuelle Vargas-Gonzalez 639769d885 expose API Reference for Pattern Equivalence for ReadTheDocs 2020-09-11 13:54:13 -04:00
Michael Chisholm 9e707a3a81 Add stix_version kwargs to the pattern equivalence functions.
This allows the patterns to be parsed using either 2.0 or 2.1+
syntax.
2020-08-14 19:55:00 -04:00
Michael Chisholm c21b230edb pre-commit hook stylistic changes 2020-08-13 17:44:42 -04:00
Michael Chisholm 16a8c544ac Add a find_equivalent_patterns() function and unit tests, in case
a user wants a more efficient search capability.  (It is more
efficient than calling equivalent_patterns() over and over in a
loop, because it doesn't repeatedly re-canonicalize the search
pattern.)
2020-08-13 17:09:04 -04:00
Michael Chisholm 6c92f670cb Fix ipv4/6 special canonicalizers to reformat IP addresses even
when a non-CIDR address is used.  Before, it left plain IP
addresses untouched.
2020-08-13 16:22:24 -04:00
Michael Chisholm 5d6c7d8c8a Add some simple context-sensitive constant canonicalization, used
as part of canonicalizing comparison expressions.  This
required adding a new comparison expression transformer callback
for leaf-node comparison expression objects, and updating all
existing comparison transformers to work (it affected all/most
of them).  The observation expression transformer which actually
does the comparison canonicalization was updated to also perform
this special canonicalization step.
2020-08-12 19:28:35 -04:00
Michael Chisholm 311fe38cea Add first cut of a pattern equivalence capability 2020-08-10 18:33:26 -04:00