Commit Graph

62 Commits (0866df05462ec13150bfe8253ec8884757f6a03f)

Author SHA1 Message Date
Chris Lenk e08a26a39c Correct variable name 2020-10-18 21:09:07 -04:00
Emmanuelle Vargas-Gonzalez 92ab1227ed docstrings, changes to equivalence.ipynb 2020-10-16 17:12:52 -04:00
Emmanuelle Vargas-Gonzalez fb705c4885
Graph Equivalence (#449)
* new packages for graph and object-based semantic equivalence

* new method graphically_equivalent for Environment, move equivalence methods out

* object equivalence function, methods used for object-based moved here.

* new graph_equivalence methods

* add notes

* add support for versioning checks (default disabled)

* new tests to cover graph equivalence and new methods

* added more imports to environment.py to prevent breaking changes

* variable changes, new fields for checks, reset depth check per call

* flexibility when object is not available on graph.

* refactor debug logging message

* new file stix2.equivalence.graph_equivalence.rst and stix2.equivalence.object_equivalence.rst for docs

* API documentation for new modules

* additional text required to build docs

* add more test methods for list_semantic_check an graphically_equivalent/versioning

* add logging debug messages, code clean-up

* include individual scoring on results dict, fix issue on list_semantic_check not keeping highest score

* include results as summary in prop_scores, minor tweaks

* Update __init__.py

doctrings update

* apply feedback from pull request

- rename semantic_check to reference_check
- rename modules to graph and object respectively to eliminate redundancy
- remove created_by_ref and object_marking_refs from graph WEIGHTS and rebalance

* update docs/ entries

* add more checks, make max score based on actual objects checked instead of the full list, only create entry when type is present in WEIGHTS dictionary

update tests to reflect changes

* rename package patterns -> pattern

* documentation, moving weights around

* more documentation moving

* rename WEIGHTS variable for graph_equivalence
2020-10-16 11:35:26 -04:00
Emmanuelle Vargas-Gonzalez bf66744db3 remove round from `fuzz.token_sort_ratio(str1, str2)` call 2020-07-08 11:02:56 -04:00
Chris Lenk 457085a115
Merge pull request #377 from maxbachmann-forks/master
use rapidfuzz instead of fuzzywuzzy
2020-07-08 10:49:41 -04:00
Chris Lenk b29527cae1 Fix for isort v5.0.0 2020-07-07 09:03:07 -04:00
Emmanuelle Vargas-Gonzalez 7ef3a02971 update docstrings environment.py and parsing.py 2020-06-30 22:32:04 -04:00
Chris Lenk 50df6f1474 Rename core.py -> parsing.py 2020-03-27 05:53:39 -04:00
maxbachmann d11b812a0e
round result 2020-03-26 20:28:09 +01:00
maxbachmann 9388a9820f
use rapidfuzz instead of fuzzywuzzy 2020-03-23 02:16:36 +01:00
Chris Lenk 62cd4fd33c Change string semantic comparison algorithm
Use `fuzzywuzzy`'s Token Sort Ratio instead of Jaro-Winkler.
2019-12-23 17:00:52 -05:00
Chris Lenk 457564f2f9 Update SemEq test, use dict for property weights 2019-12-20 17:01:21 -05:00
Chris Lenk 77eda29471 Add default weight_dict to documentation
for semantic equivalence
2019-12-11 13:13:36 -05:00
Desai, Kartikey H f6e75cd8f8 Add debug logging messages and add documentation to equivalence.ipynb 2019-12-06 10:46:27 -05:00
Desai, Kartikey H c09bd071d0 Make requested changes, except documentation, which is coming soon 2019-12-06 10:46:27 -05:00
Desai, Kartikey H 2b180c40b5 Remove unnecessary functions 2019-12-06 10:46:27 -05:00
Desai, Kartikey H f5d199bedf Generalize checking functionality within environment.py and add prop_scores dict so all scoring info is one python object 2019-12-06 10:46:27 -05:00
Emmanuelle Vargas-Gonzalez 024e023967 update semantic equivalence approach to:
- add more detailed output via the logging module
- don't fail hard if an object sent to the semantically_equivalent() method
- remove specific exception related to Semantic Equivalence and tests
2019-10-15 12:54:41 -04:00
Emmanuelle Vargas-Gonzalez dc79a1f869 add docstrings for new public methods. add test with disabled spec_version check.
fix calculation for distance, using incorrect algorithm. update package settings, tox settings
2019-09-23 23:13:50 -04:00
Emmanuelle Vargas-Gonzalez 4eaaee89dc make changes according to feedback. allow for custom objects to be supplied to method 2019-09-23 09:44:09 -04:00
Emmanuelle Vargas-Gonzalez 98ecdf53e3 update timestamp comparison method 2019-09-17 11:08:01 -04:00
Emmanuelle Vargas-Gonzalez e8eb7bcca2 fix logging messages, typos and add tests for the semantic equivalence method 2019-09-16 14:35:14 -04:00
Emmanuelle Vargas-Gonzalez 6fa77adfe3 wrote all default weights, actually computing the equivalence score
logging for unsupported objects, finished implementing some methods. Missing to implement patterning.
2019-09-10 15:04:07 -04:00
Emmanuelle Vargas-Gonzalez 93aa709b68 write down some of the semantic-equivalence work. WIP 2019-07-26 16:01:45 -04:00
Emmanuelle Vargas-Gonzalez 71a2aa2611 update project documentation. 2018-11-28 16:51:00 -05:00
Emmanuelle Vargas-Gonzalez 51a499cb33 Formatting changes made by the new pre-commit hook 'add trailing commas'
closes #189
2018-07-13 11:10:05 -04:00
Greg Back 3e159abd4d WIP: Merge branch 'master' into stix2.1 2018-05-23 10:43:52 -05:00
= 31fc1c369a still WIP 2018-04-12 12:03:07 -04:00
Chris Lenk f83d9a56b5 Clean up documentation 2018-04-09 13:29:53 -04:00
Chris Lenk 589c00064b Remove query_by_type
It's not that much of a shortcut and we can add it back in later if it
makes sense.
2018-04-04 14:09:54 -04:00
Chris Lenk e48e0886a8 Improve code coverage slightly
Environment will always have a CompositeDataSource, so the try/catches
in add_filter/s did not make sense.
2018-03-30 12:02:27 -04:00
Chris Lenk 61733ad899 Add functions to set ObjectFactory default values 2018-03-30 12:02:27 -04:00
Chris Lenk fd6d9f74e9 Move query_by_type() to DataStoreMixin 2018-03-30 12:02:27 -04:00
Chris Lenk eb2d87ce71 Rename "DataStore" class to "DataStoreMixin"
Fix #116.
2018-03-01 11:27:37 -05:00
Chris Lenk 5f703509be Rename "sources" subpackage to "datastore"
Fix #114.
2018-03-01 09:04:42 -05:00
Emmanuelle Vargas-Gonzalez 722d46c6c5 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into stix2.1 2017-12-08 09:36:59 -05:00
Chris Lenk 4a6bca4486 Merge branch 'master' into 89-relationships 2017-11-21 16:41:21 -05:00
Chris Lenk d355d1827e Move `creator_of` to DataStore/DataSource 2017-11-21 16:29:06 -05:00
Chris Lenk f0331f8b9b Reorganize DataStore code for dereferencing
- Move `relationships()` to DataStore like `related_to()` is. If a
  DataStore implementation needs a different way to handle relationship
  dereferencing (e.g. TAXII in the future, or CompositeDataSource), it
  can overwrite these functions.
- Reduce code duplication.
- Check for presence of Data Source/Sink in all DataStores, not just in
  Environment.
2017-11-16 16:25:57 -05:00
Chris Lenk 29dec997a0 Add related_to()
Function for calling relationships() but instead of just returning the
Relationship objects, returns the STIX objects being refered to in those
Relationships.
2017-11-16 14:58:59 -05:00
Chris Lenk 55cf00d7f0 Move relationships() to DataSources 2017-11-15 10:37:17 -05:00
Chris Lenk f9ad7ceb65 Add relationships() function to Environment 2017-11-13 11:09:56 -05:00
Chris Lenk 7495ea6588 Add inter-documentation links
So when users read the guide docs, they can click a link to go to a
relevant section in the API reference.
2017-11-09 15:42:59 -05:00
Emmanuelle Vargas-Gonzalez f6f7d0aed8 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 2017-11-02 07:48:37 -04:00
Chris Lenk 5dffe74867 Clean up creator_of
(renamed from created_by)
2017-10-24 14:20:42 -04:00
Chris Lenk e1e368c0d2 Add created_by() function to Environment 2017-10-20 09:13:04 -04:00
= ffa2242878 code changes brought about by discussion of pull request of 'bug_fixes' branch 2017-09-29 11:24:19 -04:00
= 94df10bf8d docstrings redone; code changes for adding/returning STIX objects to FileSystem, Memory, TAXII; code changes for query sets and filters 2017-09-22 11:29:17 -04:00
Chris Lenk be07c32500 Add parse() to Environment layer
The Environment layer does not keep its own mapping of custom object types.
If we think it likely that users will want to maintain separate lists of
custom object types between two or more Environments, we can add this
later.
2017-09-08 12:40:01 -04:00
Chris Lenk f60331fb77 Improve docstrings for Environment layer 2017-09-08 11:15:10 -04:00