a751df32c6
Fix Location semantic equivalence check for Location objects without the latitude and longitude properties ( #467 )
...
* Fix Location semantic equivalence check for Location objects without the latitude and longitude properties.
Uses contribution from @zrush-mitre (#464 ).
Fixes #462 .
* Remove a line
2020-11-10 12:55:17 -05:00
1180da7cc7
update to test cases
...
removing invalid test cases per new changes. updating some to reflect new changes to the Extensions Property, other minor ones
2020-11-09 20:35:22 -05:00
fb705c4885
Graph Equivalence ( #449 )
...
* new packages for graph and object-based semantic equivalence
* new method graphically_equivalent for Environment, move equivalence methods out
* object equivalence function, methods used for object-based moved here.
* new graph_equivalence methods
* add notes
* add support for versioning checks (default disabled)
* new tests to cover graph equivalence and new methods
* added more imports to environment.py to prevent breaking changes
* variable changes, new fields for checks, reset depth check per call
* flexibility when object is not available on graph.
* refactor debug logging message
* new file stix2.equivalence.graph_equivalence.rst and stix2.equivalence.object_equivalence.rst for docs
* API documentation for new modules
* additional text required to build docs
* add more test methods for list_semantic_check an graphically_equivalent/versioning
* add logging debug messages, code clean-up
* include individual scoring on results dict, fix issue on list_semantic_check not keeping highest score
* include results as summary in prop_scores, minor tweaks
* Update __init__.py
doctrings update
* apply feedback from pull request
- rename semantic_check to reference_check
- rename modules to graph and object respectively to eliminate redundancy
- remove created_by_ref and object_marking_refs from graph WEIGHTS and rebalance
* update docs/ entries
* add more checks, make max score based on actual objects checked instead of the full list, only create entry when type is present in WEIGHTS dictionary
update tests to reflect changes
* rename package patterns -> pattern
* documentation, moving weights around
* more documentation moving
* rename WEIGHTS variable for graph_equivalence
2020-10-16 11:35:26 -04:00
d17d01d165
Merge pull request #444 from chisholm/comp_ds_unversioned_objs
...
Update CompositeDataSource and deduplicate() support for unversioned objects
2020-10-15 08:57:55 -04:00
ddb25c544a
Merge pull request #456 from chisholm/fix_comparison_expression_root_type
...
Fix object type tracking for AST comparison expression 'AND'
2020-09-18 09:59:53 -04:00
7d64764ae3
Fix object type tracking for comparison expressions in the pattern
...
AST module.
2020-09-14 15:42:36 -04:00
bad42e5b78
pre-commit stylistic junk
2020-09-12 19:33:56 -04:00
ff35e8a01b
Add some unit tests for the AST make_constant() function.
2020-09-12 19:14:29 -04:00
b5015b74ba
pre-commit stylistic fixes
2020-08-19 12:10:51 -04:00
3c25410a9d
pre-commit stylistic fix
2020-08-19 11:51:47 -04:00
7fa3c78dea
Update CompositeDataSource and deduplicate() to handle unversioned
...
objects, including 2.1 SCOs. Updated some unit tests to test
this. Fixed a typo in a 2.0 unit test (2.0 deduplicate() test).
2020-08-17 18:38:29 -04:00
b6c2206491
Add some unit test suites for pattern equivalence which use some
...
STIX version-specific pattern features.
2020-08-14 19:56:49 -04:00
c21b230edb
pre-commit hook stylistic changes
2020-08-13 17:44:42 -04:00
1948b38eec
Merge pull request #438 from oasis-open/issue_437
...
hack for issue_435
Fixes #435 .
2020-08-07 09:55:41 -04:00
1f9a844941
Merge pull request #439 from maybe-sybr/fix/customs-class-name
...
fix: Respect name of `@Custom*` decorated defs
2020-08-04 07:32:03 -04:00
8f76a84bbf
handle quoted path components
2020-07-30 15:32:06 -04:00
15344527aa
fix: Respect name of `@Custom*` decorated defs
2020-07-28 11:13:59 +10:00
8cdbfed5e4
Merge pull request #431 from oasis-open/filesys-write-custom
...
Fix bug when adding custom object to FileSystemSink if the object type hasn't been registered
2020-07-27 09:43:38 -04:00
b7a30befdc
add tests and fix introduced bug
2020-07-25 14:47:40 -04:00
084941dd41
handle mixed boolean expressions
2020-07-24 11:40:21 -04:00
8093898a3d
move serialization-related methods to serialization.py
...
update tests that call specific methods from this area
2020-07-22 15:36:48 -04:00
55ea84ece2
Fix bug when adding custom obj to FileSystemSink
...
... if the object type hasn't been registered.
Related: #439 .
2020-07-20 00:04:32 -04:00
d888eb60e2
Remove protocol_family and its uses from the v21 network
...
socket SCO extension. It's a 2.0 property that got copied into
the 2.1 code by mistake.
2020-07-08 14:26:02 -04:00
bf66744db3
remove round from `fuzz.token_sort_ratio(str1, str2)` call
2020-07-08 11:02:56 -04:00
3397805367
Consolidate version-independent property tests
2020-07-01 09:41:10 -04:00
e25734d2b6
Fix trailing commas
2020-06-26 16:43:19 -04:00
3ef63d5e17
Update Workbench for 2.1
2020-06-26 16:43:19 -04:00
d62f5ee141
Remove 'encoding' param from json.loads
...
It's ignored since Python 3.1 and we've dropped 2.7.
2020-06-26 16:43:19 -04:00
f78b396648
Fix parsing tests
2020-06-26 16:43:19 -04:00
b42b1cafa7
Move a couple tests for organization
2020-06-26 16:43:19 -04:00
b2ee33208f
Set STIX 2.1 as default, fix indicator.valid_from
2020-06-26 16:43:19 -04:00
9d05c9d3e2
Merge pull request #401 from chisholm/fix_versionable_detection
...
Fix versionability detection
2020-06-10 10:22:59 -04:00
00b78e7a47
Stylistic pre-commit junk
2020-06-08 17:51:13 -04:00
99bf4215a3
Change all uses of ObjectReferenceProperty in 2.1 SCO classes
...
to ReferenceProperty. Add a unit test to ensure that when a
stix2 object is passed as a value of a ReferenceProperty, that
the ID is properly pulled out and used as the actual property
value, and that it doesn't affect the deterministic ID
calculation.
2020-06-08 17:24:33 -04:00
8bde22fa1f
Merge pull request #407 from emmanvg/flexibility-selector-syntax
...
Update SELECTOR_REGEX for GranularMarkings
2020-06-08 13:54:39 -04:00
aa18575c83
add `id` as special-case option in regex. closes #406
2020-06-08 09:27:12 -04:00
9c5f950d5b
Add a new_version() restriction preventing SCO ID contributing
...
properties from being changed, if a UUIDv5 is in use. Changing
one of those properties would imply an ID change, which is not
allowed across versions. Also:
- add a trailing comma
- change unchangable_properties to a set instead of a list,
in case there are dupe props between STIX_UNMOD_PROPERTIES and
sco_locked_props
2020-06-05 14:24:16 -04:00
5522050899
Improve versionability detection when creating new versions
...
of objects. Move versioning-related stuff out of stix2.utils
to its own module. Some misc versioning-related fixes.
2020-06-05 14:24:16 -04:00
5a5484d464
Move/merge some deterministic ID tests from the observed-data SDO
...
test suite to the test suite specific to testing deterministic
IDs. This keeps the tests for that specific system in one place.
2020-06-05 14:14:44 -04:00
f17f782134
Bugfixes to the json un-escaping function. Added some unit tests
...
for that.
2020-06-05 14:14:44 -04:00
2179028c91
Fix deterministic ID behavior when an empty "hashes" property is
...
given. Now, an exception is raised. Added a unit test for this.
2020-06-05 14:14:44 -04:00
c94feae623
Add a deterministic ID unit test with an embedded object
2020-06-05 14:14:43 -04:00
fa981affa9
stupid python 3.8 pre-commit still not adding the right trailing
...
commas....
2020-06-05 14:14:43 -04:00
9dcffd9fad
Add some unit tests for deterministic IDs.
2020-06-05 14:14:43 -04:00
7955a41997
Drop python-dateutil as a dependency and switch to the builtin
...
datetime module for parsing timestamps. Dateutil proved too
slow.
2020-05-20 15:06:53 -04:00
b4dbc419f6
Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 391-ssdeep-hash-case
2020-05-14 12:51:02 -04:00
68f7ca6377
resolve problem with SSDEEP vocab use for 2.1, closes #391
2020-05-13 18:17:17 -04:00
998b4c0725
Change streamlined_obj_vals list to streamlined_object dict
2020-05-13 12:45:16 -05:00
de3fa99a12
Add property names to canonicalization for deterministic id gen
2020-05-13 11:20:16 -05:00
df92770d25
Merge pull request #384 from oasis-open/365-versioned-classes
...
Validate custom type/property name formats
2020-04-03 17:30:24 -04:00
2b0d63c4b1
update test_datastore_taxii.py conftest.py for latest changes in medallion. add extra data used by filter
2020-04-03 17:19:36 -04:00
c7fb79d195
Fix some TAXII DataStore tests
2020-04-03 15:58:56 -04:00
9145bdf5e8
Merge pull request #374 from chisholm/version_precision
...
Support STIX 2.1 version precision
2020-04-03 15:52:42 -04:00
0d770972cf
Merge pull request #382 from oasis-open/more-pattern-tests
...
More pattern tests
2020-04-03 11:24:43 -04:00
e730d45d44
Use DEFAULT_VERSION in create_pattern_object()
2020-04-03 10:45:36 -04:00
14540c0ea1
Clean up _register_* functions
...
Made them consistent with _register_observable_extension, by:
- moving validation logic there from _custom_*_builder functions
- using a new function for ensuring properties are dict-like
- using the library default spec version instead of None
Fix #371 , fix #372 , fix #373 .
2020-04-02 14:15:45 -04:00
d33adbcc71
Rename test files to align with module renaming
2020-04-02 08:22:49 -04:00
13cddf9d6d
Move TypeProperty format checks to __init__
...
TypeProperty uses a fixed value, so check() was never called. This way
also runs the check at object registration time because the wrapper
creates an instance of TypeProperty and doesn't have to wait for the
object to be instantiated so clean() can be called.
Also fix some tests.
2020-04-02 08:17:34 -04:00
03cb225932
Merge branch 'master' into 365-versioned-classes
2020-04-02 06:02:20 -04:00
c494a2e477
Use TypeProperty.clean() to verify type format
2020-04-01 21:52:04 -04:00
c911cff97f
Add duplicate checking to markings and observable extensions, and fix some tests and add some tests. Fixes #363
2020-03-27 14:58:18 -04:00
1a2b1367cf
flaky 2
2020-03-27 14:06:24 -04:00
9933f88975
few more pattern op tests
2020-03-27 13:59:03 -04:00
e3ebb6393d
flaky
2020-03-27 12:33:24 -04:00
202111acdf
more pattern tests
2020-03-27 11:22:00 -04:00
50df6f1474
Rename core.py -> parsing.py
2020-03-27 05:53:39 -04:00
b06bc1afc1
Fix import issues
2020-03-20 17:32:18 -04:00
f37b84a564
Pull in updates from master
2020-03-20 16:52:21 -04:00
1260c7b45e
Fix existing tests and add new tests. Fixes #363
2020-03-20 16:49:20 -04:00
9e5e998c3d
don't allow leading '_' on custom properties, whenever allow_custom is true
2020-03-20 12:49:20 -04:00
2c4e47de56
remove leading - from type name re
2020-03-20 11:56:09 -04:00
6e4151aeeb
flaky
2020-03-19 16:49:46 -04:00
f60e4170fd
finish 365
2020-03-19 16:11:52 -04:00
844ec2c3bf
more on issue 365
2020-03-19 14:16:48 -04:00
9699c78ad8
issue-365
2020-03-19 10:40:35 -04:00
1741cc9f6b
Fix import sort order for the import sorter precommit hook
2020-03-17 20:26:21 -04:00
6f43814918
Add xfail mark to a unit test which trips a Python 3.6 bug.
...
https://bugs.python.org/issue32404
2020-03-17 20:21:09 -04:00
f99665f2ba
One more comma, because python 3.8's add-trailing-comma
...
pre-commit hook doesn't add all the commas Travis's hook
script expects...
2020-03-17 19:45:39 -04:00
cf9aef59c2
More flake8 style fixes
2020-03-17 18:28:38 -04:00
a9ac7ce838
pre-commit hook changes, e.g. trailing commas, import sorting,
...
flake8 style.
2020-03-17 18:26:57 -04:00
4aa69fa7c9
Add support for enforcing STIX 2.1 minimum precision requirement
...
on versioning timestamps.
2020-03-16 20:25:38 -04:00
6842abb371
Merge pull request #370 from chisholm/observable_extension_names_ext
...
New STIX 2.1 SCO extension name requirement: must end with "-ext"
2020-03-12 17:26:26 -04:00
15316e7933
Added "x-" to SCO extension names in unit tests, to illustrate
...
best practice and follow a spec "should" rule.
2020-03-12 16:20:32 -04:00
5abc139e79
Merge branch 'khdesai-fix_issue_338'; Close #347
2020-03-12 09:30:52 -04:00
8885a757cb
Fix properties spec version back to 2.1, and re-adjust tests. Fixes #338
2020-03-12 09:24:43 -04:00
36f7035785
Fixes #338
2020-03-12 09:24:43 -04:00
e782d095ea
Merge pull request #369 from chisholm/malware_os_refs
...
Change software SCO: os_execution_envs -> operating_system_refs
2020-03-11 23:47:14 -04:00
94e3cd7ca6
Merge pull request #360 from chisholm/enforce_hash_keys
...
Enforce hash keys on 2.1 external-references
2020-03-11 23:13:55 -04:00
87c5ef30ad
Merge pull request #358 from chisholm/software_cpe_swid
...
Add swid property to the software SCO
2020-03-11 23:03:18 -04:00
2472af387b
Change a SWID tagId in a unit test from a UUID to something
...
more plausible.
2020-03-11 15:21:34 -04:00
33fb31421b
Merge pull request #357 from chisholm/malware_analysis_result
...
Update malware-analysis SDO's av_result property
2020-03-11 09:16:34 -04:00
bdf7cab8fe
Merge pull request #356 from chisholm/malware_analysis_sample_ref
...
Add the "sample_ref" property to malware-analysis SDOs
2020-03-11 09:12:02 -04:00
2429533e4f
Merge pull request #355 from chisholm/optional_type_properties
...
Changed several *_types properties to be optional due to STIX spec change
2020-03-11 09:08:42 -04:00
d708537b85
Add enforcement of a new STIX 2.1 SCO extension name requirement:
...
that it must end with "-ext".
2020-03-10 20:24:53 -04:00
792cc570d7
Change the os_execution_envs property of software SCOs to
...
operating_system_refs, and add a test for it.
2020-03-06 19:43:47 -05:00
e32b074bc9
Fix stylistic issues for pre-commit hooks.
2020-03-05 17:39:35 -05:00
22f2b241a7
Add a missing required property to fix up an external-reference
...
test.
2020-03-05 17:38:03 -05:00
3803e4bdd7
Merge pull request #343 from chisholm/sco_tlo_filesystemstore
...
Fix the filesystem store to support the new top-level 2.1 SCOs.
2020-03-05 17:08:20 -05:00
cdde664434
Merge branch 'master' into fix_indicator_test
2020-03-05 10:51:03 -05:00
a5dc514403
Fix external-references to force hash keys to come from
...
hash-algorithm-ov.
2020-03-04 20:55:52 -05:00
Chris Lenk