Commit Graph

1227 Commits (e513081100a067fab7292d6d9960ab1f0971c054)

Author SHA1 Message Date
Chris Lenk 53dbc5835e Add test for coverage
Tests creating a property with both required and default.
2020-07-01 09:54:50 -04:00
Chris Lenk 3397805367 Consolidate version-independent property tests 2020-07-01 09:41:10 -04:00
Emmanuelle Vargas-Gonzalez 7ef3a02971 update docstrings environment.py and parsing.py 2020-06-30 22:32:04 -04:00
Emmanuelle Vargas-Gonzalez 21e109d881 remove `core` and added `parsing`, `versioning` to autosummary 2020-06-30 22:16:15 -04:00
Emmanuelle Vargas-Gonzalez 9a0cb02ab3 nit-pick on v21 Indicator constructor 2020-06-30 13:58:19 -04:00
Chris Lenk e25734d2b6 Fix trailing commas 2020-06-26 16:43:19 -04:00
Chris Lenk d5d664ba41 Update docs for 2.1 as default 2020-06-26 16:43:19 -04:00
Chris Lenk 10bdfe9e25 Fix KeyError in TAXII DataStore 2020-06-26 16:43:19 -04:00
Chris Lenk 3ef63d5e17 Update Workbench for 2.1 2020-06-26 16:43:19 -04:00
Chris Lenk d62f5ee141 Remove 'encoding' param from json.loads
It's ignored since Python 3.1 and we've dropped 2.7.
2020-06-26 16:43:19 -04:00
Chris Lenk e80d3bad2c Fix 2.0 HashesProperties 2020-06-26 16:43:19 -04:00
Chris Lenk f78b396648 Fix parsing tests 2020-06-26 16:43:19 -04:00
Chris Lenk b42b1cafa7 Move a couple tests for organization 2020-06-26 16:43:19 -04:00
Chris Lenk b2ee33208f Set STIX 2.1 as default, fix indicator.valid_from 2020-06-26 16:43:19 -04:00
Michael Chisholm bf5176f370 Add some new unit tests corresponding to ListProperty revamp. 2020-06-17 16:31:03 -04:00
Michael Chisholm bc51cd47bd Revamp ListProperty so its logic makes more sense. 2020-06-17 16:11:30 -04:00
Chris Lenk 9d05c9d3e2
Merge pull request #401 from chisholm/fix_versionable_detection
Fix versionability detection
2020-06-10 10:22:59 -04:00
Michael Chisholm 00b78e7a47 Stylistic pre-commit junk 2020-06-08 17:51:13 -04:00
Michael Chisholm 99bf4215a3 Change all uses of ObjectReferenceProperty in 2.1 SCO classes
to ReferenceProperty.  Add a unit test to ensure that when a
stix2 object is passed as a value of a ReferenceProperty, that
the ID is properly pulled out and used as the actual property
value, and that it doesn't affect the deterministic ID
calculation.
2020-06-08 17:24:33 -04:00
Chris Lenk 8bde22fa1f
Merge pull request #407 from emmanvg/flexibility-selector-syntax
Update SELECTOR_REGEX for GranularMarkings
2020-06-08 13:54:39 -04:00
Emmanuelle Vargas-Gonzalez aa18575c83 add `id` as special-case option in regex. closes #406 2020-06-08 09:27:12 -04:00
Michael Chisholm c13cb182db versioning related updates:
- remove var 'properties_to_change' since it's unnecessary
- delete most of remove_custom_stix() since it was unnecessary,
  greatly simplify it
2020-06-05 14:24:16 -04:00
Michael Chisholm 9c5f950d5b Add a new_version() restriction preventing SCO ID contributing
properties from being changed, if a UUIDv5 is in use.  Changing
one of those properties would imply an ID change, which is not
allowed across versions.  Also:

- add a trailing comma
- change unchangable_properties to a set instead of a list,
  in case there are dupe props between STIX_UNMOD_PROPERTIES and
  sco_locked_props
2020-06-05 14:24:16 -04:00
Michael Chisholm eaa7f17ee4 Fix versionability detection: _properties is an OrderedDict not
a list of tuples! (despite what it looks like if you glance at it
quickly.)
2020-06-05 14:24:16 -04:00
Michael Chisholm 15e75cb4de Python 2 compatibility fix in versionability detection. In
python2, Mapping.keys() returns a list instead of a set!
2020-06-05 14:24:16 -04:00
Michael Chisholm a05dbc42d5 Improve the docstring for new_version(). Especially with the
three-value logic for allow_custom, it needed some much better
documentation.
2020-06-05 14:24:16 -04:00
Michael Chisholm d4cecd6f51 In remove_custom_stix(), when creating the new version,
use allow_custom=False to force custom properties to be
disallowed.  This is extra insurance against a custom prop
accidentally getting through.
2020-06-05 14:24:16 -04:00
Michael Chisholm 78d3b2c4c5 Add stix2.utils import to v20/test_versioning.py since the module
was used but not imported there.  Also import sorting from the
pre-commit hook.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5522050899 Improve versionability detection when creating new versions
of objects.  Move versioning-related stuff out of stix2.utils
to its own module.  Some misc versioning-related fixes.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5a5484d464 Move/merge some deterministic ID tests from the observed-data SDO
test suite to the test suite specific to testing deterministic
IDs.  This keeps the tests for that specific system in one place.
2020-06-05 14:14:44 -04:00
Michael Chisholm f17f782134 Bugfixes to the json un-escaping function. Added some unit tests
for that.
2020-06-05 14:14:44 -04:00
Michael Chisholm 2179028c91 Fix deterministic ID behavior when an empty "hashes" property is
given.  Now, an exception is raised.  Added a unit test for this.
2020-06-05 14:14:44 -04:00
Michael Chisholm c94feae623 Add a deterministic ID unit test with an embedded object 2020-06-05 14:14:43 -04:00
Michael Chisholm fa981affa9 stupid python 3.8 pre-commit still not adding the right trailing
commas....
2020-06-05 14:14:43 -04:00
Michael Chisholm 9dcffd9fad Add some unit tests for deterministic IDs. 2020-06-05 14:14:43 -04:00
Michael Chisholm 1f5e8defd3 Argh, I fixed an RFC URL but forgot to fix the comment which
described it and gave the RFC number...
2020-06-05 14:14:43 -04:00
Michael Chisholm 22c0d5806c Trailing comma for the pre-commit hook 2020-06-05 14:14:43 -04:00
Michael Chisholm e22c37a16b Update a RFC URL to the latest version. I'd accidentally used
one which had been obsoleted by a newer one.
2020-06-05 14:14:43 -04:00
Michael Chisholm 6c2c4781e7 Revamp deterministic ID generation code to fix bugs. 2020-06-05 14:14:43 -04:00
Chris Lenk 41525f9be0
Merge pull request #400 from oasis-open/issue-398
fixed start/stop qualifier constant issue
Fixes #398.
2020-06-05 10:57:59 -04:00
Rich Piazza 53dfe40c30 only test start/stop timestamps if they are strings (2.0) 2020-06-05 09:25:06 -04:00
Rich Piazza 04af273d71 flaky 2020-06-05 09:17:42 -04:00
Michael Chisholm 7def04fd6f Add __all__'s to v20 and v21/__init__.py to ensure that
star-imports from those modules get the right symbols.  "base"
was a problem because it was getting imported via
stix2/__init__.py and overwriting stix2.base, making the actual
stix2/base.py module inaccessible.
2020-06-03 17:39:09 -04:00
Rich Piazza 977107e713 2.0 Start/Stop Quailifer uses StringConstants 2020-06-01 22:34:40 -04:00
Rich Piazza cfe0648c98 fixed start/stop qualifier constant issue 2020-05-26 15:12:02 -04:00
Michael Chisholm 7955a41997 Drop python-dateutil as a dependency and switch to the builtin
datetime module for parsing timestamps.  Dateutil proved too
slow.
2020-05-20 15:06:53 -04:00
Emmanuelle Vargas-Gonzalez b4dbc419f6 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 391-ssdeep-hash-case 2020-05-14 12:51:02 -04:00
Emmanuelle Vargas-Gonzalez 68f7ca6377 resolve problem with SSDEEP vocab use for 2.1, closes #391 2020-05-13 18:17:17 -04:00
Desai, Kartikey H 998b4c0725 Change streamlined_obj_vals list to streamlined_object dict 2020-05-13 12:45:16 -05:00
Desai, Kartikey H 9ce299b660 Fixes #389 2020-05-13 11:40:37 -05:00
Desai, Kartikey H 65d4060e6a Fixes #389 2020-05-13 11:23:26 -05:00
Desai, Kartikey H 0b1297b14a Fixes #389 2020-05-13 11:22:51 -05:00
Desai, Kartikey H de3fa99a12 Add property names to canonicalization for deterministic id gen 2020-05-13 11:20:16 -05:00
Chris Lenk 31cb2f85be Bump version: 1.3.1 → 1.4.0 2020-04-03 17:44:52 -04:00
Chris Lenk df92770d25
Merge pull request #384 from oasis-open/365-versioned-classes
Validate custom type/property name formats
2020-04-03 17:30:24 -04:00
Emmanuelle Vargas-Gonzalez 2b0d63c4b1 update test_datastore_taxii.py conftest.py for latest changes in medallion. add extra data used by filter 2020-04-03 17:19:36 -04:00
Chris Lenk c7fb79d195 Fix some TAXII DataStore tests 2020-04-03 15:58:56 -04:00
Chris Lenk 9145bdf5e8
Merge pull request #374 from chisholm/version_precision
Support STIX 2.1 version precision
2020-04-03 15:52:42 -04:00
Chris Lenk 0d770972cf
Merge pull request #382 from oasis-open/more-pattern-tests
More pattern tests
2020-04-03 11:24:43 -04:00
Chris Lenk e730d45d44 Use DEFAULT_VERSION in create_pattern_object() 2020-04-03 10:45:36 -04:00
Chris Lenk 14540c0ea1 Clean up _register_* functions
Made them consistent with _register_observable_extension, by:
- moving validation logic there from _custom_*_builder functions
- using a new function for ensuring properties are dict-like
- using the library default spec version instead of None

Fix #371, fix #372, fix #373.
2020-04-02 14:15:45 -04:00
Chris Lenk bbf0f81d5f
Merge pull request #376 from khdesai/fix_issue_363
Fix existing tests and add new tests. Fixes #363
2020-04-02 13:42:02 -04:00
Chris Lenk d33adbcc71 Rename test files to align with module renaming 2020-04-02 08:22:49 -04:00
Chris Lenk 13cddf9d6d Move TypeProperty format checks to __init__
TypeProperty uses a fixed value, so check() was never called. This way
also runs the check at object registration time because the wrapper
creates an instance of TypeProperty and doesn't have to wait for the
object to be instantiated so clean() can be called.
Also fix some tests.
2020-04-02 08:17:34 -04:00
Chris Lenk 03cb225932 Merge branch 'master' into 365-versioned-classes 2020-04-02 06:02:20 -04:00
Chris Lenk 897e884217 Fix some testing 2020-04-02 04:46:11 -04:00
Chris Lenk c494a2e477 Use TypeProperty.clean() to verify type format 2020-04-01 21:52:04 -04:00
Desai, Kartikey H c911cff97f Add duplicate checking to markings and observable extensions, and fix some tests and add some tests. Fixes #363 2020-03-27 14:58:18 -04:00
Rich Piazza 1a2b1367cf flaky 2 2020-03-27 14:06:24 -04:00
Rich Piazza 9933f88975 few more pattern op tests 2020-03-27 13:59:03 -04:00
Rich Piazza e3ebb6393d flaky 2020-03-27 12:33:24 -04:00
Rich Piazza 202111acdf more pattern tests 2020-03-27 11:22:00 -04:00
Rich Piazza 46219bf072 add 2.1 links 2020-03-27 09:36:10 -04:00
Chris Lenk b4700e6d00 Fix import errors
And pin medallion version for testing.
2020-03-27 06:33:29 -04:00
Chris Lenk 50df6f1474 Rename core.py -> parsing.py 2020-03-27 05:53:39 -04:00
Chris Lenk 01ba190525 Reorganize bases, use isinstance to check version
Renamed STIXDomainObject -> _DomainObject.
Renamed STIXRelationshipObject -> _RelationshipObject.
2020-03-27 02:40:42 -04:00
Desai, Kartikey H a7e9a7dde5 Merge branch 'master' of https://github.com/oasis-open/cti-python-stix2 into fix_issue_363 2020-03-26 23:27:51 -04:00
maxbachmann d11b812a0e
round result 2020-03-26 20:28:09 +01:00
maxbachmann 9388a9820f
use rapidfuzz instead of fuzzywuzzy 2020-03-23 02:16:36 +01:00
Chris Lenk e8035863b8
Make swid an id-contributing property 2020-03-21 23:56:09 -04:00
Chris Lenk e31634c32b Rework spec version detection for _STIXBase objs 2020-03-21 22:22:36 -04:00
Desai, Kartikey H 1a1ad90388 Fixes #363 2020-03-20 17:37:15 -04:00
Desai, Kartikey H b06bc1afc1 Fix import issues 2020-03-20 17:32:18 -04:00
Desai, Kartikey H f37b84a564 Pull in updates from master 2020-03-20 16:52:21 -04:00
Desai, Kartikey H 1260c7b45e Fix existing tests and add new tests. Fixes #363 2020-03-20 16:49:20 -04:00
Rich Piazza 2dea4caf00 fix re so they begin with ^ 2020-03-20 14:24:16 -04:00
Rich Piazza d8a9fc2306 flaky 2020-03-20 13:15:42 -04:00
Rich Piazza 9e5e998c3d don't allow leading '_' on custom properties, whenever allow_custom is true 2020-03-20 12:49:20 -04:00
Rich Piazza 2c4e47de56 remove leading - from type name re 2020-03-20 11:56:09 -04:00
Rich Piazza 6e4151aeeb flaky 2020-03-19 16:49:46 -04:00
Rich Piazza fe919049b8 fix marking test 2020-03-19 16:43:37 -04:00
Rich Piazza f60e4170fd finish 365 2020-03-19 16:11:52 -04:00
Rich Piazza 844ec2c3bf more on issue 365 2020-03-19 14:16:48 -04:00
Rich Piazza 9699c78ad8 issue-365 2020-03-19 10:40:35 -04:00
Michael Chisholm 1741cc9f6b Fix import sort order for the import sorter precommit hook 2020-03-17 20:26:21 -04:00
Michael Chisholm 6f43814918 Add xfail mark to a unit test which trips a Python 3.6 bug.
https://bugs.python.org/issue32404
2020-03-17 20:21:09 -04:00
Michael Chisholm f99665f2ba One more comma, because python 3.8's add-trailing-comma
pre-commit hook doesn't add all the commas Travis's hook
script expects...
2020-03-17 19:45:39 -04:00
Michael Chisholm cf9aef59c2 More flake8 style fixes 2020-03-17 18:28:38 -04:00
Michael Chisholm a9ac7ce838 pre-commit hook changes, e.g. trailing commas, import sorting,
flake8 style.
2020-03-17 18:26:57 -04:00
Michael Chisholm 4aa69fa7c9 Add support for enforcing STIX 2.1 minimum precision requirement
on versioning timestamps.
2020-03-16 20:25:38 -04:00
Chris Lenk 6842abb371
Merge pull request #370 from chisholm/observable_extension_names_ext
New STIX 2.1 SCO extension name requirement: must end with "-ext"
2020-03-12 17:26:26 -04:00
Michael Chisholm 15316e7933 Added "x-" to SCO extension names in unit tests, to illustrate
best practice and follow a spec "should" rule.
2020-03-12 16:20:32 -04:00
Chris Lenk 3dda25e976
Merge pull request #362 from chisholm/file_id_contrib_props
Add parent_directory_ref as an ID contributing property for file SCOs
2020-03-12 09:36:16 -04:00
Chris Lenk 5abc139e79 Merge branch 'khdesai-fix_issue_338'; Close #347 2020-03-12 09:30:52 -04:00
Chris Lenk 3dd9351d38 Bring back lang, confidence for Course of Action 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 82517ae284 Fixes #338 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 8885a757cb Fix properties spec version back to 2.1, and re-adjust tests. Fixes #338 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 36f7035785 Fixes #338 2020-03-12 09:24:43 -04:00
Chris Lenk e782d095ea
Merge pull request #369 from chisholm/malware_os_refs
Change software SCO: os_execution_envs -> operating_system_refs
2020-03-11 23:47:14 -04:00
Chris Lenk 94e3cd7ca6
Merge pull request #360 from chisholm/enforce_hash_keys
Enforce hash keys on 2.1 external-references
2020-03-11 23:13:55 -04:00
Chris Lenk 87c5ef30ad
Merge pull request #358 from chisholm/software_cpe_swid
Add swid property to the software SCO
2020-03-11 23:03:18 -04:00
Michael Chisholm 2472af387b Change a SWID tagId in a unit test from a UUID to something
more plausible.
2020-03-11 15:21:34 -04:00
Chris Lenk 33fb31421b
Merge pull request #357 from chisholm/malware_analysis_result
Update malware-analysis SDO's av_result property
2020-03-11 09:16:34 -04:00
Chris Lenk bdf7cab8fe
Merge pull request #356 from chisholm/malware_analysis_sample_ref
Add the "sample_ref" property to malware-analysis SDOs
2020-03-11 09:12:02 -04:00
Chris Lenk 2429533e4f
Merge pull request #355 from chisholm/optional_type_properties
Changed several *_types properties to be optional due to STIX spec change
2020-03-11 09:08:42 -04:00
Michael Chisholm 371bf0b9a4 Add trailing commas for git commit hook... 2020-03-10 21:21:53 -04:00
Michael Chisholm d708537b85 Add enforcement of a new STIX 2.1 SCO extension name requirement:
that it must end with "-ext".
2020-03-10 20:24:53 -04:00
Michael Chisholm 792cc570d7 Change the os_execution_envs property of software SCOs to
operating_system_refs, and add a test for it.
2020-03-06 19:43:47 -05:00
Chris Lenk 380926cff5 Bump version: 1.3.0 → 1.3.1 2020-03-06 09:50:09 -05:00
Michael Chisholm e32b074bc9 Fix stylistic issues for pre-commit hooks. 2020-03-05 17:39:35 -05:00
Michael Chisholm 22f2b241a7 Add a missing required property to fix up an external-reference
test.
2020-03-05 17:38:03 -05:00
Michael Chisholm a862b930be Add parent_directory_ref as an ID contributing property for the
file SCO.
2020-03-05 17:18:32 -05:00
Chris Lenk 3803e4bdd7
Merge pull request #343 from chisholm/sco_tlo_filesystemstore
Fix the filesystem store to support the new top-level 2.1 SCOs.
2020-03-05 17:08:20 -05:00
Chris Lenk cdde664434
Merge branch 'master' into fix_indicator_test 2020-03-05 10:51:03 -05:00
Michael Chisholm a5dc514403 Fix external-references to force hash keys to come from
hash-algorithm-ov.
2020-03-04 20:55:52 -05:00
Desai, Kartikey H a5cd0fdc50 Change location of None-check for id_contrib_props. Fixes #351 2020-03-04 14:46:55 -05:00
Desai, Kartikey H fc95b400ff Change default parameters from empty lists to None. Fixes #351 2020-03-04 14:29:35 -05:00
Desai, Kartikey H 8810983ca0 Merge branch 'master' of https://github.com/oasis-open/cti-python-stix2 into fix_issue_351 2020-03-04 14:16:54 -05:00
Chris Lenk 30a59ad776
Merge pull request #344 from chisholm/fix_ast_builder
Fix the pattern AST creation function
2020-03-04 13:49:16 -05:00
Chris Lenk 4f00c7ca4f Fix patterning test 2020-03-04 13:33:54 -05:00
Michael Chisholm 4e2b018272 Add a property to the software SCO, due to STIX spec change. 2020-03-02 16:57:18 -05:00
Michael Chisholm d2bff4d411 Update malware-analysis SDO's av_result property: replace it with
result and result_name properties.  Per:
https://github.com/oasis-tcs/cti-stix2/issues/213
2020-02-27 17:26:04 -05:00
Michael Chisholm 50eb188190 Add the "sample_ref" property to malware-analysis SDOs, per:
https://github.com/oasis-tcs/cti-stix2/issues/210
2020-02-27 16:40:56 -05:00
Desai, Kartikey H 055ad97a7a Add tests for _id_contributing_properties for custom observables 2020-02-27 15:15:37 -05:00
Michael Chisholm 93a8caa09d Remove unused import 2020-02-25 20:19:30 -05:00
Michael Chisholm 31c37a9b12 Changed several *_types properties which were formerly required,
to be optional, due to a STIX spec change.  Updated unit tests
accordingly.
2020-02-25 20:07:47 -05:00
Desai, Kartikey H 41e541959d Add _id_contributing_properties functionality to custom SCOs. Tests coming soon. Fixes #351 2020-02-24 21:11:42 -05:00
Michael Chisholm 274abc52e9 An exception message changed as a result of a pattern-validator
update.  This broke a unit test which was testing the message.
I updated the test.
2020-02-24 20:02:26 -05:00
Desai, Kartikey H a0a8b7d0e1 Fixes #334 2020-02-21 15:40:38 -05:00
Desai, Kartikey H 796a4e20fa Correct bug in recursive dict loop. Fixes #334 2020-02-21 15:26:19 -05:00
Michael Chisholm 1959cc6097 Removed a bunch of no-longer-used imports from pattern_visitor.py 2020-02-19 16:45:15 -05:00
Michael Chisholm 76a6eb5873 Greatly simplify the create_pattern_object() function to take
advantage of the pattern validator library's Pattern.visit()
method.
2020-02-19 16:39:15 -05:00
Desai, Kartikey H 1084c75d33 Fixes #334 2020-02-19 16:29:13 -05:00
Michael Chisholm 14daa1edae Add a test case to test parse exceptions from
create_pattern_object().
2020-02-19 15:39:23 -05:00
Desai, Kartikey H 8219b34ea4 Fix formatting issues. Fixes #336 2020-02-19 09:24:27 -05:00
Desai, Kartikey H 86f9e51a42 Fixes #336 2020-02-19 09:11:30 -05:00
Michael Chisholm cfb7c4c73b Fix stix2.pattern_visitor.create_pattern_object() so its
documentation at least isn't wrong, and it behaves better.
2020-02-17 19:26:21 -05:00
Michael Chisholm 4c67142b92 Fix the filesystem store to support the new top-level 2.1 SCOs. 2020-02-15 19:02:53 -05:00
Michael Chisholm be5274878d Add trailing commas for pre-commit hook... 2020-02-13 17:37:59 -05:00
Michael Chisholm 98a654884d Fix the memory store to support the new top-level 2.1 SCOs. 2020-02-13 17:11:58 -05:00