Commit Graph

1157 Commits (6c92f670cb824700c34e04b1493215cd5aaae80d)

Author SHA1 Message Date
Emmanuelle Vargas-Gonzalez aa18575c83 add `id` as special-case option in regex. closes #406 2020-06-08 09:27:12 -04:00
Michael Chisholm c13cb182db versioning related updates:
- remove var 'properties_to_change' since it's unnecessary
- delete most of remove_custom_stix() since it was unnecessary,
  greatly simplify it
2020-06-05 14:24:16 -04:00
Michael Chisholm 9c5f950d5b Add a new_version() restriction preventing SCO ID contributing
properties from being changed, if a UUIDv5 is in use.  Changing
one of those properties would imply an ID change, which is not
allowed across versions.  Also:

- add a trailing comma
- change unchangable_properties to a set instead of a list,
  in case there are dupe props between STIX_UNMOD_PROPERTIES and
  sco_locked_props
2020-06-05 14:24:16 -04:00
Michael Chisholm eaa7f17ee4 Fix versionability detection: _properties is an OrderedDict not
a list of tuples! (despite what it looks like if you glance at it
quickly.)
2020-06-05 14:24:16 -04:00
Michael Chisholm 15e75cb4de Python 2 compatibility fix in versionability detection. In
python2, Mapping.keys() returns a list instead of a set!
2020-06-05 14:24:16 -04:00
Michael Chisholm a05dbc42d5 Improve the docstring for new_version(). Especially with the
three-value logic for allow_custom, it needed some much better
documentation.
2020-06-05 14:24:16 -04:00
Michael Chisholm d4cecd6f51 In remove_custom_stix(), when creating the new version,
use allow_custom=False to force custom properties to be
disallowed.  This is extra insurance against a custom prop
accidentally getting through.
2020-06-05 14:24:16 -04:00
Michael Chisholm 78d3b2c4c5 Add stix2.utils import to v20/test_versioning.py since the module
was used but not imported there.  Also import sorting from the
pre-commit hook.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5522050899 Improve versionability detection when creating new versions
of objects.  Move versioning-related stuff out of stix2.utils
to its own module.  Some misc versioning-related fixes.
2020-06-05 14:24:16 -04:00
Michael Chisholm 5a5484d464 Move/merge some deterministic ID tests from the observed-data SDO
test suite to the test suite specific to testing deterministic
IDs.  This keeps the tests for that specific system in one place.
2020-06-05 14:14:44 -04:00
Michael Chisholm f17f782134 Bugfixes to the json un-escaping function. Added some unit tests
for that.
2020-06-05 14:14:44 -04:00
Michael Chisholm 2179028c91 Fix deterministic ID behavior when an empty "hashes" property is
given.  Now, an exception is raised.  Added a unit test for this.
2020-06-05 14:14:44 -04:00
Michael Chisholm c94feae623 Add a deterministic ID unit test with an embedded object 2020-06-05 14:14:43 -04:00
Michael Chisholm fa981affa9 stupid python 3.8 pre-commit still not adding the right trailing
commas....
2020-06-05 14:14:43 -04:00
Michael Chisholm 9dcffd9fad Add some unit tests for deterministic IDs. 2020-06-05 14:14:43 -04:00
Michael Chisholm 1f5e8defd3 Argh, I fixed an RFC URL but forgot to fix the comment which
described it and gave the RFC number...
2020-06-05 14:14:43 -04:00
Michael Chisholm 22c0d5806c Trailing comma for the pre-commit hook 2020-06-05 14:14:43 -04:00
Michael Chisholm e22c37a16b Update a RFC URL to the latest version. I'd accidentally used
one which had been obsoleted by a newer one.
2020-06-05 14:14:43 -04:00
Michael Chisholm 6c2c4781e7 Revamp deterministic ID generation code to fix bugs. 2020-06-05 14:14:43 -04:00
Chris Lenk 41525f9be0
Merge pull request #400 from oasis-open/issue-398
fixed start/stop qualifier constant issue
Fixes #398.
2020-06-05 10:57:59 -04:00
Rich Piazza 53dfe40c30 only test start/stop timestamps if they are strings (2.0) 2020-06-05 09:25:06 -04:00
Rich Piazza 04af273d71 flaky 2020-06-05 09:17:42 -04:00
Michael Chisholm 7def04fd6f Add __all__'s to v20 and v21/__init__.py to ensure that
star-imports from those modules get the right symbols.  "base"
was a problem because it was getting imported via
stix2/__init__.py and overwriting stix2.base, making the actual
stix2/base.py module inaccessible.
2020-06-03 17:39:09 -04:00
Rich Piazza 977107e713 2.0 Start/Stop Quailifer uses StringConstants 2020-06-01 22:34:40 -04:00
Rich Piazza cfe0648c98 fixed start/stop qualifier constant issue 2020-05-26 15:12:02 -04:00
Michael Chisholm 7955a41997 Drop python-dateutil as a dependency and switch to the builtin
datetime module for parsing timestamps.  Dateutil proved too
slow.
2020-05-20 15:06:53 -04:00
Emmanuelle Vargas-Gonzalez b4dbc419f6 Merge branch 'master' of github.com:oasis-open/cti-python-stix2 into 391-ssdeep-hash-case 2020-05-14 12:51:02 -04:00
Emmanuelle Vargas-Gonzalez 68f7ca6377 resolve problem with SSDEEP vocab use for 2.1, closes #391 2020-05-13 18:17:17 -04:00
Desai, Kartikey H 998b4c0725 Change streamlined_obj_vals list to streamlined_object dict 2020-05-13 12:45:16 -05:00
Desai, Kartikey H 9ce299b660 Fixes #389 2020-05-13 11:40:37 -05:00
Desai, Kartikey H 65d4060e6a Fixes #389 2020-05-13 11:23:26 -05:00
Desai, Kartikey H 0b1297b14a Fixes #389 2020-05-13 11:22:51 -05:00
Desai, Kartikey H de3fa99a12 Add property names to canonicalization for deterministic id gen 2020-05-13 11:20:16 -05:00
Chris Lenk 31cb2f85be Bump version: 1.3.1 → 1.4.0 2020-04-03 17:44:52 -04:00
Chris Lenk df92770d25
Merge pull request #384 from oasis-open/365-versioned-classes
Validate custom type/property name formats
2020-04-03 17:30:24 -04:00
Emmanuelle Vargas-Gonzalez 2b0d63c4b1 update test_datastore_taxii.py conftest.py for latest changes in medallion. add extra data used by filter 2020-04-03 17:19:36 -04:00
Chris Lenk c7fb79d195 Fix some TAXII DataStore tests 2020-04-03 15:58:56 -04:00
Chris Lenk 9145bdf5e8
Merge pull request #374 from chisholm/version_precision
Support STIX 2.1 version precision
2020-04-03 15:52:42 -04:00
Chris Lenk 0d770972cf
Merge pull request #382 from oasis-open/more-pattern-tests
More pattern tests
2020-04-03 11:24:43 -04:00
Chris Lenk e730d45d44 Use DEFAULT_VERSION in create_pattern_object() 2020-04-03 10:45:36 -04:00
Chris Lenk 14540c0ea1 Clean up _register_* functions
Made them consistent with _register_observable_extension, by:
- moving validation logic there from _custom_*_builder functions
- using a new function for ensuring properties are dict-like
- using the library default spec version instead of None

Fix #371, fix #372, fix #373.
2020-04-02 14:15:45 -04:00
Chris Lenk bbf0f81d5f
Merge pull request #376 from khdesai/fix_issue_363
Fix existing tests and add new tests. Fixes #363
2020-04-02 13:42:02 -04:00
Chris Lenk d33adbcc71 Rename test files to align with module renaming 2020-04-02 08:22:49 -04:00
Chris Lenk 13cddf9d6d Move TypeProperty format checks to __init__
TypeProperty uses a fixed value, so check() was never called. This way
also runs the check at object registration time because the wrapper
creates an instance of TypeProperty and doesn't have to wait for the
object to be instantiated so clean() can be called.
Also fix some tests.
2020-04-02 08:17:34 -04:00
Chris Lenk 03cb225932 Merge branch 'master' into 365-versioned-classes 2020-04-02 06:02:20 -04:00
Chris Lenk 897e884217 Fix some testing 2020-04-02 04:46:11 -04:00
Chris Lenk c494a2e477 Use TypeProperty.clean() to verify type format 2020-04-01 21:52:04 -04:00
Desai, Kartikey H c911cff97f Add duplicate checking to markings and observable extensions, and fix some tests and add some tests. Fixes #363 2020-03-27 14:58:18 -04:00
Rich Piazza 1a2b1367cf flaky 2 2020-03-27 14:06:24 -04:00
Rich Piazza 9933f88975 few more pattern op tests 2020-03-27 13:59:03 -04:00
Rich Piazza e3ebb6393d flaky 2020-03-27 12:33:24 -04:00
Rich Piazza 202111acdf more pattern tests 2020-03-27 11:22:00 -04:00
Rich Piazza 46219bf072 add 2.1 links 2020-03-27 09:36:10 -04:00
Chris Lenk b4700e6d00 Fix import errors
And pin medallion version for testing.
2020-03-27 06:33:29 -04:00
Chris Lenk 50df6f1474 Rename core.py -> parsing.py 2020-03-27 05:53:39 -04:00
Chris Lenk 01ba190525 Reorganize bases, use isinstance to check version
Renamed STIXDomainObject -> _DomainObject.
Renamed STIXRelationshipObject -> _RelationshipObject.
2020-03-27 02:40:42 -04:00
Desai, Kartikey H a7e9a7dde5 Merge branch 'master' of https://github.com/oasis-open/cti-python-stix2 into fix_issue_363 2020-03-26 23:27:51 -04:00
maxbachmann d11b812a0e
round result 2020-03-26 20:28:09 +01:00
maxbachmann 9388a9820f
use rapidfuzz instead of fuzzywuzzy 2020-03-23 02:16:36 +01:00
Chris Lenk e8035863b8
Make swid an id-contributing property 2020-03-21 23:56:09 -04:00
Chris Lenk e31634c32b Rework spec version detection for _STIXBase objs 2020-03-21 22:22:36 -04:00
Desai, Kartikey H 1a1ad90388 Fixes #363 2020-03-20 17:37:15 -04:00
Desai, Kartikey H b06bc1afc1 Fix import issues 2020-03-20 17:32:18 -04:00
Desai, Kartikey H f37b84a564 Pull in updates from master 2020-03-20 16:52:21 -04:00
Desai, Kartikey H 1260c7b45e Fix existing tests and add new tests. Fixes #363 2020-03-20 16:49:20 -04:00
Rich Piazza 2dea4caf00 fix re so they begin with ^ 2020-03-20 14:24:16 -04:00
Rich Piazza d8a9fc2306 flaky 2020-03-20 13:15:42 -04:00
Rich Piazza 9e5e998c3d don't allow leading '_' on custom properties, whenever allow_custom is true 2020-03-20 12:49:20 -04:00
Rich Piazza 2c4e47de56 remove leading - from type name re 2020-03-20 11:56:09 -04:00
Rich Piazza 6e4151aeeb flaky 2020-03-19 16:49:46 -04:00
Rich Piazza fe919049b8 fix marking test 2020-03-19 16:43:37 -04:00
Rich Piazza f60e4170fd finish 365 2020-03-19 16:11:52 -04:00
Rich Piazza 844ec2c3bf more on issue 365 2020-03-19 14:16:48 -04:00
Rich Piazza 9699c78ad8 issue-365 2020-03-19 10:40:35 -04:00
Michael Chisholm 1741cc9f6b Fix import sort order for the import sorter precommit hook 2020-03-17 20:26:21 -04:00
Michael Chisholm 6f43814918 Add xfail mark to a unit test which trips a Python 3.6 bug.
https://bugs.python.org/issue32404
2020-03-17 20:21:09 -04:00
Michael Chisholm f99665f2ba One more comma, because python 3.8's add-trailing-comma
pre-commit hook doesn't add all the commas Travis's hook
script expects...
2020-03-17 19:45:39 -04:00
Michael Chisholm cf9aef59c2 More flake8 style fixes 2020-03-17 18:28:38 -04:00
Michael Chisholm a9ac7ce838 pre-commit hook changes, e.g. trailing commas, import sorting,
flake8 style.
2020-03-17 18:26:57 -04:00
Michael Chisholm 4aa69fa7c9 Add support for enforcing STIX 2.1 minimum precision requirement
on versioning timestamps.
2020-03-16 20:25:38 -04:00
Chris Lenk 6842abb371
Merge pull request #370 from chisholm/observable_extension_names_ext
New STIX 2.1 SCO extension name requirement: must end with "-ext"
2020-03-12 17:26:26 -04:00
Michael Chisholm 15316e7933 Added "x-" to SCO extension names in unit tests, to illustrate
best practice and follow a spec "should" rule.
2020-03-12 16:20:32 -04:00
Chris Lenk 3dda25e976
Merge pull request #362 from chisholm/file_id_contrib_props
Add parent_directory_ref as an ID contributing property for file SCOs
2020-03-12 09:36:16 -04:00
Chris Lenk 5abc139e79 Merge branch 'khdesai-fix_issue_338'; Close #347 2020-03-12 09:30:52 -04:00
Chris Lenk 3dd9351d38 Bring back lang, confidence for Course of Action 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 82517ae284 Fixes #338 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 8885a757cb Fix properties spec version back to 2.1, and re-adjust tests. Fixes #338 2020-03-12 09:24:43 -04:00
Desai, Kartikey H 36f7035785 Fixes #338 2020-03-12 09:24:43 -04:00
Chris Lenk e782d095ea
Merge pull request #369 from chisholm/malware_os_refs
Change software SCO: os_execution_envs -> operating_system_refs
2020-03-11 23:47:14 -04:00
Chris Lenk 94e3cd7ca6
Merge pull request #360 from chisholm/enforce_hash_keys
Enforce hash keys on 2.1 external-references
2020-03-11 23:13:55 -04:00
Chris Lenk 87c5ef30ad
Merge pull request #358 from chisholm/software_cpe_swid
Add swid property to the software SCO
2020-03-11 23:03:18 -04:00
Michael Chisholm 2472af387b Change a SWID tagId in a unit test from a UUID to something
more plausible.
2020-03-11 15:21:34 -04:00
Chris Lenk 33fb31421b
Merge pull request #357 from chisholm/malware_analysis_result
Update malware-analysis SDO's av_result property
2020-03-11 09:16:34 -04:00
Chris Lenk bdf7cab8fe
Merge pull request #356 from chisholm/malware_analysis_sample_ref
Add the "sample_ref" property to malware-analysis SDOs
2020-03-11 09:12:02 -04:00
Chris Lenk 2429533e4f
Merge pull request #355 from chisholm/optional_type_properties
Changed several *_types properties to be optional due to STIX spec change
2020-03-11 09:08:42 -04:00
Michael Chisholm 371bf0b9a4 Add trailing commas for git commit hook... 2020-03-10 21:21:53 -04:00
Michael Chisholm d708537b85 Add enforcement of a new STIX 2.1 SCO extension name requirement:
that it must end with "-ext".
2020-03-10 20:24:53 -04:00
Michael Chisholm 792cc570d7 Change the os_execution_envs property of software SCOs to
operating_system_refs, and add a test for it.
2020-03-06 19:43:47 -05:00
Chris Lenk 380926cff5 Bump version: 1.3.0 → 1.3.1 2020-03-06 09:50:09 -05:00
Michael Chisholm e32b074bc9 Fix stylistic issues for pre-commit hooks. 2020-03-05 17:39:35 -05:00