* [tests] Quick fix on embedded galaxies in attributes tests, as the opposite references handling creates uncertainty in relationships order. [Christian Studer]
* [stix2 import] Removed the UUID handling methods in the parsers directory to keep using the original ones from `importparser` as a MISP event also need some of those methods. [Christian Studer]
* [tests] Fixed STIX 2.0 test method names. [Christian Studer]
* [stix2 export] Some more pycodestyle to make the mapping cleaner. [Christian Studer]
* [stix2 import] Some quick pycodestyle to make the mapping cleaner. [Christian Studer]
* [stix2 import] Fixed debugging messages handling in the command-line feature. [Christian Studer]
### Other
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'parser_feature' of github.com:MISP/misp-stix into dev. [Christian Studer]
* Merge branch 'main' of github.com:misp/misp-stix into parser_feature. [Christian Studer]
* Merge branch 'dev' of github.com:misp/misp-stix into parser_feature. [Christian Studer]
* [stix import] Simplified data path. [Christian Studer]
* [tests] Updated tests for sightings import. [Christian Studer]
- Also changed some sample to have different order
with Identity objects in order to test properly
our recent changes on loading and converting the
sightings, which purpose was to avoid issues
with the STIX objects order
* [stix2 import] Updated the External STIX 2 Identity objects mapping to MISP `organization` objects following recent updates on that template. [Christian Studer]
* [tests] Removed some results writing in files which were used at some point for debugging purposes and forgotten in the code vastness. [Christian Studer]
* [stix2 import] Shorter obervable types extraction while still including the recent fix to avoid issues with observables that are of `dict` type. [Christian Studer]
* [stix1 export] Fixed backward compatibility with old object templates. [Christian Studer]
* [stix2 export] Fixed backward compatibility with old object templates. [Christian Studer]
* [stix import] Sanitised the import variables declaration to avoid issue with wrong value format. [Christian Studer]
* [stix2 import] Better `Identity` object's identity class field handling. [Christian Studer]
* [stix2 import] Using the Galaxy Cluster adding method to add cluster instead of appending it the the list of clusters. [Christian Studer]
### Other
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:misp/misp-stix. [Christian Studer]
* Merge pull request #45 from SYNchroACK/fix/wrong-import. [Christian Studer]
Fix wrong stix observables import
* Fix wrong stix observables import. [Tomas Lima]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix into dev. [Christian Studer]
* Wip: [stix2 import] Updated the import conversion of internal STIX 2.x Identity objects to better support recent changes on the `organization` template. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge pull request #46 from dragsu/fix-observable-access-dict-type. [Christian Studer]
fix: `type` access for dict type Observables
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Add: [stix2 import] Importing Identity objects with `identity_class` set to organization as `organization` object. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix into dev. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'main' of github.com:MISP/misp-stix into dev. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* [stix1 export] Fixed Package header writting for methods used to replicate the MISP pagination - used with collections export helpers. [Christian Studer]
* [stix1 export] Reusing methods from the framing to generate packages (& handling namespaces) [Christian Studer]
* [stix1 export] Handling cases when there is no STIX header. [Christian Studer]
- In this specific case, the STIX package in XML
format is a single xml tag with the included
`/` closing character... so we remove it
- ( JSON >>>>> XML definitely :) )
* [stix1 export] Added option to generate a Package with no header. [Christian Studer]
* [stix1 export] Fixed the creation process of the STIX package used to serve as container for related packages. [Christian Studer]
* [stix export] Made STIX framing methods more modular. [Christian Studer]
* [stix2 export] Returning the result files in a traceback message as list. [Christian Studer]
* [stix2 export] Fixed some statements in the MISP collections export to STIX 2 helper. [Christian Studer]
- Including fixes on:
- the single file handling (regarding the single
file name)
- the default directory for collections export
results
- the input files argument of the function
* [stix1 export] Fixed arguments passed to the MISP collections export to STIX 1. [Christian Studer]
* [stix1 export] Added a use case to support the use of the events collection export even with a single file. [Christian Studer]
* [stix1 export] Fixed name for the result STIX 1 event collections export & added a missing traceback. [Christian Studer]
* [stix1 export] Making sure we avoid exceptions with the fails catching on traceback messages. [Christian Studer]
* [stix2 import] Better handling of the `single_event` variable inside of the STIX 2 to MISP parser. [Christian Studer]
* [stix2 import] Added missing `campaign` type in the list of STIX object types to look for. [Christian Studer]
* [stix2 import] Fixed the observable registry key values parsing in case of a single key imported as `regkey|value` attribute. [Christian Studer]
* [stix2 import] Catching parsing issues that appear while the STIX file is loaded. [Christian Studer]
* [stix export] Galaxies mapping are now also using the uninstantiated mapping classmethods. [Christian Studer]
* [tests] Using the uninstantiated mapping classes with their classmethods. [Christian Studer]
* [stix2 import] Fixed the `from_misp` test that defines whether a STIX file has been generated with the MISP to STIX conversion feature or not. [Christian Studer]
* [stix2 import] Fixed the email or IP address observable objects from internal STIX content parsing. [Christian Studer]
- Could fail previously with some content generated
from a previous version of the MISP to STIX
conversion feature
* [stix2 import] Fixed marking definition parsing, as we store the tag and not the marking definition object. [Christian Studer]
* [tests] Fixed tests to avoid issues with STIX 2 to MISP mappings, following the recent changes on them. [Christian Studer]
* [stix2 import] Revert change to fix the pattern assertion operator check. [Christian Studer]
- Revert of a part of the code that was staged for
a previous commit while it should not have been
- For now the pattern assertion check will remain
as is even tough there is an ongoing work to
improve it.
* [stix2 import] Fixed missing variable name change. [Christian Studer]
* [stix2 import] Using non instantiated external STIX 2 to MISP mapping. [Christian Studer]
- As they were missing in the export mapping, they
were exported as custom objects, but we simply
have to take the pattern and export it as is,
like we do for sigma or yara patterns for
instance in STIX 2.1
- In this case, it applies to both STIX 2.0 & 2.1
* [stix2 export] Made the `created` & `modified` fields in custom galaxy objects optional. [Christian Studer]
* [stix2 export] Using the property for `identity_id` instead of the 'private' variable. [Christian Studer]
* [stix2 export] Same as the previous commit, for standalone attributes from feeds. [Christian Studer]
* [stix2 export] Fixed the orgc parsing for attributes collections. [Christian Studer]
- The `created_by_ref` values were missing on all
objects because the statement used to wait for
a value where the recent changes made the
related method return nothing anymore
* [stix2 export] Better Orgc & info handling for instance when they are empty. [Christian Studer]
* [stix2 export] Avoiding issues with unset `timestamp` value in MISP Event. [Christian Studer]
* [stix2 export] Checking `Orgc` fields before trying to generate the Identity object which will be used as `created_by_ref` object reference. [Christian Studer]
* [stix2 import] To avoid any possible issue in MISP with float version, we just made the generic Galaxies & Clusters version int. [Christian Studer]
* [tests] Fixed Galaxies & Clusters tests following all the recent changes on generic conversion from STIX 2.0 & 2.1. [Christian Studer]
* [stix2 import] Fixed the galaxy creation method for external STIX content to avoid issues with `region` and `country` galaxies. [Christian Studer]
* [stix2 import] Fixed the clusters creation method to avoid issues with unassigned cluster value. [Christian Studer]
* [stix2 import] Added missing `self` param in the clusters creation method. [Christian Studer]
* [stix2 import] Syntax fixed in f-string. [Christian Studer]
* [stix2 import] The Galaxy args creation is better and handles some of the formerly missing required field to validate a Galaxy in MISP. [Christian Studer]
* [stix2 import] Quick improvement on a `hasattr` that can be directly replaced by a `getattr` with a default value. [Christian Studer]
* [stix2 import] Fixed the generic info method. [Christian Studer]
- The way it is implemented, it has to be a
property rather than a classmethod in order to
avoid the info field to be null because as a
classmethod, the returned value was a bound
method
### Other
* Merge branch 'main' of github.com:MISP/misp-stix. [Christian Studer]
* Merge pull request #42 from MISP/dev. [Christian Studer]
A few changes and improvement
* Merge branch 'main' of github.com:MISP/misp-stix into dev. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Merge branch 'dev' of github.com:MISP/misp-stix. [Christian Studer]
* Fix; [stix export] Fixed fail messages as the command line feature wants lists. [Christian Studer]
* Fix; [stix1 export] Fixed the input files argument for the collections export as STIX 1 helpers. [Christian Studer]
* [misp_stix_converter] Added quick comments & made the `_from_misp` utility available to import from the library. [Christian Studer]
* [misp_stix_converter] Moved the command line feature to `misp_stix_converter.py` to avoid all the related utility functions to be exposed while importing the python library. [Christian Studer]
* [stix2 import] Using the `from_dict` method as much as possible to populate the different MISP Object or Event fields. [Christian Studer]
- It introduces some changes on the format of the
datetime fields which are now properly defined
as datetime with the right format and the
timezone info
* [stix2 import] Extracted the object case handling to make it callable. [Christian Studer]
* [stix2 import] Better STIX objects as Galaxy import handling. [Christian Studer]
- Instead of testing if we have to import the
tag names or the full Galaxy object each time
we parse a single STIX object, we set a variable
from the beginning to redirect to the related
parsing function
### Fix
* [stix2 export] Export the `source` of a sighting as `x_misp_source` as defined in the Custom STIX 2.0 object. [Christian Studer]
- Fixes #28
* [stix2 import] Fixed Galaxy parsing as tag names variable typo. [Christian Studer]
* [misp_stix_converter] Better output names handling. [Christian Studer]
* [misp_stix_converter] Some clean-up. [Christian Studer]
* [stix2 import] Added the missing `entrypoin-address` attribute. [Christian Studer]
* [stix2 import] Making sure we won't have MISP objects rejected for having the same UUID. [Christian Studer]
- `pe` & `pe-section` objects are converted from
the same observable object or pattern as the
`file` object that contains them.
If we create the different MISP objects the same
way we do for the file, they will all have the
same UUID and MISP will reject them
* [tests] Updated tests to handle the recent changes on the datetime values format. [Christian Studer]
* [tests] Fixed tests for internal file with pe & sections objects following recent changes on the related parsing functions. [Christian Studer]
* [stix2 import] Fixed `_add_misp_attribute` function called names. [Christian Studer]
* [stix2 import] Updated the `process` object attributes used to force the MISP content being an object to align with the `requiredOneOf` field of the template. [Christian Studer]
* [stix2 import] Fixed STIX 2 Observable objects to MISP mapping for `Domain Name` with `Network Traffic` objects. [Christian Studer]
* [stix2 import] Fixed the internal STIX 2 objects conversion as MISP Galaxy. [Christian Studer]
- We have to check whether the `description` field
does contain the `|` as separation caracter,
because it is not the case for internal
`Identity` objects with the `identity_class`
field set to 'class' imported as `sector` galaxy
* [tests] Fixed the galaxies export tests to avoid issues with potential missing `description` & `meta` fields within the cluster definition. [Christian Studer]
* [stix2 export] Fixed the `sector` galaxy parsing to avoid issues with the `description` field within the galaxy cluster definition. [Christian Studer]
* [stix2 export] Making the sector galaxy export available for both STIX 2.0 & 2.1. [Christian Studer]
* [stix2 export] Using the STIX objects adding function instead of dealing with the private variable. [Christian Studer]
* [stix2 import] STIX 2 import mapping classes renames for more clarity. [Christian Studer]
* [tests] Fixed the tags test to go with the recent changes on some galaxy test samples. [Christian Studer]
* [tests] Added specific testing methods for clusters meta fields. [Christian Studer]
* [tests] Fixed tests for MISP galaxies export as STIX 2, following the recent updates and improvements on their parsing. [Christian Studer]
* [stix2 export] Fixed the `kill_chain` parsing in clusters meta fields. [Christian Studer]
* [stix2 export] Fixed one of the missing attack-pattern object creation that was missed and still using the previous creation function. [Christian Studer]
* [stix2 export] Removed no longer necessary argument of some STIX 2 object creation function. [Christian Studer]
- Which also made unnecessary some of thoses
functions being no longer specific to galaxies
* [stix2 import] Avoiding Custom Objects converted as Attributes to be modified while they are parsed. [Christian Studer]
* [stix2 import] Fixed some loading definitions. [Christian Studer]
* [stix2 import] Fixed variable that should not be self. [Christian Studer]
* [tests] Simply avoiding issues with the custom galaxies not exported in STIX 1 (for now at least) [Christian Studer]
* [tests] Added tests to make sure custom galaxies are correctly exported when embedded in attributes or object attributes. [Christian Studer]
* [stix2 export] Added the missing custom galaxies handler for attributes galaxies. [Christian Studer]
* [stix2 export] Reverted some try/catch bypass used for debugging purposes. [Christian Studer]
* [stix2 export] Clarification on some incomplete MISP Galaxies typing. [Christian Studer]
* [stix2 export] Quick fix & improvement on the custom galaxies export. [Christian Studer]
* [stix2 export] Simply a quick clean-up. [Christian Studer]
* [stix2 export] Fixing the `EventReport` references handling. [Christian Studer]
- When there is no actual reference to a MISP
attribute, object or galaxy in the Event report,
the `object_refs` field is empty, which is not
allowed, so we add a reference to the report or
grouping to avoid raising an exception
* [stix2 export] Fixing the `EventReport` references handling. [Christian Studer]
- When there is no actual reference to a MISP
attribute, object or galaxy in the Event report,
the `object_refs` field is empty, which is not
allowed, so we add a reference to the report or
grouping to avoid raising an exception
* [tests] Fixed tests for `registry-key` objects export as STIX 2.0 following the recent mapping change on the `last-modified` attribute. [Christian Studer]
* [stix2 export] Fixed the `registry-key` object mapping regarding the `last-modified` attribute export as STIX 2.0. [Christian Studer]
* [tests] Fixed tests for `registry-key` objects export as STIX 2.0 following the recent mapping change on the `last-modified` attribute. [Christian Studer]
* Wip: [stix2 import] Using the MISP Galaxy & Cluster classes to convert STIX objects meant to be galaxy clusters, and no longer using the tag names. [Christian Studer]
* Wip: [stix2 import] Removed the synonyms to tag_names mapping. [Christian Studer]
- We will now use the PyMISP classses to create
galaxies and clusters attached to the related
containers (Event & Attributes)
- The galaxies checking for existing galaxies and
references will be processed in MISP directly
* Wip: [stix2 import] Introducing a new way of parsing content converted into Galaxies. [Christian Studer]
* Wip: [stix2 export] More Hash values checking. [Christian Studer]
- We also check now Hash values in the case of a
conversion as Observable objects
* Wip: [stix2 export] Introducing a hash value checking function to avoid issues with invalid hashes. [Christian Studer]
* Wip: [stix2 import] Added some helpers to parse content in STIX 2 patterns. [Christian Studer]
- Loading patterns for now
## v2.4.162 (2022-09-19)
### Changes
* [package] Updated to latest version to publish. [Christian Studer]
* [stix2 export] Returning warning as a dictionary of lists instead of sets. [Christian Studer]
* [setup, poetry] Aligning with the package features that are actually used on pypi. [Christian Studer]
* [tests] Ported all STIX 1 export tests to support both JSON & MISP inputs. [Christian Studer]
* [stix2 export] Made the timestamp values checking common to all export classes and moved the test is the values are datetime to this common function. [Christian Studer]
* [tests] Duplicated tests for attributes, objects & galaxies export as STIX 2 to support both JSON & MISP input. [Christian Studer]
* [tests] Tests for interoperability & feeds now support both JSON and MISP inputs. [Christian Studer]
* [stix2 export] Added correct typing to functions receiving attributes, objects or events. [Christian Studer]
* [stix2 export] Added missing use case making available Attributes parsing in some situations while giving the input as file instead of as loaded dict. [Christian Studer]
- It avoids for instance issues with the command
line script when giving a file containing an
attributes collection
* [stix2 export] Fixed edge case when the `send-date` attribute within an `email` object is not a correctly formatted datetime value. [Christian Studer]
* [tests] Fixed tests for composite attributes exported as STIX 2 indicator that received a tiny change. [Christian Studer]
* [stix1 export] Fixed composite attribute values parsing to avoid issues with values not formatted the right way. [Christian Studer]
* [stix2 export] Fixed parsing of composite attributes which require some attribute type handling. [Christian Studer]
- The composite attribute type will indeed always
have the standard `|` as separator
* [stix2 export] Handling composite attribute values when they are not formatted as they should be with a `|` [Christian Studer]
* [stix2 export] Added the missing `interoperability` parameter in the Relationship object arguments. [Christian Studer]
* [stix2 export] Fixed `annotation` object export as STIX 2.1 when there is no object reference. [Christian Studer]
* [stix2 import] Fixed the `add_attribute` method that was missing the `**` prefix that is required when you pass a dict directly to it. [Christian Studer]
### Other
* Merge pull request #21 from netantho/patch-1. [Christian Studer]
Add setuptools as a build-system dependency
* Add setuptools as a build-system dependency. [Anthony VEREZ]
* Wip: [stix2 import] Better handling of external references from `attack-pattern` objects. [Christian Studer]
- Instead of having a common parsing function for
all STIX 2 attack pattern external references,
we parse those references depending on whether
it is external STIX data or not, to have 1 very
specific parsing function for content we know,
and a more flexible one for external content in
order to avoid issues with that kind of data
## v2.4.160 (2022-08-05)
### New
* [logo] new japanese misp-stix logo. [Alexandre Dulaunoy]
### Changes
* [poetry] Bumped latest dependencies version in lock file. [Christian Studer]
* [documentation] Added the new STIX -> MISP import mapping documentation. [Christian Studer]
* [tests, documentation] Updated the documentation auto-update in tests. [Christian Studer]
- MISP -> STIX export mapping documentation now
has a different structure from the STIX -> MISP
import mapping documentation, so we have in the
import documentation the difference between how
STIX content is converted into MISP data
depending on the STIX object type
* [documentation] Regenerated documentation with `sigma` objects supported in the STIX 2.1 export mapping & updates on the `yara` object mapping. [Christian Studer]
* [poetry] Bumped latest versions in lock file. [Christian Studer]
* [stix2 import] Making flake8 happy. [Christian Studer]
- Eventhough the `if pattern.startwith('(')` case
comes always first, flake8 does not like the
`reference` declaration statement being after
the other cases
* [tests] Fixed `first-packet-seen` attribute in `netflow` sample test object. [Christian Studer]
* [documentation] Updated documentation for `netflow` objects export as Indicator. [Christian Studer]
* [tests] Fixed tests for `netflow` objects export as Indicator to include the recent changes on protocols handling. [Christian Studer]
* [stix2 export] Converting protocol in lower case while exporting `netflow` objects as Indicator pattern. [Christian Studer]
* [documentation] Fixed mapping documentation for `netflow` objects export as STIX 2, following the recent changes on the related mapping. [Christian Studer]
* [stix2 export] Making sure we do not miss some required network-traffic fields if there is only the IP attribute(s) in the http-request object. [Christian Studer]
* [stix2 export] Quick change on some attributes parsing order for the `http-request` object parsing. [Christian Studer]
* [stix2 export] Making pycodestyle happy. [Christian Studer]
* [stix2 import] Added a few missing imports. [Christian Studer]
* [tests] Fixed confidence tags tests to avoid errors with a random order in the list of tags. [Christian Studer]
* [stix1 export] Same as 05dd0d4 but for STIX 1 attributes export. [Christian Studer]
* [stix2 export] More straight forward tags and confidence score handling. [Christian Studer]
- We just store confidence scores during the
execution of the tags parsing function instead
of storing the related tags separately. Thus,
those tags are now directly handled
- Since the markings handling function is the same
for every concerned MISP data structure (event,
attributes, objects), it does not require more
specific function for each different structure
* [stix1 export] Making sure we have simple marking before raising a KeyError exception. [Christian Studer]
* [stix1 export] Typo while handling confidence tags from campaign-name attribute. [Christian Studer]
* [stix2 import] A few unnecessary lines removed to make pep8 happy. [Christian Studer]
* [stix2 import] Moved all loading functions to the common STIX 2 import class instead of the specific one for internal content and removed duplicated function. [Christian Studer]
* [documentation] Galaxies mapping documentation re-generated automatically while running the tests. [Christian Studer]
* [tests, documentation] Added missing documentation auto-generation function call from within the `x509` objects import tests. [Christian Studer]
* [stix2 export] Passing the x509 object `hidden` attribute boolean value directly since the Boolean property class will handle it. [Christian Studer]
* [tests] Added tests for the `hidden` attribute value from process objects export as STIX 1 & 2. [Christian Studer]
* [stix2 export] Exporting `hidden` attributes from the `process` object template as `is_hidden` within the Process Observable object or patterning language. [Christian Studer]
* [stix1 export] Exporting `hidden` attributes from the `process` MISP object as the `is_hidden` field of STIX 1 Process objects. [Christian Studer]
* [tests, documentation] Making sure the `data` field is not null while sanitizing data to update for the documentation. [Christian Studer]
* [tests] Simplified the timestamp test since we do test on MISP's side and not STIX. [Christian Studer]
* [stix2 import] A few fixes for the timestamp values in objects and the multiple attributes in object templates parsing. [Christian Studer]
* [stix2 import] Some minor changes on variable name and making the stix object param of the MISP object creation function optional. [Christian Studer]
* [stix2 import] Fixed File hashes mapping to avoid `ssdeep` to be skipped. [Christian Studer]
- For some reason in STIX 2.0 this hash type is
not expressed in capital letters as for the
other hash types in the File observable object
### Other
* Wip: [tests] Tests for `sigma` objects import from STIX 2 Indicators. [Christian Studer]
* Add: [github actions] Added template for issues to report a bug. [Christian Studer]
* Add: [readme] Added a few badges. [Christian Studer]
* Add: [github actions] Added the STIX to MISP import tests. [Christian Studer]
* Wip: [tests] Tests for MISP objects import from custom objects. [Christian Studer]
* Wip: [tests] Tests for attributes import from STIX 2 `custom-attribute` objects. [Christian Studer]
* Wip: [tests] Tests for object references. [Christian Studer]
* Wip: [tests] Tests for attributes with embedded galaxies. [Christian Studer]
* Wip: [stix2 import] Parsing Relationships objects to extract embedded galaxies as well as object references. [Christian Studer]
* Wip: [tests] Tests for MISP galaxies import from STIX 2 objects. [Christian Studer]
* Add: [stix2 import] Added exception handling functions for errors with Intrusion Set and Threat Actor objects. [Christian Studer]
* Wip: [stix2 import] Importing MISP Galaxies from several STIX 2 objects. [Christian Studer]
- Importing for now Galaxies at event level
- To make it very straight forward we import tag
names instead of parsing and re-generating the
galaxy with its cluster, since MISP will better
accept the tag names
* Add: [tests, documentation] Galaxies documentation is now auto-generated during the related tests. [Christian Studer]
* Fix; [tests, documentation] Fixed names used for variables where the attributes and objects documentation is stored during the tests procedure, in order to avoid confusions between both STIX 2 versions. [Christian Studer]
* Wip: [tests] Tests for `annotation` objects import from STIX 2.1 Note objects. [Christian Studer]
* [poetry] Updated poetry config file & lock file to the latest. [Christian Studer]
* [tests] Changed samples used for `email` objects import from STIX 2 Observable objects. [Christian Studer]
* [tests] Updated tests for attributes export as STIX1 URI objects or STIX2 URL objects. [chrisr3d]
* [tests] Added more attributes types to be converted as STIX URL / URI objects. [chrisr3d]
* [stix2 import] Added a reusable function to fetch observable objects. [chrisr3d]
* [tests] Added more hash attribute types to be tested & fixed the tests for thoses attributes export as STIX 1 at the same time. [chrisr3d]
* [stix2 export] Added `link` attribute from the `news-agency` object to the list of contact information fields within the STIX 2 Identity object. [chrisr3d]
* [stix2 import] Enhanced the `vulnerability` object import mapping. [chrisr3d]
* Tests, documentation] Modifying the documentation to keep the shortened data values even if we use the actual files in tests. [chrisr3d]
* [tests] Using the actual attachment files to declare tests samples. [chrisr3d]
* [tests] Preparing some features to be reused with more inheritance from parent classes. [chrisr3d]
* [stix2 export] Updated the `employee` object export as STIX 2 mapping. [chrisr3d]
- Now includes the recently added `full-name`
object relation
* [tests] Deduplication of test code for `attack-pattern` object tests & for some multiple assertion statements. [chrisr3d]
* [tests] Preparing some features to be reused with more inheritance from parent classes. [chrisr3d]
* [tests] Updated tests for `attack-pattern` objects export as STIX 2.0 & 2.1. [chrisr3d]
* [documentation] Re-generated the full documentation with the updated mapping. [chrisr3d]
* [tests, documentation] Populating the automated documentation from attributes & objects export as STIX 2.0 tests. [chrisr3d]
* [documentation] Used the automated documentation update from tests to regenerate the objects export as STIX 2.1 mapping. [chrisr3d]
* [documentation] Used the automated documentation update from tests to regenerate the attributes export as STIX 2.1 mapping. [chrisr3d]
* [documentation] The misp objects mapping to stix21 summary is sanitized. [chrisr3d]
* [documentation] Re-generated the full documentation with the updated mapping. [chrisr3d]
* [tests, documentation] Populating the automated documentation from attributes & objects export as STIX 2.0 tests. [chrisr3d]
* [documentation] Used the automated documentation update from tests to regenerate the objects export as STIX 2.1 mapping. [chrisr3d]
* [documentation] Used the automated documentation update from tests to regenerate the attributes export as STIX 2.1 mapping. [chrisr3d]
* [documentation] The misp objects mapping to stix21 summary is sanitized. [chrisr3d]
* [stix2 import] Made some loading functions specific to each subclass. [chrisr3d]
* [stix2 import] Merged common grouping and report parsing process into on function. [chrisr3d]
* [poetry] Added missing `codecov` dependency that was removed by error. [Christian Studer]
* [github actions] Typo. [Christian Studer]
* [misp-stix] Typo. [Christian Studer]
* [misp-stix] Fixed a few typos and variable name issues. [Christian Studer]
* [tests] Fixed tests for `email` objects import from indicator objects following the recent changes on the related mapping & parsing. [Christian Studer]
* [stix2 export] Fixed `user-account` objects export to indicator where characters were not escaped. [Christian Studer]
* [stix2 import] Added missing Observed Data object in the STIX 2.1 email samples. [Christian Studer]
* [tests] Removed print used for debugging. [Christian Studer]
* [tests] Fixed space missing to make pep8 happy. [Christian Studer]
* [tests] Added tests for the content_disposition fields within the email-message objects body_multipart. [Christian Studer]
* [stix2 export] Exporting content disposition in the body_multipart field within email-message objects while exporting email objects as indicator, to keep the object_relation field. [Christian Studer]
* [documentation] Fixed documentation auto-generation by checking the Observed Data version. [Christian Studer]
* [documentation] Regenerated documentation with the recent changes on documentation mapping. [Christian Studer]
* [documentation] Updated documentation mapping for `domain-ip` objects export as STIX 2 Indicators. [Christian Studer]
* [tests] Fixed tests for `domain-ip` objects export as STIX2 Indicators. [Christian Studer]
* [stix2 export] Fixed `domain-ip` objects export as Indicator to avoid confusions. [Christian Studer]
- When `domain` and `hostname` attributes are both
present, we want to avoid confusions between the
domain attribute and the hostname attribute
* [stix2 import] Fixed the `twitter-account` object mapping. [Christian Studer]
* [documentation] The MISP objects export as STIX 2 documentation mapping has been regenerated with the recent changes on the user & account object samples. [chrisr3d]
* [documentation] The `link` attributes export as STIX 2 documentation has been fixed with the documentation auto-regeneration. [chrisr3d]
* [tests] Fixed tests for user & account objects export as STIX 2. [chrisr3d]
* [stix2 export] Fixed some user & account objects mapping as STIX 2. [chrisr3d]
* [stix2 import] Made pep8 more happy with some code style fixed. [chrisr3d]
* [tests] In STIX 2 samples: getting the data fields by base64-encoding the related files instead of copy-pasting the base64-encoded string. [chrisr3d]
* [stix2 import] Skipping timeline fields parsing for `observed_data` objects when the `first_observed` and `last_observed` values are the same as `modified` [chrisr3d]
* [stix2 import] Avoiding to raise the unknown STIX object exception with a test against a list of observable object types. [chrisr3d]
* [documentation] Updated attributes export as STIX 2 mapping. [chrisr3d]
* [tests] Fixed wrong category for the link attribute export. [chrisr3d]
* [tests] Just a quick function name fix. [chrisr3d]
* [tests] Removed unused variable in some MISP to STIX 1 export features tests. [chrisr3d]
* [documentation] Attributes export as STIX 2 documentation updated following the recent changes on tests. [chrisr3d]
* [stix2 export] Fixed hash attribute types mapping with the `filename|telfhash` type that does not exist. [chrisr3d]
* [tests] For tests using loops over attributes and stix objects, we assert the number of converted attributes first to make sure we do not loop over an empty list (which does not raise any assertion error) [chrisr3d]
* [stix2 export] Simplified the `pe-section` hash attributes handling with only the supported hash types, and no longer the full list of existing hash types. [chrisr3d]
* [documentation] Fixed documentation with non existing attribute type removed. [chrisr3d]
* [tests] Fixed hash attributes tests since `filename|telfhash` is not an existing MISP attribute type. [chrisr3d]
* [tests] Better automation on tests for multiple single attributes export. [chrisr3d]
* [stix2 export] Enhanced the list of supported hash attribute types to be exported. [chrisr3d]
* [tests] Removed utility function that had already been moved in the parent class. [chrisr3d]
* [tests] Fixed tests for `suricata` objects export as STIX 2.1 and added more attributes to the `suricata` & `yara` test object samples to be tested. [chrisr3d]
* [stix2 export] Fixed the `suricata` object export as STIX 2.1 mapping. [chrisr3d]
* [stix2 import] Fixed patterning language objects parsing for external STIX content. [chrisr3d]
* [tests] Fixed tests for `news-agency` objects export as STIX 2.0 & 2.1 following the changes on the contact information field for this object. [chrisr3d]
* [tests] A few changes in the test function names & added unit tests for the MISP object names. [chrisr3d]
* [tests] Fixed tests for `employee` objects import from STIX 2 Identity objects, following the recent changes on the `contact_information` field handling. [chrisr3d]
* [stix2 import] Fixed the Identity object error message. [chrisr3d]
* [stix2 import] Fixed contact information field handling in the STIX 2 Identity object import as MISP employee object. [chrisr3d]
* [tests] Fixed documentation auto-generation from tests for user account objects. [chrisr3d]
* [tests] Fixed tests for `legal-entity` export as STIX 2.0 & 2.1. [chrisr3d]
* [stix2 export] Fixed the `legal-entity` objects export as STIX 2 mapping, with the `website` attribute now being part of the contact information mapping for this object. [chrisr3d]
* [stix2 export] Fixed `employee` objects export as STIX 2 mapping, with the `email-address` attribute being now part of the contact information mapping for this object. [chrisr3d]
* [stix2 export] Added missing specific mapping list for employee objects export as STIX 2.0 & 2.1. [chrisr3d]
* [stix2 export] Fixed `employee` object export of the contact information STIX 2 field. [chrisr3d]
* [stix2 import] Fixed a variable name. [chrisr3d]
* [stix2 import] Better handling of STIX objects loaded in a dict with a `used` flag. [chrisr3d]
* [tests] Putting the `AttackPattern` objects checking function at the right place. [chrisr3d]
- In this case, this is a testing function for
specific STIX 2 objects generated from MISP
* [stix2 import] Avoiding any issue with the `type` feature in mappings. [chrisr3d]
* [stix2 import] Added `force_timestamps` parameter at the creation of MISP events and objects to make sure the timestamps will be preserved once ingested in MISP format. [chrisr3d]
* [stix2 export] Fixed `attack-pattern` export as STIX 1 tests following the recent changes on the sample objects. [chrisr3d]
* [documentation, tests] Sanitized the automated documentation generation from the tests. [chrisr3d]
* [documentation, tests] Stripped data fields values to make them more convenient to be used in a documentation. [chrisr3d]
* [documentation, tests] Forcing some summary definition in the objects documentation. [chrisr3d]
* [tests] Better variables handling in some attributes export tests. [chrisr3d]
* [tests] Fixed variable name. [chrisr3d]
* [documentation, tests] Fixed the `mac-address` Observed Data documentation automation. [chrisr3d]
* [tests] Removed test print. [chrisr3d]
* [stix2 export] Fixed the suricata object mapping. [chrisr3d]
* [stix2 export] Using the parent class property to get the `identity_id` since the "private" attribute is not known by the children classes. [chrisr3d]
* [git] Fixed gitmodules file. [chrisr3d]
* [tests] Quick grouping features testing simplification. [chrisr3d]
* [stix2 export] Fixed cti library path following the recent path changes for this git submodule. [chrisr3d]
* [stix2 export] Simplified one tmp variable that was not necessary. [chrisr3d]
* [stix2 export] Fixed typo with `Sighting` fields. [chrisr3d]
* [documentation] Making sure we don't face any path issue in case the documentation generation is ran from another path. [chrisr3d]
* [documentation] Updated summary. [chrisr3d]
* [documentation, tests] Some typos which generated a broken documentation update. [chrisr3d]
* [tests] Just a quick summary update. [chrisr3d]
* [tests] A few copy paste and variable name issues. [chrisr3d]
* [tests] Reusing declared variables. [chrisr3d]
* [tests] Removed or used unused variables. [chrisr3d]
* [documentation, tests] Sanitized the automated documentation generation from the tests. [chrisr3d]
* [documentation, tests] Stripped data fields values to make them more convenient to be used in a documentation. [chrisr3d]
* [documentation, tests] Forcing some summary definition in the objects documentation. [chrisr3d]
* [tests] Better variables handling in some attributes export tests. [chrisr3d]
* [tests] Fixed variable name. [chrisr3d]
* [documentation, tests] Fixed the `mac-address` Observed Data documentation automation. [chrisr3d]
* [tests] Removed test print. [chrisr3d]
* [stix2 export] Fixed the suricata object mapping. [chrisr3d]
* [stix2 export] Using the parent class property to get the `identity_id` since the "private" attribute is not known by the children classes. [chrisr3d]
* [stix2 import] A few changes on the `single_event` parameter and the number of report or grouping objects. [chrisr3d]
* [git] Fixed gitmodules file. [chrisr3d]
* [tests] Quick grouping features testing simplification. [chrisr3d]
* [stix2 export] Fixed cti library path following the recent path changes for this git submodule. [chrisr3d]
* [stix2 export] Fixed typo with `Sighting` fields. [chrisr3d]
* [stix2 import] Clarification on various mapping variable names. [chrisr3d]
* [stix2 import] Changed the pattern type exception catching to an error instead of a warning since we cannot call the stix2-pattern object creation function in this case. [chrisr3d]
* [stix2 import] Typo. [chrisr3d]
* [stix2 export] Simplified one tmp variable that was not necessary. [chrisr3d]
* [stix2 import] Quick fix on vulnerability object parameter that is a ref and not the vulnerability object directly. [chrisr3d]
* [stix2 import] Making the MISP object creation function an attribute of the parent class, available for both children classes. [chrisr3d]
* [stix2 import] A few errors fixed, like a missing import or a wrong variable name etc. [chrisr3d]
* [stix2 import] Made the list of unsupported pattern separation key words a property of the external STIX files parsing mapping. [chrisr3d]
* [stix2 import] This typing variable is now going to be needed in the parent class. [chrisr3d]
* [stix2 import] Better separation in catching exceptions while looping over report or grouping object_refs. [chrisr3d]
* [stix2 import] Fixed a few variable names issues. [chrisr3d]
* [stix2 import] Fixed function name change that was missing. [chrisr3d]
* [stix1 export] Better errors handling for objects to parse as the same improvement has been made to STIX2 recently. [chrisr3d]
* [stix1 export] Better errors handling for objects to parse as the same improvement has been made to STIX2 recently. [chrisr3d]
* [stix export] Enhanced handling of MISP object which encountered a parsing issue. [chrisr3d]
- Avoiding those objects to be skipped
- They're exported as custom objects instead
* [stix2 export] Enhanced the pattern values sanitisation. [chrisr3d]
- Generalised the sanitisation made on registry
key values to all the pattern since they may
contain characted like `%` and `\` which are
particularly tricky to handle in STIX patterns
* [stix2 export] Enhanced the pattern values sanitisation. [chrisr3d]
- Generalised the sanitisation made on registry
key values to all the pattern since they may
contain characted like `%` and `\` which are
particularly tricky to handle in STIX patterns
* [stix2 export] Better exceptions catching while handling MISP objects to parse. [chrisr3d]
- Most of the objects are parsed on the go and
directly converted into a STIX object, but some
objects have specific relations that require
special care. It is the case for file objects
with pe and pe-section objects. Since they are
exported into a single STIX file object with an
extension, we need to store them until we are
sure all MISP objects have been handled (parsed
or stored) and we do have all the referenced
objects to start the special parsing. Then they
are parsed together using the `ObjectReference`
field of each one of them. For this specific use
case, we were missing some exception catching
since they're out of the standard objects
resolving loop
* [tests] Making sure the recent changes on STIX objects labels don't break the tests. [chrisr3d]
* [stix2 import] Updated the `stix2_to_misp` helper function. [chrisr3d]
- We already wrote previously a skeleton for this
function to take a filename using its name and
to call the parsing function which takes the
STIX2 bundle object. We simply updated it with
the recent STIX2 to MISP parsing features
development
* [stix2 import] Variable names typo. [chrisr3d]
* [stix2 import] Wrong variable name. [chrisr3d]
* [tests] Fixed tests on labels. [chrisr3d]
* [stix2 export] Better markings handling to avoid issues with unrecognised tlp tags. [chrisr3d]
* [stix2 import] Syntax fixed. [chrisr3d]
* [stix2 export] Better markings handling to avoid issues with unrecognised tlp tags. [chrisr3d]
* [stix1 export] Transforming into upper case TLP tags only. [chrisr3d]
- TLP tags that are not parsed as TLPMarkings are
then exported as SimpleMarking with no uppercase
conversion, which keeps the tag as is
- It also avoids the `.upper()` for every test ran
on each tag, and limits this conversion into
uppercase only when needed
* [stix1 export] Transforming into upper case TLP tags only. [chrisr3d]
- TLP tags that are not parsed as TLPMarkings are
then exported as SimpleMarking with no uppercase
conversion, which keeps the tag as is
- It also avoids the `.upper()` for every test ran
on each tag, and limits this conversion into
uppercase only when needed
* [stix1 export] Fixed tags parsing to avoid issues with TLP tags. [chrisr3d]
- Parsing as TLPMarking only the supported TLP tags
- The other ones are exported as SimpleMarkings
* [stix1 export] Fixed tags parsing to avoid issues with TLP tags. [chrisr3d]
- Parsing as TLPMarking only the supported TLP tags
- The other ones are exported as SimpleMarkings
* [tests] Fixed orgname testing in every different test. [chrisr3d]
- The orgname value used to define the information
source and reporter identity remains the same
- The orgname value used to define every STIX
object id is correctly sanitized
* [stix1 export] Fixed missing import and typo. [chrisr3d]
* [stix1 export] Fixed STIX objects ID identifier. [chrisr3d]
- Making sure the orgname used is sanitised and
does not contain any space
* [stix1 framing] Fixed STIX 1 XML Header framing. [chrisr3d]
* [stix2 export] Making sure observable object ids are correctly parsed. [chrisr3d]
- Making also sure those ids are correctly
fetched if there are event reports, so they are
correctly referenced in the `object_refs` field
* [stix2 export] Better handling of object ids used in the `object_refs` field within the Note objects generated from the event reports parsing. [chrisr3d]
* [stix2 export] Making `parent-pid` attribute prioritary over `parent-command-line` to define which attribute uuid is used to define the parent process id while parsing process objects. [chrisr3d]
* [tests] Fixed tests for `legal-entity` objects export. [chrisr3d]
* [tests] Added missing `legal-entity` test object that is necessary for the related tests. [chrisr3d]
* [tests] Fixed tests for `malware-sample` attributes & object attributes tests following the recent updates on the conversion of this type of attribute. [chrisr3d]
* [stix2 export] Added missing `created_by_ref` field in Note & Location objects. [chrisr3d]
* [stix2 export] Better handling of custom features with potential data field in STIX objects or Observable objects. [chrisr3d]
* [tests] Testing the location object id with the grouping refs. [chrisr3d]
* [tests] Fixed tests for objects which recently got there STIX conversion to contain a `to_ids` tag. [chrisr3d]
* [stix2 export] Added the global `to_ids` tag fetched from object attributes even in STIX objects that are not dependant from this tag. [chrisr3d]
- As opposed to `Indicator` & `Observable` objects
which are directly depending on the `to_ids`
value, other objects were not getting the value
as additional tag value. As it does not cost
much more to at least get the info whether there
was a `to_ids` flag in the object attributes, we
add this tag in some objects that were missing it
* [tests] Testing precisely the observable ids within observable compositions while exporting MISP into STIX 1. [chrisr3d]
* [tests] Changed ids of observable objects within observable composition objects to comply with the recent changes on observable ids in that specific case. [chrisr3d]
* [tests] Properly testing the observable features in the case of an export of a domain|ip attribute. [chrisr3d]
* Wip: [stix2 import] Updated the STIX 2 objects mapping handling. [chrisr3d]
* Merge branch 'main' of github.com:MISP/misp-stix into dev. [chrisr3d]
* Add: [documentation] MISP objects export as STIX 2.0 & 2.1 mappings are automatically updated with the recent changes on tests. [chrisr3d]
* Add: [tests] Added tests for `script` objects export as STIX 2.0 & 2.1. [chrisr3d]
* Add: [stix2 export] Added `script` objects to the export as STIX 2.0 & 2.1 export mapping. [chrisr3d]
* Wip: [documentation] Updated documentation has been regenerated. [chrisr3d]
* Wip: [documentation] Replaced the attributes & objects export as STIX 2.0 & 2.1 summaries with the formatting headers so they are generated from the recently added summary mappings. [chrisr3d]
* Wip: [documentation] Added the auto generation of the attributes & objects export as STIX 2.0 & 2.1 mapping summary. [chrisr3d]
* Add: [documentation] Added the attributes & objects export as STIX 2.0 summary autogenerated with tests. [chrisr3d]
* Wip: [documentation] Updated the MISP objects export as STIX 2.0 documentation using the documentation automated update from tests. [chrisr3d]
* Wip: [documentation] Updated the attributes export to STIX 2.0 documentation regenerated with the tests automated documentation update. [chrisr3d]
* Wip: [documentation, tests] Updated the automated documentation generation to support STIX 2.0. [chrisr3d]
* Fix; [tests] Removed or used unused variables. [chrisr3d]
* Iadd: [documentation] Added summary mapping for attributes & objects export as STIX 2.1. [chrisr3d]
* Wip: [documentation, tests] Populating the objects documentation while running STIX 2.1 tests. [chrisr3d]
* Wip: [documentation, tests] Outsourced the documentation update process to an external class and script. [chrisr3d]
* Wip: [documentation, tests] Testing if the attributes conversion as STIX 2.1 mapping from documentation if different from the mapping built from tests before replacing it. [chrisr3d]
* Wip: [documentation, tests] Replacing attribute to STIX 2.1 mapping with the samples used in tests. [chrisr3d]
* Wip: [tests] Initiated an automated way to check if the mapping documentation is up-to-date using the tests. [chrisr3d]
- Started with the tests for attributes export as STIX 2.1
* Add: [tests] Added tests for patterning language objects export as STIX 2.1. [chrisr3d]
* Add: [tests] Test samples for objects converted into indicator with a specific pattern type. [chrisr3d]
* Add: [stix2 export] Added suricata & yara to the list of supported MISP object templates for export as STIX 2.1. [chrisr3d]
* Add: [git] Added tmp dir & a gitignore file that contains the tmp dir for now. [chrisr3d]
* Add: [documentation] MISP objects export as STIX 2.0 & 2.1 mappings are automatically updated with the recent changes on tests. [chrisr3d]
* Add: [tests] Added tests for `script` objects export as STIX 2.0 & 2.1. [chrisr3d]
* Add: [stix2 export] Added `script` objects to the export as STIX 2.0 & 2.1 export mapping. [chrisr3d]
* Wip: [documentation] Updated documentation has been regenerated. [chrisr3d]
* Wip: [documentation] Replaced the attributes & objects export as STIX 2.0 & 2.1 summaries with the formatting headers so they are generated from the recently added summary mappings. [chrisr3d]
* Wip: [documentation] Added the auto generation of the attributes & objects export as STIX 2.0 & 2.1 mapping summary. [chrisr3d]
* Add: [documentation] Added the attributes & objects export as STIX 2.0 summary autogenerated with tests. [chrisr3d]
* Wip: [documentation] Updated the MISP objects export as STIX 2.0 documentation using the documentation automated update from tests. [chrisr3d]
* Wip: [documentation] Updated the attributes export to STIX 2.0 documentation regenerated with the tests automated documentation update. [chrisr3d]
* Wip: [documentation, tests] Updated the automated documentation generation to support STIX 2.0. [chrisr3d]
* Fix; [tests] Removed or used unused variables. [chrisr3d]
* Iadd: [documentation] Added summary mapping for attributes & objects export as STIX 2.1. [chrisr3d]
* Wip: [documentation, tests] Populating the objects documentation while running STIX 2.1 tests. [chrisr3d]
* Wip: [documentation, tests] Outsourced the documentation update process to an external class and script. [chrisr3d]
* Wip: [documentation, tests] Testing if the attributes conversion as STIX 2.1 mapping from documentation if different from the mapping built from tests before replacing it. [chrisr3d]
* Wip: [documentation, tests] Replacing attribute to STIX 2.1 mapping with the samples used in tests. [chrisr3d]
* Wip: [tests] Initiated an automated way to check if the mapping documentation is up-to-date using the tests. [chrisr3d]
- Started with the tests for attributes export as STIX 2.1
* Add: [tests] Added tests for patterning language objects export as STIX 2.1. [chrisr3d]
* Add: [tests] Test samples for objects converted into indicator with a specific pattern type. [chrisr3d]
* Add: [stix2 export] Added suricata & yara to the list of supported MISP object templates for export as STIX 2.1. [chrisr3d]
* Add: [git] Added tmp dir & a gitignore file that contains the tmp dir for now. [chrisr3d]
* Wip: [stix2 import] Better pattern type handling & redirection to the `stix2-pattern` object creation in case of parsing exception. [chrisr3d]
* Wip: [stix2 import] Some pieces of documentation for the main parsing function used for external STIX 2. [chrisr3d]
* Wip: [stix2 import] Considering the possibility some producers of STIX data still use the deprecated `objects` field instead of `object_refs` [chrisr3d]
* Wip: [stix2 import] Added a first version of observable & pattern mappings for STIX objects from external STIX files. [chrisr3d]
* [stix1 export] Making the STIX1 framing available for attributes colletions export. [chrisr3d]
* [stix1 export] Better XML formatting for several STIX object types. [chrisr3d]
* [stix mapping] Making mapping dicts immutable. [chrisr3d]
- Some mapping features are tuples, and thus
immutable, and the `@property` decorator is good
for preventing the class variables to be changed
but does not prevent changes on the dictionaries
(new key/value, `pop`, `update`, etc.)
* [poetry] Bumped lock file. [chrisr3d]
### Fix
* [tests] Recursively testing all features while exporting domain|ip attributes in order to avoid issues with the ids of the observable objects embedded in observable composition. [chrisr3d]
* [tests] Fixed tests to avoid issues with the Observable objects id within observable compositions. [chrisr3d]
* [stix1 export] Fixed Observables header & footer that are used for attributes collections export. [chrisr3d]
* [stix1 export] A simple typing clarification. [chrisr3d]
* [stix1 export] Avoiding Observable objects' id duplication in Observable composition while exporting `domain|ip` attributes. [chrisr3d]
* [tests] Updated tests for attributes & events collections export as STIX 1.1.1 & 1.2 following the recent changes on the related function. [chrisr3d]
* [stix1 export] Made events collections export as STIX1 function's parameters the same as for attributes collections. [chrisr3d]
* [stix1 export] Avoiding KeyError exceptions if the attributes collections are not embedded within a `response` field. [chrisr3d]
* [stix1 export] Using the latest version of the `_get_events` helper to get STIX 1 content converted from MISP events. [chrisr3d]
* [stix1 export] Attributes collections export helper function is now supporting the recent changes on the other getter functions (framing & `to_xml` or `to_json` calls) [chrisr3d]
* [stix1 export] Harmonising the attributes export framing for STIX 1 with the events export framing. [chrisr3d]
* [stix1 export] Making the STIX1 content getter functions callable. [chrisr3d]
* [stix2 export] Grouped markings parsing function that did not require to be split into the STIX 2.0 and 2.1 parsing subclasses. [chrisr3d]
* [stix2 export] Using the class property for unique ids as much as possible when there is no change to it. [chrisr3d]
* [stix2 export] Handling markings once they are already parsed. [chrisr3d]
- We already parsed markings and stored them into
a dictionary, we now added them in the list of
parsed STIX2 objects
* [stix2 export] Better Galaxy clusters meta fields parsing. [chrisr3d]
* [stix1 export] Avoiding `KeyError` exceptions if `meta` field is not set in galaxy clusters. [chrisr3d]
* [stix1 export] Added missing `_ids` class variable to the attributes parser class. [chrisr3d]
* [stix2 export] Making markigns available during attributes collections export. [chrisr3d]
* [stix export] Removed unused import. [chrisr3d]
* [tests] Fixed test to support the case of custom objects containing fields that have been sanitised to avoid issues with unauthorised characters. [chrisr3d]
* [stix2 export] Avoiding issues with the `report` object export as custom STIX2 object. [chrisr3d]
- The `report-file` attribute was `report-file(s)`
and has been changed with b0eb077, but we need
to keep the backward compatibility
* [tests] Just a quick variables simplication. [chrisr3d]
* [stix2 export] Fixed email objects exports in the case of multiple `from` attributes. [chrisr3d]
- `From` attributes, like `To` and `Cc`, are
associated with their uuid in order to properly
reference the Email Address Cyber Observable
objects corresponding to the export of those
attributes.
- When the first `from` attribute is associated
with the `from` field of the Email Message
object, the other `from` attributes, if they
exist are exported in a custom fields. In this
case we need to remove the uuids and keep the
attribute values only
* [stix2 export] Fixed filename|hash attributes export as indicator. [chrisr3d]
- We cannot remove the escaping for hash composite
attributes otherwise the filename is not
properly escaped
* [stix2 export] Registry-key objects export mapping updated accordingly to the latest changes applied to the parsing functions. [chrisr3d]
* [stix2 export] Better parsing of values to escape for registry-key objects. [chrisr3d]
- We separated the registry key & data value that
require some specific escaping. The standard
escaping is now only for the other attributes
- The escaping is only for attributes and objects
exported as indicators, but the parsing of
the registry-key object attributes exported
as observable objects has also been enhanced
* [stix2 export] Fixed parsing of hash values exported in indicator patterns. [chrisr3d]
- Hash values must be validated anyway, so instead
of escaping values that could be invalid, we
simply removed them since they would raise an
issue even escaped
* [stix2 export] Removed attribute values escaping for object attributes exported in observable objects. [chrisr3d]
* [stix2 export] Removed double escaping for attribute values supposed to be exported as indicator patterns. [chrisr3d]
* [stix1 export] Enabled pe object not referenced by file objects to be parsed and exported as WindowsExecutableFile objects with their sections. [chrisr3d]
- Nonetheless, sections are bound to their pe object
which references them, thus they will not be parsed
and exported alone in a WindowsExecutableFile object
* [tests] Updated hash & hash composite attributes tests with some more hash types tested. [chrisr3d]
* [tests] Making the network-socket test objects compliant with the STIX2 export tests. [chrisr3d]
- Added the uuid to each object attributes since
they are required for some of them in STIX 2.1
- Changed the address-family attribute value to
avoid enumeration issues within the STIX2
socket extension
* [tests] Updated the network-socket object export as STIX1 tests. [chrisr3d]
* [stix export] Made all the lists used to help extracting object attributes immutable and declared in the mapping script. [chrisr3d]
- Instead of redefining them each times the
functions are called, they are declared once in
the mapping script and are called from there.
* [stix export] Reusing the single feature selection function in STIX1 export by making it available in the parent class common for STIX1 & STIX2. [chrisr3d]
* [stix1 export] Moving functions to parent class in order to be reused for stix2 parsing. [chrisr3d]
* [tests] Testing that `is_multipart` is set to False when there is no multipart in a STIX 2.1 Email object exported from a MISP email object. [chrisr3d]
* [stix2 export] Avoid `is_multipart` to be True when there is actually no multipart. [chrisr3d]
* [stix2 export] Handling the display names parsing input differences between STIX 2.0 & STIX 2.1 parsing functions. [chrisr3d]
- No difference in the way display names are
parsed and matched with email addresses, but
STIX 2.1 email addresses are associated with
their attribute uuid which makes the input of
the display names parsing function different
from the STIX 2.0 version
* [stix2 export] Added missing support of `message-id` object attribute when exporting email objects as observed data objects. [chrisr3d]
* [stix1 export] Added missing `message-id` object relation to the mapping of supported object attributes from email objects. [chrisr3d]
* [tests] Fixed tests for registry-key objects export as STIX 2.0 & 2.1. [chrisr3d]
* [stix1 framing] Fixed STIX1 xml header. [chrisr3d]
* [stix2 export] Fixed malware & tool objects creation when the interoperability flag is set. [chrisr3d]
* [tests] Just a quick pep8 compliance fix. [chrisr3d]
* [tests] Fixed the example file for attributes collections export as STIX 2.1. [chrisr3d]
* [stix2 export] Fixed Attributes collections export with attributes exported as Observed Data which where actually missing the cybox observable objects. [chrisr3d]
* [framing, stix1 export] Added missing fix already used for the validation of some previous commits, fixing the attributes collections export as JSON STIX1. [chrisr3d]
* [stix1 export] Fixed Observables parsing while exporting multiple attributes collection files as STIX1. [chrisr3d]
- We want to avoid empty content to add `\n` to
the result file each time the `observables` field
is set but empty of observables (only the cybox
information is present)
* [stix1 export] Fixed attributes collection export footer handling. [chrisr3d]
* [stix1 export] Enhanced the attributes collections export for multiple collections in order to fix the export as JSON STIX. [chrisr3d]
* [stix2 export] Considering the case where there is no file name to get from a pe object to populate the 'name' field of the STIX file object. [chrisr3d]
* [tests] Function name typo. [chrisr3d]
* [documentation] Correctly documented how time fields are exported in STIX 2.0 & 2.1 Indicators & Observed Data objects. [chrisr3d]
* [tests] Fixed vulnerability object export tests to include the created and modified attributes exported as STIX vulnerability object fields. [chrisr3d]
* [stix2 export] Some typo, variable name and naming fixes. [chrisr3d]
* [stix2 export] Exporting created and modified attribute objects from vulnerability objects. [chrisr3d]
- Also fixed some datetime parsing features
* [stix2 export] Added missing object references parsing when the object is exported as observed data. [chrisr3d]
- 'is_family' is a STIX 2.1 Malware Object required field
* [stix2 export] Added missing timestamp while defining the list of target IDs & relationship type for a given list of relationships related to a source ID. [chrisr3d]
* [stix1 export] making pep8 happy with the STIX1 mapping. [chrisr3d]
* [tests] Testing that the created & modified time of the Identity object used as creator are the actual event timestamp. [chrisr3d]
* [stix2 export] Giving the Identity object generated out of the Orgc of the event the actual timestamp of the event as creation and modified time. [chrisr3d]
* [tests] Fixed tests for attributes exported as Custom objects. [chrisr3d]
* [stix1 export] Clearer identification of the type of STIX objects when they get a related_ttp from an attribute galaxy or object attributes galaxies. [chrisr3d]
* [stix2 export] Added header to the report creation functions. [chrisr3d]
* [stix1 export] Fixed raw_header & raw_body fields condition as well as their corresponding tests. [chrisr3d]
* [tests] Changes on the email-body attributes export tests according to the recent changes on their export. [chrisr3d]
* [stix2 export] More straight forward way to handle email-body export. [chrisr3d]
* [tests] Using event timestamp to test stix report timestamp. [chrisr3d]
* [stix2 export] Fixed time related fields for ObservedData & Indicator objects. [chrisr3d]
* [stix2 export] A few missing functions and variables issues fixed. [chrisr3d]
- Right now the events collections export as STIX1
is broken since we removed also the STIX1 class
that is going to be easily included in the code
we already have
* Merge branch 'dev' of github.com:chrisr3d/MISP-STIX-Converter into dev. [chrisr3d]
* Wip: [tests] Updated tests to include the sanitizing function of registry keys and data. [chrisr3d]
* Wip: [stix2 export] Fixed registry keys and data values parsing with a sanitizing function that should avoid issues with special characters. [chrisr3d]
* Wip: [stix2 export] Some other `allow_custom` management within file objects about custom hash types. [chrisr3d]
* Wip: [stix2 export] A few fixes on `allow_custom` values to follow the recent changes on the cti-python-stix2 library. [chrisr3d]
* Merge branch 'dev' of github.com:chrisr3d/MISP-STIX-Converter into dev. [chrisr3d]
* Wip: [stix2 export] Better external references handling when dealing with galaxies. [chrisr3d]
* Wip: [stix2 export] Fixed galaxies matching as STIX objects from the cti catalog + some variable names fixes & clean up. [chrisr3d]
* Wip: [stix2 export] Taking STIX objects to export galaxies when they are defined in the cti catalog. [chrisr3d]
* Wip: [stix2 export] Cleaned up some functions parameters. [chrisr3d]
* Merge branch 'dev' of github.com:chrisr3d/MISP-STIX-Converter into dev. [chrisr3d]
* Wip: [stix2 export] Changed the build of the cti catalog to make the relevant fields more accessible. [chrisr3d]
* Merge branch 'dev' of github.com:chrisr3d/MISP-STIX-Converter into dev. [chrisr3d]
* Wip: [documentation] Added documentation for intrusion-set galaxies export as STIX 2.0 & 2.1. [chrisr3d]
* Wip: [tests] Tests for intrusion-set galaxies export as STIX 2.0 & 2.1. [chrisr3d]
* Wip: [stix2 export] Added missing intrusion-set galaxies to the export as STIX 2.0 & 2.1 mapping. [chrisr3d]
* Wip: [stix2 export] Working on the mapping between MISP galaxies and objects loaded from the cti catalog. [chrisr3d]
- We start with the full cti catalog loading whenever
the interoperability flag is set
- If the flag is not set, the behavior remains the
same and each MISP galaxy is processed
- Adjustment will probably come soon to make sure
we have all the parameters we need to make the
association with an object from the catalog as
accurate as possible
* Wip: [stix2 export] Submodules the cti catalog of attack technic for further implementation. [chrisr3d]
- The goal is to use the already defined STIX objects
to export Galaxy clusters, by trying to find a match
on the name, instead of processing them
* Wip: [tests] Tests for events collections export as STIX 2.0 & 2.1. [chrisr3d]
* Wip: [stix export] Helpers to get a STIX 2.0 or 2.1 bundle and write the result of an export in the output file. [chrisr3d]
* Wip: [stix2 export] A few changes on the STIX 2 parser. [chrisr3d]
- Moved some class variables to allow multiple
calls of the main parsing function while the
class only needs to be declared once. This
avoids the multiple declaration of the class
for each event when we want to export an
events collection.
- Some variables to store lists of ids have been
merged in one unique variable since the purpose
of the list is to store unique ids of orgs and
galaxies to avoid processing them multiple times
- Concerning the export of events collections and
the storage of unique ids, this list of unique
ids is simply declared with the class and can
be populated for each event. It is also possible
to return this list to, if we want to use it in
another call of the class, which should happen
for instance when we want to export a large
number of events in a collection: MISP is going
to split the collection and call the parser
multiple times; we can then pass this list of
unique ids to skip some object ids that have
already been processed with a previous call of
the parser
* Wip: [stix export] Helpers to get the STIX 2.0 or 2.1 bundle from the export of a MISP event or a collection of events. [chrisr3d]
- Also cleared the parent class used for STIX1 too
* Wip: [documentation] Added documentation for events export as STIX 2.0 & 2.1. [chrisr3d]
- Including events with embedded attribute galaxies,
events with embedded object attribute galaxies,
and events with objects referencing each others
* Wip: [documentation] Regenerated the full documentation. [chrisr3d]
* Add: [documentation] Updated code to generate the objects export documentation. [chrisr3d]
* Wip: [documentation] Added documentation for the mutex objects export as STIX1. [chrisr3d]
* Wip: [tests] Tests for mutex objects export as STIX1. [chrisr3d]
* Wip: [stix1 export] Exporting mutex objects which were missing in the export mapping. [chrisr3d]
* Wip: [documentation] Mapping for MISP objects export as STIX 2.0 & 2.1. [chrisr3d]
* Wip: [tests] Tests for pe & section objects export as STIX 2.0 & 2.1 in windows pebinary extension. [chrisr3d]
* Wip: [stix2 export] Exporting pe object and their sections even with no file object referencing them. [chrisr3d]
* Wip: [tests] Tests for pe objects and their sections to be exported as STIX1 WindowsExecutableFile objects without being referenced by a file object. [chrisr3d]
* Wip: [documentation] Regenerated the Attributes export documentations. [chrisr3d]
* Wip: [documentation] Populated the STIX 2.0 & 2.1 documentations with the missing hash, hash composite, link & uri attributes. [chrisr3d]
* Wip: [documentation] Regenerated documentation with the updates on attributes export as STIX1. [chrisr3d]
* Wip: [stix1 export] Added check for objects which should not be parsed the usual way + decommented try catch statement that has been commented for test purposes. [chrisr3d]
* Wip: [tests] Added tests for the credential and domain-ip objects that have been added recently to the stix1 export. [chrisr3d]
* Wip: [stix1 export] Population the objects export mapping. [chrisr3d]
* Wip: [tests] Testing asn object export + some slight changes to go with. [chrisr3d]
