mirror of https://github.com/MISP/misp-website
chg: [MISP] release 2.4.128 - changelog
parent
ebc2e8dd84
commit
83246a2111
798
Changelog.txt
798
Changelog.txt
|
|
@ -2,6 +2,804 @@ Changelog
|
||||||
=========
|
=========
|
||||||
|
|
||||||
|
|
||||||
|
v2.4.128 (2020-06-22)
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
New
|
||||||
|
~~~
|
||||||
|
- [correlations] Enable CIDR correlations for ip-src|port and ip-
|
||||||
|
dst|port types. [Jakub Onderka]
|
||||||
|
|
||||||
|
Changes
|
||||||
|
~~~~~~~
|
||||||
|
- [version] bump. [iglocska]
|
||||||
|
- [PyMISP] Bump. [Raphaël Vinot]
|
||||||
|
- [stix2 import] Parsing external pattern made with 'OR' separators the
|
||||||
|
same way we do for pattern with 'AND' [chrisr3d]
|
||||||
|
|
||||||
|
- Also slight update of some mapping dictionaries
|
||||||
|
to go with the changes introduced with this
|
||||||
|
commit on the main script
|
||||||
|
- [stix2 tests] Bumped the latest MISP & STIX2 test files. [chrisr3d]
|
||||||
|
- [correlations] Faster IPv4 CIDR correlation. [Jakub Onderka]
|
||||||
|
- [correlations] Faster IPv6 correlation. [Jakub Onderka]
|
||||||
|
- [correlations] Big speedup when correlating CIDR. [Jakub Onderka]
|
||||||
|
- [widget] remove unused var, make test pass. [Jean-Louis Huynen]
|
||||||
|
- [stix2 import] Moved all the mapping dictionaries to the mapping
|
||||||
|
script. [chrisr3d]
|
||||||
|
- [stix2 import] Temporary rework stix2 to misp script should now be
|
||||||
|
ready to replace the original stix2 to misp script. [chrisr3d]
|
||||||
|
- Bump PyMISP. [Raphaël Vinot]
|
||||||
|
- [stix2 import] Better parsing for patterns we always import as single
|
||||||
|
attributes. [chrisr3d]
|
||||||
|
- [stix2 import] Generic way of dealing with payloads in external file &
|
||||||
|
artifact patterns. [chrisr3d]
|
||||||
|
|
||||||
|
- After struggling a lot with the different use
|
||||||
|
cases, we ended up with the following process:
|
||||||
|
- checking if any file:content_ref is there
|
||||||
|
and grouping the content refs features
|
||||||
|
together if possible
|
||||||
|
- After all the content refs have been parsed,
|
||||||
|
we check if there still is some payloads
|
||||||
|
- [stix2 export] Moved the Attributes parsing functions into the main
|
||||||
|
script. [chrisr3d]
|
||||||
|
|
||||||
|
- Also checked the mapping to find potential bugs,
|
||||||
|
and fixed/updated some fields (in observed-data
|
||||||
|
and indicators)
|
||||||
|
- [stix2 import] Observable single attributes parsing functions are now
|
||||||
|
in the main script. [chrisr3d]
|
||||||
|
|
||||||
|
- Also update of the mapping dictionary with the
|
||||||
|
latest updated functions moved from the mapping
|
||||||
|
script to the main script
|
||||||
|
- [stixtest] JQing MISP event result from a STIX import for more visual
|
||||||
|
ease. [chrisr3d]
|
||||||
|
- [stix1 import] Better parsing of ttps, threat actors & courses of
|
||||||
|
action. [chrisr3d]
|
||||||
|
- [stixtest] Updated the STIX1 test files following the changes on the
|
||||||
|
test MISP events. [chrisr3d]
|
||||||
|
- [stixtest] Updated stix2 test files with the most recent changes on
|
||||||
|
the related MISP events, and on the export script. [chrisr3d]
|
||||||
|
- [stixtest] Test MISP events up-to-date. [chrisr3d]
|
||||||
|
- [stix2 export] Exporting Course of Action object attributes as custom
|
||||||
|
properties if not supported. [chrisr3d]
|
||||||
|
- [stix2] Bumped latest stix2 python library. [chrisr3d]
|
||||||
|
- [stixtest] Updated the stix1 test files. [chrisr3d]
|
||||||
|
- [stix1 export] Exporting Galaxies per TTP, Threat Actor or COA.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Exporting each galaxy as one TTP, Threat Actor,
|
||||||
|
or Course of Action instead of exporting each
|
||||||
|
Galaxy Cluster individually
|
||||||
|
- All clusters of a same galaxy are exported in
|
||||||
|
the same TTP, Threat Actor or Course of Action
|
||||||
|
- [stix2 import] Mapping galaxy cluster names with their corresponding
|
||||||
|
tag names. [chrisr3d]
|
||||||
|
|
||||||
|
- We map existing cluster names with the json file
|
||||||
|
of synonyms mapped with tag names generated by
|
||||||
|
MISP and introduced in the latest commit
|
||||||
|
- If there is not association, we just add a tag
|
||||||
|
looking like a galaxy tag name. The difference
|
||||||
|
is this tag will not be recognized as a proper
|
||||||
|
galaxy tag name and will stay as a tag
|
||||||
|
- [stix import] Passing a mapping of cluster name with tag names as
|
||||||
|
parameter of the import scripts. [chrisr3d]
|
||||||
|
|
||||||
|
- We map GalaxyCluster names and synonyms with the
|
||||||
|
associated tag names
|
||||||
|
- This mapping will be used in the python stix 1&2
|
||||||
|
import scripts so they can return the correct
|
||||||
|
tag names about galaxies
|
||||||
|
|
||||||
|
Fix
|
||||||
|
~~~
|
||||||
|
- [stix2 import] Quick issues fixing. [chrisr3d]
|
||||||
|
|
||||||
|
- Fixed issue that could happen sometimes during
|
||||||
|
an external pattern parsing when we split the
|
||||||
|
identifier of the pattern from the value.
|
||||||
|
We now make sure the identifier is stripped, so
|
||||||
|
we avoid issues with the mapping dictionaries
|
||||||
|
that could not recognize it
|
||||||
|
- Also displaying a warning message when we have
|
||||||
|
no attributes resulting from the parsing of an
|
||||||
|
external pattern or observable object
|
||||||
|
- [stix2 import] Small update on the mapping to work with some external
|
||||||
|
patterns seen recently. [chrisr3d]
|
||||||
|
- [stix2 import] Fixed some external observable objects import.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Fixed no longer existing variable in the mapping
|
||||||
|
script. [chrisr3d]
|
||||||
|
- [stix2 import] Added the function to parse external email-address
|
||||||
|
observable objects, that was missing. [chrisr3d]
|
||||||
|
- [UI] Typo. [Jakub Onderka]
|
||||||
|
- [stix2 import] Better way of parsing some attributes and objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- For single attributes that could be part of an
|
||||||
|
object and would lose some context if imported
|
||||||
|
as single attribute without their object
|
||||||
|
relation (mostly attributes of type 'text'),
|
||||||
|
we decide to import them as object anyway to
|
||||||
|
avoid the increase of context-less attributes
|
||||||
|
- Also cleaner way to parse observable objects and
|
||||||
|
patterns that will alwyas give single attributes
|
||||||
|
- [stix2 export] Typo in variable name. [chrisr3d]
|
||||||
|
- [stix2 export] Reverted the email object attribute 'from' export as
|
||||||
|
observable object. [chrisr3d]
|
||||||
|
|
||||||
|
- From-ref is always a single value, we cannot use
|
||||||
|
a list of references
|
||||||
|
- [stix2 export] Fixed email object attributes export into pattern.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Avoiding issues with attributes with no Galaxy field.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Fixed x509 object export. [chrisr3d]
|
||||||
|
|
||||||
|
- x509 fingerprint hashes parsing was pointing to
|
||||||
|
a part of a mapping dict which does not exist
|
||||||
|
- [stix2 import] Fixed external pattern parsing for pe section
|
||||||
|
attributes. [chrisr3d]
|
||||||
|
|
||||||
|
- As an example, instead of storing the full
|
||||||
|
pattern identifiers, like:
|
||||||
|
"file:extensions.'windows-pebinary-ext'.section.name"
|
||||||
|
we only store what is usefull (name) for the
|
||||||
|
parsing part where we check the mapping dict
|
||||||
|
to find the corresponding attribute type and
|
||||||
|
object_relation
|
||||||
|
- [stix2 import] Importing external vulnerabilities as single attribute
|
||||||
|
or object depending on the case. [chrisr3d]
|
||||||
|
|
||||||
|
- In other words, we made available the import of
|
||||||
|
vulnerabilities as single attributes when only
|
||||||
|
a name is present in the STIX object
|
||||||
|
- Was only importing vulnerability objects before,
|
||||||
|
which does not change if there is more than only
|
||||||
|
the name within the STIX vulnerability object
|
||||||
|
- [stix2 import] Removed unused variable that was used for debug
|
||||||
|
purposes. [chrisr3d]
|
||||||
|
- [stix2 import] Cleaner autonomous system observable import. [chrisr3d]
|
||||||
|
|
||||||
|
(for STIX documents generated with MISP)
|
||||||
|
- [stix2 import] Parsing timeline features on single attributes.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- As it is parsed for imported objects
|
||||||
|
- It adds timestamp, first_seen & last_seen values
|
||||||
|
on single attributes accordingly
|
||||||
|
- [stix2 import] Fixed email reply-to single attribute import.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Fixed payload_bin import into single MISP attribute.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Fixed email-reply-to export in observable object.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Removed unused import. [chrisr3d]
|
||||||
|
- [stix2 import] Importing PyMISP from the submoduled library.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- As it is in the currently used stix2 import
|
||||||
|
script which is going to be replaced by this one
|
||||||
|
- Avoids issues when the python library is not
|
||||||
|
installed with pip
|
||||||
|
- [stix2 import] More generic network-traffic references parsing.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also fixing some edge cases of reference parsing
|
||||||
|
with the wrong mapping
|
||||||
|
(network_traffic_references_mapping no longer exists)
|
||||||
|
- [stix2 import] Fixed single attributes import following changes on the
|
||||||
|
export part. [chrisr3d]
|
||||||
|
- [stix2 export] Making sure we have the required name field set while
|
||||||
|
exporting regkey values from a MISP regkey object to a STIX observed
|
||||||
|
data. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed regkey|value expor. [chrisr3d]
|
||||||
|
|
||||||
|
- Revert to the initial mapping that has been
|
||||||
|
changed to the wrong field: the value should be
|
||||||
|
mapped to the data field and instead of name
|
||||||
|
- [stix2 export] Removed unused mapping dictionary fields. [chrisr3d]
|
||||||
|
- [stix2 export] Removed object attributes added in file patterns for
|
||||||
|
test purposes. [chrisr3d]
|
||||||
|
- [stix2 export] Removed object attributes added for test purposes.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Writing import results as expected in the result file.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Fixed relationships parsing. [chrisr3d]
|
||||||
|
|
||||||
|
- Using iterators is good for a single iteration,
|
||||||
|
but not for more, including an if test
|
||||||
|
- Using tuples instead is better and avoids then
|
||||||
|
losing our relationships
|
||||||
|
- [stix1 import] Better parsing of malware instances within ttps.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- In some cases when malware instances within ttps
|
||||||
|
do not have a title but one or more name(s), we
|
||||||
|
need to use them instead of the title
|
||||||
|
- [stix1 import] Fixed malware instance parsing. [chrisr3d]
|
||||||
|
- [stix2 import] Typo. [chrisr3d]
|
||||||
|
- [stix2 test] Typo. [chrisr3d]
|
||||||
|
- [stix2 import] Handling external STIX file pattern properly.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- If there is no extension (case which has been
|
||||||
|
fixed in the few last commit), we need to check
|
||||||
|
if we have to create a MISP attribute or object
|
||||||
|
- We then check if we exctracted one attribute
|
||||||
|
from the pattern or more, and create respectively
|
||||||
|
a MISP attribute or object
|
||||||
|
- [stix2 import] Fixed monkey issues... [chrisr3d]
|
||||||
|
- [stix2 import] Using the expected parameters to handle the file, pe &
|
||||||
|
sections objects. [chrisr3d]
|
||||||
|
- [stix1 import] Some quick fixes on MISP objects parsing. [chrisr3d]
|
||||||
|
|
||||||
|
- Better handling on MISP object name parsing
|
||||||
|
- Importing properly MISP object uuid for course
|
||||||
|
of action objects
|
||||||
|
- [stix2 import] Importing event uuid from report. [chrisr3d]
|
||||||
|
|
||||||
|
- The event uuid is set when there is one report
|
||||||
|
- [stix2 import] Fixed timestamp parsing following the latest changes on
|
||||||
|
STIX2 export. [chrisr3d]
|
||||||
|
- [stix2 import] Fixed timestamp parsing. [chrisr3d]
|
||||||
|
|
||||||
|
- Fixed timestamp parsing of custom objects
|
||||||
|
- [stix2 import] Fixed attack-pattern & course-of-action object
|
||||||
|
attributes parsing. [chrisr3d]
|
||||||
|
|
||||||
|
- Avoids setting the ids flag to false when object
|
||||||
|
attributes do not come from an observable object
|
||||||
|
- [stix2 import] Fixed attack-pattern external_references parsing.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Fixed attack-pattern object export. [chrisr3d]
|
||||||
|
|
||||||
|
- Fixed the id attribute export
|
||||||
|
- Supporting expport of the newest 'references'
|
||||||
|
attribute added to the object template
|
||||||
|
- [stix2 import] Fixed file objects import. [chrisr3d]
|
||||||
|
|
||||||
|
- As it has been updated for file objects export,
|
||||||
|
we now better support potential multiple fields
|
||||||
|
like filename, path and fullpath
|
||||||
|
- Also handling properly the special case of a
|
||||||
|
file object with an extension field
|
||||||
|
- [stix2 export] Fixed special case of file with a path property and a
|
||||||
|
PE extension. [chrisr3d]
|
||||||
|
|
||||||
|
- If a file object had a path property and a PE
|
||||||
|
extension, the extension could be added to the
|
||||||
|
wrong part of the observable object
|
||||||
|
- We make sure here the extension is attached to
|
||||||
|
the observable object related to the file, and
|
||||||
|
not to the directory referenced by the file as
|
||||||
|
its path
|
||||||
|
- [stix2 export] Fixed files objects export (patterns & observable
|
||||||
|
object) [chrisr3d]
|
||||||
|
|
||||||
|
- Better handling of the data field for attributes
|
||||||
|
like malware-sample and attachment
|
||||||
|
- Support of path & fullpath attributes export
|
||||||
|
- Better handling of potential multiple attributes
|
||||||
|
like filename, path and fullpath
|
||||||
|
- [stix2 export] Fixed artifact name export in pattern as custom
|
||||||
|
property. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed x509-fingerprint-sha1 single attribute export.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Fixed regkey|data attribute export. [chrisr3d]
|
||||||
|
- [stix2 import] Fixed regkey values observable objects parsing.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Fixed & cleaned network traffic objects. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed reference typo in network traffic pattern.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Passing mapping variable name instead of the
|
||||||
|
dictionary. [chrisr3d]
|
||||||
|
|
||||||
|
- For all the generic parsing functions, we pass
|
||||||
|
the mapping variable name and get the attribute
|
||||||
|
afterwards instead of passing the dictionary
|
||||||
|
- [stix2 import] Fixed some observable and pattern parsing issues.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Quick custom property in pattern parsing fixed
|
||||||
|
- Fixed file and network socket observable objects
|
||||||
|
parsing
|
||||||
|
- [stix2 export] Fixed SocketExt properties exceptions catching.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- address_family is a required property, thus we
|
||||||
|
need to handle it separately
|
||||||
|
- protocol_family is optional and thus easier to
|
||||||
|
handle
|
||||||
|
- [stix2 import] Some patterns import fixed. [chrisr3d]
|
||||||
|
|
||||||
|
- AS attribute in asn object is now imported with
|
||||||
|
the 'AS' prefix
|
||||||
|
- Importing properly attachment attributes in file
|
||||||
|
objects
|
||||||
|
- pe mapping enhanced
|
||||||
|
- [stix2 export] Fixed file & vulnerability patterns export. [chrisr3d]
|
||||||
|
- [stix2 import] Fixed malware sample import in file objects. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed custom properties for vulnerability and attack
|
||||||
|
pattern objects. [chrisr3d]
|
||||||
|
|
||||||
|
- Dashes ('-') in object relations should be
|
||||||
|
replaced by underscores as custom properties
|
||||||
|
only accept underscores
|
||||||
|
- [stix2 export] Better file objects export and joining patterns from
|
||||||
|
list instead of concatenating strings. [chrisr3d]
|
||||||
|
- [stix2 import] Better import for some objects. [chrisr3d]
|
||||||
|
|
||||||
|
- Support of custom properties that are lists
|
||||||
|
- Support of protocol attribute in network socket
|
||||||
|
object
|
||||||
|
- Support of group attribute in user account
|
||||||
|
object
|
||||||
|
- [stix2 export] Better export for object attributes of vulnerability
|
||||||
|
and attack pattern objects. [chrisr3d]
|
||||||
|
|
||||||
|
- Need to use custom properties in some cases
|
||||||
|
- [stix2 import] Importing pe attributes from patterns within the pe
|
||||||
|
object and not in the file object. [chrisr3d]
|
||||||
|
- [stix2 import] Fixed file pattern import. [chrisr3d]
|
||||||
|
- [stix2 import] Better network connection patterns parsing. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed export of port attribute in network traffic
|
||||||
|
patterns. [chrisr3d]
|
||||||
|
- [stix2 export] Fixed network traffic references in patterns.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 import] Fixed network socket pattern values parsing. [chrisr3d]
|
||||||
|
|
||||||
|
- We do not want to import the single quotes that
|
||||||
|
are all around the pattern values
|
||||||
|
- [stix2 export] Exporting TLP tags as marking definition. [chrisr3d]
|
||||||
|
|
||||||
|
- Marking definition in the case of TLP was
|
||||||
|
missing and only the reference to the marking
|
||||||
|
definition was exported, but not the actual
|
||||||
|
marking definition object
|
||||||
|
- [stix2 import] Fixed MISP Object creation. [chrisr3d]
|
||||||
|
|
||||||
|
- MISP Object creation function used for attack
|
||||||
|
pattern & course of action objects, so they get
|
||||||
|
the correct uuid
|
||||||
|
- MISP Object creation trying to parse the first
|
||||||
|
seen & last seen values without raising issues
|
||||||
|
when the object parsed does not have any
|
||||||
|
- [stix2 export] Fixed file content ref for malware sample exports.
|
||||||
|
[chrisr3d]
|
||||||
|
- [stix2 export] Fixed email attachment export. [chrisr3d]
|
||||||
|
- [stix2 export] Exporitng process attributes in patterns as intended.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Handling the child ref(s), parent ref, and image
|
||||||
|
name values in process patterns
|
||||||
|
- [stix1 import] Parsing COA_Taken objects as MISP object. [chrisr3d]
|
||||||
|
- [stix1 export] Exporting category and value in STIX objects title and
|
||||||
|
not the attribute/object id. [chrisr3d]
|
||||||
|
- [stix1 export] No longer exporting object IDs. [chrisr3d]
|
||||||
|
|
||||||
|
- ThreatActors and TTPs titles only use categories
|
||||||
|
and values of the attribute/object to define the
|
||||||
|
title, and not the attribute/object id anymore
|
||||||
|
- [stix1 export] Various fixes. [chrisr3d]
|
||||||
|
|
||||||
|
- Got rid of some variables and calling some
|
||||||
|
functions directly to parse data withtout
|
||||||
|
storing it
|
||||||
|
- TTPs, Courses of action and Threat Actors are
|
||||||
|
now referenced in related objects only when they
|
||||||
|
come from attributes/objects in MISP and not
|
||||||
|
when they come from galaxies
|
||||||
|
- [stix1 export] Tiny fixes. [chrisr3d]
|
||||||
|
|
||||||
|
- Making sure adding an indicator type fails
|
||||||
|
because of the mapping between attribute types
|
||||||
|
and indicator types does not support a specific
|
||||||
|
type, and not because of the indicator not
|
||||||
|
accepting the type we want it to have
|
||||||
|
- Making sure CAPEC IDs are always starting with
|
||||||
|
'CAPEC' in the AttackPattern objects we create
|
||||||
|
- [stix1 export] Fixed email attachment related objects uuid. [chrisr3d]
|
||||||
|
|
||||||
|
- Email message related objects representing the
|
||||||
|
email attachments now have the correct uuid of
|
||||||
|
the attachment attribute
|
||||||
|
- Before, a random uuid was used, due to the file
|
||||||
|
object losing its parent properties while being
|
||||||
|
switched from a File object type to a related
|
||||||
|
object type
|
||||||
|
- [stix1 export] Referencing COAs as RelatedCOAs. [chrisr3d]
|
||||||
|
- [stix1 export] Removed function no longer used. [chrisr3d]
|
||||||
|
- [stix1 export] Export only cluster values as name or title. [chrisr3d]
|
||||||
|
|
||||||
|
- No longer exporting the Galaxy name within the
|
||||||
|
name or title, since we have this information
|
||||||
|
somewhere else and the name or title should
|
||||||
|
only be the Galaxy cluster name value
|
||||||
|
- [stix import] Some strings are defined in a cleaner way. [chrisr3d]
|
||||||
|
- [stix2 import] Skipping adding Galaxy info in the Galaxy field and
|
||||||
|
only importing it as tag. [chrisr3d]
|
||||||
|
- [ACL] unpublished_private global setting tightened to include
|
||||||
|
correlations. [iglocska]
|
||||||
|
|
||||||
|
- Thanks to Jakub Onderka for reporting and providing a fix to this!
|
||||||
|
- [security] missing ACL lookup on attribute correlations. [iglocska]
|
||||||
|
|
||||||
|
- attribute correlation ACL checks are skipped when querying the attribute restsearch API revealing metadata about a correlating but unreachable attribute.
|
||||||
|
|
||||||
|
- Thanks to Jakub Onderka for his tireless work and for reporting this!
|
||||||
|
|
||||||
|
Other
|
||||||
|
~~~~~
|
||||||
|
- Merge branch '2.4' of github.com:MISP/MISP into 2.4. [iglocska]
|
||||||
|
- Merge branch 'rework_stix' into 2.4. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge pull request #6028 from JakubOnderka/patch-115. [Andras Iklody]
|
||||||
|
|
||||||
|
fix: [UI] Typo
|
||||||
|
- Merge pull request #6022 from MISP/rework_stix. [Andras Iklody]
|
||||||
|
|
||||||
|
STIX parsing updates
|
||||||
|
- Cleanup: [stix] Cleaned up the recently changed scripts. [chrisr3d]
|
||||||
|
|
||||||
|
Including:
|
||||||
|
- Removed some unused imports and variable
|
||||||
|
- Renamed some variable which could have been
|
||||||
|
built-in methods redefinition
|
||||||
|
- Typos
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge pull request #5916 from JakubOnderka/patch-100. [Andras Iklody]
|
||||||
|
|
||||||
|
chg: [correlations] Big speedup when correlating CIDR
|
||||||
|
- Merge pull request #6019 from D4-project/2.4. [Andras Iklody]
|
||||||
|
|
||||||
|
add [widget] Authentication failure widget
|
||||||
|
- Add [widget] Authentication failure widget. [Jean-Louis Huynen]
|
||||||
|
- Wip: [stix2 import] More complete external patterns mapping.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external domain, ip & network traffic
|
||||||
|
patterns. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external network traffic patterns.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external email patterns. [chrisr3d]
|
||||||
|
|
||||||
|
- Parsing function to split attachments fields
|
||||||
|
from all the other fields already implemented,
|
||||||
|
we just added the attachment parsing and the
|
||||||
|
attributes handling at the end
|
||||||
|
- Also slight fixes on the from, to and cc refs
|
||||||
|
following the last fix on the export side
|
||||||
|
- Wip: [stix2 import] Handling import case for indicators of which we
|
||||||
|
already parsed the pattern. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external process indicators. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external url indicator based on the
|
||||||
|
pattern mapping already implemented. [chrisr3d]
|
||||||
|
|
||||||
|
- tl;dr: We just took the parsed attributes and
|
||||||
|
callled the appropriate function to handle the
|
||||||
|
import case (attribute or object)
|
||||||
|
- Wip: [stix2 import] Importing external user-account indicators.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also fixed some user-account and credential
|
||||||
|
mapping dictionaries
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Fix"[stix2 import] Fixed process observable objects parsing for STIX
|
||||||
|
documents generated with MISP. [chrisr3d]
|
||||||
|
|
||||||
|
- Little typo and copy-paste issue
|
||||||
|
- Wip: [stix2 import] Parsing external process observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also changed parsing of process observable
|
||||||
|
objects from STIX documents generated with MISP
|
||||||
|
to apply the same logic to both use cases
|
||||||
|
- Wip: [stix2 import] Parsing external user_account observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Mapping into credential or user-account MISP
|
||||||
|
objects depending on the case
|
||||||
|
- Wip: [stix2 import] Finally parsing properly external network traffic
|
||||||
|
observable objects with their references and potential extensions.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- After struggling a lot on it, we ended up
|
||||||
|
parsing external network traffic observable
|
||||||
|
objects independently depending on the actual
|
||||||
|
references they have or not
|
||||||
|
- Chosing this approach instead of the common
|
||||||
|
parsing function handling the different use
|
||||||
|
cases, we can parse each observable object
|
||||||
|
depending on the case, and use common function
|
||||||
|
then when we are sure we determined the actual
|
||||||
|
situation
|
||||||
|
- We no longer start from a common function trying
|
||||||
|
to determine the case using lots of tests, we
|
||||||
|
already know which case it is and go to the
|
||||||
|
common point afterwards
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Network traffic references parsing function for
|
||||||
|
further reuse. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external autonomous system observable
|
||||||
|
objects. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external x509 observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing mac-address external observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also changed the recently changed mutex import
|
||||||
|
to reuse a function to parse all observable
|
||||||
|
objects of an observed-data object at once to
|
||||||
|
import single attributes
|
||||||
|
- Wip: [stix2 import] Importing external mutex observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also change on a function name for more clarity
|
||||||
|
and to differenciate more easily functions for
|
||||||
|
observable objects and patterns
|
||||||
|
- Wip: [stix2 import] Importing external registry-key observable
|
||||||
|
objects. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Updated external observable mapping: files with
|
||||||
|
artifact & directory references. [chrisr3d]
|
||||||
|
|
||||||
|
- The parsing logic is already there since files
|
||||||
|
with artifact references and files with directory
|
||||||
|
references are supported. We just updated here
|
||||||
|
the mapping dictionary
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing external url observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Added warning message if not all the observable
|
||||||
|
objects are referenced by an email-message object. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Import of external email message & address
|
||||||
|
observable objects. [chrisr3d]
|
||||||
|
|
||||||
|
- Reuse of some parsing functions for external and
|
||||||
|
MISP generated STIX files
|
||||||
|
- Added an email references mapping dict to help
|
||||||
|
parsing email addresses, body & content refs
|
||||||
|
references by email message objects
|
||||||
|
- Fixed another indentation issue
|
||||||
|
- Wip: [stix2 import] Import of domain and ip observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also quick indentation fix
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Import of network-traffic and ip external
|
||||||
|
observable objects. [chrisr3d]
|
||||||
|
|
||||||
|
- Ongoing rework for external observable objects
|
||||||
|
and patterns in progress
|
||||||
|
- Wip: [stix2 import] Import of external file observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Support of PE extension to create PE object(s)
|
||||||
|
with the corresponding section(s) alongside the
|
||||||
|
file object import
|
||||||
|
- As always with pe and sections, the appropriate
|
||||||
|
references are added too
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Starting parsing external observable objects.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Started with file observables
|
||||||
|
- Making 'filter_main_object' function available
|
||||||
|
for both subclasses to split the observable
|
||||||
|
object type we want and all the references
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Struggling with the files and payloads import.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Removed unused mapping dict + moved constant to
|
||||||
|
the mapping script. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 export] Moved dictionaries in the mapping file & using the
|
||||||
|
complete import path instead of import * from the mapping file.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- We control and know which mapping dictionary we
|
||||||
|
call and that they come from the mapping script
|
||||||
|
- Started moving all the mapping dictionaries in
|
||||||
|
the mapping file
|
||||||
|
- Attributes parsing function will be moved into
|
||||||
|
the main script
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Moving small parsing functions to the main script.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
- Also passing the function names only instead of
|
||||||
|
storing functions themselves in the dictionary
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Parsing single external IP v4 or v6 address.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Parsing external relationships, galaxies, tags &
|
||||||
|
reports. [chrisr3d]
|
||||||
|
|
||||||
|
(+ Quick fix on internal tags handling)
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Add: [stix2test] New argument to evaluate events using filenames only
|
||||||
|
and avoid to query MISP. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Handling File objects with PE extension &
|
||||||
|
sections. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Separating file extensions to be parsed later.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Better attack-pattern external references parsing
|
||||||
|
+ parsing external galaxies. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Parsing attack-pattern, course-of-action and
|
||||||
|
vulnerability objects from external stix files. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Making difference between external and from MISP
|
||||||
|
for some STIX object types. [chrisr3d]
|
||||||
|
|
||||||
|
- Including Attack Pattern, Course of Action and
|
||||||
|
Vulnerability
|
||||||
|
- Also better file pattern parsing
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Better parsing for more external patterns.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Some more external pattern mapped. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Starting parsing external patterns. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Some quick clean-up. [chrisr3d]
|
||||||
|
|
||||||
|
- Preparing for the future 2.1 import
|
||||||
|
- Removing mapping variables no longer used
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Importing reports external references as links.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Proper parsing of galaxies, and tags. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into rework_stix.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Loading relationships in a dictionary. [chrisr3d]
|
||||||
|
|
||||||
|
- Thus we can parse them afterwards depending on
|
||||||
|
the type of objects they put into relationship
|
||||||
|
- Wip: [stix2 import] Properly loading galaxies as tags. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Import of CourseOfAction, AttackPattern and
|
||||||
|
Vulnerability as objects reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 export] Defining relationships between observed-data and
|
||||||
|
galaxy objects. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] Updated mapping library + removed
|
||||||
|
disable_correlation flags. [chrisr3d]
|
||||||
|
|
||||||
|
- Since we use the object templates directly for
|
||||||
|
the objects creation, we do not need to have
|
||||||
|
the flag here.
|
||||||
|
- Wip: [stix2 import] Observable import rework completed. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Process observables import reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] More observable objects reworked. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] User Account objects import reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] ASN observable import reworked + functions
|
||||||
|
ordered. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Credential observable import + standard observable
|
||||||
|
parsing function reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Network socket import reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Import of network connection objects from
|
||||||
|
observable. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Started reworking observable objects import.
|
||||||
|
[chrisr3d]
|
||||||
|
- Wip: [stix2 import] All known MISP objects mapped with STIX patterning
|
||||||
|
are now reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Email pattern import. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] File patterns import reworked. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Cleaner pattern import into objects. [chrisr3d]
|
||||||
|
- Add: [stix2 export] Exporting process image attribute in observable
|
||||||
|
objects. [chrisr3d]
|
||||||
|
- Wip: [stix2 import] Reworking stix2 import. [chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
- Add: [stix1 export] Added malpedia in the list of mapped galaxies.
|
||||||
|
[chrisr3d]
|
||||||
|
- Merge branch '2.4' of https://github.com/MISP/MISP into 2.4.
|
||||||
|
[chrisr3d]
|
||||||
|
|
||||||
|
|
||||||
v2.4.127 (2020-06-16)
|
v2.4.127 (2020-06-16)
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue