mirror of https://github.com/MISP/PyMISP
Merge remote-tracking branch 'upstream/master'
Tom King
commit
545d5cc1b7
|
|
@ -2,6 +2,103 @@ Changelog
|
|||
=========
|
||||
|
||||
|
||||
v2.4.99 (2018-12-06)
|
||||
--------------------
|
||||
|
||||
New
|
||||
~~~
|
||||
- Auto generate doc for PyMISPExpanded. [Raphaël Vinot]
|
||||
- Search_index in ExpandedPyMISP, cleanup, update jupyter. [Raphaël
|
||||
Vinot]
|
||||
- Add log search. [Raphaël Vinot]
|
||||
- Add test for pushing an event to ZMQ. [Raphaël Vinot]
|
||||
- Change_distribution method. [Raphaël Vinot]
|
||||
- Add test cases for sightings, cleanup. [Raphaël Vinot]
|
||||
- [example] Added sighting rest search example. [Sami Mokaddem]
|
||||
- [sighting] Added support of sighting REST API. [Sami Mokaddem]
|
||||
- Allow to pass csv to return_format in search. [Raphaël Vinot]
|
||||
- Page/limit in search. [Raphaël Vinot]
|
||||
|
||||
Changes
|
||||
~~~~~~~
|
||||
- Bump Changelog. [Raphaël Vinot]
|
||||
- Bump version. [Raphaël Vinot]
|
||||
- Bump misp-objects & describeTypes. [Raphaël Vinot]
|
||||
- Bump Changelog. [Raphaël Vinot]
|
||||
- Version bump. [Raphaël Vinot]
|
||||
- Bump misp-objects. [Raphaël Vinot]
|
||||
- Add test cases for default distribution levels. [Raphaël Vinot]
|
||||
- Include proposals in attributes search. [Dawid Czarnecki]
|
||||
|
||||
Add includeProposals param to the search method
|
||||
- Bump misp-objects. [Raphaël Vinot]
|
||||
- Update readme to document testing. [Raphaël Vinot]
|
||||
- Fixes & update Jupyter. [Raphaël Vinot]
|
||||
- [tuto] Update search. [Raphaël Vinot]
|
||||
- Add a script to load the API key from the file system (training VM)
|
||||
[Raphaël Vinot]
|
||||
- Bump misp-objects. [Raphaël Vinot]
|
||||
- Add print in testlive to debug travis. [Raphaël Vinot]
|
||||
- Bump objects. [Raphaël Vinot]
|
||||
|
||||
Fix
|
||||
~~~
|
||||
- Auto generate doc for PyMISPExpanded. [Raphaël Vinot]
|
||||
- Test failing on travis... [Raphaël Vinot]
|
||||
- Properly handle errors on event creation/update. [Raphaël Vinot]
|
||||
- Test case. [Raphaël Vinot]
|
||||
- Do not run the zmq test on travis. [Raphaël Vinot]
|
||||
- Type of quick_filter. [Raphaël Vinot]
|
||||
- Quick_filter was broken. [Raphaël Vinot]
|
||||
- Properly initialize the config when jupyter runs on the VM. [Raphaël
|
||||
Vinot]
|
||||
- Travis run. [Raphaël Vinot]
|
||||
- Readme update + python3 + pep8. [Christophe Vandeplas]
|
||||
|
||||
align python path to readme specifying python3
|
||||
- Feed-generator gitignore. [Christophe Vandeplas]
|
||||
- Test cases. [Raphaël Vinot]
|
||||
|
||||
Other
|
||||
~~~~~
|
||||
- Merge branch 'master' of github.com:MISP/PyMISP. [Raphaël Vinot]
|
||||
- Merge pull request #310 from DragonDev1906/master. [Raphaël Vinot]
|
||||
|
||||
Added get_object & get_attribute (by ID)
|
||||
- Dded get_object & get_attribute. [DragonDev1906]
|
||||
- Merge pull request #307 from garanews/patch-1. [Raphaël Vinot]
|
||||
|
||||
fix for last pymisp version
|
||||
- Fix for last pymisp version. [garanews]
|
||||
- Merge branch 'master' of github.com:MISP/PyMISP. [Raphaël Vinot]
|
||||
- Merge pull request #305 from dawid-
|
||||
czarnecki/feature/include_proposals. [Raphaël Vinot]
|
||||
|
||||
chg: Include proposals in attributes search
|
||||
- Merge pull request #301 from deralexxx/patch-7. [Raphaël Vinot]
|
||||
|
||||
mention virtualenv
|
||||
- Mention virtualenv. [Alexander J]
|
||||
|
||||
mide make sense for people who want to use it with virtualenv
|
||||
- Merge branch 'master' of github.com:MISP/PyMISP. [Raphaël Vinot]
|
||||
- Be more precise with the supported time indicators. [Sascha
|
||||
Rommelfangen]
|
||||
- Fixed documentation bug. [Sascha Rommelfangen]
|
||||
- Merge branch 'master' of github.com:MISP/PyMISP. [Raphaël Vinot]
|
||||
- Merge pull request #295 from 3c7/fix/search_index_date. [Raphaël
|
||||
Vinot]
|
||||
|
||||
Fixes date parameters for search_index() function
|
||||
- Fixes date parameters for search_index() function. [Nils Kuhnert]
|
||||
- Merge branch 'sightingAPI' [Raphaël Vinot]
|
||||
- Merge branch 'master' into sightingAPI. [Raphaël Vinot]
|
||||
- Merge pull request #285 from juju4/devel. [Raphaël Vinot]
|
||||
|
||||
align examples on custom usage of misp_verifycert
|
||||
- Align examples on custom usage of misp_verifycert. [juju4]
|
||||
|
||||
|
||||
v2.4.96 (2018-10-12)
|
||||
--------------------
|
||||
|
||||
|
|
@ -22,6 +119,7 @@ New
|
|||
|
||||
Changes
|
||||
~~~~~~~
|
||||
- Bump changelog. [Raphaël Vinot]
|
||||
- Bump version. [Raphaël Vinot]
|
||||
- Bump misp-objects. [Raphaël Vinot]
|
||||
- Allow to pass a json string to direct_call. [Raphaël Vinot]
|
||||
|
|
@ -32,6 +130,7 @@ Changes
|
|||
|
||||
Fix
|
||||
~~~
|
||||
- Test cases sample files. [Raphaël Vinot]
|
||||
- Prevent checking length on a integer. [Sami Mokaddem]
|
||||
- Direct call & add example. [Raphaël Vinot]
|
||||
- Disable test for travis, take 2. [Raphaël Vinot]
|
||||
|
|
|
|||
|
|
@ -14,6 +14,12 @@ PyMISP
|
|||
.. autoclass:: PyMISP
|
||||
:members:
|
||||
|
||||
PyMISPExpanded (Python 3.6+ only)
|
||||
---------------------------------
|
||||
|
||||
.. autoclass:: PyMISPExpanded
|
||||
:members:
|
||||
|
||||
MISPAbstract
|
||||
------------
|
||||
|
||||
|
|
|
|||
|
|
@ -20,10 +20,17 @@ if __name__ == '__main__':
|
|||
args = parser.parse_args()
|
||||
|
||||
pymisp = PyMISP(misp_url, misp_key, misp_verifycert)
|
||||
template = pymisp.get_object_templates_list()
|
||||
if 'response' in template.keys():
|
||||
template = template['response']
|
||||
try:
|
||||
template_id = [x['ObjectTemplate']['id'] for x in pymisp.get_object_templates_list() if x['ObjectTemplate']['name'] == args.type][0]
|
||||
template_ids = [x['ObjectTemplate']['id'] for x in template if x['ObjectTemplate']['name'] == args.type]
|
||||
if len(template_ids) > 0:
|
||||
template_id = template_ids[0]
|
||||
else:
|
||||
raise IndexError
|
||||
except IndexError:
|
||||
valid_types = ", ".join([x['ObjectTemplate']['name'] for x in pymisp.get_object_templates_list()])
|
||||
valid_types = ", ".join([x['ObjectTemplate']['name'] for x in template])
|
||||
print ("Template for type %s not found! Valid types are: %s" % (args.type, valid_types))
|
||||
exit()
|
||||
|
||||
|
|
|
|||
|
|
@ -1,8 +1,12 @@
|
|||
#!/usr/bin/env python
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
from pymisp import PyMISP
|
||||
from keys import url, key
|
||||
from pymisp import ExpandedPyMISP
|
||||
try:
|
||||
from keys import url, key
|
||||
except ImportError:
|
||||
url = 'http://localhost:8080'
|
||||
key = '8h0gHbhS0fv6JUOlTED0AznLXFbf83TYtQrCycqb'
|
||||
import argparse
|
||||
import tools
|
||||
|
||||
|
|
@ -13,7 +17,7 @@ if __name__ == '__main__':
|
|||
parser.add_argument("-a", "--attribute", type=int, help="Number of attributes per event (default 3000)")
|
||||
args = parser.parse_args()
|
||||
|
||||
misp = PyMISP(url, key, True, 'json')
|
||||
misp = ExpandedPyMISP(url, key, True)
|
||||
|
||||
if args.limit is None:
|
||||
args.limit = 1
|
||||
|
|
|
|||
|
|
@ -4,6 +4,7 @@
|
|||
import random
|
||||
from random import randint
|
||||
import string
|
||||
from pymisp import MISPEvent
|
||||
|
||||
|
||||
def randomStringGenerator(size, chars=string.ascii_lowercase + string.digits):
|
||||
|
|
@ -63,12 +64,16 @@ def create_dummy_event(misp):
|
|||
|
||||
|
||||
def create_massive_dummy_events(misp, nbattribute):
|
||||
event = misp.new_event(0, 4, 0, 'massive dummy event')
|
||||
eventid = event['Event']['id']
|
||||
event = MISPEvent()
|
||||
event.info = 'massive dummy event'
|
||||
event = misp.add_event(event)
|
||||
print(event)
|
||||
eventid = event.id
|
||||
distribution = '0'
|
||||
functions = [floodtxt, floodip, flooddomain, flooddomainip, floodemail, floodattachment]
|
||||
for i in range(nbattribute):
|
||||
choice = randint(0, 5)
|
||||
if choice == 5:
|
||||
floodattachment(misp, eventid, event['Event']['distribution'], False, 'Payload delivery', '', event['Event']['info'], event['Event']['analysis'], event['Event']['threat_level_id'])
|
||||
floodattachment(misp, eventid, distribution, False, 'Payload delivery', '', event.info, event.analysis, event.threat_level_id)
|
||||
else:
|
||||
functions[choice](misp, event)
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
#!/usr/bin/env python
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
import argparse
|
||||
|
||||
from pymisp import PyMISP
|
||||
from pymisp import ExpandedPyMISP
|
||||
from keys import misp_url, misp_key, misp_verifycert
|
||||
|
||||
|
||||
|
|
@ -14,12 +14,20 @@ if __name__ == '__main__':
|
|||
parser.add_argument("-o", "--object_attribute", nargs='+', help="Object attribute column names")
|
||||
parser.add_argument("-t", "--misp_types", nargs='+', help="MISP types to fetch (ip-src, hostname, ...)")
|
||||
parser.add_argument("-c", "--context", action='store_true', help="Add event level context (tags...)")
|
||||
parser.add_argument("-i", "--ignore", action='store_true', help="Returns the attributes even if the event isn't published, or the attribute doesn't have the to_ids flag")
|
||||
parser.add_argument("-f", "--outfile", help="Output file to write the CSV.")
|
||||
|
||||
args = parser.parse_args()
|
||||
pymisp = PyMISP(misp_url, misp_key, misp_verifycert, debug=True)
|
||||
response = pymisp.get_csv(args.event_id, args.attribute, args.object_attribute, args.misp_types, args.context, args.ignore)
|
||||
pymisp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert, debug=True)
|
||||
attr = []
|
||||
if args.attribute:
|
||||
attr += args.attribute
|
||||
if args.object_attribute:
|
||||
attr += args.object_attribute
|
||||
if not attr:
|
||||
attr = None
|
||||
print(args.context)
|
||||
response = pymisp.search(return_format='csv', eventid=args.event_id, requested_attributes=attr,
|
||||
type_attribute=args.misp_types, include_context=args.context)
|
||||
|
||||
if args.outfile:
|
||||
with open(args.outfile, 'w') as f:
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
__version__ = '2.4.96'
|
||||
__version__ = '2.4.99'
|
||||
import logging
|
||||
import functools
|
||||
import warnings
|
||||
|
|
@ -32,7 +32,7 @@ def deprecated(func):
|
|||
|
||||
|
||||
try:
|
||||
from .exceptions import PyMISPError, NewEventError, NewAttributeError, MissingDependency, NoURL, NoKey, InvalidMISPObject, UnknownMISPObjectTemplate, PyMISPInvalidFormat, MISPServerError, PyMISPNotImplementedYet, PyMISPUnexpectedResponse # noqa
|
||||
from .exceptions import PyMISPError, NewEventError, NewAttributeError, MissingDependency, NoURL, NoKey, InvalidMISPObject, UnknownMISPObjectTemplate, PyMISPInvalidFormat, MISPServerError, PyMISPNotImplementedYet, PyMISPUnexpectedResponse, PyMISPEmptyResponse # noqa
|
||||
from .api import PyMISP # noqa
|
||||
from .abstract import AbstractMISP, MISPEncode, MISPTag, Distribution, ThreatLevel, Analysis # noqa
|
||||
from .mispevent import MISPEvent, MISPAttribute, MISPObjectReference, MISPObjectAttribute, MISPObject, MISPUser, MISPOrganisation, MISPSighting, MISPLog # noqa
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ from io import BytesIO, open
|
|||
import zipfile
|
||||
|
||||
from . import __version__, deprecated
|
||||
from .exceptions import PyMISPError, SearchError, NoURL, NoKey
|
||||
from .exceptions import PyMISPError, SearchError, NoURL, NoKey, PyMISPEmptyResponse
|
||||
from .mispevent import MISPEvent, MISPAttribute, MISPUser, MISPOrganisation, MISPSighting, MISPFeed, MISPObject
|
||||
from .abstract import AbstractMISP, MISPEncode
|
||||
|
||||
|
|
@ -157,6 +157,11 @@ class PyMISP(object):
|
|||
if data is None:
|
||||
req = requests.Request(request_type, url)
|
||||
else:
|
||||
if not isinstance(data, str):
|
||||
if isinstance(data, dict):
|
||||
# Remove None values.
|
||||
data = {k: v for k, v in data.items() if v is not None}
|
||||
data = json.dumps(data)
|
||||
req = requests.Request(request_type, url, data=data)
|
||||
if self.asynch and background_callback is not None:
|
||||
local_session = FuturesSession
|
||||
|
|
@ -227,6 +232,8 @@ class PyMISP(object):
|
|||
json_response = response.json()
|
||||
except ValueError:
|
||||
# If the server didn't return a JSON blob, we've a problem.
|
||||
if not len(response.text):
|
||||
raise PyMISPEmptyResponse('The server returned an empty response. \n{}\n{}\n'.format(response.request.headers, response.request.body))
|
||||
raise PyMISPError(everything_broken.format(response.request.headers, response.request.body, response.text))
|
||||
|
||||
errors = []
|
||||
|
|
@ -336,6 +343,24 @@ class PyMISP(object):
|
|||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
def get_object(self, obj_id):
|
||||
"""Get an object
|
||||
|
||||
:param obj_id: Object id to get
|
||||
"""
|
||||
url = urljoin(self.root_url, 'objects/view/{}'.format(obj_id))
|
||||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
def get_attribute(self, att_id):
|
||||
"""Get an attribute
|
||||
|
||||
:param att_id: Attribute id to get
|
||||
"""
|
||||
url = urljoin(self.root_url, 'attributes/view/{}'.format(att_id))
|
||||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
def add_event(self, event):
|
||||
"""Add a new event
|
||||
|
||||
|
|
@ -1212,12 +1237,14 @@ class PyMISP(object):
|
|||
query['event_timestamp'] = kwargs.pop('event_timestamp', None)
|
||||
query['includeProposals'] = kwargs.pop('includeProposals', None)
|
||||
|
||||
if kwargs:
|
||||
logger.info('Some unknown parameters are in kwargs. appending as-is: {}'.format(', '.join(kwargs.keys())))
|
||||
# Add all other keys as-is.
|
||||
query.update({k: v for k, v in kwargs.items()})
|
||||
|
||||
# Cleanup
|
||||
query = {k: v for k, v in query.items() if v is not None}
|
||||
|
||||
if kwargs:
|
||||
raise SearchError('Unused parameter: {}'.format(', '.join(kwargs.keys())))
|
||||
|
||||
# Create a session, make it async if and only if we have a callback
|
||||
return self.__query('restSearch/download', query, controller, async_callback)
|
||||
|
||||
|
|
@ -1410,6 +1437,15 @@ class PyMISP(object):
|
|||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
def get_users_statistics(self, context='data'):
|
||||
"""Get users statistics from the MISP instance"""
|
||||
availables_contexts = ['data', 'orgs', 'users', 'tags', 'attributehistogram', 'sightings', 'attackMatrix']
|
||||
if context not in availables_contexts:
|
||||
context = 'data'
|
||||
url = urljoin(self.root_url, 'users/statistics/{}.json'.format(context))
|
||||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
# ############## Sightings ##################
|
||||
|
||||
def sighting_per_id(self, attribute_id):
|
||||
|
|
@ -2270,11 +2306,9 @@ class PyMISP(object):
|
|||
|
||||
def get_object_template_id(self, object_uuid):
|
||||
"""Gets the template ID corresponting the UUID passed as parameter"""
|
||||
templates = self.get_object_templates_list()
|
||||
for t in templates:
|
||||
if t['ObjectTemplate']['uuid'] == object_uuid:
|
||||
return t['ObjectTemplate']['id']
|
||||
raise Exception('Unable to find template uuid {} on the MISP instance'.format(object_uuid))
|
||||
url = urljoin(self.root_url, 'objectTemplates/view/{}'.format(object_uuid))
|
||||
response = self._prepare_request('GET', url)
|
||||
return self._check_response(response)
|
||||
|
||||
def update_object_templates(self):
|
||||
url = urljoin(self.root_url, '/objectTemplates/update')
|
||||
|
|
|
|||
|
|
@ -6,7 +6,6 @@ from .api import PyMISP, everything_broken
|
|||
from .mispevent import MISPEvent, MISPAttribute, MISPSighting, MISPLog
|
||||
from typing import TypeVar, Optional, Tuple, List, Dict
|
||||
from datetime import date, datetime
|
||||
import json
|
||||
import csv
|
||||
|
||||
import logging
|
||||
|
|
@ -70,7 +69,7 @@ class ExpandedPyMISP(PyMISP):
|
|||
# The server returns a json message with the error details
|
||||
error_message = response.json()
|
||||
logger.error(f'Something went wrong ({response.status_code}): {error_message}')
|
||||
return {'errors': [(response.status_code, error_message)]}
|
||||
return {'errors': (response.status_code, error_message)}
|
||||
|
||||
# At this point, we had no error.
|
||||
|
||||
|
|
@ -80,11 +79,15 @@ class ExpandedPyMISP(PyMISP):
|
|||
logger.debug(response)
|
||||
if isinstance(response, dict) and response.get('response') is not None:
|
||||
# Cleanup.
|
||||
return response.get('response')
|
||||
response = response['response']
|
||||
return response
|
||||
except Exception:
|
||||
if logger.isEnabledFor(logging.DEBUG):
|
||||
logger.debug(response.text)
|
||||
if not len(response.content):
|
||||
# Empty response
|
||||
logger.error('Got an empty response.')
|
||||
return {'errors': 'The response is empty.'}
|
||||
return response.text
|
||||
|
||||
def get_event(self, event_id: int):
|
||||
|
|
@ -97,6 +100,8 @@ class ExpandedPyMISP(PyMISP):
|
|||
created_event = super().add_event(event)
|
||||
if isinstance(created_event, str):
|
||||
raise NewEventError(f'Unexpected response from server: {created_event}')
|
||||
elif 'errors' in created_event:
|
||||
return created_event
|
||||
e = MISPEvent()
|
||||
e.load(created_event)
|
||||
return e
|
||||
|
|
@ -105,6 +110,8 @@ class ExpandedPyMISP(PyMISP):
|
|||
updated_event = super().update_event(event.uuid, event)
|
||||
if isinstance(updated_event, str):
|
||||
raise UpdateEventError(f'Unexpected response from server: {updated_event}')
|
||||
elif 'errors' in updated_event:
|
||||
return updated_event
|
||||
e = MISPEvent()
|
||||
e.load(updated_event)
|
||||
return e
|
||||
|
|
@ -113,6 +120,8 @@ class ExpandedPyMISP(PyMISP):
|
|||
updated_attribute = super().update_attribute(attribute.uuid, attribute)
|
||||
if isinstance(updated_attribute, str):
|
||||
raise UpdateAttributeError(f'Unexpected response from server: {updated_attribute}')
|
||||
elif 'errors' in updated_attribute:
|
||||
return updated_attribute
|
||||
a = MISPAttribute()
|
||||
a.from_dict(**updated_attribute)
|
||||
return a
|
||||
|
|
@ -172,10 +181,7 @@ class ExpandedPyMISP(PyMISP):
|
|||
query['includeEvent'] = include_event_meta
|
||||
|
||||
url = urljoin(self.root_url, url_path)
|
||||
# Remove None values.
|
||||
# TODO: put that in self._prepare_request
|
||||
query = {k: v for k, v in query.items() if v is not None}
|
||||
response = self._prepare_request('POST', url, data=json.dumps(query))
|
||||
response = self._prepare_request('POST', url, data=query)
|
||||
normalized_response = self._check_response(response)
|
||||
if isinstance(normalized_response, str) or (isinstance(normalized_response, dict) and
|
||||
normalized_response.get('errors')):
|
||||
|
|
@ -348,10 +354,7 @@ class ExpandedPyMISP(PyMISP):
|
|||
query['includeContext'] = include_context
|
||||
query['headerless'] = headerless
|
||||
url = urljoin(self.root_url, f'{controller}/restSearch')
|
||||
# Remove None values.
|
||||
# TODO: put that in self._prepare_request
|
||||
query = {k: v for k, v in query.items() if v is not None}
|
||||
response = self._prepare_request('POST', url, data=json.dumps(query))
|
||||
response = self._prepare_request('POST', url, data=query)
|
||||
normalized_response = self._check_response(response)
|
||||
if return_format == 'csv' and pythonify and not headerless:
|
||||
return self._csv_to_dict(normalized_response)
|
||||
|
|
@ -420,10 +423,7 @@ class ExpandedPyMISP(PyMISP):
|
|||
query['id'] = query.pop('log_id')
|
||||
|
||||
url = urljoin(self.root_url, 'admin/logs/index')
|
||||
# Remove None values.
|
||||
# TODO: put that in self._prepare_request
|
||||
query = {k: v for k, v in query.items() if v is not None}
|
||||
response = self._prepare_request('POST', url, data=json.dumps(query))
|
||||
response = self._prepare_request('POST', url, data=query)
|
||||
normalized_response = self._check_response(response)
|
||||
if not pythonify:
|
||||
return normalized_response
|
||||
|
|
@ -477,10 +477,7 @@ class ExpandedPyMISP(PyMISP):
|
|||
query['timestamp'] = self.make_timestamp(timestamp)
|
||||
|
||||
url = urljoin(self.root_url, 'events/index')
|
||||
# Remove None values.
|
||||
# TODO: put that in self._prepare_request
|
||||
query = {k: v for k, v in query.items() if v is not None}
|
||||
response = self._prepare_request('POST', url, data=json.dumps(query))
|
||||
response = self._prepare_request('POST', url, data=query)
|
||||
normalized_response = self._check_response(response)
|
||||
|
||||
if not pythonify:
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load Diff
|
|
@ -1 +1 @@
|
|||
Subproject commit 7fe77c02affc0abe14cc67fe9f14400e8b72561c
|
||||
Subproject commit b25388c406e2f8dfcba94fe742a3ca90360315fd
|
||||
|
|
@ -67,3 +67,7 @@ class PyMISPNotImplementedYet(PyMISPError):
|
|||
|
||||
class PyMISPUnexpectedResponse(PyMISPError):
|
||||
pass
|
||||
|
||||
|
||||
class PyMISPEmptyResponse(PyMISPError):
|
||||
pass
|
||||
|
|
|
|||
|
|
@ -464,13 +464,7 @@ class MISPEvent(AbstractMISP):
|
|||
event = json_event
|
||||
if not event:
|
||||
raise PyMISPError('Invalid event')
|
||||
# Invalid event created by MISP up to 2.4.52 (attribute_count is none instead of '0')
|
||||
if (event.get('Event') and
|
||||
'attribute_count' in event.get('Event') and
|
||||
event.get('Event').get('attribute_count') is None):
|
||||
event['Event']['attribute_count'] = '0'
|
||||
e = event.get('Event')
|
||||
self.from_dict(**e)
|
||||
self.from_dict(**event)
|
||||
if validate:
|
||||
jsonschema.validate(json.loads(self.to_json()), self.__json_schema)
|
||||
|
||||
|
|
@ -489,17 +483,19 @@ class MISPEvent(AbstractMISP):
|
|||
raise NewEventError('Invalid format for the date: {} - {}'.format(date, type(date)))
|
||||
|
||||
def from_dict(self, **kwargs):
|
||||
if kwargs.get('Event'):
|
||||
kwargs = kwargs.get('Event')
|
||||
# Required value
|
||||
self.info = kwargs.pop('info', None)
|
||||
if self.info is None:
|
||||
raise NewAttributeError('The info field of the new event is required.')
|
||||
raise NewEventError('The info field of the new event is required.')
|
||||
|
||||
# Default values for a valid event to send to a MISP instance
|
||||
self.distribution = kwargs.pop('distribution', None)
|
||||
if self.distribution is not None:
|
||||
self.distribution = int(self.distribution)
|
||||
if self.distribution not in [0, 1, 2, 3, 4]:
|
||||
raise NewAttributeError('{} is invalid, the distribution has to be in 0, 1, 2, 3, 4'.format(self.distribution))
|
||||
raise NewEventError('{} is invalid, the distribution has to be in 0, 1, 2, 3, 4'.format(self.distribution))
|
||||
|
||||
if kwargs.get('threat_level_id') is not None:
|
||||
self.threat_level_id = int(kwargs.pop('threat_level_id'))
|
||||
|
|
@ -1005,6 +1001,8 @@ class MISPObject(AbstractMISP):
|
|||
raise PyMISPError('All the attributes have to be of type MISPObjectReference.')
|
||||
|
||||
def from_dict(self, **kwargs):
|
||||
if kwargs.get('Object'):
|
||||
kwargs = kwargs.get('Object')
|
||||
if self._known_template:
|
||||
if kwargs.get('template_uuid') and kwargs['template_uuid'] != self.template_uuid:
|
||||
if self._strict:
|
||||
|
|
@ -1017,6 +1015,12 @@ class MISPObject(AbstractMISP):
|
|||
else:
|
||||
self._known_template = False
|
||||
|
||||
if 'distribution' in kwargs and kwargs['distribution'] is not None:
|
||||
self.distribution = kwargs.pop('distribution')
|
||||
self.distribution = int(self.distribution)
|
||||
if self.distribution not in [0, 1, 2, 3, 4, 5]:
|
||||
raise NewAttributeError('{} is invalid, the distribution has to be in 0, 1, 2, 3, 4, 5'.format(self.distribution))
|
||||
|
||||
if kwargs.get('timestamp'):
|
||||
if sys.version_info >= (3, 3):
|
||||
self.timestamp = datetime.datetime.fromtimestamp(int(kwargs.pop('timestamp')), datetime.timezone.utc)
|
||||
|
|
|
|||
|
|
@ -3,22 +3,24 @@
|
|||
|
||||
import unittest
|
||||
|
||||
from pymisp import ExpandedPyMISP, MISPEvent, MISPOrganisation, MISPUser, Distribution, ThreatLevel, Analysis
|
||||
from pymisp import ExpandedPyMISP, MISPEvent, MISPOrganisation, MISPUser, Distribution, ThreatLevel, Analysis, MISPObject
|
||||
from pymisp.tools import make_binary_objects
|
||||
from datetime import datetime, timedelta, date
|
||||
from io import BytesIO
|
||||
|
||||
import time
|
||||
from uuid import uuid4
|
||||
|
||||
try:
|
||||
from keys import url, key
|
||||
verifycert = False
|
||||
travis_run = True
|
||||
except ImportError as e:
|
||||
print(e)
|
||||
url = 'http://localhost:8080'
|
||||
key = 'BSip0zVadeFDeolkX2g7MHx8mrlr0uE04hh6CQj0'
|
||||
|
||||
from uuid import uuid4
|
||||
|
||||
travis_run = True
|
||||
key = '8h0gHbhS0fv6JUOlTED0AznLXFbf83TYtQrCycqb'
|
||||
verifycert = False
|
||||
travis_run = False
|
||||
|
||||
|
||||
class TestComprehensive(unittest.TestCase):
|
||||
|
|
@ -27,7 +29,7 @@ class TestComprehensive(unittest.TestCase):
|
|||
def setUpClass(cls):
|
||||
cls.maxDiff = None
|
||||
# Connect as admin
|
||||
cls.admin_misp_connector = ExpandedPyMISP(url, key, debug=False)
|
||||
cls.admin_misp_connector = ExpandedPyMISP(url, key, verifycert, debug=False)
|
||||
# Creates an org
|
||||
org = cls.admin_misp_connector.add_organisation(name='Test Org')
|
||||
cls.test_org = MISPOrganisation()
|
||||
|
|
@ -36,12 +38,12 @@ class TestComprehensive(unittest.TestCase):
|
|||
usr = cls.admin_misp_connector.add_user(email='testusr@user.local', org_id=cls.test_org.id, role_id=3)
|
||||
cls.test_usr = MISPUser()
|
||||
cls.test_usr.from_dict(**usr)
|
||||
cls.user_misp_connector = ExpandedPyMISP(url, cls.test_usr.authkey)
|
||||
cls.user_misp_connector = ExpandedPyMISP(url, cls.test_usr.authkey, verifycert, debug=False)
|
||||
# Creates a publisher
|
||||
pub = cls.admin_misp_connector.add_user(email='testpub@user.local', org_id=cls.test_org.id, role_id=4)
|
||||
cls.test_pub = MISPUser()
|
||||
cls.test_pub.from_dict(**pub)
|
||||
cls.pub_misp_connector = ExpandedPyMISP(url, cls.test_pub.authkey)
|
||||
cls.pub_misp_connector = ExpandedPyMISP(url, cls.test_pub.authkey, verifycert)
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls):
|
||||
|
|
@ -439,6 +441,51 @@ class TestComprehensive(unittest.TestCase):
|
|||
self.admin_misp_connector.delete_event(first.id)
|
||||
self.admin_misp_connector.delete_event(second.id)
|
||||
|
||||
def test_default_distribution(self):
|
||||
'''The default distributions on the VM are This community only for the events and Inherit from event for attr/obj)'''
|
||||
if travis_run:
|
||||
return
|
||||
first = self.create_simple_event()
|
||||
del first.distribution
|
||||
o = first.add_object(name='file')
|
||||
o.add_attribute('filename', value='foo.exe')
|
||||
try:
|
||||
# Event create
|
||||
first = self.user_misp_connector.add_event(first)
|
||||
self.assertEqual(first.distribution, Distribution.this_community_only.value)
|
||||
self.assertEqual(first.attributes[0].distribution, Distribution.inherit.value)
|
||||
self.assertEqual(first.objects[0].distribution, Distribution.inherit.value)
|
||||
self.assertEqual(first.objects[0].attributes[0].distribution, Distribution.inherit.value)
|
||||
# Event edit
|
||||
first.add_attribute('ip-dst', '12.54.76.43')
|
||||
o = first.add_object(name='file')
|
||||
o.add_attribute('filename', value='foo2.exe')
|
||||
first = self.user_misp_connector.update_event(first)
|
||||
self.assertEqual(first.attributes[1].distribution, Distribution.inherit.value)
|
||||
self.assertEqual(first.objects[1].distribution, Distribution.inherit.value)
|
||||
self.assertEqual(first.objects[1].attributes[0].distribution, Distribution.inherit.value)
|
||||
# Attribute create
|
||||
attribute = self.user_misp_connector.add_named_attribute(first, 'comment', 'bar')
|
||||
# FIXME: Add helper that returns a list of MISPAttribute
|
||||
self.assertEqual(attribute[0]['Attribute']['distribution'], str(Distribution.inherit.value))
|
||||
# Object - add
|
||||
o = MISPObject('file')
|
||||
o.add_attribute('filename', value='blah.exe')
|
||||
new_obj = self.user_misp_connector.add_object(first.id, o.template_uuid, o)
|
||||
# FIXME: Add helper that returns a MISPObject
|
||||
self.assertEqual(new_obj['Object']['distribution'], str(Distribution.inherit.value))
|
||||
self.assertEqual(new_obj['Object']['Attribute'][0]['distribution'], str(Distribution.inherit.value))
|
||||
# Object - edit
|
||||
clean_obj = MISPObject(**new_obj['Object'])
|
||||
clean_obj.from_dict(**new_obj['Object'])
|
||||
clean_obj.add_attribute('filename', value='blah.exe')
|
||||
new_obj = self.user_misp_connector.edit_object(clean_obj)
|
||||
for a in new_obj['Object']['Attribute']:
|
||||
self.assertEqual(a['distribution'], str(Distribution.inherit.value))
|
||||
finally:
|
||||
# Delete event
|
||||
self.admin_misp_connector.delete_event(first.id)
|
||||
|
||||
def test_simple_event(self):
|
||||
'''Search a bunch of parameters:
|
||||
* Value not existing
|
||||
|
|
@ -517,7 +564,7 @@ class TestComprehensive(unittest.TestCase):
|
|||
|
||||
# quickfilter
|
||||
events = self.user_misp_connector.search(timestamp=timeframe,
|
||||
quickfilter='%bar%', pythonify=True)
|
||||
quickfilter='%foo blah%', pythonify=True)
|
||||
# FIXME: should return one event
|
||||
# print(events)
|
||||
# self.assertEqual(len(events), 1)
|
||||
|
|
@ -709,7 +756,7 @@ class TestComprehensive(unittest.TestCase):
|
|||
second = self.user_misp_connector.add_event(second)
|
||||
|
||||
response = self.user_misp_connector.fast_publish(first.id, alert=False)
|
||||
self.assertEqual(response['errors'][0][1]['message'], 'You do not have permission to use this functionality.')
|
||||
self.assertEqual(response['errors'][1]['message'], 'You do not have permission to use this functionality.')
|
||||
|
||||
# Default search, attribute with to_ids == True
|
||||
first.attributes[0].to_ids = True
|
||||
|
|
@ -835,6 +882,24 @@ class TestComprehensive(unittest.TestCase):
|
|||
self.admin_misp_connector.enable_tag(tag['id'])
|
||||
# FIXME: returns the tag with ID 1
|
||||
|
||||
def test_add_event_with_attachment(self):
|
||||
first = self.create_simple_event()
|
||||
try:
|
||||
first = self.user_misp_connector.add_event(first)
|
||||
file_obj, bin_obj, sections = make_binary_objects('tests/viper-test-files/test_files/whoami.exe', standalone=False)
|
||||
first.add_object(file_obj)
|
||||
first.add_object(bin_obj)
|
||||
for s in sections:
|
||||
first.add_object(s)
|
||||
self.assertEqual(len(first.objects[0].references), 1)
|
||||
self.assertEqual(first.objects[0].references[0].relationship_type, 'included-in')
|
||||
first = self.user_misp_connector.update_event(first)
|
||||
self.assertEqual(len(first.objects[0].references), 1)
|
||||
self.assertEqual(first.objects[0].references[0].relationship_type, 'included-in')
|
||||
finally:
|
||||
# Delete event
|
||||
self.admin_misp_connector.delete_event(first.id)
|
||||
|
||||
def test_taxonomies(self):
|
||||
# Make sure we're up-to-date
|
||||
self.admin_misp_connector.update_taxonomies()
|
||||
|
|
|
|||
Loading…
Reference in New Issue