Commit Graph

1562 Commits (7e61dd3f30d9492d5977f45b895ce0c08606a30c)

Author SHA1 Message Date
Raphaël Vinot db1ffe7be6 new: method to get the raw object template 2021-05-11 12:30:00 -07:00
Raphaël Vinot 286712d0e1 fix: first-seen and last-seen on attributes and objects were not checked for sanity 2021-05-11 07:28:54 -07:00
Raphaël Vinot 2f1cf24eaa chg: Bump objects templates 2021-05-11 07:28:00 -07:00
Raphaël Vinot 18300f8aed chg: Bump version 2021-04-26 10:52:56 +02:00
Raphaël Vinot 18049212a5 new: Support for correlation exclusion list
Fix #732
2021-04-22 10:47:58 +02:00
Raphaël Vinot b471633acb fix: Enable/disable feeds 2021-04-20 15:36:11 +02:00
Raphaël Vinot cc1af2573f chg: Bump objects templates 2021-04-19 23:12:27 +02:00
Raphaël Vinot 95e31bd2e3 chg: Add comment for controller attribute in search 2021-04-06 20:05:10 +02:00
Raphaël Vinot f0b2a2b943 fix bump version, deps, templates 2021-04-02 16:35:22 +02:00
Raphaël Vinot 5cc994e253 chg: get_uuid_or_id_from_abstract_misp accepts dict 2021-03-30 14:31:31 +02:00
Raphaël Vinot c68ee576b3 fix: use get_uuid_or_id_from_abstract_misp in tag methods
Fix #725
2021-03-30 14:23:32 +02:00
Raphaël Vinot 51edb8ab33 chg: Remove references to ExpandedPyMISP
Fix #721
2021-03-16 18:32:50 +01:00
Raphaël Vinot 00ba313eae chg: Follow best practices and remove the logging handler.
Fixes: #717
Reference: https://docs.python.org/3/howto/logging.html#configuring-logging-for-a-library
Documentation: https://docs.python.org/3/howto/logging.html
2021-03-16 18:28:04 +01:00
Raphaël Vinot 3252361b3c fix: Skip nameless sections in ELF
Related: #678
2021-03-16 17:56:06 +01:00
Raphaël Vinot 31608b1480 chg: strip NULL string from value
https://github.com/MISP/PyMISP/issues/678
2021-03-15 14:09:50 +01:00
Raphaël Vinot aee6945e95 fix: enable taxonomy failed if global pythonify is on 2021-03-09 16:35:00 +01:00
Raphaël Vinot 2734224958 chg: Raise exception on missing template in CSVLoader
Related: #714
2021-03-05 19:33:27 +01:00
Raphaël Vinot b5b2f7015b chg: Bump templates 2021-03-05 18:18:03 +01:00
Raphaël Vinot 2397732b03 chg: re-bump objects 2021-03-05 15:59:23 +01:00
Raphaël Vinot 100eeec77a chg: Bump object templates 2021-03-05 15:51:04 +01:00
Raphaël Vinot a0bda8736a chg: Add test case, fix mypy 2021-03-05 12:11:00 +01:00
Raphaël Vinot 59946a6a6d chg: take simple_value as value in MISPObject.add_attribute 2021-03-05 11:58:58 +01:00
Raphaël Vinot bbd341539a fix: properly pass content-type 2021-03-05 11:42:24 +01:00
Raphaël Vinot 0697f1470b fix: Re-enable support for uploading STIX 1 documents
Fix #711
2021-03-04 12:35:52 +01:00
Raphaël Vinot 36369f779a chg: Bump version 2021-03-03 10:39:21 +01:00
Alexandre Dulaunoy 3c141e1fdb
Merge branch 'main' of github.com:MISP/PyMISP into main 2021-03-03 09:46:53 +01:00
Alexandre Dulaunoy 4b3e93089f
chg: [describetypes] updated 2021-03-03 09:46:27 +01:00
Raphaël Vinot fe87d4293b chg: Bump object templates 2021-03-03 09:44:09 +01:00
Raphaël Vinot 4a2367ec96 fix: Make mypy happy in python 3.6 and 3.7 2021-03-02 12:37:35 +01:00
Raphaël Vinot 9f7282e8f4 fix: cosmetic changes, fix mypy 2021-03-02 12:21:59 +01:00
Raphaël Vinot d3bdb46587 chg: Bump objects templates 2021-03-02 12:21:17 +01:00
Raphaël Vinot 3067b818ff Merge branch 'tomking2-feature/misp-galaxy-2' into main 2021-03-02 11:50:43 +01:00
Raphaël Vinot 8137389452 chg: Bump tests for galaxy cluster 2021-03-02 11:49:31 +01:00
Raphaël Vinot 94ce4a367b chg: Remove legacy stix converter. 2021-03-01 15:10:56 +01:00
Raphaël Vinot 1533da3558 chg: Improve Pydoc on search method's timestamp parameter
Fix #708
2021-02-27 14:53:15 +01:00
Raphaël Vinot 2e05a1b24f new: soft delete object in MISPEvent
Fix #706
2021-02-27 14:53:15 +01:00
Raphaël Vinot 28fed5c778 fix: support text search again
Fix #705
2021-02-27 14:53:15 +01:00
Alexandre Dulaunoy 125961a670 chg: [data] describeTypes updated 2021-02-27 14:53:15 +01:00
Raphaël Vinot e183dbc577 fix: Do not add the serial-number twice.
Related: #678
2021-02-27 14:53:15 +01:00
Raphaël Vinot b9f7bd9dc1 chg: Add deprecation warning for Python < 3.8 2021-02-27 14:53:15 +01:00
Raphaël Vinot 1b675bb512 fix: Skip PE section if name is none AND size is 0.
Related: #678
2021-02-27 14:53:15 +01:00
Raphaël Vinot 59bb0a7bb6 fix: urllib3.__version__ may not have a patch number
fix https://github.com/MISP/PyMISP/issues/698
2021-02-27 14:53:15 +01:00
Raphaël Vinot d71b0945e2 chg: Improve Pydoc on search method's timestamp parameter
Fix #708
2021-02-26 17:57:39 +01:00
Raphaël Vinot d01c17abf8 new: soft delete object in MISPEvent
Fix #706
2021-02-26 17:55:13 +01:00
Raphaël Vinot cdcbe9bf32 fix: support text search again
Fix #705
2021-02-26 17:13:20 +01:00
Alexandre Dulaunoy 2ceb38c741
chg: [data] describeTypes updated 2021-02-20 17:28:50 +01:00
Raphaël Vinot d0a050263e fix: Do not add the serial-number twice.
Related: #678
2021-02-16 18:34:58 +01:00
Raphaël Vinot 3d3e9abc1d chg: Add deprecation warning for Python < 3.8 2021-02-15 16:12:44 +01:00
Raphaël Vinot 4730452ce0 fix: Skip PE section if name is none AND size is 0.
Related: #678
2021-02-15 16:11:18 +01:00
Raphaël Vinot e52263b75a fix: urllib3.__version__ may not have a patch number
fix https://github.com/MISP/PyMISP/issues/698
2021-02-15 12:00:10 +01:00
Tom King 5445479960 chg: Don't parse the meta key into cluster elements on a MISPEvent, but allow users to manually perform this action 2021-02-08 11:52:08 +00:00
Tom King a94b81ae72 Merge remote-tracking branch 'upstream/main' into feature/misp-galaxy-2 2021-02-08 11:50:38 +00:00
Raphaël Vinot 37449226f9 chg: Bump version 2021-02-08 11:59:49 +01:00
Raphaël Vinot 3125af9065 chg: Bump version 2021-02-04 19:42:24 +01:00
Raphaël Vinot 39d7f0e57a chg: Bump objects 2021-02-04 19:41:44 +01:00
Raphaël Vinot 6c9234846f chg: add kw_params to tags 2021-02-04 19:41:26 +01:00
Raphaël Vinot 9e2b748b02 chg: Bump objects 2021-02-02 15:26:08 +01:00
Raphaël Vinot f675e20961 chg: Bump version 2021-02-02 11:43:47 +01:00
Raphaël Vinot c91033eb8d chg: Bump objects 2021-02-02 11:40:01 +01:00
Raphaël Vinot 05bb34623f chg: Bump version 2021-02-01 14:25:57 +01:00
Raphaël Vinot 4cf1e9afc3 fix: flake error 2021-02-01 14:16:55 +01:00
Raphaël Vinot 7e4c15ee4d chg: Make mypy happy 2021-02-01 13:45:53 +01:00
Raphaël Vinot 3494e38987 chg: Make clear that to_json returns str 2021-02-01 13:43:39 +01:00
Raphaël Vinot f6b943cb9a chg: Disable correlation on malware-sample for FileObject 2021-02-01 13:43:39 +01:00
Raphaël Vinot c59f18606c chg: Bump objects templates 2021-02-01 13:43:39 +01:00
Tom King 6d11164acf chg: Add in delete function for a MISP Object 2021-02-01 13:43:39 +01:00
Raphaël Vinot c5218c1ce2 chg: Fix return of delete_event_report 2021-02-01 13:43:24 +01:00
Raphaël Vinot 78402394e5 chg: Remove critical warning if lief is not installed
Fix https://github.com/MISP/MISP/issues/6908
2021-02-01 13:43:01 +01:00
Raphaël Vinot 47382d01c0 fix: Better warning if lief is outdated. 2021-02-01 13:43:01 +01:00
Tom King a8169a42c0 chg: Allow response of delete to be pythonify, add in nosetest 2021-02-01 13:23:19 +01:00
Tom King c949c09225 chg: Add ability to get event reports from the Event ID 2021-02-01 13:23:19 +01:00
Tom King 7e7f463d77 fix: Call the AbstractMISP.from_dict at the end of the function to ensure the edited flag remains false 2021-02-01 13:23:19 +01:00
Tom King f71c250402 new: Add in ability to create/update/delete MISP Event Reports 2021-02-01 13:21:03 +01:00
Raphaël Vinot fa4fdb13f7 new: hard delete flag for objects
Related: https://github.com/MISP/PyMISP/issues/666
2021-02-01 13:21:03 +01:00
Raphaël Vinot 25053b2286 chg: Remove critical warning if lief is not installed
Fix https://github.com/MISP/MISP/issues/6908
2021-02-01 13:21:03 +01:00
Raphaël Vinot 48d8165263 chg: Bump version 2021-02-01 13:21:03 +01:00
Raphaël Vinot 696a13e3fc fix: Better warning if lief is outdated. 2021-02-01 13:21:03 +01:00
Raphaël Vinot 5886a29351 new: Fail if a duplicate object is added to an event. 2021-02-01 13:21:03 +01:00
Raphaël Vinot 644492ace1 chg: Improve docstring for get_event
fix #686
2021-02-01 13:21:03 +01:00
Raphaël Vinot fc43d7ba60 chg: Bump version 2021-02-01 13:21:03 +01:00
Raphaël Vinot ae1bdda67c chg: Show size when the json is not loadable. 2021-02-01 13:21:03 +01:00
Raphaël Vinot 5bdaf47175 chg: Use lief 0.11.0, generate authenticode entries 2021-02-01 13:21:03 +01:00
Raphaël Vinot a619fdfeca chg: Bump objects 2021-02-01 13:21:03 +01:00
Raphaël Vinot d29a28ba6e chg: Bump deps, objects templates 2021-02-01 13:21:03 +01:00
Tom King 7d4cfc40b7 chg: Add in nosetests for MISP Galaxy functions, check default key as a dict attribute not MISPAbstract attribute 2021-01-30 15:34:29 +00:00
Tom King 96636639c4 chg: Add in more Galaxy 2.0 functions and code cleanup 2021-01-30 13:56:40 +00:00
Tom King eb28f01f01 Merge remote-tracking branch 'upstream/main' into feature/misp-galaxy-2 2021-01-29 10:56:27 +00:00
Raphaël Vinot 03ebbbedce chg: Fix return of delete_event_report 2021-01-28 14:48:23 +01:00
Raphaël Vinot 86a5d3acc7 new: hard delete flag for objects
Related: https://github.com/MISP/PyMISP/issues/666
2021-01-28 14:45:36 +01:00
Raphaël Vinot 281a7f0d23 chg: Remove critical warning if lief is not installed
Fix https://github.com/MISP/MISP/issues/6908
2021-01-28 14:45:36 +01:00
Raphaël Vinot d7b80decf7 chg: Bump version 2021-01-28 14:45:36 +01:00
Raphaël Vinot cff25c7f57 fix: Better warning if lief is outdated. 2021-01-28 14:45:36 +01:00
Raphaël Vinot e916b332f8 new: Fail if a duplicate object is added to an event. 2021-01-28 14:45:36 +01:00
Raphaël Vinot d21e43bc59 chg: Improve docstring for get_event
fix #686
2021-01-28 14:45:36 +01:00
Raphaël Vinot c67da842d3 chg: Bump version 2021-01-28 14:45:36 +01:00
Raphaël Vinot 6f0c942800 chg: Show size when the json is not loadable. 2021-01-28 14:45:36 +01:00
Raphaël Vinot 8c09a5bbc9 chg: Use lief 0.11.0, generate authenticode entries 2021-01-28 14:45:36 +01:00
Raphaël Vinot c195b7cc61 chg: Bump objects 2021-01-28 14:45:36 +01:00
Raphaël Vinot 961fb77de1 chg: Bump deps, objects templates 2021-01-28 14:45:36 +01:00
Tom King cc102675bb chg: Add in add_cluster function and ability to search clusters within a galaxy 2021-01-25 13:18:12 +00:00
Tom King cff7e7b285 new: Add in ability to add a new cluster relation 2021-01-16 16:11:41 +00:00
Tom King 164791e980 new: MISP Galaxy 2.0 capability 2021-01-16 15:56:30 +00:00
Jakub Onderka 361d8d0944 new: Support brotli compression 2021-01-15 20:19:19 +01:00
Tom King 07f00a68f1 chg: Allow response of delete to be pythonify, add in nosetest 2021-01-15 15:26:41 +00:00
Tom King 120f3917e3 chg: Add ability to get event reports from the Event ID 2021-01-15 09:42:08 +00:00
Tom King e6cb4ff9ee fix: Call the AbstractMISP.from_dict at the end of the function to ensure the edited flag remains false 2021-01-14 18:58:35 +00:00
Tom King 12c29e6a06 new: Add in ability to create/update/delete MISP Event Reports 2021-01-14 16:45:25 +00:00
Raphaël Vinot de6125a623 fix: Do not fail if extract_msg is missing 2021-01-11 14:57:22 +01:00
Raphaël Vinot fa95c9d84f fix: Properly decode the body depending on the encoding of the email
Fix #671
2021-01-11 14:15:34 +01:00
Raphaël Vinot c50bbd5d1c chg: Add controller argument to get_csv script 2021-01-11 11:49:12 +01:00
seamus tuohy 87c02da0d7 Updated emailobject.
Email object no longer requires extra php libraries for install.
Tests have been expanded to improve coverage.
RTF encapsulated HTML and Plain Text will now be de-encapsulated.
The raw MSG binary will now be included in the extracted email object.
2020-12-28 13:47:21 -05:00
Alexandre Dulaunoy b9df83a384
chg: [misp-objects] updated 2020-12-24 12:01:29 +01:00
Alexandre Dulaunoy f72c2d2ff9
chg: [type] favicon-mmh3 is the murmur3 hash of a favicon as used in Shodan. 2020-12-24 12:00:17 +01:00
Alexandre Dulaunoy a46feebb32
chg: [misp-objects] updated to the latest version 2020-12-20 11:05:14 +01:00
Raphaël Vinot 649e068fd8 chg: clarify misp_objects_template_custom 2020-12-01 14:32:03 +01:00
Raphaël Vinot 3375c9d519 chg: Add docstring for misp_objects_template_custom 2020-12-01 14:14:21 +01:00
Raphaël Vinot 2c5f5b8662
Merge pull request #665 from nighttardis/main
adding check if "from" is in the "received" header row
2020-12-01 14:11:03 +01:00
Raphaël Vinot 0d86a4339f new: Allow to pass an object template to MISPObject.__init__
MISPObject part of #6670
2020-12-01 14:01:32 +01:00
nighttardis 2a4b215026 adding check if "from" is in the "received" header row 2020-11-30 18:45:53 -06:00
Raphaël Vinot babb04cbc2 chg: Improve documentation of MISPAttribute.malware_binary 2020-11-30 09:53:49 +01:00
Raphaël Vinot c8cb3bb589
chg: remove trailing space 2020-11-28 11:28:22 +01:00
Raphaël Vinot 201eeeb729
Update mispevent.py 2020-11-28 02:06:48 +01:00
Raphaël Vinot fe91e10ced chg: on-demand decryption of malware-binary, speeds up pythonify. 2020-11-26 13:31:10 +01:00
Raphaël Vinot 4c2ee4fd2f fix: Properly match IO in load event 2020-11-25 13:34:13 +01:00
Raphaël Vinot f254e15bd4 fix: Typing on recent mypy 2020-11-25 13:19:19 +01:00
Raphaël Vinot 3e1cfc1461 fix: Typing edge case 2020-11-25 09:23:33 +01:00
Raphaël Vinot ded44278af fix: Add attribute dict as proposal 2020-11-24 20:03:01 +01:00
Raphaël Vinot ad40915a79 chg: Bump version 2020-11-24 15:03:13 +01:00
Raphaël Vinot 9046b08a3c fix: Do not fail on PyMISP import when mail-parser is not present 2020-11-24 14:56:29 +01:00
Raphaël Vinot 7b2e78246a chg: Improve typing 2020-11-24 14:40:00 +01:00
Raphaël Vinot 35860b49bd chg: Improve add_attribute with a list
Related: #655
2020-11-24 13:50:14 +01:00
Raphaël Vinot 346f8d4b03 chg: Bump version 2020-11-24 12:39:05 +01:00
Alexandre Dulaunoy 39d471b58d
chg: [type] process-state added 2020-11-24 12:22:37 +01:00
Raphaël Vinot 71fe62b466 fix: Make mail-parser really optional 2020-11-24 12:18:35 +01:00
Alexandre Dulaunoy 0a08925a1a
chg: [misp-objects] updated 2020-11-24 11:57:16 +01:00
Alexandre Dulaunoy f3a408ce11
chg: [types] jarm-fingerprint added 2020-11-24 11:28:02 +01:00
Raphaël Vinot 80e13df3fa chg: Bump version, travis install 2020-11-24 11:17:23 +01:00
Raphaël Vinot 9fed66eb2b chg: Make mail-parser an optional dependency 2020-11-24 11:17:23 +01:00
Raphaël Vinot 5b0d42d6b2 chg: Bump version 2020-11-23 10:05:32 +01:00
Raphaël Vinot 75a7774887 chg: Improve documentation of search_index
Related: #656
2020-11-19 11:48:18 +01:00
Raphaël Vinot b55370cdad chg: Improve error handling for Outlook emails
Related: #631
2020-11-19 11:38:35 +01:00
Raphaël Vinot ac9b117f36
Merge pull request #631 from JakubOnderka/emailobject-tool-upgrade
Emailobject tool upgrade
2020-11-19 10:46:42 +01:00
Raphaël Vinot 02eff91c1e chg: Bump object templates 2020-11-18 00:24:23 +01:00
Raphaël Vinot ef845926b1 chg: Do not split a string into a list in complex query builder
fix #597
2020-11-17 14:39:58 +01:00
Jakub Onderka 2d4debe23c
fix: Path for event creating and editing 2020-11-16 17:22:10 +01:00
Raphaël Vinot 3b130bd973 fix: object_uuid could be None
Fix #640
2020-11-10 12:04:45 +01:00
Raphaël Vinot b646f0c5da
Merge pull request #651 from JakubOnderka/new-api
New API
2020-11-09 10:53:34 +01:00
Jakub Onderka 6c1f476bdd new: Method to check attribute and object existence 2020-11-07 10:17:16 +01:00
Raphaël Vinot 0d8467920f fix: Missing f-string marker 2020-11-06 11:01:08 +01:00
Raphaël Vinot 70de680912 chg: Use REST search for the tags
Related to comments on a1326f2cf2
2020-11-05 16:51:58 +01:00
Raphaël Vinot bdcc19c5fb chg: Add typing meta 2020-11-03 13:30:50 +01:00
Remy Dewailly be2b8b4ce7 We can now upload stix object directly. File is not necessary. 2020-11-03 13:17:16 +01:00
Remy Dewailly 115bc59425 We can now upload stix object directly. File is not necessary. 2020-11-03 13:13:32 +01:00
Jakub Onderka 5e4dd2b974 new: Allow to get just event metadata after add_event and edit_event 2020-11-03 12:10:38 +01:00
Jakub Onderka 495af1fd9c new: Method to check event existence 2020-11-03 12:10:37 +01:00
Raphaël Vinot 7e84c36406 fix: Docstring improvment based on @chrisinmtown's feedback 2020-11-02 14:48:51 +01:00
Raphaël Vinot a1326f2cf2 new: Add method to search for tags.
fix #648
2020-11-02 12:47:56 +01:00
Raphaël Vinot 15b9569ccb chg: Bump version 2020-11-02 10:52:04 +01:00
Raphaël Vinot d1a2dd10ab chg: Bump misp-objects 2020-11-02 10:47:50 +01:00
Raphaël Vinot f46d44aaa6
Merge pull request #649 from JakubOnderka/keep-alive
chg: Keep connection alive between requests
2020-11-02 09:55:24 +01:00
Jakub Onderka 9aa119e080 chg: Keep connection alive between requests 2020-10-29 13:40:23 +01:00
Lott, Christopher (cl778h) aa206d0009 chg: format docstrings in mispevent.py
Add ":param " prefix to parameters to improve ReadTheDocs output.
Fix some minor typos in docstrings.
2020-10-27 11:14:06 -04:00
Friedrich Lindenberg 5016858201 Drop `encoding=` in Python 3.9 2020-10-27 12:24:29 +01:00
Jakub Onderka 9fd3d8a3e3 fix: [emailobject] Correctly parse multiple addresses 2020-10-24 17:24:18 +02:00
Jakub Onderka 055ef16e41 new: Test parsing just email header 2020-10-24 17:24:18 +02:00
Jakub Onderka 5e0ad0a47f new: Test parsing outlook message format 2020-10-24 17:24:18 +02:00
Jakub Onderka f598865ce4 new: Refactored emailobject generator 2020-10-24 17:24:17 +02:00
Jakub Onderka d39d4caf7d new: Export display name from email 2020-10-24 17:16:16 +02:00
Jakub Onderka c2fedc3850 new: Parse date from email 2020-10-24 17:16:16 +02:00
Raphaël Vinot d428858f1e fix: Do now fail on requests returning plain text
Fix #639
2020-10-21 15:16:17 +02:00
Raphaël Vinot 624c6e0422 chg: Bump object templates 2020-10-16 13:13:43 +02:00
Raphaël Vinot e683ceabf7 chg: Bump version 2020-10-16 13:09:29 +02:00
Raphaël Vinot 8392a84c83 Merge branch 'main' of github.com:MISP/PyMISP into main 2020-10-16 13:01:29 +02:00
Alexandre Dulaunoy 83b8172dc6
chg: [type] updated 2020-10-15 15:12:47 +02:00
Tom King e5d413ca4f Merge remote-tracking branch 'upstream/main' into feature/tagdelete_searchsg 2020-10-14 17:14:52 +01:00
Alexandre Dulaunoy 1d83f38725
chg: [data] misp-objects updated 2020-10-13 22:57:38 +02:00
Alexandre Dulaunoy 85c2600bd7
new: [attribute type] telfhash added 2020-10-13 22:34:24 +02:00
Alexandre Dulaunoy 77e7111c29
chg: [type] new type added 2020-10-01 15:08:45 +02:00
garanews cd785aab09 fix typo
fix typo
2020-10-01 13:45:29 +02:00
Raphaël Vinot 516e7472bb chg: Bump deps, objects 2020-09-29 11:17:16 +02:00
Raphaël Vinot c39328f30a fix: Do not modify default_attributes_parameters in MISPObject 2020-09-15 17:01:56 +02:00
Lott, Christopher (cl778h) f1de0fb794 chg: add docstrings and extend conf.py for RTD
Add minimal docstrings to public methods so ReadTheDocs will display them.
Add autodoc mock import for lief so RTD can generate HTML for tools.

This fixes issue #626
2020-09-15 10:40:21 -04:00
Raphaël Vinot 18474a2144 chg: Add comments to ELF, PE, and MachO object generators. 2020-09-15 12:39:59 +02:00
Raphaël Vinot 50e5f156bd chg: Improve error message, add comments, rename whitelist->allowedlist 2020-09-15 12:31:22 +02:00
Raphaël Vinot 9c48079d88 new: Method to get the new version of the templates 2020-09-10 15:26:34 +02:00
Raphaël Vinot e3815a41f1 fix: Make flake8 happy 2020-09-09 15:41:42 +02:00
Raphaël Vinot cab202e1da
Merge pull request #624 from seamustuohy/fix-badly-encoded-emails
Attempt to decode utf-8-sig encoded emails.
2020-09-09 15:02:18 +02:00
seamus tuohy 07137209e2 Attempt to decode utf-8-sig encoded emails.
eml files downloaded from Windows Online security on some Windows 11
systems are automatically encoded in UTF with a byte order mark (BOM)
at the front of the file. This will cause the email parser to fail.

This is a somewhat isolated problem. It only will affects a small
subset of Windows users who download and re-upload eml files. But,
this small subset of users is the target user-base for the MISP
email module: low expertiese users who wish to quickly share
high-value indicators on an ad-hoc basis.

While this fix could be tacked onto the MISP email module instead of
here, I beleive that this fix is more appropriate in the PyMISP object
code. As the "email" object parser this object should be built to
parse all manner of emails that it may encounter. This includes common
malformations such as this one and, even horrors such as, the .msg
format. This commit adds a generically named "attempt_decoding"
function which can be expanded to address all manner of sins that
are encountered in the future.
2020-09-09 07:45:07 -04:00
Raphaël Vinot 49aede3947 chg: Bump version 2020-09-08 12:43:25 +02:00
Raphaël Vinot 07fed2fbb4 chg: Bump objects 2020-09-08 11:18:40 +02:00
Raphaël Vinot 7cc868bc8d Merge branch 'main' of github.com:MISP/PyMISP into main 2020-09-08 10:55:32 +02:00
Raphaël Vinot cd93d6b868 chg: Bump objects 2020-09-08 10:55:20 +02:00
Alexandre Dulaunoy c7edf4e33a
chg: [describeTypes] updated 2020-09-04 16:33:11 +02:00
Alexandre Dulaunoy 5598351a8b
chg: [describeTypes] updated 2020-09-04 16:00:41 +02:00
Raphaël Vinot 3cbd906520 chg: Bump objects 2020-09-02 15:06:59 +02:00
Raphaël Vinot 918f841087 chg: Rename blacklist -> blocklist 2020-09-01 19:29:12 +02:00
Alexandre Dulaunoy 92c5d11f47
new: [describeTypes] sha3 added 2020-08-24 10:38:25 +02:00
Raphaël Vinot 29af8645f7 chg: Bump version 2020-08-20 13:01:00 +02:00
Raphaël Vinot f52ee0e0e7 chg: Bump objects 2020-08-20 12:44:35 +02:00
Raphaël Vinot 6e4bf35bda chg: Bump types 2020-08-20 12:22:12 +02:00
deku dd6922fd3a Exclude section correlation .rsrc and zero-filled 2020-08-14 11:13:53 -04:00
Raphaël Vinot be8c94e6e7 chg: Cleanup blocklist methods 2020-08-04 12:20:21 +02:00
Raphaël Vinot 2bbf888ca7 new: Blacklist methods 2020-08-03 15:59:54 +02:00
Raphaël Vinot 83273b6ce8 new: Add list of missing calls 2020-07-30 16:48:37 +02:00
Paal Braathen ff62f1c19c Linting/Add missing whitespace 2020-07-28 20:05:42 +02:00
Raphaël Vinot 706e553f5d
Merge pull request #607 from paalbra/remove-unnecessary-logic
Remove explicit loglevel checking
2020-07-28 12:42:44 +02:00
Paal Braathen e8d34ea337 Remove explicit loglevel checking 2020-07-28 11:18:43 +02:00
Paal Braathen 96881f216b Remove explicit traceback printing 2020-07-28 11:03:59 +02:00
Raphaël Vinot b2e8cffd0b fix: Add STIX XML output for the search
Use stix-xml as return_format.

Fix #600 https://github.com/MISP/MISP/issues/5618
2020-07-17 14:19:15 +02:00
louis b6322c0d0c chg: Make get_object return a not standalone object 2020-06-30 13:07:38 +02:00
louis f8589061cb chg: Remove standalone default value from MISPObject children c'tor
MISPObject.__init__ sets standalone=True by default, so there is no
need to do it in its child classes.
2020-06-30 12:40:08 +02:00
louis 67d2e47b3b chg: Make MISPObject standalone by default
standalone defaults to True in MISPObject.__init__, and is set to False
when the object is added to an event.
2020-06-30 12:36:19 +02:00
louis 86f758e5b4 new: Add MISPObject.standalone property
Setting MISPObject.standalone updates MISPObject._standalone and
add/removes "ObjectReference" from AbstractMISP.__not_jsonable using
update_not_jsonable/_remove_from_not_jsonable.
2020-06-29 18:55:07 +02:00
louis aa1c95f344 chg: Add MISPObject._standalone type 2020-06-29 18:38:27 +02:00
louis 0bbfac6143 new: Add AbstractMISP._remove_from_not_jsonable 2020-06-29 18:35:37 +02:00
Raphaël Vinot fc101aa790 chg: Bump version 2020-06-22 14:31:02 +02:00
Raphaël Vinot 3177d05c5d chg: Bump objects 2020-06-21 21:46:16 +02:00
Raphaël Vinot f94e247771 chg: Bump version 2020-06-19 15:33:23 +02:00
Raphaël Vinot ef91d3d966 chg: Bump misp-objects 2020-06-19 15:32:41 +02:00
Raphaël Vinot 578801e50d fix: Keep deleted key in MISPObject and MISPObjectAttribute 2020-06-19 14:12:03 +02:00
Raphaël Vinot c8d66365c5 chg: Update comments for search 2020-06-19 11:32:02 +02:00
Raphaël Vinot 16cbb93867 chg: Rename master -> main 2020-06-16 14:58:38 +02:00
Raphaël Vinot bbfe9d5b1f chg: Bump version 2020-06-16 14:22:22 +02:00
Raphaël Vinot b1fad98ab2 chg: Bump misp-objects 2020-06-16 14:20:45 +02:00
Troy Ross 17ebfe86ab Previously file object was reporting the libmagic description of a file
instead of the mimetype. According to [MISP DataModels](https://www.misp-project.org/datamodels/#types)
```
mime-type: A media type (also MIME type and content type) is a two-part identifier for file formats and format contents transmitted on the Internet
```
more precisely defined in [RFC2045](https://tools.ietf.org/html/rfc2045) and others.

The description returned by libmagic is more useful than the generic mime-type,
but I did not find a place to put the description in the current data model.
2020-06-14 10:48:29 -06:00
Raphaël Vinot 23d732e398 chg: Remove extra parameter in change_user_password 2020-06-02 10:08:17 +02:00
Raphaël Vinot 1e9eed198e fix: Do not fail if the attribute value is not a string 2020-05-29 01:23:34 +02:00
Raphaël Vinot 74a5d04bda fix: Properly strip value in MISPObject.add_attribute, take 2
Fix #546
2020-05-29 01:02:02 +02:00
Raphaël Vinot 524aa13641 fix: Properly strip value in MISPObject.add_attribute
Fix #546
2020-05-29 00:56:32 +02:00
Raphaël Vinot 5d97d7ee0c new: Add helper and test case for GitVulnFinderObject 2020-05-26 15:37:24 +02:00
Raphaël Vinot fb03cc1361 new: Add git-commit-id type 2020-05-26 14:45:59 +02:00
Raphaël Vinot 06eb92f912 fix: Deleted is not always required in the feed export 2020-05-26 11:36:53 +02:00
Raphaël Vinot 526321c8b4 new: Add deleted in field export
Fix #586
2020-05-26 10:56:46 +02:00
Raphaël Vinot 3e26d3c807 fix: Make mypy happy 2020-05-21 23:03:04 +02:00
Christophe Vandeplas 515a47a591 fix: fixes bug in timeout change 2020-05-21 22:01:26 +02:00
Christophe Vandeplas f3b3f4c13c fix: fixes bug in timeout change 2020-05-21 21:52:42 +02:00
Christophe Vandeplas 12f8fd8530 fix: fixes bug in timeout change 2020-05-21 21:49:25 +02:00
Christophe Vandeplas fa639d8aa9 fix: fixes bug in timeout change 2020-05-21 21:46:24 +02:00
Christophe Vandeplas e74a0a4269 fix: fixes bug in timeout change
hail to Rafiot
2020-05-21 21:30:28 +02:00
Christophe Vandeplas d09852fa4b fix: fixes bug in timeout change 2020-05-21 20:59:28 +02:00
Christophe Vandeplas d745d5b226 fix: fixes bug in timeout change 2020-05-21 20:44:42 +02:00
Christophe Vandeplas 50ee8d9a66 new: Timeout for connection/request, fixes #584 2020-05-21 20:31:19 +02:00
Raphaël Vinot e7166345b8 chg: Bump version 2020-05-18 12:34:09 +02:00
Raphaël Vinot 1d45ce8eb7 chg: Bump misp-object 2020-05-18 12:32:27 +02:00
Raphaël Vinot 7178d3a8a0 fix: settings is not required in MISPFeed 2020-05-15 11:44:13 +02:00
Raphaël Vinot 901afb32d9 chg: Strip empty parameters in build_complex_query
Fix #577
2020-05-14 13:10:01 +02:00
Raphaël Vinot 18c1460376 chg: Simplify delete_attribute 2020-05-14 12:43:10 +02:00
Raphaël Vinot 034a4e7d8e Merge branch 'master' of github.com:MISP/PyMISP 2020-05-14 12:41:25 +02:00
Raphaël Vinot 73693ac5f9 fix: Properly skip timestamp in __iter__ when needed 2020-05-14 12:41:19 +02:00
Bernhard E. Reiter da0373a615
Update docstring in api.py
* remove typo in ssl parameter docstring. 
 * Add hint that other certs (which are not in the default CAs, but also are not self signed in a strict sense) can also use the CA_BUNDLE function of the ssl parameter.
2020-05-14 09:42:24 +02:00
Raphaël Vinot b214c7d4c1 chg: Add comment in microblog object 2020-05-12 22:34:25 +02:00
Raphaël Vinot 5df58406ef fix: Catch exception when liblua-5.3 is not present
Related: https://github.com/MISP/misp-modules/issues/398
2020-05-12 13:21:05 +02:00
Raphaël Vinot 35257e538d fix: Make flake8 happy 2020-05-12 11:34:38 +02:00
Raphaël Vinot 14d278fff2 fix: Properly load feeds, fix undefined variable 2020-05-12 11:24:47 +02:00
Raphaël Vinot dcd1db8883 fix: make flake8 happy 2020-05-11 15:40:20 +02:00
VVX7 fff0caa330 chg: [dev] clean up how keys are accessed in self._parameters 2020-05-08 19:54:12 -04:00
VVX7 759e9196de chg: [dev] use isinstance() type check. 2020-05-08 19:31:19 -04:00
VVX7 395d6aabac chg: [dev] fix abstract generator import. add logger. 2020-05-08 19:27:42 -04:00
VVX7 de994fd944 chg: [dev] change type() == list 2020-05-08 16:32:29 -04:00
VVX7 0eb209c7df new: [dev] add microblog object tool 2020-05-08 16:10:09 -04:00
Raphaël Vinot 1d106d1a20 fix: remove extra print 2020-05-07 15:55:45 +02:00
Raphaël Vinot c098981a40 new: Very simple test case for rest search on objects 2020-05-07 13:59:45 +02:00
Raphaël Vinot 4a060b3c07 new: Self registration, object level search (initial) 2020-05-07 12:17:31 +02:00
Raphaël Vinot e020bac5f6 chg: Bump misp-objects 2020-05-05 11:05:50 +02:00
Raphaël Vinot 8980c2da3b fix: Typo, add test for extended event 2020-05-04 10:19:55 +02:00
VVX7 a76a85b616 chg: [dev] add extend_event() test. chg typo in get_event() 2020-05-03 20:58:33 -04:00
Raphaël Vinot 3ac8c5916b chg: Bump CHANGELOG 2020-04-30 10:23:31 +02:00
Raphaël Vinot 029aa8df79 chg: Bump objects, deps 2020-04-30 10:20:21 +02:00
Raphaël Vinot ed2a95fbdd new: Extended option on get event
Related to #567
2020-04-28 11:17:27 +02:00
mokaddem f965e579d7 fix: [abstract] Forces file to be read with utf8 encoding 2020-04-24 11:33:32 +02:00
DocArmoryTech 4ee4db16fe
Fixed __query_virustotal return type
__query_virustotal returned a Response object and not the json expected; modified so that report_json is returned instead of report.
2020-04-06 10:46:15 +01:00
Raphaël Vinot 92e884f15d chg: Bump version 2020-03-30 09:39:57 +02:00
Raphaël Vinot a64c79e960 chg: Bump misp-objects 2020-03-30 09:35:11 +02:00
Raphaël Vinot b5b40ae2c5 fix: Strip every string in AbstractMISP
fix #546
2020-03-24 14:34:29 +01:00
Raphaël Vinot 2a9c79a1e9 fix: Incorrect expectation of attribute value to be a str - take 2
Related #553
2020-03-13 12:02:11 +01:00
Raphaël Vinot 8cf3887d54 fix: Incorrect expectation of attribute value to be a str
Fix #553
2020-03-13 11:02:51 +01:00
Raphaël Vinot 1b4c74642d chg: Bump version 2020-03-10 14:10:38 +01:00
Raphaël Vinot 4fba2b05ad chg: Bump misp-objects 2020-03-10 10:27:52 +01:00
Raphaël Vinot eff7146b3c chg: JSON files are UTF8
Bump dev deps, update comment
2020-03-02 17:33:41 +01:00
Raphaël Vinot 67442dd503 new: Add import script for dxy data 2020-03-02 00:13:53 +01:00
Raphaël Vinot 68a2352afd chg: Bump misp-objects 2020-02-29 01:38:46 +01:00
Raphaël Vinot 0a696d8c14 chg: Bump misp-objects 2020-02-26 14:52:41 +01:00
Raphaël Vinot ffffbef69a chg: Bump misp-objects 2020-02-26 14:50:26 +01:00
Raphaël Vinot 92afc4a2a0 chg: Bump version 2020-02-26 14:39:58 +01:00
Raphaël Vinot 21a0c74443 chg: Bump misp-objects 2020-02-26 14:39:13 +01:00
Raphaël Vinot 8d6e69ce65 fix: mypy, more typing 2020-02-24 17:09:42 +01:00
Raphaël Vinot 94c2a644af fix: do not skip data in add_attribute methods 2020-02-24 14:13:10 +01:00
Raphaël Vinot 35377399e8 new: Add uuid by default in MISPEvent, add F/L seen in feed output. 2020-02-21 14:12:36 +01:00
Tom King b08d26d762 chg: Remove SG search for search() func as this doesn't support SG searching, but the index does 2020-02-13 16:35:11 +00:00
Tom King 394b7a545e Merge remote-tracking branch 'upstream/master' into feature/tagdelete_searchsg 2020-02-13 16:27:24 +00:00
Tom King 415e06f375 fix: merge SG params to allow search 2020-02-13 16:20:14 +00:00
Raphaël Vinot 55d1faac90 chg: Bump objects 2020-02-07 13:16:40 +01:00
Raphaël Vinot 0f72460d1a chg: Bump version 2020-02-07 13:15:18 +01:00
Raphaël Vinot 11353f8ae2 fix: Make lief optional again
fix #538
2020-02-07 11:51:48 +01:00
Raphaël Vinot 70dca1d408 fix: Bump objects 2020-02-06 10:58:40 +01:00
Raphaël Vinot f14963a656 chg: Bump version 2020-02-06 10:42:38 +01:00
Raphaël Vinot 8d58a50b9a chg: Bump objects 2020-02-06 10:30:16 +01:00
Raphaël Vinot 732908a1d3 fix: Remove debugging 2020-02-05 17:29:40 +01:00
Raphaël Vinot a1e96731b2 Merge branch 'master' of github.com:MISP/PyMISP 2020-02-05 13:11:56 +01:00
Raphaël Vinot fe80924d60 chg: str to int, properly load SharingGroup
Fix #535
2020-02-05 13:08:17 +01:00