Commit Graph

65 Commits (0436118747d4a199e8b42e0b19840767dc824859)

Author SHA1 Message Date
chrisr3d d885286792
Clarified functions arguments using a class 2018-03-05 19:59:30 +01:00
chrisr3d 4d7642ac91
add: Added Object References in the objects imported 2018-03-05 14:58:31 +01:00
chrisr3d 82fe8ba78c
fix: Fixed input & output of the module 2018-03-02 11:03:21 +01:00
chrisr3d 70436b7ddb Merge branch 'csvimport' of github.com:chrisr3d/misp-modules into goaml 2018-03-02 09:40:46 +01:00
chrisr3d c9ef578262
Removed print 2018-03-02 09:09:12 +01:00
chrisr3d e6c55f5dde
fix: Fixed input & output of the module
Also updated some functions
2018-03-02 09:03:51 +01:00
chrisr3d 03d20856d9
add: added goamlimport 2018-02-28 22:46:39 +01:00
chrisr3d 323f71cdd3
Fixed some details about the module output 2018-02-28 17:41:45 +01:00
chrisr3d 8f5c08e2c6
Converting GoAML into MISPEvent 2018-02-28 15:07:55 +01:00
chrisr3d cad62464c5
Now parsing all the transaction attributes 2018-02-27 11:08:37 +01:00
chrisr3d 478cd53912
add: Added dictionary to map aml types into MISP types 2018-02-26 18:13:43 +01:00
chrisr3d 81a6be17d3
chg: Structurded data 2018-02-26 11:47:35 +01:00
chrisr3d f361fb4ee3
Reading the entire document, to create a big dictionary containing the data, as a beginning 2018-02-20 17:00:13 +01:00
Thomas Gardner 69d733bb35 added csvimport to __init__.py 2018-02-01 10:22:28 -07:00
chrisr3d 71c00954d0
fix: Solved reading problems for some files 2018-01-30 11:20:28 +01:00
chrisr3d b2ec186ccb
Updated delimiter finder method 2018-01-29 17:04:32 +01:00
chrisr3d 529d22cca8
fix: skipping empty lines 2018-01-29 09:19:58 +01:00
chrisr3d 56cbd72b65
Fixed data treatment & other updates 2018-01-28 18:12:40 +01:00
chrisr3d 4d846f968f
Updated delimiter parsing & data reading functions 2018-01-26 17:11:01 +01:00
chrisr3d b9d72bb043
First version of csv import module
- If more than 1 misp type is recognized, for each one an
  attribute is created

- Needs to have header set by user as parameters of the module atm

- Review needed to see the feasibility with fields that can create
  confusion and be interpreted both as misp type or attribute field
  (for instance comment is a misp type and an attribute field)
2018-01-25 15:44:08 +01:00
Christophe Vandeplas 46975f4f16 Added ThreatAnalyzer sandbox import
Experimental module - some parts should be migrated to
2018-01-16 11:05:26 +01:00
Raphaël Vinot 951a0f974b fix: OpenIOC importer 2017-10-25 11:27:59 -04:00
seamus tuohy 40c71af637 Added support for malformed internationalized email headers
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93)

To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.

This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
2017-07-02 18:03:14 -04:00
Raphaël Vinot c42c8a800e Update travis, fix open ioc import 2017-05-24 07:39:18 +02:00
Tristan METAYER 75c02058e6 replace tab by space 2017-05-11 09:56:43 +02:00
Tristan METAYER ba1d715ad1 Add a field for user to add tag for this import 2017-05-11 09:54:25 +02:00
Tristan METAYER 96f9cb4699 typo correction 2017-05-02 15:07:33 +02:00
Tristan METAYER 4ef7261168 Add user config to not add file as attachement in a box 2017-05-02 15:04:40 +02:00
Tristan METAYER 79f48eccfe If filename add iocfilename as attachment 2017-05-02 14:41:22 +02:00
Raphaël Vinot c508e60f65 Add OpenIOC import module 2017-02-27 13:32:31 +01:00
Richard van den Berg 3a4c540a81 Updated description to reflect merging use case 2017-01-11 10:08:35 +01:00
Richard van den Berg 50bae1f549 Simple import module to import MISP JSON format 2017-01-11 10:08:35 +01:00
seamus tuohy 83a9d695ea Email import no longer unzips major compressed text document formats.
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.

It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
2017-01-10 09:55:33 -05:00
Raphaël Vinot 1051e2210b Keep zip content as binary 2017-01-07 19:30:00 -05:00
Raphaël Vinot 9f84db3659 Fix tests, cleanup 2017-01-07 18:36:08 -05:00
Raphaël Vinot 2db845c45c Improve support of email attachments
Related to #90
2017-01-07 14:39:52 -05:00
Raphaël Vinot b51806ac9f Improve support of email importer if headers are missing
Fix #88
2017-01-07 10:25:38 -05:00
Raphaël Vinot 02f5e95a98 Fix python 3.6 support 2017-01-06 20:36:09 -05:00
Raphaël Vinot 329586768b Make PEP8 happy 2017-01-06 20:10:44 -05:00
Raphaël Vinot 7a9774bff7 Add email_import in the modules loaded by default 2017-01-06 19:23:23 -05:00
Raphaël Vinot 93a49c3c1d Make PEP8 happy 2017-01-06 19:01:19 -05:00
Raphaël Vinot 3f83357a2d Fix failing test (bug in the mail parser?) 2017-01-06 18:56:29 -05:00
seamus tuohy 1a7973bc06 Add additional email parsing and tests
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
2017-01-04 10:21:36 -08:00
seamus tuohy 0ff270a3be Fixed basic errors 2016-12-26 14:33:10 -08:00
seamus tuohy 08261366b7 Merged with current master 2016-12-26 14:17:20 -08:00
seamus tuohy 86ae72c444 Added attachment and url support 2016-12-26 13:55:54 -08:00
Ubuntu b76f59edcb Added cuckooimport.py 2016-12-07 16:36:31 +00:00
Koen Van Impe 077470b8ed Merge remote-tracking branch 'MISP/master' 2016-11-30 13:06:43 +01:00
Koen Van Impe cb29506640 Extra VTI detections 2016-11-27 22:42:43 +01:00
Raphaël Vinot 79a0b9e667 Merge pull request #73 from FloatingGhost/master
Use SpooledTemp, not NamedTemp file
2016-11-21 16:37:11 +01:00