MISP
/
misp-modules
mirror of https://github.com/MISP/misp-modules
2
0
Fork 0
Code Issues Releases Wiki Activity
1,396 Commits
24 Branches
47 Tags
23 MiB
Commit Graph

156 Commits (66bf650b79e8a1c5d094a0af79fae5f2ac9aae40)

Author SHA1 Message Date
Koen Van Impe c8a4d8d76f New VMRay modules 
New JSON output format of VMRay
Prepare for automation (via PyMISP) with workflow taxonomy tags
 2019-05-01 22:44:24 +02:00
root c886247a64
fix: Fixed standard MISP csv format header 
- The csv header we can find in data produced from
  MISP restSearch csv format is the one to use to
  recognize a csv file produced by MISP
 2019-05-01 22:32:06 +02:00
root f900cb7c68
fix: Fixed introspection fields for csvimport & goamlimport 
- Added format field for goaml so the module is
  known as returning MISP attributes & objects
- Fixed introspection to make the format, user
  config and input source fields visible from
  MISP (format also added at the same time)
 2019-05-01 22:28:19 +02:00
root db74c5f49a
fix: Fixed libraries import that changed with the latest merge 2019-05-01 22:26:53 +02:00
chrisr3d 55e494c9ed Merge branch 'features_csvimport' of github.com:MISP/misp-modules into features_csvimport 2019-04-30 17:16:31 +02:00
Raphaël Vinot 454c9e0f43 fix: Pep8 related fixes. 2019-02-04 11:05:51 +01:00
Raphaël Vinot 8fc5b1fd1f fix: Make pep8 happy 2018-12-11 15:29:09 +01:00
Christophe Vandeplas 8817de4765 fix: threatanalyzer_import - bugfix for TA6.1 behavior 2018-11-16 13:29:47 +01:00
chrisr3d fcc18cbd73 Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport 2018-09-03 15:40:19 +02:00
Christophe Vandeplas 7deeb95820 fix: ta_import - bugfixes 2018-08-21 11:13:08 +02:00
Christophe Vandeplas 8d4e2025f7 ta_import - bugfixes for TA 6.1 2018-08-03 13:58:53 +02:00
chrisr3d 8b4d24ba63
fix: Fixed fields parsing to support files from csv export with additional context 2018-08-02 15:42:59 +02:00
chrisr3d 7980aa045a
fix: Handling the case of Context included in the csv file exported from MISP 2018-08-01 17:59:00 +02:00
chrisr3d 92fbcaeff6
fix: Fixed changes omissions in handler function 2018-07-28 00:07:02 +02:00
chrisr3d 63ba7580d3
chg: Updated csvimport to support files from csv export + import MISP objects 2018-07-27 23:13:47 +02:00
Christophe Vandeplas 2f27ff1244 ta_import - support for TheatAnalyzer 6.1 2018-07-27 14:44:06 +02:00
Steve Clement 562a6b1308 - Removed test modules from view 
- Moved skeleton expansion module to it's proper place
 2018-07-03 08:27:54 +02:00
Steve Clement 549f32547d - Reverted to <3.6 compatibility 2018-07-01 22:09:02 +08:00
Steve Clement 9f0313a97e - Fixed log output 2018-06-30 12:01:21 +08:00
Steve Clement 184065cf74 - Forgot to import sys 2018-06-30 11:58:44 +08:00
Steve Clement ffce2aa5cc - Added logger functionality for debug sessions 2018-06-30 11:52:12 +08:00
Steve Clement 2f5dd9928e - content was already a wand.obj 2018-06-30 11:38:26 +08:00
Steve Clement 90f2fe9d19 Merge remote-tracking branch 'upstream/master' 2018-06-30 01:05:01 +08:00
Steve Clement f97359de6a Merge branch 'master' of github.com:SteveClement/misp-modules 2018-06-30 01:04:30 +08:00
Steve Clement ef3837077e - Some more comments 
- Removed libmagic, wand can handle it better
 2018-06-30 00:58:25 +08:00
Christophe Vandeplas ff793bc221
threatanalyzer_import - order of category tuned 2018-06-29 11:17:03 +02:00
Alexandre Dulaunoy d8eeb73a4a
Merge branch 'master' into master 2018-06-29 06:49:40 +02:00
Steve Clement fbb3617f25 - Quick comment ToDo: Avoid using Magic in future releases 2018-06-29 12:01:17 +08:00
Steve Clement 60a3fbe282 - added wand requirement 
- fixed missing return png byte-stream
- move module import to handler to catch and  report errorz
 2018-06-28 23:20:38 +08:00
Steve Clement 7885017981 - fixed typo move image back in scope 2018-06-28 16:59:03 +08:00
chrisr3d 7dd8e988c0
Updated the list of modules (removed stiximport) 2018-06-28 10:51:40 +02:00
Steve Clement 59b7688bdc - Added initial PDF support, nothing is processed yet 
- Test to replace PIL with wand
 2018-06-28 16:00:14 +08:00
chrisr3d 2b509a2fd3
Updated delimiter finder function 2018-05-18 11:38:13 +02:00
chrisr3d 1fb72f3c7a
add: Added user config to specify if there is a header in the csv to import 2018-05-18 11:33:53 +02:00
chrisr3d dba8bd8c5b
fix: Avoid trying to build attributes with not intended fields 
- Previously: if the header field is not an attribute type, then
              it was added as an attribute field.
              PyMISP then used to skip it if needed

- Now: Those fields are discarded before they are put in an attribute
 2018-05-17 16:24:11 +02:00
chrisr3d c088b13f03
fix: Using userConfig to define the header instead of moduleconfig 2018-05-17 13:47:49 +02:00
Christophe Vandeplas 0593dbb408 ta import - more filter for pollution 2018-05-16 11:50:47 +02:00
Christophe Vandeplas 67cecc89d0 threatanalyzer_import - minor generic noise removal 2018-05-15 13:02:17 +02:00
Christophe Vandeplas 27a22e5d86 threatanalyzer_import - loads sample info + pollution fix 2018-05-03 09:42:38 +02:00
Christophe Vandeplas 370011c081 threatanalyzer_import - fix regkey issue 2018-05-02 12:43:34 +02:00
Koen Van Impe 6d23d4f4c7 Fix VMRay API access error 
hotfix for the "Unable to access VMRay API" error
 2018-03-30 15:11:25 +02:00
chrisr3d d885286792
Clarified functions arguments using a class 2018-03-05 19:59:30 +01:00
chrisr3d 4d7642ac91
add: Added Object References in the objects imported 2018-03-05 14:58:31 +01:00
chrisr3d 82fe8ba78c
fix: Fixed input & output of the module 2018-03-02 11:03:21 +01:00
chrisr3d 70436b7ddb Merge branch 'csvimport' of github.com:chrisr3d/misp-modules into goaml 2018-03-02 09:40:46 +01:00
chrisr3d c9ef578262
Removed print 2018-03-02 09:09:12 +01:00
chrisr3d e6c55f5dde
fix: Fixed input & output of the module 
Also updated some functions
 2018-03-02 09:03:51 +01:00
chrisr3d 03d20856d9
add: added goamlimport 2018-02-28 22:46:39 +01:00
chrisr3d 323f71cdd3
Fixed some details about the module output 2018-02-28 17:41:45 +01:00
chrisr3d 8f5c08e2c6
Converting GoAML into MISPEvent 2018-02-28 15:07:55 +01:00
chrisr3d cad62464c5
Now parsing all the transaction attributes 2018-02-27 11:08:37 +01:00
chrisr3d 478cd53912
add: Added dictionary to map aml types into MISP types 2018-02-26 18:13:43 +01:00
chrisr3d 81a6be17d3
chg: Structurded data 2018-02-26 11:47:35 +01:00
chrisr3d f361fb4ee3
Reading the entire document, to create a big dictionary containing the data, as a beginning 2018-02-20 17:00:13 +01:00
Thomas Gardner 69d733bb35 added csvimport to __init__.py 2018-02-01 10:22:28 -07:00
chrisr3d 71c00954d0
fix: Solved reading problems for some files 2018-01-30 11:20:28 +01:00
chrisr3d b2ec186ccb
Updated delimiter finder method 2018-01-29 17:04:32 +01:00
chrisr3d 529d22cca8
fix: skipping empty lines 2018-01-29 09:19:58 +01:00
chrisr3d 56cbd72b65
Fixed data treatment & other updates 2018-01-28 18:12:40 +01:00
chrisr3d 4d846f968f
Updated delimiter parsing & data reading functions 2018-01-26 17:11:01 +01:00
chrisr3d b9d72bb043
First version of csv import module 
- If more than 1 misp type is recognized, for each one an
  attribute is created

- Needs to have header set by user as parameters of the module atm

- Review needed to see the feasibility with fields that can create
  confusion and be interpreted both as misp type or attribute field
  (for instance comment is a misp type and an attribute field)
 2018-01-25 15:44:08 +01:00
Christophe Vandeplas 46975f4f16 Added ThreatAnalyzer sandbox import 
Experimental module - some parts should be migrated to
 2018-01-16 11:05:26 +01:00
Raphaël Vinot 951a0f974b fix: OpenIOC importer 2017-10-25 11:27:59 -04:00
seamus tuohy 40c71af637 Added support for malformed internationalized email headers 
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93)

To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.

This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
 2017-07-02 18:03:14 -04:00
Raphaël Vinot c42c8a800e Update travis, fix open ioc import 2017-05-24 07:39:18 +02:00
Tristan METAYER 75c02058e6 replace tab by space 2017-05-11 09:56:43 +02:00
Tristan METAYER ba1d715ad1 Add a field for user to add tag for this import 2017-05-11 09:54:25 +02:00
Tristan METAYER 96f9cb4699 typo correction 2017-05-02 15:07:33 +02:00
Tristan METAYER 4ef7261168 Add user config to not add file as attachement in a box 2017-05-02 15:04:40 +02:00
Tristan METAYER 79f48eccfe If filename add iocfilename as attachment 2017-05-02 14:41:22 +02:00
Raphaël Vinot c508e60f65 Add OpenIOC import module 2017-02-27 13:32:31 +01:00
Richard van den Berg 3a4c540a81 Updated description to reflect merging use case 2017-01-11 10:08:35 +01:00
Richard van den Berg 50bae1f549 Simple import module to import MISP JSON format 2017-01-11 10:08:35 +01:00
seamus tuohy 83a9d695ea Email import no longer unzips major compressed text document formats. 
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.

It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
 2017-01-10 09:55:33 -05:00
Raphaël Vinot 1051e2210b Keep zip content as binary 2017-01-07 19:30:00 -05:00
Raphaël Vinot 9f84db3659 Fix tests, cleanup 2017-01-07 18:36:08 -05:00
Raphaël Vinot 2db845c45c Improve support of email attachments 
Related to #90
 2017-01-07 14:39:52 -05:00
Raphaël Vinot b51806ac9f Improve support of email importer if headers are missing 
Fix #88
 2017-01-07 10:25:38 -05:00
Raphaël Vinot 02f5e95a98 Fix python 3.6 support 2017-01-06 20:36:09 -05:00
Raphaël Vinot 329586768b Make PEP8 happy 2017-01-06 20:10:44 -05:00
Raphaël Vinot 7a9774bff7 Add email_import in the modules loaded by default 2017-01-06 19:23:23 -05:00
Raphaël Vinot 93a49c3c1d Make PEP8 happy 2017-01-06 19:01:19 -05:00
Raphaël Vinot 3f83357a2d Fix failing test (bug in the mail parser?) 2017-01-06 18:56:29 -05:00
seamus tuohy 1a7973bc06 Add additional email parsing and tests 
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
 2017-01-04 10:21:36 -08:00
seamus tuohy 0ff270a3be Fixed basic errors 2016-12-26 14:33:10 -08:00
seamus tuohy 08261366b7 Merged with current master 2016-12-26 14:17:20 -08:00
seamus tuohy 86ae72c444 Added attachment and url support 2016-12-26 13:55:54 -08:00
Ubuntu b76f59edcb Added cuckooimport.py 2016-12-07 16:36:31 +00:00
Koen Van Impe 077470b8ed Merge remote-tracking branch 'MISP/master' 2016-11-30 13:06:43 +01:00
Koen Van Impe cb29506640 Extra VTI detections 2016-11-27 22:42:43 +01:00
Raphaël Vinot 79a0b9e667 Merge pull request #73 from FloatingGhost/master 
Use SpooledTemp, not NamedTemp file
 2016-11-21 16:37:11 +01:00
Hannah Ward 1f49f36205
Removed unneeded modules 2016-11-21 13:05:07 +00:00
Hannah Ward 0dfea44001
Use SpooledTemp, not NamedTemp file 2016-11-21 11:57:04 +00:00
Raphaël Vinot e78e008aa3 Merge pull request #72 from FloatingGhost/master 
Migrated stiximport to use misp-stix-converter
 2016-11-21 12:06:16 +01:00
Hannah Ward c567d1e6f2
Moved to misp_stix_converter 2016-11-21 10:59:30 +00:00
Koen Van Impe 3253d92b42 Submit malware samples 
_submit now includes malware samples (zipped content from misp)
_import checks when no vti_results are returned + bugfix
 2016-11-18 18:23:52 +01:00
Raphaël Vinot 5624104b77 Fix STIX import module 2016-11-15 16:47:17 +01:00
Raphaël Vinot c676587461 Multiple clanges in the vmray modules. 
* Generic fix to load modules requiring a local library
* Fix python3 support
* PEP8 related cleanups
 2016-11-15 16:43:11 +01:00
Koen Van Impe adda9562c0 VMRay Import & Submit module 
* First commit
* No support for archives (yet) submit
 2016-11-13 21:43:59 +01:00
seamus tuohy 5033b1a9ca Added email meta-data import module. 
This email meta-data import module collects basic meta-data from an e-mail
and populates an event with it. It populates the email subject, source
addresses, destination addresses, subject, and any attachment file names.
This commit also contains unit-tests for this module as well as updates to
the readme. Readme updates are additions aimed to make it easier for
outsiders to build modules.
 2016-10-22 17:13:20 -04:00