MISP
/
misp-modules
mirror of https://github.com/MISP/misp-modules
2
0
Fork 0
Code Issues Releases Wiki Activity
2,467 Commits
7 Branches
54 Tags
19 MiB
Commit Graph

205 Commits (e3472136f055214abf2ee49878fb3ee31160851d)

Author SHA1 Message Date
root c886247a64
fix: Fixed standard MISP csv format header 
- The csv header we can find in data produced from
  MISP restSearch csv format is the one to use to
  recognize a csv file produced by MISP
 2019-05-01 22:32:06 +02:00
root f900cb7c68
fix: Fixed introspection fields for csvimport & goamlimport 
- Added format field for goaml so the module is
  known as returning MISP attributes & objects
- Fixed introspection to make the format, user
  config and input source fields visible from
  MISP (format also added at the same time)
 2019-05-01 22:28:19 +02:00
root db74c5f49a
fix: Fixed libraries import that changed with the latest merge 2019-05-01 22:26:53 +02:00
chrisr3d 55e494c9ed Merge branch 'features_csvimport' of github.com:MISP/misp-modules into features_csvimport 2019-04-30 17:16:31 +02:00
Raphaël Vinot 454c9e0f43 fix: Pep8 related fixes. 2019-02-04 11:05:51 +01:00
Raphaël Vinot 8fc5b1fd1f fix: Make pep8 happy 2018-12-11 15:29:09 +01:00
Christophe Vandeplas 8817de4765 fix: threatanalyzer_import - bugfix for TA6.1 behavior 2018-11-16 13:29:47 +01:00
chrisr3d fcc18cbd73 Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport 2018-09-03 15:40:19 +02:00
Christophe Vandeplas 7deeb95820 fix: ta_import - bugfixes 2018-08-21 11:13:08 +02:00
Christophe Vandeplas 8d4e2025f7 ta_import - bugfixes for TA 6.1 2018-08-03 13:58:53 +02:00
chrisr3d 8b4d24ba63
fix: Fixed fields parsing to support files from csv export with additional context 2018-08-02 15:42:59 +02:00
chrisr3d 7980aa045a
fix: Handling the case of Context included in the csv file exported from MISP 2018-08-01 17:59:00 +02:00
chrisr3d 92fbcaeff6
fix: Fixed changes omissions in handler function 2018-07-28 00:07:02 +02:00
chrisr3d 63ba7580d3
chg: Updated csvimport to support files from csv export + import MISP objects 2018-07-27 23:13:47 +02:00
Christophe Vandeplas 2f27ff1244 ta_import - support for TheatAnalyzer 6.1 2018-07-27 14:44:06 +02:00
Steve Clement 562a6b1308 - Removed test modules from view 
- Moved skeleton expansion module to it's proper place
 2018-07-03 08:27:54 +02:00
Steve Clement 549f32547d - Reverted to <3.6 compatibility 2018-07-01 22:09:02 +08:00
Steve Clement 9f0313a97e - Fixed log output 2018-06-30 12:01:21 +08:00
Steve Clement 184065cf74 - Forgot to import sys 2018-06-30 11:58:44 +08:00
Steve Clement ffce2aa5cc - Added logger functionality for debug sessions 2018-06-30 11:52:12 +08:00
Steve Clement 2f5dd9928e - content was already a wand.obj 2018-06-30 11:38:26 +08:00
Steve Clement 90f2fe9d19 Merge remote-tracking branch 'upstream/master' 2018-06-30 01:05:01 +08:00
Steve Clement f97359de6a Merge branch 'master' of github.com:SteveClement/misp-modules 2018-06-30 01:04:30 +08:00
Steve Clement ef3837077e - Some more comments 
- Removed libmagic, wand can handle it better
 2018-06-30 00:58:25 +08:00
Christophe Vandeplas ff793bc221
threatanalyzer_import - order of category tuned 2018-06-29 11:17:03 +02:00
Alexandre Dulaunoy d8eeb73a4a
Merge branch 'master' into master 2018-06-29 06:49:40 +02:00
Steve Clement fbb3617f25 - Quick comment ToDo: Avoid using Magic in future releases 2018-06-29 12:01:17 +08:00
Steve Clement 60a3fbe282 - added wand requirement 
- fixed missing return png byte-stream
- move module import to handler to catch and  report errorz
 2018-06-28 23:20:38 +08:00
Steve Clement 7885017981 - fixed typo move image back in scope 2018-06-28 16:59:03 +08:00
chrisr3d 7dd8e988c0
Updated the list of modules (removed stiximport) 2018-06-28 10:51:40 +02:00
Steve Clement 59b7688bdc - Added initial PDF support, nothing is processed yet 
- Test to replace PIL with wand
 2018-06-28 16:00:14 +08:00
chrisr3d 2b509a2fd3
Updated delimiter finder function 2018-05-18 11:38:13 +02:00
chrisr3d 1fb72f3c7a
add: Added user config to specify if there is a header in the csv to import 2018-05-18 11:33:53 +02:00
chrisr3d dba8bd8c5b
fix: Avoid trying to build attributes with not intended fields 
- Previously: if the header field is not an attribute type, then
              it was added as an attribute field.
              PyMISP then used to skip it if needed

- Now: Those fields are discarded before they are put in an attribute
 2018-05-17 16:24:11 +02:00
chrisr3d c088b13f03
fix: Using userConfig to define the header instead of moduleconfig 2018-05-17 13:47:49 +02:00
Christophe Vandeplas 0593dbb408 ta import - more filter for pollution 2018-05-16 11:50:47 +02:00
Christophe Vandeplas 67cecc89d0 threatanalyzer_import - minor generic noise removal 2018-05-15 13:02:17 +02:00
Christophe Vandeplas 27a22e5d86 threatanalyzer_import - loads sample info + pollution fix 2018-05-03 09:42:38 +02:00
Christophe Vandeplas 370011c081 threatanalyzer_import - fix regkey issue 2018-05-02 12:43:34 +02:00
Koen Van Impe 6d23d4f4c7 Fix VMRay API access error 
hotfix for the "Unable to access VMRay API" error
 2018-03-30 15:11:25 +02:00
chrisr3d d885286792
Clarified functions arguments using a class 2018-03-05 19:59:30 +01:00
chrisr3d 4d7642ac91
add: Added Object References in the objects imported 2018-03-05 14:58:31 +01:00
chrisr3d 82fe8ba78c
fix: Fixed input & output of the module 2018-03-02 11:03:21 +01:00
chrisr3d 70436b7ddb Merge branch 'csvimport' of github.com:chrisr3d/misp-modules into goaml 2018-03-02 09:40:46 +01:00
chrisr3d c9ef578262
Removed print 2018-03-02 09:09:12 +01:00
chrisr3d e6c55f5dde
fix: Fixed input & output of the module 
Also updated some functions
 2018-03-02 09:03:51 +01:00
chrisr3d 03d20856d9
add: added goamlimport 2018-02-28 22:46:39 +01:00
chrisr3d 323f71cdd3
Fixed some details about the module output 2018-02-28 17:41:45 +01:00
chrisr3d 8f5c08e2c6
Converting GoAML into MISPEvent 2018-02-28 15:07:55 +01:00
chrisr3d cad62464c5
Now parsing all the transaction attributes 2018-02-27 11:08:37 +01:00
chrisr3d 478cd53912
add: Added dictionary to map aml types into MISP types 2018-02-26 18:13:43 +01:00
chrisr3d 81a6be17d3
chg: Structurded data 2018-02-26 11:47:35 +01:00
chrisr3d f361fb4ee3
Reading the entire document, to create a big dictionary containing the data, as a beginning 2018-02-20 17:00:13 +01:00
Thomas Gardner 69d733bb35 added csvimport to __init__.py 2018-02-01 10:22:28 -07:00
chrisr3d 71c00954d0
fix: Solved reading problems for some files 2018-01-30 11:20:28 +01:00
chrisr3d b2ec186ccb
Updated delimiter finder method 2018-01-29 17:04:32 +01:00
chrisr3d 529d22cca8
fix: skipping empty lines 2018-01-29 09:19:58 +01:00
chrisr3d 56cbd72b65
Fixed data treatment & other updates 2018-01-28 18:12:40 +01:00
chrisr3d 4d846f968f
Updated delimiter parsing & data reading functions 2018-01-26 17:11:01 +01:00
chrisr3d b9d72bb043
First version of csv import module 
- If more than 1 misp type is recognized, for each one an
  attribute is created

- Needs to have header set by user as parameters of the module atm

- Review needed to see the feasibility with fields that can create
  confusion and be interpreted both as misp type or attribute field
  (for instance comment is a misp type and an attribute field)
 2018-01-25 15:44:08 +01:00
Christophe Vandeplas 46975f4f16 Added ThreatAnalyzer sandbox import 
Experimental module - some parts should be migrated to
 2018-01-16 11:05:26 +01:00
Raphaël Vinot 951a0f974b fix: OpenIOC importer 2017-10-25 11:27:59 -04:00
seamus tuohy 40c71af637 Added support for malformed internationalized email headers 
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93)

To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.

This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
 2017-07-02 18:03:14 -04:00
Raphaël Vinot c42c8a800e Update travis, fix open ioc import 2017-05-24 07:39:18 +02:00
Tristan METAYER 75c02058e6 replace tab by space 2017-05-11 09:56:43 +02:00
Tristan METAYER ba1d715ad1 Add a field for user to add tag for this import 2017-05-11 09:54:25 +02:00
Tristan METAYER 96f9cb4699 typo correction 2017-05-02 15:07:33 +02:00
Tristan METAYER 4ef7261168 Add user config to not add file as attachement in a box 2017-05-02 15:04:40 +02:00
Tristan METAYER 79f48eccfe If filename add iocfilename as attachment 2017-05-02 14:41:22 +02:00
Raphaël Vinot c508e60f65 Add OpenIOC import module 2017-02-27 13:32:31 +01:00
Richard van den Berg 3a4c540a81 Updated description to reflect merging use case 2017-01-11 10:08:35 +01:00
Richard van den Berg 50bae1f549 Simple import module to import MISP JSON format 2017-01-11 10:08:35 +01:00
seamus tuohy 83a9d695ea Email import no longer unzips major compressed text document formats. 
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.

It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
 2017-01-10 09:55:33 -05:00
Raphaël Vinot 1051e2210b Keep zip content as binary 2017-01-07 19:30:00 -05:00
Raphaël Vinot 9f84db3659 Fix tests, cleanup 2017-01-07 18:36:08 -05:00
Raphaël Vinot 2db845c45c Improve support of email attachments 
Related to #90
 2017-01-07 14:39:52 -05:00
Raphaël Vinot b51806ac9f Improve support of email importer if headers are missing 
Fix #88
 2017-01-07 10:25:38 -05:00
Raphaël Vinot 02f5e95a98 Fix python 3.6 support 2017-01-06 20:36:09 -05:00
Raphaël Vinot 329586768b Make PEP8 happy 2017-01-06 20:10:44 -05:00
Raphaël Vinot 7a9774bff7 Add email_import in the modules loaded by default 2017-01-06 19:23:23 -05:00
Raphaël Vinot 93a49c3c1d Make PEP8 happy 2017-01-06 19:01:19 -05:00
Raphaël Vinot 3f83357a2d Fix failing test (bug in the mail parser?) 2017-01-06 18:56:29 -05:00
seamus tuohy 1a7973bc06 Add additional email parsing and tests 
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
 2017-01-04 10:21:36 -08:00
seamus tuohy 0ff270a3be Fixed basic errors 2016-12-26 14:33:10 -08:00
seamus tuohy 08261366b7 Merged with current master 2016-12-26 14:17:20 -08:00
seamus tuohy 86ae72c444 Added attachment and url support 2016-12-26 13:55:54 -08:00
Ubuntu b76f59edcb Added cuckooimport.py 2016-12-07 16:36:31 +00:00
Koen Van Impe 077470b8ed Merge remote-tracking branch 'MISP/master' 2016-11-30 13:06:43 +01:00
Koen Van Impe cb29506640 Extra VTI detections 2016-11-27 22:42:43 +01:00
Raphaël Vinot 79a0b9e667 Merge pull request #73 from FloatingGhost/master 
Use SpooledTemp, not NamedTemp file
 2016-11-21 16:37:11 +01:00
Hannah Ward 1f49f36205
Removed unneeded modules 2016-11-21 13:05:07 +00:00
Hannah Ward 0dfea44001
Use SpooledTemp, not NamedTemp file 2016-11-21 11:57:04 +00:00
Raphaël Vinot e78e008aa3 Merge pull request #72 from FloatingGhost/master 
Migrated stiximport to use misp-stix-converter
 2016-11-21 12:06:16 +01:00
Hannah Ward c567d1e6f2
Moved to misp_stix_converter 2016-11-21 10:59:30 +00:00
Koen Van Impe 3253d92b42 Submit malware samples 
_submit now includes malware samples (zipped content from misp)
_import checks when no vti_results are returned + bugfix
 2016-11-18 18:23:52 +01:00
Raphaël Vinot 5624104b77 Fix STIX import module 2016-11-15 16:47:17 +01:00
Raphaël Vinot c676587461 Multiple clanges in the vmray modules. 
* Generic fix to load modules requiring a local library
* Fix python3 support
* PEP8 related cleanups
 2016-11-15 16:43:11 +01:00
Koen Van Impe adda9562c0 VMRay Import & Submit module 
* First commit
* No support for archives (yet) submit
 2016-11-13 21:43:59 +01:00
seamus tuohy 5033b1a9ca Added email meta-data import module. 
This email meta-data import module collects basic meta-data from an e-mail
and populates an event with it. It populates the email subject, source
addresses, destination addresses, subject, and any attachment file names.
This commit also contains unit-tests for this module as well as updates to
the readme. Readme updates are additions aimed to make it easier for
outsiders to build modules.
 2016-10-22 17:13:20 -04:00
Hannah Ward 0521833c65
Removed useless pickle storage of stiximport 2016-09-06 14:12:09 +01:00
Hannah Ward a492d975c4
Now searches within observable_compositions 2016-08-19 17:21:12 +01:00
Hannah Ward 6db269f965
stiximport now uses temporary files to store stix data. 
Set max size in config, in bytes
 2016-08-12 13:53:23 +01:00
Raphaël Vinot c6fccf1b7e Make PEP8 happy \o/ 2016-08-12 14:09:59 +02:00
Raphaël Vinot 91675a635c Move stiximport.py to misp_modules/modules/import_mod/ 2016-08-12 14:08:47 +02:00
Raphaël Vinot 59b16950f7 Remove bin script, use cleaner way. Fix last commit. 2016-08-12 12:35:33 +02:00