Raphaël Vinot
454c9e0f43
fix: Pep8 related fixes.
2019-02-04 11:05:51 +01:00
Raphaël Vinot
8fc5b1fd1f
fix: Make pep8 happy
2018-12-11 15:29:09 +01:00
Christophe Vandeplas
8817de4765
fix: threatanalyzer_import - bugfix for TA6.1 behavior
2018-11-16 13:29:47 +01:00
chrisr3d
fcc18cbd73
Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport
2018-09-03 15:40:19 +02:00
Christophe Vandeplas
7deeb95820
fix: ta_import - bugfixes
2018-08-21 11:13:08 +02:00
Christophe Vandeplas
8d4e2025f7
ta_import - bugfixes for TA 6.1
2018-08-03 13:58:53 +02:00
chrisr3d
8b4d24ba63
fix: Fixed fields parsing to support files from csv export with additional context
2018-08-02 15:42:59 +02:00
chrisr3d
7980aa045a
fix: Handling the case of Context included in the csv file exported from MISP
2018-08-01 17:59:00 +02:00
chrisr3d
92fbcaeff6
fix: Fixed changes omissions in handler function
2018-07-28 00:07:02 +02:00
chrisr3d
63ba7580d3
chg: Updated csvimport to support files from csv export + import MISP objects
2018-07-27 23:13:47 +02:00
Christophe Vandeplas
2f27ff1244
ta_import - support for TheatAnalyzer 6.1
2018-07-27 14:44:06 +02:00
Steve Clement
562a6b1308
- Removed test modules from view
...
- Moved skeleton expansion module to it's proper place
2018-07-03 08:27:54 +02:00
Steve Clement
549f32547d
- Reverted to <3.6 compatibility
2018-07-01 22:09:02 +08:00
Steve Clement
9f0313a97e
- Fixed log output
2018-06-30 12:01:21 +08:00
Steve Clement
184065cf74
- Forgot to import sys
2018-06-30 11:58:44 +08:00
Steve Clement
ffce2aa5cc
- Added logger functionality for debug sessions
2018-06-30 11:52:12 +08:00
Steve Clement
2f5dd9928e
- content was already a wand.obj
2018-06-30 11:38:26 +08:00
Steve Clement
90f2fe9d19
Merge remote-tracking branch 'upstream/master'
2018-06-30 01:05:01 +08:00
Steve Clement
f97359de6a
Merge branch 'master' of github.com:SteveClement/misp-modules
2018-06-30 01:04:30 +08:00
Steve Clement
ef3837077e
- Some more comments
...
- Removed libmagic, wand can handle it better
2018-06-30 00:58:25 +08:00
Christophe Vandeplas
ff793bc221
threatanalyzer_import - order of category tuned
2018-06-29 11:17:03 +02:00
Alexandre Dulaunoy
d8eeb73a4a
Merge branch 'master' into master
2018-06-29 06:49:40 +02:00
Steve Clement
fbb3617f25
- Quick comment ToDo: Avoid using Magic in future releases
2018-06-29 12:01:17 +08:00
Steve Clement
60a3fbe282
- added wand requirement
...
- fixed missing return png byte-stream
- move module import to handler to catch and report errorz
2018-06-28 23:20:38 +08:00
Steve Clement
7885017981
- fixed typo move image back in scope
2018-06-28 16:59:03 +08:00
chrisr3d
7dd8e988c0
Updated the list of modules (removed stiximport)
2018-06-28 10:51:40 +02:00
Steve Clement
59b7688bdc
- Added initial PDF support, nothing is processed yet
...
- Test to replace PIL with wand
2018-06-28 16:00:14 +08:00
chrisr3d
2b509a2fd3
Updated delimiter finder function
2018-05-18 11:38:13 +02:00
chrisr3d
1fb72f3c7a
add: Added user config to specify if there is a header in the csv to import
2018-05-18 11:33:53 +02:00
chrisr3d
dba8bd8c5b
fix: Avoid trying to build attributes with not intended fields
...
- Previously: if the header field is not an attribute type, then
it was added as an attribute field.
PyMISP then used to skip it if needed
- Now: Those fields are discarded before they are put in an attribute
2018-05-17 16:24:11 +02:00
chrisr3d
c088b13f03
fix: Using userConfig to define the header instead of moduleconfig
2018-05-17 13:47:49 +02:00
Christophe Vandeplas
0593dbb408
ta import - more filter for pollution
2018-05-16 11:50:47 +02:00
Christophe Vandeplas
67cecc89d0
threatanalyzer_import - minor generic noise removal
2018-05-15 13:02:17 +02:00
Christophe Vandeplas
27a22e5d86
threatanalyzer_import - loads sample info + pollution fix
2018-05-03 09:42:38 +02:00
Christophe Vandeplas
370011c081
threatanalyzer_import - fix regkey issue
2018-05-02 12:43:34 +02:00
Koen Van Impe
6d23d4f4c7
Fix VMRay API access error
...
hotfix for the "Unable to access VMRay API" error
2018-03-30 15:11:25 +02:00
chrisr3d
d885286792
Clarified functions arguments using a class
2018-03-05 19:59:30 +01:00
chrisr3d
4d7642ac91
add: Added Object References in the objects imported
2018-03-05 14:58:31 +01:00
chrisr3d
82fe8ba78c
fix: Fixed input & output of the module
2018-03-02 11:03:21 +01:00
chrisr3d
70436b7ddb
Merge branch 'csvimport' of github.com:chrisr3d/misp-modules into goaml
2018-03-02 09:40:46 +01:00
chrisr3d
c9ef578262
Removed print
2018-03-02 09:09:12 +01:00
chrisr3d
e6c55f5dde
fix: Fixed input & output of the module
...
Also updated some functions
2018-03-02 09:03:51 +01:00
chrisr3d
03d20856d9
add: added goamlimport
2018-02-28 22:46:39 +01:00
chrisr3d
323f71cdd3
Fixed some details about the module output
2018-02-28 17:41:45 +01:00
chrisr3d
8f5c08e2c6
Converting GoAML into MISPEvent
2018-02-28 15:07:55 +01:00
chrisr3d
cad62464c5
Now parsing all the transaction attributes
2018-02-27 11:08:37 +01:00
chrisr3d
478cd53912
add: Added dictionary to map aml types into MISP types
2018-02-26 18:13:43 +01:00
chrisr3d
81a6be17d3
chg: Structurded data
2018-02-26 11:47:35 +01:00
chrisr3d
f361fb4ee3
Reading the entire document, to create a big dictionary containing the data, as a beginning
2018-02-20 17:00:13 +01:00
Thomas Gardner
69d733bb35
added csvimport to __init__.py
2018-02-01 10:22:28 -07:00
chrisr3d
71c00954d0
fix: Solved reading problems for some files
2018-01-30 11:20:28 +01:00
chrisr3d
b2ec186ccb
Updated delimiter finder method
2018-01-29 17:04:32 +01:00
chrisr3d
529d22cca8
fix: skipping empty lines
2018-01-29 09:19:58 +01:00
chrisr3d
56cbd72b65
Fixed data treatment & other updates
2018-01-28 18:12:40 +01:00
chrisr3d
4d846f968f
Updated delimiter parsing & data reading functions
2018-01-26 17:11:01 +01:00
chrisr3d
b9d72bb043
First version of csv import module
...
- If more than 1 misp type is recognized, for each one an
attribute is created
- Needs to have header set by user as parameters of the module atm
- Review needed to see the feasibility with fields that can create
confusion and be interpreted both as misp type or attribute field
(for instance comment is a misp type and an attribute field)
2018-01-25 15:44:08 +01:00
Christophe Vandeplas
46975f4f16
Added ThreatAnalyzer sandbox import
...
Experimental module - some parts should be migrated to
2018-01-16 11:05:26 +01:00
Raphaël Vinot
951a0f974b
fix: OpenIOC importer
2017-10-25 11:27:59 -04:00
seamus tuohy
40c71af637
Added support for malformed internationalized email headers
...
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93 )
To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.
This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
2017-07-02 18:03:14 -04:00
Raphaël Vinot
c42c8a800e
Update travis, fix open ioc import
2017-05-24 07:39:18 +02:00
Tristan METAYER
75c02058e6
replace tab by space
2017-05-11 09:56:43 +02:00
Tristan METAYER
ba1d715ad1
Add a field for user to add tag for this import
2017-05-11 09:54:25 +02:00
Tristan METAYER
96f9cb4699
typo correction
2017-05-02 15:07:33 +02:00
Tristan METAYER
4ef7261168
Add user config to not add file as attachement in a box
2017-05-02 15:04:40 +02:00
Tristan METAYER
79f48eccfe
If filename add iocfilename as attachment
2017-05-02 14:41:22 +02:00
Raphaël Vinot
c508e60f65
Add OpenIOC import module
2017-02-27 13:32:31 +01:00
Richard van den Berg
3a4c540a81
Updated description to reflect merging use case
2017-01-11 10:08:35 +01:00
Richard van den Berg
50bae1f549
Simple import module to import MISP JSON format
2017-01-11 10:08:35 +01:00
seamus tuohy
83a9d695ea
Email import no longer unzips major compressed text document formats.
...
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.
It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
2017-01-10 09:55:33 -05:00
Raphaël Vinot
1051e2210b
Keep zip content as binary
2017-01-07 19:30:00 -05:00
Raphaël Vinot
9f84db3659
Fix tests, cleanup
2017-01-07 18:36:08 -05:00
Raphaël Vinot
2db845c45c
Improve support of email attachments
...
Related to #90
2017-01-07 14:39:52 -05:00
Raphaël Vinot
b51806ac9f
Improve support of email importer if headers are missing
...
Fix #88
2017-01-07 10:25:38 -05:00
Raphaël Vinot
02f5e95a98
Fix python 3.6 support
2017-01-06 20:36:09 -05:00
Raphaël Vinot
329586768b
Make PEP8 happy
2017-01-06 20:10:44 -05:00
Raphaël Vinot
7a9774bff7
Add email_import in the modules loaded by default
2017-01-06 19:23:23 -05:00
Raphaël Vinot
93a49c3c1d
Make PEP8 happy
2017-01-06 19:01:19 -05:00
Raphaël Vinot
3f83357a2d
Fix failing test (bug in the mail parser?)
2017-01-06 18:56:29 -05:00
seamus tuohy
1a7973bc06
Add additional email parsing and tests
...
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
2017-01-04 10:21:36 -08:00
seamus tuohy
0ff270a3be
Fixed basic errors
2016-12-26 14:33:10 -08:00
seamus tuohy
08261366b7
Merged with current master
2016-12-26 14:17:20 -08:00
seamus tuohy
86ae72c444
Added attachment and url support
2016-12-26 13:55:54 -08:00
Ubuntu
b76f59edcb
Added cuckooimport.py
2016-12-07 16:36:31 +00:00
Koen Van Impe
077470b8ed
Merge remote-tracking branch 'MISP/master'
2016-11-30 13:06:43 +01:00
Koen Van Impe
cb29506640
Extra VTI detections
2016-11-27 22:42:43 +01:00
Raphaël Vinot
79a0b9e667
Merge pull request #73 from FloatingGhost/master
...
Use SpooledTemp, not NamedTemp file
2016-11-21 16:37:11 +01:00
Hannah Ward
1f49f36205
Removed unneeded modules
2016-11-21 13:05:07 +00:00
Hannah Ward
0dfea44001
Use SpooledTemp, not NamedTemp file
2016-11-21 11:57:04 +00:00
Raphaël Vinot
e78e008aa3
Merge pull request #72 from FloatingGhost/master
...
Migrated stiximport to use misp-stix-converter
2016-11-21 12:06:16 +01:00
Hannah Ward
c567d1e6f2
Moved to misp_stix_converter
2016-11-21 10:59:30 +00:00
Koen Van Impe
3253d92b42
Submit malware samples
...
_submit now includes malware samples (zipped content from misp)
_import checks when no vti_results are returned + bugfix
2016-11-18 18:23:52 +01:00
Raphaël Vinot
5624104b77
Fix STIX import module
2016-11-15 16:47:17 +01:00
Raphaël Vinot
c676587461
Multiple clanges in the vmray modules.
...
* Generic fix to load modules requiring a local library
* Fix python3 support
* PEP8 related cleanups
2016-11-15 16:43:11 +01:00
Koen Van Impe
adda9562c0
VMRay Import & Submit module
...
* First commit
* No support for archives (yet) submit
2016-11-13 21:43:59 +01:00
seamus tuohy
5033b1a9ca
Added email meta-data import module.
...
This email meta-data import module collects basic meta-data from an e-mail
and populates an event with it. It populates the email subject, source
addresses, destination addresses, subject, and any attachment file names.
This commit also contains unit-tests for this module as well as updates to
the readme. Readme updates are additions aimed to make it easier for
outsiders to build modules.
2016-10-22 17:13:20 -04:00
Hannah Ward
0521833c65
Removed useless pickle storage of stiximport
2016-09-06 14:12:09 +01:00
Hannah Ward
a492d975c4
Now searches within observable_compositions
2016-08-19 17:21:12 +01:00
Hannah Ward
6db269f965
stiximport now uses temporary files to store stix data.
...
Set max size in config, in bytes
2016-08-12 13:53:23 +01:00
Raphaël Vinot
c6fccf1b7e
Make PEP8 happy \o/
2016-08-12 14:09:59 +02:00
Raphaël Vinot
91675a635c
Move stiximport.py to misp_modules/modules/import_mod/
2016-08-12 14:08:47 +02:00