Commit Graph

34 Commits (529719d9d89dab3890677a2c9bf1c82e1c6ee18c)

Author SHA1 Message Date
seamus tuohy 40c71af637 Added support for malformed internationalized email headers
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93)

To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.

This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
2017-07-02 18:03:14 -04:00
seamus tuohy 3eecf9afe5 Merge branch 'master' into utf_hate 2017-07-01 18:23:01 -04:00
Raphaël Vinot c508e60f65 Add OpenIOC import module 2017-02-27 13:32:31 +01:00
seamus tuohy 0566049c63 Added unit tests for UTF emails 2017-01-11 17:53:54 -05:00
seamus tuohy 83a9d695ea Email import no longer unzips major compressed text document formats.
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.

It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
2017-01-10 09:55:33 -05:00
Raphaël Vinot 9f84db3659 Fix tests, cleanup 2017-01-07 18:36:08 -05:00
seamus tuohy 1a7973bc06 Add additional email parsing and tests
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
2017-01-04 10:21:36 -08:00
seamus tuohy 0ff270a3be Fixed basic errors 2016-12-26 14:33:10 -08:00
seamus tuohy 08261366b7 Merged with current master 2016-12-26 14:17:20 -08:00
seamus tuohy 6ec307b911 Adding basic test mockup 2016-12-26 14:09:52 -08:00
Raphaël Vinot f8bedd4554 Remove domaintools tests 2016-12-02 16:16:25 +01:00
Raphaël Vinot 2b020c55ba Add test for domaintools 2016-12-02 15:29:44 +01:00
Hannah Ward c567d1e6f2
Moved to misp_stix_converter 2016-11-21 10:59:30 +00:00
Raphaël Vinot 5624104b77 Fix STIX import module 2016-11-15 16:47:17 +01:00
seamus tuohy 5033b1a9ca Added email meta-data import module.
This email meta-data import module collects basic meta-data from an e-mail
and populates an event with it. It populates the email subject, source
addresses, destination addresses, subject, and any attachment file names.
This commit also contains unit-tests for this module as well as updates to
the readme. Readme updates are additions aimed to make it easier for
outsiders to build modules.
2016-10-22 17:13:20 -04:00
Alexandre Dulaunoy 2df8bf970e Merge pull request #47 from FloatingGhost/CEF_Export
CEF export, fixes in CountryCode, virustotal
2016-09-01 19:39:16 +02:00
Raphaël Vinot c69fae087c Add timeout for the modules, cleanup. 2016-08-25 17:36:28 +02:00
Hannah Ward 232014f221
Added virustotal tests 2016-08-17 13:01:11 +01:00
Alexandre Dulaunoy d499ac0ce6 Merge pull request #44 from Rafiot/travis
Add coverage, update logging
2016-08-12 15:20:26 +02:00
Raphaël Vinot b24b16b30a Add coverage, update logging 2016-08-12 15:15:38 +02:00
Hannah Ward 6db269f965
stiximport now uses temporary files to store stix data.
Set max size in config, in bytes
2016-08-12 13:53:23 +01:00
Hannah Ward c02a452c05
added tests, also disregards related_observables. Because they're useless 2016-08-12 12:16:49 +01:00
Raphaël Vinot ef6e3b27f8 Proper testcases 2016-06-18 15:09:42 +09:00
Raphaël Vinot 0752c5e3f4 Make it a package 2016-06-18 11:04:41 +09:00
Alexandre Dulaunoy 2699eef633 dns module test with option added 2016-04-01 08:00:56 +02:00
Raphaël Vinot be27730fd3 Add CIRCL pssl module 2016-03-25 17:38:03 +01:00
Alexandre Dulaunoy 6cf77031cc Tests updated to include CIRCL passive dns 2016-03-25 11:01:12 +01:00
Alexandre Dulaunoy 1b6afbe736 Test file for passivetotal updated 2016-03-25 09:04:23 +01:00
Alexandre Dulaunoy b505f1bd32 cve module tests added 2016-03-18 08:17:17 +01:00
Alexandre Dulaunoy e282150850 Sample JSON files reflecting config changes 2016-03-16 07:47:01 +01:00
Alexandre Dulaunoy 5cc07c0203 A minimal caching module added to cache link or url from MISP 2016-03-14 20:40:06 +01:00
Alexandre Dulaunoy d86b58165e First version of a passivetotal MISP expansion module 2016-03-02 21:17:37 +01:00
Alexandre Dulaunoy e7e8f28a03 curl is now silent 2016-02-17 18:33:33 +01:00
Alexandre Dulaunoy ad87f947a5 Basic test cases added 2016-02-17 16:05:30 +01:00