MISP
/
mail_to_misp
mirror of https://github.com/MISP/mail_to_misp
2
0
Fork 0
Code Issues Releases Wiki Activity
88 Commits
5 Branches
2 Tags
3 MiB
Go to file
Sascha Rommelfangen 4bd1d6179a config example fixed 2017-06-30 08:35:50 +02:00
MUA python3 2017-05-24 15:49:49 +02:00
README.md Update README.md 2017-06-02 08:52:25 +02:00
fake_smtp.py fixed larger size mails handling 2017-06-02 11:13:28 +02:00
hashmarker.py new functionalities (hashes, ids_flag) 2017-04-28 09:58:58 +02:00
mail_to_misp.py better url/hostname/ip handling 2017-06-16 09:59:46 +02:00
mail_to_misp_config.py-example config example fixed 2017-06-30 08:35:50 +02:00
urlmarker.py modifications to regex 2017-05-24 15:44:36 +02:00

README.md

mail_to_misp

Connect your mail infrastructure to MISP in order to create events based on the information contained within mails.

Features

  • Extraction of URLs and IP addresses (and port numbers) from free text emails
  • Extraction of hostnames from URLs
  • Extraction of hashes (MD5, SHA1, SHA256)
  • DNS expansion
  • Custom filter list for lines containing specific words
  • Subject filters
  • Respecting TLP classification mentioned in free text (including optional spelling robustness)
  • Refanging of URLs ('hxxp://...')
  • Add tags automatically based on key words (configurable)
  • Add tags automatically depending on the presence of other tags (configurable)
  • Add tags automatically depending on presence of hashes (e.g. for automatic expansion)
  • Ignore 'whitelisted' domains (configurable)
  • Specify a stop word term to no further process input
  • Configurable list of attributes not to enable the IDS flag
  • Automatically create 'external analysis' links based on filter list (e.g. VirusTotal, malwr.com)
  • Automatically create 'internal reference' links based on filter list
  • Detection of forwarded messages
  • Process attachments as malware samples
  • Logging to syslog
  • Remove "[tags]", "Re:" and "Fwd:" from subjects
  • Optionally attach entire mail to event
  • Contains now a fake-smtpd spamtrap which delivers IoCs/mails to MISP

Implementation

For the moment, the implemented workflow is:

  1. Apple Mail

Email -> Apple Mail -> Mail rule -> AppleScript -> mail_to_misp -> PyMISP -> MISP

  1. Mozilla Thunderbird

Email -> Thunderbird -> Mail rule -> filterscript -> thunderbird_wrapper -> mail_to_misp -> PyMISP -> MISP

  1. Postfix and others

Email -> mail_to_misp

Installation

Apple Mail

  1. Mail rule script
  • git clone this repository
  • open the AppleScript file MUA/Apple/Mail/MISP Mail Rule Action.txt in Apple's 'Script Editor'
  • adjust the path to the python installation and location of the mail_to_misp.py script
  • save it in ~/Library/Application Scripts/com.apple.mail/
  1. Create a mail rule based on your needs, executing the AppleScript defined before
  2. Configure mail_to_misp_config.py

Thunderbird

  1. Git clone https://github.com/rommelfs/filterscript and install plugin (instructions within the project description)
  2. Mail rule script
  • git clone this repository
  • open the bash script MUA/Mozilla/Thunderbird/thunderbird_wrapper.sh and adujst the paths
  • adjust the path to the python installation and location of the mail_to_misp.py script
  1. Create a mail rule based on your needs, executing the thunderbird_wrapper.sh script
  2. Configure mail_to_misp_config.py

You should be able to create MISP events now.

Outlook

Outlook is not implemented due to lack of test environment. However, it should be feasible to do it this way:

import win32com.client
import pythoncom
 
class Handler_Class(object):
    def OnNewMailEx(self, receivedItemsIDs):
        for ID in receivedItemsIDs.split(","):
            # Microsoft.Office.Interop.Outlook _MailItem properties:
            # https://msdn.microsoft.com/en-us/library/microsoft.office.interop.outlook._mailitem_properties.aspx
            mailItem = outlook.Session.GetItemFromID(ID)
            print "Subj: " + mailItem.Subject
            print "Body: " + mailItem.Body.encode( 'ascii', 'ignore' )
            print "========"
         
outlook = win32com.client.DispatchWithEvents("Outlook.Application", Handler_Class)
pythoncom.PumpMessages()

(from: https://blog.matthewurch.ca/?p=236)

Obviously, you would like to filter mails based on subject or from address and pass subject and body to mail_to_misp.py in order to do something useful. Pull-requests welcome for actual implementations :)

Postfix (or other MTA)

  1. Setup a new email address in the aliases file (e.g. /etc/aliases) and configure the correct path:

misp_handler: "|/path/to/mail_to_misp.py"

  1. Rebuild the DB:

$ sudo newaliases

  1. Configure mail_to_misp_config.py

You should now be able to send your IoC-containing mails to misp_handler@YOURDOMAIN.

Fake-SMTPD spamtrap

If you want to process all incoming junk mails automatically and collect the contained information in a (separate?) MISP instance, you could use the fake_smtp.py script. It listens on port 25, accepts all mails and pushes them through mail_to_misp to a MISP instance.

  1. Configure mail_to_misp_config.py

  2. Run fake_smtp.py (as root)

$ sudo python3 fake_smtp.py

Requirements

General

Thunderbird