MISP
/
PyMISP
mirror of https://github.com/MISP/PyMISP
2
0
Fork 0
Code Issues Releases Wiki Activity
2,729 Commits
13 Branches
170 Tags
94 MiB
Commit Graph

198 Commits (e2f9a7c6f9fd5d23b5bf02ff4c12e69c677f9de8)

Author SHA1 Message Date
Raphaël Vinot b251d70164 fix: Properly convert MSG to EML 2022-07-21 16:31:03 +02:00
Raphaël Vinot 3882ade918 chg: Bump deps 2022-07-21 11:18:00 +02:00
Raphaël Vinot 63d402b358 chg: Make mypy happy 2022-06-16 13:15:27 +02:00
malvidin cfded6e8bb Fix multiple_space warning 2022-06-16 09:44:25 +02:00
malvidin 2b98616982 Option to include more URLObject attributes 
Add publicsuffixlist faup for URLObject Windows support
URLObject with PSLFaup prefers IP to host/domain
 2022-06-16 09:38:39 +02:00
Sami Tainio 25fb7b5a28
chg: Removed a whitespace 2021-11-01 13:41:51 +02:00
Sami Tainio 8772c1fa5e
new: Add Blind Carbon Copy (bcc) headers 2021-11-01 13:35:39 +02:00
Raphaël Vinot a16aa03872 chg: Keep strict and generate attributes when needed 2021-10-28 16:29:27 -04:00
Thomas Dupuy c82dd6848f chg: Unified constructors 2021-10-27 14:27:38 -04:00
Raphaël Vinot 54d38df6dc fix: message_from_bytes really dislikes newline at the beginning of a mail 2021-09-30 11:16:03 +02:00
Sami Tainio f6c8e2ad0d
Remove unicode to ascii parts 2021-09-28 16:42:15 +03:00
Sami Tainio 2fb354a938
Fix #787 and add Unicode to ASCII function 
Fix #787
- Uses regex to pick up the hostnames/domains from the "Received: from" headers.

Unicode to ASCII function
- Spam messages more often than not contain junk text as unicode characters in the headers. The "from" and "subject" headers being the most common ones. Before this change the script would error on such emails or sometimes replace the unicode characters with questionmarks "?".
- Function takes argument as an input and then encodes it in ascii while ignoring any malformed data. It then returns an ASCII string without the unicode characters.
- Currently implemented for "from" and "subject" handling.
 2021-09-28 14:50:17 +03:00
Raphaël Vinot d44847b63a fix: skip IPs in Received header 
Related:  #787
 2021-09-27 10:27:14 +02:00
Raphaël Vinot 9fc4d90454 new: Add few keys to email object creator 
Fix #787
 2021-09-23 17:10:27 +02:00
iglocska 95f20939f2
Revert "chg: Remove legacy stix converter." 
This reverts commit 94ce4a367b.

- breaks misp-stix converter, reverting it for now, let's find a way to deprecate this without outright removing it
 2021-06-23 12:19:04 +02:00
Raphaël Vinot 3252361b3c fix: Skip nameless sections in ELF 
Related: #678
 2021-03-16 17:56:06 +01:00
Raphaël Vinot 2734224958 chg: Raise exception on missing template in CSVLoader 
Related: #714
 2021-03-05 19:33:27 +01:00
Raphaël Vinot 94ce4a367b chg: Remove legacy stix converter. 2021-03-01 15:10:56 +01:00
Raphaël Vinot d0a050263e fix: Do not add the serial-number twice. 
Related: #678
 2021-02-16 18:34:58 +01:00
Raphaël Vinot 4730452ce0 fix: Skip PE section if name is none AND size is 0. 
Related: #678
 2021-02-15 16:11:18 +01:00
Raphaël Vinot 78ead2f49e chg: Disable correlation on malware-sample for FileObject 2021-02-01 12:17:28 +01:00
Raphaël Vinot c41a2f1549 chg: Remove critical warning if lief is not installed 
Fix https://github.com/MISP/MISP/issues/6908
 2021-01-26 13:14:03 +01:00
Raphaël Vinot c5c1d84bcf fix: Better warning if lief is outdated. 2021-01-21 11:55:30 +01:00
Raphaël Vinot 76c4f92c17 chg: Use lief 0.11.0, generate authenticode entries 2021-01-19 15:44:58 +01:00
Raphaël Vinot de6125a623 fix: Do not fail if extract_msg is missing 2021-01-11 14:57:22 +01:00
Raphaël Vinot fa95c9d84f fix: Properly decode the body depending on the encoding of the email 
Fix #671
 2021-01-11 14:15:34 +01:00
Raphaël Vinot c50bbd5d1c chg: Add controller argument to get_csv script 2021-01-11 11:49:12 +01:00
seamus tuohy 87c02da0d7 Updated emailobject. 
Email object no longer requires extra php libraries for install.
Tests have been expanded to improve coverage.
RTF encapsulated HTML and Plain Text will now be de-encapsulated.
The raw MSG binary will now be included in the extracted email object.
 2020-12-28 13:47:21 -05:00
nighttardis 2a4b215026 adding check if "from" is in the "received" header row 2020-11-30 18:45:53 -06:00
Raphaël Vinot 9046b08a3c fix: Do not fail on PyMISP import when mail-parser is not present 2020-11-24 14:56:29 +01:00
Raphaël Vinot 71fe62b466 fix: Make mail-parser really optional 2020-11-24 12:18:35 +01:00
Raphaël Vinot 9fed66eb2b chg: Make mail-parser an optional dependency 2020-11-24 11:17:23 +01:00
Raphaël Vinot b55370cdad chg: Improve error handling for Outlook emails 
Related: #631
 2020-11-19 11:38:35 +01:00
Jakub Onderka 9fd3d8a3e3 fix: [emailobject] Correctly parse multiple addresses 2020-10-24 17:24:18 +02:00
Jakub Onderka 055ef16e41 new: Test parsing just email header 2020-10-24 17:24:18 +02:00
Jakub Onderka 5e0ad0a47f new: Test parsing outlook message format 2020-10-24 17:24:18 +02:00
Jakub Onderka f598865ce4 new: Refactored emailobject generator 2020-10-24 17:24:17 +02:00
Jakub Onderka d39d4caf7d new: Export display name from email 2020-10-24 17:16:16 +02:00
Jakub Onderka c2fedc3850 new: Parse date from email 2020-10-24 17:16:16 +02:00
garanews cd785aab09 fix typo 
fix typo
 2020-10-01 13:45:29 +02:00
Raphaël Vinot 18474a2144 chg: Add comments to ELF, PE, and MachO object generators. 2020-09-15 12:39:59 +02:00
Raphaël Vinot 9c48079d88 new: Method to get the new version of the templates 2020-09-10 15:26:34 +02:00
Raphaël Vinot e3815a41f1 fix: Make flake8 happy 2020-09-09 15:41:42 +02:00
seamus tuohy 07137209e2 Attempt to decode utf-8-sig encoded emails. 
eml files downloaded from Windows Online security on some Windows 11
systems are automatically encoded in UTF with a byte order mark (BOM)
at the front of the file. This will cause the email parser to fail.

This is a somewhat isolated problem. It only will affects a small
subset of Windows users who download and re-upload eml files. But,
this small subset of users is the target user-base for the MISP
email module: low expertiese users who wish to quickly share
high-value indicators on an ad-hoc basis.

While this fix could be tacked onto the MISP email module instead of
here, I beleive that this fix is more appropriate in the PyMISP object
code. As the "email" object parser this object should be built to
parse all manner of emails that it may encounter. This includes common
malformations such as this one and, even horrors such as, the .msg
format. This commit adds a generically named "attempt_decoding"
function which can be expanded to address all manner of sins that
are encountered in the future.
 2020-09-09 07:45:07 -04:00
deku dd6922fd3a Exclude section correlation .rsrc and zero-filled 2020-08-14 11:13:53 -04:00
Paal Braathen ff62f1c19c Linting/Add missing whitespace 2020-07-28 20:05:42 +02:00
louis f8589061cb chg: Remove standalone default value from MISPObject children c'tor 
MISPObject.__init__ sets standalone=True by default, so there is no
need to do it in its child classes.
 2020-06-30 12:40:08 +02:00
Troy Ross 17ebfe86ab Previously file object was reporting the libmagic description of a file 
instead of the mimetype. According to [MISP DataModels](https://www.misp-project.org/datamodels/#types)
```
mime-type: A media type (also MIME type and content type) is a two-part identifier for file formats and format contents transmitted on the Internet
```
more precisely defined in [RFC2045](https://tools.ietf.org/html/rfc2045) and others.

The description returned by libmagic is more useful than the generic mime-type,
but I did not find a place to put the description in the current data model.
 2020-06-14 10:48:29 -06:00
Raphaël Vinot 5d97d7ee0c new: Add helper and test case for GitVulnFinderObject 2020-05-26 15:37:24 +02:00
Raphaël Vinot b214c7d4c1 chg: Add comment in microblog object 2020-05-12 22:34:25 +02:00