MISP
/
misp-modules
mirror of https://github.com/MISP/misp-modules
2
0
Fork 0
Code Issues Releases Wiki Activity
1,353 Commits
7 Branches
55 Tags
20 MiB
Commit Graph

723 Commits (5b1ac3dc516d29249bcf8a035f9d06708c91e3b1)

Author SHA1 Message Date
Sascha Rommelfangen ef781f59f8
fixed typo 
via #220
 2018-09-06 14:05:55 +02:00
chrisr3d fcc18cbd73 Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport 2018-09-03 15:40:19 +02:00
chrisr3d ba728f7120
fix: Fixed 1 variable misuse + cleaned up variable names 
- Fixed use of 'domain' variable instead of 'email'
- Cleaned up variable names to avoid redefinition
  of built-in variables
 2018-09-03 14:43:51 +02:00
chrisr3d cdf2f434ce
fix: Avoiding adding attributes that are already in the event 2018-09-03 14:30:33 +02:00
chrisr3d 33181bc52b
fix: Fixed quick variable issue 2018-09-03 14:29:42 +02:00
chrisr3d 0ab38feade
fix: Cleaned up test function not used anymore 2018-09-03 13:17:48 +02:00
chrisr3d 936e30b15b
fix: Multiple attributes parsing support 
- Fixing one of my previous changes not processing
  multiple attributes parsing
 2018-09-03 12:03:42 +02:00
chrisr3d 2af947a2de
fix: Removed print 2018-09-03 10:23:05 +02:00
chrisr3d bc2a73c5cf Merge branch 'master' of github.com:MISP/misp-modules 2018-09-02 20:21:01 +02:00
chrisr3d 179430d69d
fix: Some cleanup and output types fixed 
- hashes types specified in output
 2018-08-31 21:38:53 +02:00
SuRb0 b0be965e57
Update urlscan.py 
Added hash to the search so you can take advantage of the new file down load function on urlscan.io.  You can use this to pivot on file hashes and find out domains that hosting the same malicious file.
 2018-08-30 19:41:34 -05:00
chrisr3d 35f3a5e43f
fix: Quick cleanup 2018-08-30 20:45:29 +02:00
chrisr3d d15cbe58fe
fix: Quick cleanup 2018-08-30 20:41:49 +02:00
chrisr3d 18bad54603 Merge branch 'master' of github.com:MISP/misp-modules 2018-08-30 09:11:25 +02:00
Christophe Vandeplas 7deeb95820 fix: ta_import - bugfixes 2018-08-21 11:13:08 +02:00
David J a697f65382
Add error handling for DNS failures, reduce imports, and simplify misp_comments 2018-08-14 10:51:15 -05:00
David J bdbf538893
Create urlscan.py 2018-08-10 16:00:01 -05:00
chrisr3d 3a57d11745 Merge branch 'chrisr3d_patch' of github.com:chrisr3d/misp-modules 2018-08-08 17:41:07 +02:00
chrisr3d 90baa1dd5a
add: Added DBL spamhaus module documentation and in expansion init file 2018-08-08 17:05:22 +02:00
chrisr3d 61232ad93e
new: Expansion hover module to check spamhaus DBL for a domain name 2018-08-08 17:00:10 +02:00
chrisr3d 9acf66053e Merge branch 'master' of github.com:MISP/misp-modules 2018-08-08 16:51:42 +02:00
chrisr3d 0666a60b3d
fix: [cleanup] Quick clean up on exception type 2018-08-07 18:15:15 +02:00
chrisr3d bb6002a3ff
fix: [cleanup] Quick clean up on yaml load function 2018-08-07 18:14:29 +02:00
chrisr3d 57af98720d
fix: [cleanup] Quick clean up on exception type 2018-08-07 18:13:25 +02:00
Christophe Vandeplas 8d4e2025f7 ta_import - bugfixes for TA 6.1 2018-08-03 13:58:53 +02:00
chrisr3d 8b4d24ba63
fix: Fixed fields parsing to support files from csv export with additional context 2018-08-02 15:42:59 +02:00
chrisr3d 0045645c96 Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport 2018-08-01 18:02:15 +02:00
chrisr3d 7980aa045a
fix: Handling the case of Context included in the csv file exported from MISP 2018-08-01 17:59:00 +02:00
chrisr3d 63c3252062
fix: Put the report location parsing in a try/catch statement as it is an optional field 2018-07-30 14:22:40 +02:00
chrisr3d 92fbcaeff6
fix: Fixed changes omissions in handler function 2018-07-28 00:07:02 +02:00
chrisr3d 63ba7580d3
chg: Updated csvimport to support files from csv export + import MISP objects 2018-07-27 23:13:47 +02:00
Christophe Vandeplas 2f27ff1244 ta_import - support for TheatAnalyzer 6.1 2018-07-27 14:44:06 +02:00
sebdraven d7fac002af
Merge branch 'master' into dnstrails 2018-07-18 11:07:44 +02:00
Sebdraven 804e59ed8d change type of status 2018-07-18 10:58:51 +02:00
Sebdraven c8e20d9087 remove print 2018-07-18 10:51:47 +02:00
Sebdraven f2df6dc538 last commit for release 2018-07-18 10:47:42 +02:00
Sebdraven 88859a0ba7 add logs 2018-07-17 18:43:52 +02:00
Sebdraven 8cbeda40a5 add searching_stats 2018-07-17 18:42:01 +02:00
Sebdraven 9d603344c2 add searching_stats 2018-07-17 18:32:50 +02:00
Sebdraven c785cae89b correct key 2018-07-17 17:22:48 +02:00
Sebdraven 2706c4a82a correct key 2018-07-17 17:21:38 +02:00
Sebdraven 431c1511a3 correct param 2018-07-17 17:20:30 +02:00
Sebdraven 999ae1f6f0 add searching domains 2018-07-17 17:09:01 +02:00
Sebdraven a41cf59e0c add searching domains 2018-07-17 15:05:15 +02:00
Sebdraven 966f9603a9 add return 2018-07-12 15:02:46 +02:00
Sebdraven 7f52a15d16 add logs 2018-07-12 14:59:50 +02:00
Sebdraven 3eda712193 add whois expand to test 2018-07-12 14:58:48 +02:00
Sebdraven 5a422c2e5b add whois expand to test 2018-07-12 14:57:37 +02:00
Sebdraven db35c9b091 correct index error 2018-07-12 14:55:56 +02:00
Sebdraven 0341bdc398 error call functions 2018-07-12 14:52:01 +02:00
Sebdraven 2f5381d7b2 add logs 2018-07-12 14:49:51 +02:00
Sebdraven 0b0137829a add logs 2018-07-12 14:48:15 +02:00
Sebdraven 86d236f859 add status_ok to true 2018-07-12 14:47:34 +02:00
Sebdraven aa89a7fc4d add logs 2018-07-12 14:44:19 +02:00
Sebdraven 86d9427816 add logs 2018-07-12 14:42:33 +02:00
Sebdraven a0cf9de590 add logs 2018-07-12 14:38:38 +02:00
Sebdraven 9de201375b add logs 2018-07-12 14:37:09 +02:00
Sebdraven d56bf55038 add logs 2018-07-12 14:33:52 +02:00
Sebdraven 844b25b4cd correct out of bound returns 2018-07-12 14:32:56 +02:00
Sebdraven 9063da88cd correct key and return of functions 2018-07-12 14:27:59 +02:00
Sebdraven 731c06a939 add logs 2018-07-12 14:17:16 +02:00
Sebdraven fb595c08aa add logs 2018-07-12 14:16:19 +02:00
Sebdraven 41587bd568 correct typo 2018-07-12 14:14:43 +02:00
Sebdraven 4b0daee6f1 test whois history 2018-07-12 14:04:37 +02:00
Sebdraven 576b3c9b9b history whois dns 2018-07-12 13:40:51 +02:00
chrisr3d 32419c398e Merge branch 'master' of github.com:MISP/misp-modules 2018-07-12 00:05:01 +02:00
chrisr3d a62078aad1
add: Experimental expansion module to display the SIEM signatures from a sigma rule 2018-07-11 23:43:42 +02:00
Sebdraven 51067039da correct typo 2018-07-11 13:03:47 +02:00
Sebdraven 3a2aab6d71 rename misp modules 2018-07-11 12:41:54 +02:00
Sebdraven a8ae6e06e9 add a test to check if the list is not empty 2018-07-11 12:09:34 +02:00
Sebdraven f0a4c71908 add a test to check if the list is not empty 2018-07-11 12:08:01 +02:00
Sebdraven dbeec4682e add logs 2018-07-11 12:02:31 +02:00
Sebdraven fb262b451f debug whois 2018-07-11 12:00:59 +02:00
Sebdraven 80e71f582c debug ipv4 or ipv6 2018-07-11 11:58:42 +02:00
Sebdraven 386d38c88f add debug 2018-07-11 11:55:59 +02:00
Sebdraven 45decc728d debug 2018-07-11 11:55:31 +02:00
Sebdraven 45c473aef5 change status 2018-07-11 11:52:10 +02:00
Sebdraven 64e7f9c8b6 change history dns 2018-07-11 11:47:10 +02:00
Sebdraven 560dacbf7e add logs to debug 2018-07-11 11:40:22 +02:00
Sebdraven 74c611d2fb correct call function 2018-07-11 11:37:07 +02:00
Sebdraven f47a64b364 add history mx and soa 2018-07-11 11:24:49 +02:00
Sebdraven 43a49dafc6 add history dns and handler exception 2018-07-11 09:48:14 +02:00
Sebdraven 54d996cb00 add history dns 2018-07-11 09:39:09 +02:00
Sebdraven dcdb6e5895 switch type ip 2018-07-11 09:02:47 +02:00
Sebdraven 42c362d2fd refactoring expand_whois 2018-07-11 09:00:23 +02:00
Sebdraven 41635d43c7 correct typo 2018-07-11 08:49:59 +02:00
Sebdraven 3a96e189ed add ipv6 and ipv4 2018-07-11 08:43:23 +02:00
Sebdraven f2333a4978 change type 2018-07-10 16:55:13 +02:00
Sebdraven 9e6162a434 change type 2018-07-10 16:53:06 +02:00
Sebdraven 26950ea7de change loop 2018-07-10 16:51:31 +02:00
Sebdraven e9747a3379 add time sleep in each request 2018-07-10 16:41:44 +02:00
Sebdraven 602da3d1a3 control return of records 2018-07-10 16:35:01 +02:00
Sebdraven 495c720d0f add history ipv4 2018-07-10 16:31:39 +02:00
Sebdraven 21794249d0 add logs 2018-07-10 15:17:37 +02:00
Sebdraven b677cd5fc7 change categories 2018-07-10 15:16:02 +02:00
Sebdraven 1d100833a4 concat results 2018-07-10 15:12:27 +02:00
Sebdraven 1223d93d52 change name keys 2018-07-10 15:07:54 +02:00
Sebdraven 714c15f079 change return value 2018-07-10 15:05:10 +02:00
Sebdraven e1a1648f14 add logs 2018-07-10 15:01:04 +02:00
Sebdraven f710162bed change errors 2018-07-10 14:59:39 +02:00
Sebdraven 2a8fb76e84 add logs 2018-07-10 14:56:20 +02:00
Steve Clement 562a6b1308 - Removed test modules from view 
- Moved skeleton expansion module to it's proper place
 2018-07-03 08:27:54 +02:00
chrisr3d 90e42c0305
fix: Put the stix2-pattern library import in a try statement 
--> Error more easily caught
 2018-07-02 12:14:21 +02:00
chrisr3d 08d8459e1a
add: STIX2 pattern syntax validator 2018-07-02 11:38:33 +02:00
Steve Clement 549f32547d - Reverted to <3.6 compatibility 2018-07-01 22:09:02 +08:00
Steve Clement 9f0313a97e - Fixed log output 2018-06-30 12:01:21 +08:00
Steve Clement 184065cf74 - Forgot to import sys 2018-06-30 11:58:44 +08:00
Steve Clement ffce2aa5cc - Added logger functionality for debug sessions 2018-06-30 11:52:12 +08:00
Steve Clement 2f5dd9928e - content was already a wand.obj 2018-06-30 11:38:26 +08:00
Steve Clement 90f2fe9d19 Merge remote-tracking branch 'upstream/master' 2018-06-30 01:05:01 +08:00
Steve Clement f97359de6a Merge branch 'master' of github.com:SteveClement/misp-modules 2018-06-30 01:04:30 +08:00
Steve Clement ef3837077e - Some more comments 
- Removed libmagic, wand can handle it better
 2018-06-30 00:58:25 +08:00
Sebdraven 34da5cdb76 add expand whois 2018-06-29 17:57:11 +02:00
Sebdraven f1c6095914 typo 2018-06-29 17:26:56 +02:00
Sebdraven 78d6de9b7a add categories and comments 2018-06-29 17:25:37 +02:00
Sebdraven 0965def6bf add expand subdomains 2018-06-29 17:22:19 +02:00
Sebdraven 64847a8a04 add expand subdomains 2018-06-29 17:19:21 +02:00
Sebdraven 2d1adf4aa9 change categories 2018-06-29 16:30:47 +02:00
Sebdraven 0275e3ecd8 changes keys 2018-06-29 16:20:35 +02:00
Sebdraven f3962d2d05 add status ! 2018-06-29 16:17:32 +02:00
Sebdraven 09c52788b8 add methods 2018-06-29 16:11:24 +02:00
Sebdraven cfe971a271 add expand domains 2018-06-29 15:50:26 +02:00
Sebdraven 60f772b905 add new module dnstrails 2018-06-29 11:27:36 +02:00
Christophe Vandeplas ff793bc221
threatanalyzer_import - order of category tuned 2018-06-29 11:17:03 +02:00
Alexandre Dulaunoy d8eeb73a4a
Merge branch 'master' into master 2018-06-29 06:49:40 +02:00
Steve Clement c7c93b53e8 - Set tornado timeout to 300 seconds. 2018-06-29 12:02:08 +08:00
Steve Clement fbb3617f25 - Quick comment ToDo: Avoid using Magic in future releases 2018-06-29 12:01:17 +08:00
Steve Clement 60a3fbe282 - added wand requirement 
- fixed missing return png byte-stream
- move module import to handler to catch and  report errorz
 2018-06-28 23:20:38 +08:00
Steve Clement 7885017981 - fixed typo move image back in scope 2018-06-28 16:59:03 +08:00
chrisr3d 7dd8e988c0
Updated the list of modules (removed stiximport) 2018-06-28 10:51:40 +02:00
chrisr3d b1c90b411e
add: Sigma syntax validator expansion module 
--> Checks sigma rules syntax
- Updated the expansion modules list as well
- Updated the requirements list
 2018-06-28 10:41:32 +02:00
chrisr3d 7c691af807
Updated the list of expansion modules 2018-06-28 10:39:40 +02:00
Steve Clement 59b7688bdc - Added initial PDF support, nothing is processed yet 
- Test to replace PIL with wand
 2018-06-28 16:00:14 +08:00
milkmix 349dd99d47 added support for scheduledtasks 2018-06-24 21:13:56 +02:00
milkmix 7c037ed090 added support for service-displayname, regkey|value 2018-06-24 21:09:42 +02:00
milkmix 0c6a205136 initial implementation supporting regkey. mutexes support waiting osquery table 2018-06-23 15:51:38 +02:00
Sebdraven 785aac3e6b add return handle domains 2018-06-22 16:18:23 +02:00
Sebdraven 87b07b89b5 add search 2018-06-22 16:15:34 +02:00
Sebdraven 396b71ef3b add domain to expand 2018-06-22 16:06:34 +02:00
Sebdraven de6a81d488 correct bugs 2018-06-22 16:04:14 +02:00
Sebdraven 83999d6402 add domain expansion 2018-06-22 15:57:52 +02:00
Sebdraven 96c829470d add comment 2018-06-22 15:14:44 +02:00
Sebdraven 8d03354399 correct bugs 2018-06-22 15:12:10 +02:00
Sebdraven e9c18b3d5f correct comments 2018-06-22 13:03:09 +02:00
Sebdraven e230c88c15 add threat list expansion 2018-06-22 11:59:09 +02:00
Sebdraven 1d1fd36569 change method to concat methods 2018-06-20 18:05:28 +02:00
Sebdraven e712a31760 set status after requests 2018-06-20 18:04:12 +02:00
Sebdraven a9b7a10c41 set status after requests 2018-06-20 18:03:34 +02:00
Sebdraven 4166475f9e add logs 2018-06-20 18:02:12 +02:00
Sebdraven fe00f099f6 add logs 2018-06-20 17:59:49 +02:00
Sebdraven 153d8bd340 add logs 2018-06-20 17:56:19 +02:00
Sebdraven 9195887f98 pep 8 2018-06-20 17:51:46 +02:00
Sebdraven 2afd2b8aaf correct bug 2018-06-20 17:50:28 +02:00
Sebdraven 04e932cce0 add datascan expansion 2018-06-20 17:47:11 +02:00
Sebdraven b56f8cfa36 add reverse infos 2018-06-20 16:30:56 +02:00
Sebdraven d4be9d9fda add reverse infos 2018-06-20 16:29:04 +02:00
Sebdraven 4a8a79c560 add reverse infos 2018-06-20 16:26:09 +02:00
Sebdraven 0d120af647 add reverse infos 2018-06-20 16:24:17 +02:00
Sebdraven a24b529868 add forward infos 2018-06-20 15:33:21 +02:00
Sebdraven d0f42c1772 add comment of attributes 2018-06-20 15:07:55 +02:00
Sebdraven 915747073a add comment of attributes 2018-06-20 15:05:00 +02:00
Sebdraven 7eba7c0386 error loops 2018-06-20 14:53:08 +02:00
Sebdraven d1e72676f1 error method 2018-06-20 14:50:48 +02:00
Sebdraven 3a4294391f error type 2018-06-20 14:48:18 +02:00
Sebdraven 9427c76603 error keys 2018-06-20 14:45:06 +02:00
Sebdraven e1bc67afad add expansion synscan 2018-06-20 14:41:57 +02:00
Sebdraven 5426ec5380 change key access domains 2018-06-20 12:40:52 +02:00
Sebdraven 7a3c4b1084 change add in results 2018-06-20 12:38:41 +02:00
Sebdraven e8aefde2ee add logs 2018-06-20 12:36:32 +02:00
Sebdraven 7195f33f5d correct error keys 2018-06-20 12:34:07 +02:00
Sebdraven c14d05adef test patries expansion 2018-06-20 12:32:54 +02:00
Sebdraven 8ae7210aef add onyphe full module 2018-06-20 11:07:33 +02:00
Sebdraven 023c35f5d8 add onyphe full module and code the stub 2018-06-14 16:47:11 +02:00
Sebdraven 14695bbeb9 correct codecov 2018-06-11 13:34:45 +02:00
Sebdraven 755d907580 pep 8 compliant 2018-06-11 13:21:21 +02:00
Sebdraven f6b8655f64 correct type of comments 2018-06-11 12:29:51 +02:00
Sebdraven 43402fde26 correct typo 2018-06-11 12:28:40 +02:00
Sebdraven e0631c9651 correct typo 2018-06-11 12:02:34 +02:00
Sebdraven 59b49f9d20 add domains forward 2018-06-11 12:00:46 +02:00
Sebdraven d9ee5286e3 add domains 2018-06-11 11:59:00 +02:00
Sebdraven 2e0e63fad6 add targeting os 2018-06-11 11:25:17 +02:00
Sebdraven 7580c63433 add category for AS number 2018-06-11 10:59:06 +02:00
Sebdraven f069cd9bf4 change keys 2018-06-11 10:56:40 +02:00
Sebdraven 0a543ca0d5 change type 2018-06-11 10:55:44 +02:00
Sebdraven ef035d051b add category 2018-06-11 10:54:06 +02:00
Sebdraven 735e626058 add as number with onyphe 2018-06-11 10:41:05 +02:00
Sebdraven 04032d110c add as number with onyphe 2018-06-08 18:31:08 +02:00
Sebdraven cad35b5332 error indentation 2018-06-08 18:11:04 +02:00
Sebdraven 3ec1535897 correct key in map result 2018-06-08 18:09:59 +02:00
Sebdraven f18f8fe05a correct a bug 2018-06-08 18:01:58 +02:00
Sebdraven 6eeca0fba1 add pastebin url imports 2018-06-08 17:53:50 +02:00
Sebdraven e6bac113ba add onyphe module 2018-06-08 16:38:41 +02:00
Andras Iklody 0b0f57b30c
Update countrycode.py 2018-06-06 08:31:41 +02:00
Alexandre Dulaunoy 2d9b0cd172
Merge branch 'master' of github.com:MISP/misp-modules 2018-05-29 21:59:25 +02:00
Alexandre Dulaunoy 9664127b85
add: new expansion module to check hashes against hashdd.com including NSLR dataset. 2018-05-29 21:54:22 +02:00
chrisr3d 2b509a2fd3
Updated delimiter finder function 2018-05-18 11:38:13 +02:00
chrisr3d 1fb72f3c7a
add: Added user config to specify if there is a header in the csv to import 2018-05-18 11:33:53 +02:00
chrisr3d dba8bd8c5b
fix: Avoid trying to build attributes with not intended fields 
- Previously: if the header field is not an attribute type, then
              it was added as an attribute field.
              PyMISP then used to skip it if needed

- Now: Those fields are discarded before they are put in an attribute
 2018-05-17 16:24:11 +02:00
chrisr3d c088b13f03
fix: Using userConfig to define the header instead of moduleconfig 2018-05-17 13:47:49 +02:00
Christophe Vandeplas 0593dbb408 ta import - more filter for pollution 2018-05-16 11:50:47 +02:00
Christophe Vandeplas 67cecc89d0 threatanalyzer_import - minor generic noise removal 2018-05-15 13:02:17 +02:00
Christophe Vandeplas 27a22e5d86 threatanalyzer_import - loads sample info + pollution fix 2018-05-03 09:42:38 +02:00
Christophe Vandeplas 370011c081 threatanalyzer_import - fix regkey issue 2018-05-02 12:43:34 +02:00
Nick Driver 252d190714
fix missing comma 
fix ip-dst and vulnerability input
 2018-03-30 14:27:37 -04:00
Koen Van Impe 6d23d4f4c7 Fix VMRay API access error 
hotfix for the "Unable to access VMRay API" error
 2018-03-30 15:11:25 +02:00
Fred Morris d0f618b648 Add exception blocks for query errors. 2018-03-08 15:26:39 -08:00
x41\x43 0436118747
Improving regex (validating e-mail) 
Line 48:
The previous regex ` ^[\w\.\+\-]+\@[\w]+\.[a-z]{2,3}$ ` matched only a small subset of valid e-mail address (e.g.: didn't match domain names longer than 3 chars or user@this-domain.de or user@multiple.level.dom) and needed to be with start (^) and end ($).
This ` [a-zA-Z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-zA-Z0-9!#$%&'*+\/=?^_`{|}~-]+)*@(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])?\.)+[a-zA-Z0-9](?:[a-zA-Z0-9-]*[a-zA-Z0-9])? ` is not perfect (e.g: can't match oriental chars), but imho is much more complete. 

Regex tested with several e-mail addresses with Python 3.6.4 and Python 2.7.14 on Linux 4.14.
 2018-03-06 18:12:36 +01:00
chrisr3d d885286792
Clarified functions arguments using a class 2018-03-05 19:59:30 +01:00
chrisr3d 4d7642ac91
add: Added Object References in the objects imported 2018-03-05 14:58:31 +01:00
chrisr3d 82fe8ba78c
fix: Fixed input & output of the module 2018-03-02 11:03:21 +01:00
chrisr3d 70436b7ddb Merge branch 'csvimport' of github.com:chrisr3d/misp-modules into goaml 2018-03-02 09:40:46 +01:00
chrisr3d c9ef578262
Removed print 2018-03-02 09:09:12 +01:00
chrisr3d 8d345d8cf5 Merge branch 'master' of github.com:MISP/misp-modules into csvimport 2018-03-02 09:05:46 +01:00
chrisr3d e6c55f5dde
fix: Fixed input & output of the module 
Also updated some functions
 2018-03-02 09:03:51 +01:00
chrisr3d 03d20856d9
add: added goamlimport 2018-02-28 22:46:39 +01:00
chrisr3d 323f71cdd3
Fixed some details about the module output 2018-02-28 17:41:45 +01:00
chrisr3d 8f5c08e2c6
Converting GoAML into MISPEvent 2018-02-28 15:07:55 +01:00
chrisr3d cad62464c5
Now parsing all the transaction attributes 2018-02-27 11:08:37 +01:00
chrisr3d a02dbd6a8d
fix: Fixed typo of the aml type for country codes 2018-02-26 18:52:28 +01:00
chrisr3d 478cd53912
add: Added dictionary to map aml types into MISP types 2018-02-26 18:13:43 +01:00
chrisr3d 5df2d309a0
typo 2018-02-26 15:58:53 +01:00
chrisr3d 81a6be17d3
chg: Structurded data 2018-02-26 11:47:35 +01:00
chrisr3d 359ac9100e
fix: typo in references mapping dictionary 2018-02-23 15:58:04 +01:00
Christian Studer 983b7da7b7
fix: Added an object checking 
- Checking if there are objects in the event, and then if there is at least 1 transaction object
- This prevents the module from crashing, but does not guaranty having a valid GoAML file (depending on objects and their relations)
 2018-02-22 16:55:52 +01:00
chrisr3d b2b0fccd47
fix: Added an object checking 
- Checking if there are objects in the event, and then
  if there is at least 1 transaction object
- This prevents the module from crashing, but does not
  guaranty having a valid GoAML file (depending on
  objects and their relations)
 2018-02-22 16:37:27 +01:00
chrisr3d 53b4a43448 Merge branch 'master' of github.com:chrisr3d/misp-modules into aml_import 2018-02-22 14:29:36 +01:00
chrisr3d c942013812
chg: Modified the mapping dictionary to support misp-objects updates 2018-02-22 01:23:08 +01:00
chrisr3d 5995458aab
fix: Added the moduleinfo field need to have MISP event in standard format 2018-02-21 17:14:26 +01:00
Alexandre Dulaunoy c3ac53a069
fix: goamlexport added 2018-02-20 17:18:36 +01:00
chrisr3d f361fb4ee3
Reading the entire document, to create a big dictionary containing the data, as a beginning 2018-02-20 17:00:13 +01:00
chrisr3d 02b8938b2a
typo 2018-02-20 16:57:27 +01:00
chrisr3d 11dddb974b Merge branch 'master' of github.com:MISP/misp-modules 2018-02-20 15:18:45 +01:00
chrisr3d eb9e06f1cc
explicit name 
Avoiding confusion with the coming import module for goaml
 2018-02-20 15:18:12 +01:00
Andras Iklody 978903f911
Quick fix to the invalid hash types offered on all returned hashes, hopefully fixes #162 2018-02-20 14:08:14 +01:00
chrisr3d 92ab1d5c23
Added "t_to" and "t_from" required fields: funds code & country 2018-02-14 21:30:48 +01:00
chrisr3d be1b541966
Added a required field & the latest attributes in transaction 2018-02-14 12:18:12 +01:00
chrisr3d 43e9010858
Added report expected information fields 2018-02-13 16:39:19 +01:00
chrisr3d d4538382d0
Simplified ObjectReference dictionary reading 2018-02-13 13:41:22 +01:00
chrisr3d b7098d1cff Merge branch 'master' of github.com:MISP/misp-modules 2018-02-13 11:58:56 +01:00
chrisr3d a97eeb44fe
Added some report information 
Also changed the ObjectReference parser to replace
all the if conditions by a dictionary reading
 2018-02-13 11:51:34 +01:00
Dennis Rand 43db92dbe6 Added Yara syntax validation expansion module 2018-02-12 19:11:54 +00:00
chrisr3d 8569c3d702
Suporting the recent objects added to misp-objects 
- Matching the aml documents structure
- Some parts of the document still need to be added
 2018-02-12 13:40:49 +01:00
chrisr3d 8983ebc4b2
wip: added location & signatory information 2018-02-05 15:51:03 +01:00
chrisr3d 54ebb8a96f Merge branch 'master' of github.com:MISP/misp-modules into test 2018-02-04 17:16:25 +01:00
Thomas Gardner 69d733bb35 added csvimport to __init__.py 2018-02-01 10:22:28 -07:00
chrisr3d 8dce7935ae
Outputting xml format 
Also mapping MISP and GoAML types
 2018-02-01 14:55:48 +01:00
chrisr3d 48869335ee
first tests for the GoAML export module 2018-01-31 18:09:45 +01:00
chrisr3d 71c00954d0
fix: Solved reading problems for some files 2018-01-30 11:20:28 +01:00
chrisr3d b2ec186ccb
Updated delimiter finder method 2018-01-29 17:04:32 +01:00
chrisr3d 529d22cca8
fix: skipping empty lines 2018-01-29 09:19:58 +01:00
chrisr3d 56cbd72b65
Fixed data treatment & other updates 2018-01-28 18:12:40 +01:00
chrisr3d 4d846f968f
Updated delimiter parsing & data reading functions 2018-01-26 17:11:01 +01:00
chrisr3d b9d72bb043
First version of csv import module 
- If more than 1 misp type is recognized, for each one an
  attribute is created

- Needs to have header set by user as parameters of the module atm

- Review needed to see the feasibility with fields that can create
  confusion and be interpreted both as misp type or attribute field
  (for instance comment is a misp type and an attribute field)
 2018-01-25 15:44:08 +01:00
Christophe Vandeplas 8a1a860cda added CrowdStrike Falcon Intel Indicators expansion module 2018-01-19 14:42:25 +01:00
chrisr3d d045cf7d5f
chg: Modified output format 2018-01-16 19:46:52 +01:00
chrisr3d dcab9aa150 Merge github.com:MISP/misp-modules 2018-01-16 17:15:36 +01:00
Alexandre Dulaunoy c3823b74cf
Merge pull request #149 from cvandeplas/master 
Added ThreatAnalyzer sandbox import
 2018-01-16 17:11:38 +01:00
chrisr3d 18523c4ada
Check an IPv4 address against known RBLs 2018-01-16 17:08:44 +01:00
Christophe Vandeplas 0be1886444
fix farsight_passivedns - rdata 404 not found 2018-01-16 15:13:17 +01:00
Christophe Vandeplas 46975f4f16 Added ThreatAnalyzer sandbox import 
Experimental module - some parts should be migrated to
 2018-01-16 11:05:26 +01:00
Alexandre Dulaunoy 5c4df3075e
Fix the __init__ import 2018-01-08 20:31:26 +01:00
Robert Nixon 85f1a9bd91
Update threatStream_misp_export.py 2018-01-08 12:09:23 -05:00
Robert Nixon 1d2f3d9c3c
Updated __init__.py 
Added reference to new ThreatStream export module
 2018-01-08 11:03:42 -05:00
Robert Nixon 49d5520fa3
Added threatStream_misp_export.py 2018-01-08 11:01:16 -05:00
Christophe Vandeplas 4cdb143733 fixes missing init file in dnsdb library folder 2017-12-06 09:23:44 +01:00
Christophe Vandeplas 0ec8339d7a New Farsight DNSDB Passive DNS expansion module 2017-12-05 16:41:41 +01:00
Raphaël Vinot 02253e5a87 Merge branch 'master' of github.com:MISP/misp-modules 2017-11-20 14:57:18 +01:00
Jericho 32958324ca
minor touch-ups on error messages for user friendliness 2017-11-16 23:04:41 -07:00
Koen Van Impe 74e660d61b VulnDB Queries 
Search on CVE at https://vulndb.cyberriskanalytics.com/
    https://www.riskbasedsecurity.com/
Get extended CVE info, links + CPE
 2017-11-06 14:23:03 +01:00
Raphaël Vinot 37d9b3831c Add quick and dirty pdf export 2017-10-26 16:54:20 -04:00
Raphaël Vinot c09135d251 Merge pull request #139 from Rafiot/master 
fix: OpenIOC importer
 2017-10-25 11:41:46 -04:00
Raphaël Vinot 951a0f974b fix: OpenIOC importer 2017-10-25 11:27:59 -04:00
Alexandre Dulaunoy 03baa0b84d
fix: #137 when a CVE is not found, a return message is given 2017-10-21 19:52:19 +02:00
Viktor von Drakk 113ac21a5d added default parameter for new -m flag 2017-09-01 07:44:53 -07:00
Viktor von Drakk 76a733fa66 Added code to allow 3rd party modules 
The new '-m pip.module.name' feature allows a pip-installed module to be specified on the command line and then loaded into the available modules without having to copy-paste files into the appropriate directories of this package.
 2017-08-25 05:45:57 -07:00
Thomas Gardner 72c52da7ed added threat_connect_export to export_mod.__init__ 2017-08-06 08:15:17 -06:00
Thomas Gardner 529719d9d8 added threat_connect_export.py 2017-08-03 16:21:26 -06:00
Raphaël Vinot 4c2cda9903 Merge pull request #129 from seamustuohy/utf_hate 
Added support for malformed internationalized email headers
 2017-07-18 10:06:08 +02:00
Chris Doman c4fe78b39d Add AlienVault OTX and ThreatCrowd Expansions 2017-07-11 18:16:45 +01:00
seamus tuohy 40c71af637 Added support for malformed internationalized email headers 
When an emails contains headers that use Unicode without properly crafing
them to comform to RFC-6323 the email import module would crash.
(See issue #119 & issue #93)

To address this I have added additional layers of encoding/decoding to
any possibly internationalized email headers. This decodes properly
formed and malformed UTF-8, UTF-16, and UTF-32 headers appropriately.
When an unknown encoding is encountered it is returned as an 'encoded-word'
per RFC2047.

This commit also adds unit-tests that tests properly formed and malformed
UTF-8, UTF-16, UTF-32, and CJK encoded strings in all header fields; UTF-8,
UTF-16, and UTF-32 encoded message bodies; and emoji testing for headers
and attachment file names.
 2017-07-02 18:03:14 -04:00
Raphaël Vinot c42c8a800e Update travis, fix open ioc import 2017-05-24 07:39:18 +02:00
Tristan METAYER 75c02058e6 replace tab by space 2017-05-11 09:56:43 +02:00
Tristan METAYER ba1d715ad1 Add a field for user to add tag for this import 2017-05-11 09:54:25 +02:00
Tristan METAYER 96f9cb4699 typo correction 2017-05-02 15:07:33 +02:00
Tristan METAYER 4ef7261168 Add user config to not add file as attachement in a box 2017-05-02 15:04:40 +02:00
Tristan METAYER 79f48eccfe If filename add iocfilename as attachment 2017-05-02 14:41:22 +02:00
Alexandre Dulaunoy 3cb12d6962 Merge pull request #118 from truckydev/master 
Add indent field for export
 2017-04-23 12:21:16 +02:00
Tristan METAYER 24c51a6e21 Add indent field for export 2017-04-21 15:53:48 +02:00
Hannah Ward 648c6414c3
fix: Use the proper formatting method and not the horrible % one 2017-03-08 16:35:03 +00:00
kx499 aa3a11cd5f bug fixes 2017-03-08 04:08:23 +01:00
kx499 31a8fb0fe4 threatminer initial commit 2017-03-06 21:36:00 -05:00
Raphaël Vinot 44867b2adc Cosmetic changes 2017-03-05 18:59:36 +01:00
Raphaël Vinot ad49fd3819 Merge pull request #111 from kx499/master 
Handful of changes to VirusTotal module
 2017-03-05 18:31:50 +01:00
kx499 3ecd095d1e bug fixes, tweaks, and python3 learning curve :) 2017-03-04 03:10:45 +01:00
kx499 01fdf3e52b Initial commit of IPRep module 2017-03-03 15:55:52 -05:00
kx499 bc1eab3520 fixed spacing, addressed error handling for public api, added subdomains, and added context comment 2017-02-28 22:04:24 -05:00
Raphaël Vinot c508e60f65 Add OpenIOC import module 2017-02-27 13:32:31 +01:00
Tristan METAYER 20cb534203 Exclude internal reference 2017-02-21 17:12:17 +01:00
Tristan METAYER dd2646a0f4 Add lite Export module 2017-02-21 16:48:09 +01:00
rmarsollier b5b7e09ef4 Some improvements of virustotal plugin 2017-02-10 14:16:39 +01:00
Joerg Stephan de3495ea6c passed local run check 2017-02-01 14:05:29 +01:00
Joerg Stephan 68250094ff v1 2017-01-31 16:57:16 +01:00
Joerg Stephan dad73feaa4 python3 changes 2017-01-31 16:34:41 +01:00
Joerg Stephan 3590504821 XForce Exchange v1 (alpha) 2017-01-21 23:31:19 +01:00
Richard van den Berg 3a4c540a81 Updated description to reflect merging use case 2017-01-11 10:08:35 +01:00
Richard van den Berg 50bae1f549 Simple import module to import MISP JSON format 2017-01-11 10:08:35 +01:00
seamus tuohy 83a9d695ea Email import no longer unzips major compressed text document formats. 
Let this commit serve as a warning about the perils of duck typing.
Word documents (docx,odt,etc) were being uncompressed when they were
attached to emails. The email importer now checks a list of well known
extensions and will not attempt to unzip them.

It is stuck using a list of extensions instead of using file magic because
many of these formats produce an application/zip mimetype when scanned.
 2017-01-10 09:55:33 -05:00
Raphaël Vinot 1051e2210b Keep zip content as binary 2017-01-07 19:30:00 -05:00
Raphaël Vinot 9f84db3659 Fix tests, cleanup 2017-01-07 18:36:08 -05:00
Raphaël Vinot 2db845c45c Improve support of email attachments 
Related to #90
 2017-01-07 14:39:52 -05:00
Hannah Ward 727f302dd1 Standardised key checking 2017-01-07 10:38:28 -05:00
Hannah Ward 20fd05a231 Fixed checking for submission_names in VT JSON 2017-01-07 10:37:57 -05:00
CheYenBzh d7b33532eb Update virustotal.py 2017-01-07 10:37:47 -05:00
Raphaël Vinot b51806ac9f Improve support of email importer if headers are missing 
Fix #88
 2017-01-07 10:25:38 -05:00
Raphaël Vinot 02f5e95a98 Fix python 3.6 support 2017-01-06 20:36:09 -05:00
Raphaël Vinot 329586768b Make PEP8 happy 2017-01-06 20:10:44 -05:00
Raphaël Vinot 7a9774bff7 Add email_import in the modules loaded by default 2017-01-06 19:23:23 -05:00
Raphaël Vinot 93a49c3c1d Make PEP8 happy 2017-01-06 19:01:19 -05:00
Raphaël Vinot 3f83357a2d Fix failing test (bug in the mail parser?) 2017-01-06 18:56:29 -05:00
seamus tuohy 1a7973bc06 Add additional email parsing and tests 
Added additional attribute parsing and corresponding unit-tests.
E-mail attachment and url extraction added in this commit. This includes
unpacking zipfiles and simple password cracking of encrypted zipfiles.
 2017-01-04 10:21:36 -08:00
seamus tuohy 0ff270a3be Fixed basic errors 2016-12-26 14:33:10 -08:00
seamus tuohy 08261366b7 Merged with current master 2016-12-26 14:17:20 -08:00
seamus tuohy 86ae72c444 Added attachment and url support 2016-12-26 13:55:54 -08:00
Raphaël Vinot 9bf1c936cf Do not crash if the dat file is not available 2016-12-16 15:22:16 +01:00
Raphaël Vinot 064c3e3649 Fix path to config file 2016-12-16 15:14:48 +01:00
Raphaël Vinot 29bedc7faa Merge branch 'master' of https://github.com/amuehlem/misp-modules into amuehlem-master 2016-12-16 15:05:45 +01:00
Raphaël Vinot 60d3e0a1ac Better error reporting 2016-12-16 12:02:28 +01:00
Raphaël Vinot ffc0a97126 Catch exception 2016-12-16 11:52:51 +01:00
Raphaël Vinot 467e50327d Add reverse lookup 2016-12-16 11:22:22 +01:00
Raphaël Vinot 4a8ccb54fb Refactoring of domaintools expansion module 2016-12-15 16:49:56 +01:00
Ubuntu b76f59edcb Added cuckooimport.py 2016-12-07 16:36:31 +00:00
Andreas Muehlemann cc58b05d6e added empty line to end of config file 2016-12-07 17:28:16 +01:00
Andreas Muehlemann 98a27ac3ff removed DEFAULT section from configfile 2016-12-07 16:36:02 +01:00
Andreas Muehlemann 6853d67a43 fixed more typos 2016-12-07 16:13:46 +01:00
Andreas Muehlemann 6dcc77ba5d fixed typo 2016-12-07 15:48:08 +01:00
Andreas Muehlemann a95af26424 changed configparser from python2 to python3 2016-12-07 15:30:49 +01:00
Andreas Muehlemann 1e1796b414 updated missing parenthesis 2016-12-07 15:19:54 +01:00
Andreas Muehlemann bb62394c1e Merge branch 'geoip_country' 2016-12-07 14:54:33 +01:00
Andreas Muehlemann d09c2f3d44 removed unneeded config option for misp 2016-12-07 14:29:11 +01:00
Andreas Muehlemann 6ea7acc5e4 removed debug message 2016-12-07 14:28:27 +01:00
Andreas Muehlemann f8c7271467 added config option to geoip_country.py 2016-12-07 14:18:21 +01:00
Raphaël Vinot ac33940628 Merge pull request #75 from Rafiot/domtools 
Add Domain Tools module
 2016-12-01 17:52:04 +01:00
Raphaël Vinot 2e3119b5f4 Add domaintools to the import list 2016-12-01 17:36:40 +01:00
Raphaël Vinot 0f8fa4aaec Fix Typo 2016-12-01 16:44:29 +01:00
Raphaël Vinot 17205a1913 Add domain profile and reputation 2016-12-01 16:41:50 +01:00
Raphaël Vinot 7db1216efb Add more comments 2016-12-01 13:45:14 +01:00
Raphaël Vinot 9dbd241e63 fix typo 2016-12-01 12:14:16 +01:00
Raphaël Vinot 6db5436c62 remove json.dumps 2016-12-01 11:54:04 +01:00
Raphaël Vinot afd8b71349 Avoid passing None in comments 2016-12-01 10:26:40 +01:00
Raphaël Vinot 7c6153478e Add comments to fields when possible 2016-11-30 18:09:11 +01:00