Commit Graph

738 Commits (f5452055f607fcdba8e407a78fa106ff172c1155)

Author SHA1 Message Date
chrisr3d 7b1c35d583
fix: Fixed cvss-score object relation name 2019-07-30 09:55:36 +02:00
chrisr3d 3367e47490
fix: Avoid issues when there is no pe field in a windows file sample analysis
- For instance: doc file
2019-07-25 17:57:36 +02:00
chrisr3d 3d41104d5b
fix: Avoid adding file object twice if a KeyError exception comes for some unexpected reasons 2019-07-25 17:47:08 +02:00
chrisr3d ddeb04bd74
add: Parsing linux samples and their elf data 2019-07-25 17:46:21 +02:00
chrisr3d 41bbbeddfb
fix: Testing if file & registry activities fields exist before trying to parse it 2019-07-25 17:44:32 +02:00
chrisr3d 4c8fe9d8ef
fix: Testing if there is some screenshot data before trying to fetch it 2019-07-25 17:43:11 +02:00
chrisr3d e2a0f27d75
fix: Fixed direction of the relationship between files, PEs and their sections
- The file object includes a PE, and the PE
  includes sections, not the other way round
2019-07-24 14:58:45 +02:00
chrisr3d 42b95c4210
fix: Fixed variable names 2019-07-24 12:21:58 +02:00
chrisr3d 27f5c9ceeb Merge branch 'master' of github.com:MISP/misp-modules 2019-07-24 12:08:28 +02:00
chrisr3d 5602cf1759
add: Parsing apk samples and their permissions 2019-07-24 11:59:11 +02:00
chrisr3d fc8a573ba7
fix: Changed the way references added at the end are saved
- Some references are saved until they are added
  at the end, to make it easier when needed
- Here we changed the way they are saved, from a
  dictionary with some keys to identify each part
  to the actual dictionary with the keys the
  function add_reference needs, so we can directly
  use this dictionary as is when the references are
  added to the different objects
2019-07-24 11:14:12 +02:00
chrisr3d 4ee0cbe4c5
add: Added virustotal_public to the list of available modules 2019-07-24 11:10:25 +02:00
Raphaël Vinot 80ce0a58b5 fix: Skip tests on haveibeenpwned.com if 403. Make pep8 happy. 2019-07-24 09:49:05 +02:00
chrisr3d 92d90e8e1c
add: TODO comment for the next improvement 2019-07-23 09:42:10 +02:00
chrisr3d 14cf39d8b6
chg: Updated the module to work with the updated VirusTotal API
- Parsing functions updated to support the updated
  format of the VirusTotal API responses
- The module can now return objects
- /!\ This module requires a high number of
  requests limit rate to work as expected /!\
2019-07-22 16:22:29 +02:00
chrisr3d 1fa37ea712
fix: Avoiding issues with non existing sample types 2019-07-22 11:43:35 +02:00
chrisr3d 675e0815ff
add: Parsing communicating samples returned by domain reports 2019-07-22 11:42:52 +02:00
chrisr3d c9c2027a57
fix: Undetected urls are represented in lists 2019-07-22 11:39:46 +02:00
chrisr3d 6fdfcb0a29
fix: Changed function name to avoid confusion with the same variable name 2019-07-22 09:53:19 +02:00
chrisr3d 729c86c336
fix: Quick fix on siblings & url parsing 2019-07-22 09:16:04 +02:00
chrisr3d 9aa721bc37
fix: typo 2019-07-19 16:20:24 +02:00
chrisr3d 641dda0103
add: Parsing downloaded samples as well as the referrer ones 2019-07-18 21:38:17 +02:00
chrisr3d 795edb7457
chg: Adding references between a domain and their siblings 2019-07-17 20:40:56 +02:00
chrisr3d 8de350744b
chg: Getting domain siblings attributes uuid for further references 2019-07-16 22:39:35 +02:00
chrisr3d a61d09db8b
fix: Parsing detected & undetected urls 2019-07-15 23:44:25 +02:00
chrisr3d d9b03a7aa5
fix: Various fixes about typo, variable names, data types and so on 2019-07-12 10:59:19 +02:00
chrisr3d f862a14ce6
add: Object for VirusTotal public API queries
- Lighter analysis of the report to avoid reaching
  the limit of queries per minute while recursing
  on the different elements
2019-07-11 22:59:07 +02:00
chrisr3d 3edc323836
fix: Making pep8 happy 2019-07-10 15:29:31 +02:00
chrisr3d 5703253961
new: First version of an advanced CVE parser module
- Using cve.circl.lu as well as the initial module
- Going deeper into the CVE parsing
- More parsing to come with the CWE, CAPEC and so on
2019-07-10 15:20:22 +02:00
chrisr3d 181e6383a3
fix: Added missing add_attribute function 2019-07-03 11:14:46 +02:00
chrisr3d 9a6d484188
add: Added screenshot of the behavior of the analyzed sample 2019-06-21 10:53:12 +02:00
chrisr3d 9e45d302b1
fix: Testing if an object is not empty before adding it the the event 2019-06-18 09:45:59 +02:00
chrisr3d 9fdd6c5e58
fix: Making travis happy 2019-06-15 08:17:29 +02:00
chrisr3d 2f3ce1b615
fix: Support of the latest version of sigmatools 2019-06-15 08:06:47 +02:00
chrisr3d 1ac85a4879
fix: We will display galaxies with tags 2019-06-15 08:05:14 +02:00
chrisr3d b7223abe78 Merge branch 'new_module' of github.com:MISP/misp-modules into new_module 2019-06-07 15:30:19 +02:00
chrisr3d de966eac51
fix: Returning tags & galaxies with results
- Tags may exist with the current version of the
  parser
- Galaxies are not yet expected from the parser,
  nevertheless the principle is we want to return
  them as well if ever we have some galaxies from
  parsing a JoeSandbox report. Can be removed if
  we never galaxies at all
2019-06-07 15:22:11 +02:00
chrisr3d b52e17fa8d
fix: Removed duplicate finalize_results function call 2019-06-07 11:38:50 +02:00
Georg Schölly efb0a88eeb joesandbox_query.py: improve behavior in unexpected circumstances 2019-06-04 11:29:40 +02:00
chrisr3d aa3e873845
fix: Making pep8 happy + added joe_import module in the init list 2019-06-04 11:33:42 +10:00
chrisr3d 42bc6f8d2b
fix: Fixed variable name typo 2019-06-04 11:32:21 +10:00
chrisr3d ee48d99845
add: New expansion module to query Joe Sandbox API with a report link 2019-06-04 09:48:50 +10:00
chrisr3d 07698e5c72
fix: Fixed references between domaininfo/ipinfo & their targets
- Fixed references when no target id is set
- Fixed domaininfo parsing when no ip is defined
2019-06-03 18:38:58 +10:00
chrisr3d 0d40830a7f
fix: Some quick fixes
- Fixed strptime matching because months are
  expressed in abbreviated format
- Made data loaded while the parsing function is
  called, in case it has to be called multiple
  times at some point
2019-06-03 18:35:58 +10:00
chrisr3d 74b73f9332
chg: Moved JoeParser class to make it reachable from expansion & import modules 2019-05-29 11:26:14 +10:00
chrisr3d f541b1f4ba Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-05-29 10:50:39 +10:00
Georg Schölly 9377a892f4 support url analyses 2019-05-28 16:19:35 +02:00
Georg Schölly 380b8d46ba improve forwards-compatibility 2019-05-28 16:14:59 +02:00
chrisr3d 8ac651562e
fix: Making pep8 & travis happy 2019-05-23 16:13:49 +02:00
chrisr3d be05de62c0
add: Parsing MITRE ATT&CK tactic matrix related to the Joe report 2019-05-23 15:59:52 +02:00
chrisr3d e608107a09
add: Parsing domains, urls & ips contacted by processes 2019-05-22 17:12:49 +02:00
chrisr3d cfec9a6b1c
fix: Added references between processes and the files they drop 2019-05-22 15:27:04 +02:00
chrisr3d 191034d311
add: Starting parsing dropped files 2019-05-21 23:37:53 +02:00
Georg Schölly 1745d33ee4 add expansion for joe sandbox 2019-05-21 21:14:21 +02:00
chrisr3d 417c306ace
fix: Avoiding network connection object duplicates 2019-05-20 15:59:18 +02:00
chrisr3d 72e5f0099d
fix: Avoid creating a signer info object when the pe is not signed 2019-05-20 10:52:34 +02:00
chrisr3d 54f5fa6fa9
fix: Avoiding dictionary indexes issues
- Using tuples as a dictionary indexes is better
  than using generators...
2019-05-20 09:19:38 +02:00
chrisr3d 0d5f867825
add: Starting parsing network behavior fields 2019-05-17 22:18:11 +02:00
chrisr3d f9515c14d0
fix: Avoiding attribute & reference duplicates 2019-05-16 16:14:25 +02:00
chrisr3d 2246fc0d02
add: Parsing registry activities under processes 2019-05-16 16:11:43 +02:00
chrisr3d 067b229224
fix: Handling case of multiple processes in behavior field
- Also starting parsing file activities
2019-05-15 22:06:55 +02:00
chrisr3d d195b554a5
fix: Testing if some fields exist before trying to import them
- Testing for pe itself, pe versions and pe signature
2019-05-15 22:05:03 +02:00
chrisr3d fc8a56d1d9
fix: Removed test print 2019-05-15 15:49:29 +02:00
chrisr3d df7047dff0
fix: Fixed output format to match with the recent changes on modules 2019-05-14 10:50:11 +02:00
chrisr3d 29e681ef81
add: Parsing processes called by the file analyzed in the joe sandbox report 2019-05-13 17:30:01 +02:00
chrisr3d d39fb7da18
add: Parsing some object references at the end of the process 2019-05-13 17:29:07 +02:00
chrisr3d 728386d8a0
add: [new_module] Module to import data from Joe sandbox reports
- Parsing file, pe and pe-section objects from the
  report file info field
- Deeper file info parsing to come
- Other fields parsing to come as well
2019-05-08 16:52:49 +02:00
chrisr3d 77db21cf18
fix: Making pep8 happy 2019-05-07 09:37:21 +02:00
chrisr3d f1b5f05bb3
fix: Checking not MISP header fields
- Rejecting fields not recognizable by MISP
2019-05-07 09:35:56 +02:00
chrisr3d 6608671a01 Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-05-07 08:38:16 +02:00
chrisr3d 28eb92da53
fix: Using pymisp classes & methods to parse the module results 2019-05-06 22:16:14 +02:00
chrisr3d ae5bd8d06a
fix: Clearer user config messages displayed in the import view 2019-05-06 22:15:14 +02:00
Koen Van Impe 1cd60790fd Bugfix for "sources" ; do not include as IDS for "access" registry keys
- Bugfix to query "operations" in files, mutex, registry
- Do not set IDS flag for registry 'access' operations
2019-05-06 16:36:26 +02:00
chrisr3d d4bc85259d
fix: Removed unused library 2019-05-02 14:15:12 +02:00
chrisr3d 6f4b88606b
fix: Make pep8 happy 2019-05-02 14:07:36 +02:00
chrisr3d a5ff849950 Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-05-02 13:23:24 +02:00
Steve Clement 559ed786ba
chg: [pep8] try/except # noqa
Not sure how to make flake happy on this one.
2019-05-02 11:44:32 +09:00
Steve Clement 9af06fd24c
fix: [pep8] More fixes 2019-05-02 11:23:49 +09:00
Steve Clement 81ffabd621
fix: [pep8] More pep8 happiness 2019-05-02 11:06:32 +09:00
Steve Clement 553cf44337
fix: [pep8] Fixes 2019-05-02 10:37:48 +09:00
Koen Van Impe c8a4d8d76f New VMRay modules
New JSON output format of VMRay
Prepare for automation (via PyMISP) with workflow taxonomy tags
2019-05-01 22:44:24 +02:00
root c886247a64
fix: Fixed standard MISP csv format header
- The csv header we can find in data produced from
  MISP restSearch csv format is the one to use to
  recognize a csv file produced by MISP
2019-05-01 22:32:06 +02:00
root f900cb7c68
fix: Fixed introspection fields for csvimport & goamlimport
- Added format field for goaml so the module is
  known as returning MISP attributes & objects
- Fixed introspection to make the format, user
  config and input source fields visible from
  MISP (format also added at the same time)
2019-05-01 22:28:19 +02:00
root db74c5f49a
fix: Fixed libraries import that changed with the latest merge 2019-05-01 22:26:53 +02:00
root 92351e6679
add: Added urlhaus in the expansion modules init list 2019-05-01 22:22:10 +02:00
chrisr3d ed7a14b057 Merge branch 'features_csvimport' of github.com:MISP/misp-modules into new_module 2019-04-30 17:19:34 +02:00
chrisr3d ee560155a4 Merge branch 'master' of github.com:MISP/misp-modules into features_csvimport 2019-04-30 17:16:48 +02:00
chrisr3d 55e494c9ed Merge branch 'features_csvimport' of github.com:MISP/misp-modules into features_csvimport 2019-04-30 17:16:31 +02:00
chrisr3d 922782f24b Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-04-30 08:57:19 +02:00
Raphaël Vinot 48c158271b new: Devel mode.
Fix #293
2019-04-26 13:48:41 +02:00
Alexandre Dulaunoy ec766f571c
chg: [init] cleanup for pep 2019-04-26 13:36:53 +02:00
Alexandre Dulaunoy 63c12f34e6
chg: [pdf-enrich] updated 2019-04-26 13:36:07 +02:00
Sascha Rommelfangen fc339c888d removed trailing whitespaces 2019-04-26 12:14:56 +02:00
Sascha Rommelfangen 722ec88b45 Merge branch 'master' of https://github.com/MISP/misp-modules 2019-04-26 12:09:39 +02:00
Sascha Rommelfangen 1d4f8a6989 new modules added 2019-04-26 12:09:16 +02:00
Sascha Rommelfangen f55d7946df introduction of new modules 2019-04-26 12:07:55 +02:00
Raphaël Vinot c3c5b75157 Merge branch 'master' of github.com:MISP/misp-modules 2019-04-26 11:35:44 +02:00
Raphaël Vinot 2c64e5ca67 fix: CTRL+C is working again
Fix #292
2019-04-26 11:35:06 +02:00
Sascha Rommelfangen 06036b7fe5 Merge branch 'master' of https://github.com/MISP/misp-modules 2019-04-24 15:01:03 +02:00
Sascha Rommelfangen 07f759b07a renamed file 2019-04-24 14:53:16 +02:00
Sascha Rommelfangen 5104bce451 renamed module 2019-04-24 14:53:03 +02:00
Alexandre Dulaunoy 81b0082ae5
chg: [init] removed trailing whitespace 2019-04-24 14:01:48 +02:00
Alexandre Dulaunoy 614fc1354b
chg: [ocr] re module not used - removed 2019-04-24 14:01:08 +02:00
Sascha Rommelfangen 7171c8ce92 initial version of OCR expansion module 2019-04-24 13:54:21 +02:00
Alexandre Dulaunoy 18a2370ae3
Merge pull request #291 from Evert0x/submitcuckoo
Expansion module - File/URL submission to Cuckoo Sandbox
2019-04-23 19:36:28 +02:00
Sascha Rommelfangen 2d8aaf09c2
brackets are difficult... 2019-04-23 15:40:22 +02:00
Alexandre Dulaunoy e55ae11a1e
chg: [qrcode] added to the __init__ 2019-04-23 14:45:12 +02:00
Alexandre Dulaunoy 44050ec4da
chg: [qrcode] flake8 needs some drugs 2019-04-23 14:44:00 +02:00
Alexandre Dulaunoy d5180e7e79
chg: [qrcode] various fixes to make it PEP compliant 2019-04-23 14:37:27 +02:00
Alexandre Dulaunoy a0fce1bc90
Merge branch 'qr-code-module' of https://github.com/rommelfs/misp-modules into rommelfs-qr-code-module 2019-04-23 14:33:06 +02:00
Sascha Rommelfangen c85ab8d93c
initial version of QR code reader
Module accepts attachments and processes pictures. It tries to identify and analyze an existing QR code.
Identified values can be inserted into the event.
2019-04-23 11:38:56 +02:00
Ricardo van Zutphen e6326185d5 Use double quotes and provide headers correctly 2019-04-19 16:24:30 +02:00
Ricardo van Zutphen 49acb53745 Update Cuckoo module to support files and URLs 2019-04-19 14:06:35 +02:00
Evert0x e243edb503
Update __init__.py 2019-04-18 14:25:05 +02:00
Evert0x eefa35c65d
Create cuckoo_submit.py 2019-04-18 00:23:38 +02:00
Raphaël Vinot f5167c2f23 fix: Make flake8 happy. 2019-04-16 11:25:39 +02:00
iceone23 d24a6e2e24
Create cisco_firesight_manager_ACL_rule_export.py
Cisco Firesight Manager ACL Rule Export module
2019-04-15 06:17:27 -07:00
chrisr3d 4955698c63 Merge branch 'new_module' of github.com:MISP/misp-modules into new_module 2019-04-03 22:02:44 +02:00
chrisr3d f492465c00 Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-04-03 22:00:40 +02:00
Raphaël Vinot f82933779f fix: pep8 foobar. 2019-04-02 16:01:36 +02:00
Raphaël Vinot 9cb21f98e1 fix: Add the new module sin the list of modules availables. 2019-04-02 15:46:17 +02:00
Raphaël Vinot c64f514a6f fix: Typos in variable names 2019-04-02 15:39:27 +02:00
Raphaël Vinot b89d068c04 new: Modules for greynoise, haveibeenpwned and macvendors
Source: https://github.com/src7/misp-modules
2019-04-02 15:30:11 +02:00
root 38fc479d12 Merge branch 'master' of https://github.com/MISP/misp-modules into new_module 2019-04-01 16:29:10 +02:00
root 2439d5f75d
fix: Fixed object_id variable name typo 2019-04-01 16:28:19 +02:00
chrisr3d 756a794087 Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-03-25 15:35:10 +01:00
Raphaël Vinot 1c0984eaec fix: Remove unused import 2019-03-15 11:06:11 +01:00
chrisr3d d87a67c6f3 Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-03-14 19:04:32 +01:00
chrisr3d 0b92fd5a53
fix: Making json_decode even happier with full json format
- Using MISPEvent because it is cleaner & easier
- Also cleaner implementation globally
2019-03-14 18:48:13 +01:00
Sascha Rommelfangen 5af667edff Merge branch 'master' of https://github.com/MISP/misp-modules 2019-03-14 14:41:24 +01:00
Sascha Rommelfangen eb2dcca12b fixed a bug when checking malformed BTC addresses 2019-03-14 14:39:58 +01:00
chrisr3d 62bc45e03a
fix: Using to_dict on attributes & objects instead of to_json to make json_decode happy in the core part 2019-03-14 14:31:38 +01:00
chrisr3d 9c8ee1f3d7
new: Expansion module to query urlhaus API
- Using the next version of modules, taking a
  MISP attribute as input and able to return
  attributes and objects
- Work still in process in the core part
2019-03-13 09:57:28 +01:00
Falconieri 32e10ee273 fix: [exportpdf] custom path parameter 2019-03-05 10:39:07 +01:00
Falconieri a30bcc5dd2 fix: [exportpdf] add parameters 2019-03-04 12:36:18 +01:00
Falconieri 7d7c90143e fix: [exportpdf] mising whitespace 2019-03-01 09:25:02 +01:00
Falconieri aef8dbbe2e fix: [exportpdf] problem on one line 2019-03-01 09:17:38 +01:00
Falconieri a2716bc05d fix: [exportpdf] add configmodule parameter for galaxy 2019-03-01 09:11:34 +01:00
Falconieri a937b7c85d fix: [reportlab] Textual description parameter 2019-02-27 12:45:22 +01:00
Alexandre Dulaunoy e7fd7e8eb2
chg: [pdfexport] make flake8 happy 2019-02-25 21:18:26 +01:00
Falconieri a3a871f2fa fix [exportpdf] update parameters for links generation 2019-02-25 15:51:33 +01:00
Falconieri 40cd32f1b8 tidy: Remove old dead export code 2019-02-22 10:25:12 +01:00
Falconieri a93b34208f fix: [pdfexport] Bugfix on PyMisp exportpdf call 2019-02-22 10:14:22 +01:00
Falconieri 2d29ce11bb Test 1 - PDF call 2019-02-21 15:42:18 +01:00
Vincent-CIRCL be01d54779 print values 2019-02-18 15:23:57 +01:00
Vincent-CIRCL 2753f354ab test update 2019-02-18 14:27:16 +01:00
Alexandre Dulaunoy 0bf27c1b69
chg: [btc_scam_check] fix spacing for making flake 8 happy 2019-02-11 14:23:18 +01:00
chrisr3d 74594f29aa
Merge branch 'master' of github.com:MISP/misp-modules into new_module 2019-02-11 09:28:49 +01:00
Alexandre Dulaunoy f0ccfd2027
chg: [backscatter.io] blind fix regarding undefined value 2019-02-10 16:56:01 +01:00
Alexandre Dulaunoy 7b1a837b10
chg: [backscatter.io] remove blank line at the end of the file 2019-02-10 16:40:06 +01:00