Commit Graph

338 Commits (feature-feedgenerator-exclude-malwares)

Author SHA1 Message Date
Raphaël Vinot 5c6314c45c new: Add email object generator 2018-03-18 23:21:29 +01:00
Sami Mokaddem fdd9833cd0
Update README.md
Replaced WHAT by Description
2018-03-13 17:26:55 +01:00
Sami Mokaddem cd85238b29
Update README.md
Added example of flush operation
2018-03-13 17:24:19 +01:00
Alexandre Dulaunoy ba98c71abc
Merge pull request #204 from mokaddem/redis-feed-generator
Realtime feed generator
2018-03-12 17:07:57 +01:00
Sami Mokaddem 6553519e3b Added more examples 2018-03-12 16:55:21 +01:00
Sami Mokaddem 364d685e0c Added usage in README 2018-03-12 16:40:06 +01:00
Sami Mokaddem 91262662c4 Added MISPItemToRedis and updated readme accordingly 2018-03-12 16:13:34 +01:00
Sami Mokaddem 39fc05aad9 Updated readme 2 2018-03-12 15:41:02 +01:00
Sami Mokaddem 80517aaf41 Updated readme 2018-03-12 15:34:12 +01:00
Sami Mokaddem 38c22ba954 Moved object constructor into their own folder 2018-03-12 15:22:58 +01:00
Sami Mokaddem d898bb3857 feature: Added support of MISP object constructor instead of the generic_generator 2018-03-12 15:17:25 +01:00
Sami Mokaddem 81d3532877 Added brief object description 2018-03-12 10:25:25 +01:00
Sami Mokaddem aa3f3b85f0 removed unused function 2018-03-12 10:19:05 +01:00
weslambert 58dd9878de
fix typo(s) 2018-03-10 08:04:18 -05:00
Sami Mokaddem e1a9fe1d85 Generator handles file flushing itself 2018-03-09 17:06:00 +01:00
Sami Mokaddem f6828c4394 Added description of generator object 2018-03-09 15:51:26 +01:00
Sami Mokaddem 828aa8e6e3 Updated README 2018-03-09 15:39:19 +01:00
Sami Mokaddem fdaa4c790c Creation of the generator object which permit to easily add attributes
and objects to daily events, stored as a MISP feed.
Plus, script fromredis which pops queue element in redis to put them in
the feed
2018-03-09 15:31:13 +01:00
Sami Mokaddem 61ce67cd1c Added install script 2018-03-08 17:39:14 +01:00
Sami Mokaddem c04a3709f9 Added support of MISP Object 2018-03-08 17:33:39 +01:00
Sami Mokaddem 22efb64f14 Overhall seems to work, need testing 2018-03-08 14:19:28 +01:00
Sami Mokaddem 188c452a39 Init draft of redis to feed 2018-03-08 12:01:35 +01:00
Raphaël Vinot 7195c6580a
Merge pull request #197 from RichieB2B/misp2cef
Add misp2cef example
2018-02-26 17:26:54 +01:00
Richard van den Berg 7dd2f54196 Add misp2cef example 2018-02-26 16:51:14 +01:00
Richard van den Berg a04388f99a Use from_dict 2018-02-26 11:25:14 +01:00
Raphaël Vinot 6a3b05fd25 fix: do not try to upload objects in case make_binary_objects fails
Fix #192
2018-02-23 11:17:54 +01:00
Koen Van Impe b6eb65c77f Prevent unpublished events to be included in feed
Change default proposed config
2018-02-06 21:41:03 +01:00
Raphaël Vinot e937c3ae81 new: Add bindings for Galaxies and Taxonimies 2018-01-26 17:02:47 +01:00
Raphaël Vinot 250190e8a8 new: Add bindings to PyMISPWarninglists 2018-01-25 17:56:30 +01:00
Raphaël Vinot e2bb66d01c chg: Cleanup new sbsignature generator 2018-01-23 11:07:36 +01:00
garanews db235899bf sb-signature library
Created sb-signature library with relative example for testing.
Thanks @dadokkio
2018-01-23 10:35:21 +01:00
Andras Iklody 89e900671c
Update settings.default.py 2018-01-11 11:58:50 +01:00
Eric Jaw 66ccf54c12 fix: Typo in error output text description 2017-12-06 11:07:36 -05:00
Raphaël Vinot 9c7923fe0a new: Add get CSV method. 2017-12-01 12:01:42 +01:00
Raphaël Vinot 0875ad4a5f chg: Add example file to push OpenIOC file to MISP
chg: Add some imports in the tool's init file
2017-11-28 11:54:08 +01:00
Raphaël Vinot bfe9867b2e chg: Add a generic MISP object generator 2017-11-15 17:37:17 +01:00
Raphaël Vinot 0f21a561b0 chg: Allow to add multiple attribute of the same type 2017-11-15 09:41:20 +01:00
iglocska 195cd6d7fc Rework of the feed generator
- use objects, attribute tags and object references correctly
- generate quickhashlist for fast lookups / future MISP caching mechanism
- saner structure (herp-a-derp)
2017-11-04 14:18:15 +01:00
Raphaël Vinot ea327ceffb chg: Update asciidoctor generator 2017-10-28 16:58:50 -04:00
Thomas Gardner d293476c6a Merge branch 'master' of https://github.com/MISP/PyMISP 2017-10-25 11:34:23 -04:00
Thomas Gardner e2d690d0ef added vtreportobject and vt_to_misp example 2017-10-25 09:48:18 -04:00
garanews 4152435250 Created add_generic_object.py
usage: add_generic_object.py [-h] -e EVENT -t TYPE -d DICT

Examples:
python3 add_generic_object.py -e 1683 -t email -d '{"subject":"The Pink Letter", "to":"jon@snow.org"}'
python3 add_generic_object.py -e 2343 -t person -d '{"first-name":"Daenerys", "last-name":"Targaryen", "place-of-birth":"Dragonstone"}'
python3 add_generic_object.py -e 3596 -t "domain|ip" -d '{"domain":"stormborn.org", "ip":"50.63.202.33"}'
2017-10-25 09:43:17 -04:00
Raphaël Vinot 2bfd091774 Merge branch 'master' of github.com:MISP/PyMISP 2017-10-24 18:09:27 -04:00
Raphaël Vinot 6517081fab chg: Add simple asciidoc generator for MISP event 2017-10-24 18:09:10 -04:00
garanews 94e3419c39 Created add_generic_object.py
usage: add_generic_object.py [-h] -e EVENT -t TYPE -d DICT

Examples:
python3 add_generic_object.py -e 1683 -t email -d '{"subject":"The Pink Letter", "to":"jon@snow.org"}'
python3 add_generic_object.py -e 2343 -t person -d '{"first-name":"Daenerys", "last-name":"Targaryen", "place-of-birth":"Dragonstone"}'
python3 add_generic_object.py -e 3596 -t "domain|ip" -d '{"domain":"stormborn.org", "ip":"50.63.202.33"}'
2017-10-20 09:55:46 +02:00
Raphaël Vinot b1989f16f2 Merge branch 'objects' 2017-09-18 11:43:48 +01:00
Raphaël Vinot 2bc0745fbf Refactoring in order to load objects 2017-08-28 19:16:20 +02:00
Raphaël Vinot f06bfd310b Do not fail if pymisp is not installed 2017-08-25 16:38:12 +02:00
Raphaël Vinot f66af15c62 Update get_template_id, cleanup 2017-08-25 09:45:56 +02:00
Raphaël Vinot c09ce0032c Refactor all the things
Add script for MISP core, make everything generic.
2017-08-24 19:21:52 +02:00
Raphaël Vinot 77845bd813 Update file/pe/pe-sections objects creator. 2017-08-23 15:37:04 +02:00
Raphaël Vinot 2fd3b05202 Update accordingly to the current server implementation 2017-07-24 17:16:40 +02:00
Sebastian Wagner 5a85788c5d
Fix shebangs and executable permissions
Files containing a shebang should be executable (examples/*.py)
Non-executable files should not contain a shebang (pymisp/...)

spotted with rpmlint
2017-07-24 13:27:28 +02:00
Raphaël Vinot 9f595251d5 Add sample for get_attachment 2017-07-18 11:15:28 +02:00
Raphaël Vinot a0273b8a43 Merge branch 'master' of github.com:CIRCL/PyMISP 2017-07-11 16:15:59 +02:00
obsidianpentesting 17e44c1c74 Example script to invoke the cache_all_feeds() from PyMISP. 2017-07-06 16:07:34 -05:00
raw-data 8b90a85254 fix args.quiet and status msgs 2017-07-03 21:16:38 +01:00
raw-data 73b66af0d3 add multithreaded suricata search code, fetching ids rules based on parameters and terms 2017-06-28 14:21:43 +01:00
Alexandre Dulaunoy 4f66996366 Merge pull request #92 from deralexxx/patch-4
use misp_verifycert
2017-06-20 12:00:32 +02:00
Alexander J ef1eda5028 Create README.md 2017-06-20 11:24:02 +02:00
Alexander J 41b159b596 use misp_verifycert
misp_verifycert
2017-06-19 16:27:07 +02:00
Raphaël Vinot 72a484ca32 Add support for freetext import in the API. 2017-06-13 15:37:39 +02:00
CheYenBzh 5c74a2474f Create fetch_events_feed.py 2017-05-29 14:03:21 +02:00
Paul A 36cf46acd9 Fixed the JSON output format (\n breaks JSON loading afterwards) 2017-05-02 16:27:37 +02:00
Hannah Ward 3e3e8b1306 Merge branch 'master' of github.com:MISP/PyMISP 2017-04-07 16:28:17 +01:00
Hannah Ward 3da2a54ea1
fix: Update script had `latest`'s docstrings 2017-04-07 16:09:38 +01:00
Paul dd3ce6c758 Update last.py 2017-04-06 14:23:04 +02:00
Paul 51f49ddcaa Updated last.py to dump json results straight away
Output was not usable with cli utilities such as: ```cat results.json | python -m simplejson.tool```. 
It's now usable and works perfectly.
2017-04-06 14:20:00 +02:00
Student CIRCL 4d2861780e Treemap.py requirements updated in the README.MD file 2017-04-03 17:07:52 +02:00
Raphaël Vinot 6dc422de72 Cleanup misp2clamav 2017-03-27 17:43:11 +02:00
Richard van den Berg 9fb9715c8e Add misp2clamav 2017-03-27 16:50:56 +02:00
Nick Driver 9aec74b01c Example using the search() function
Accepts specific parameters from search() instead of just using search_all().
2017-03-09 15:57:15 -05:00
Raphaël Vinot fc80e711a9 Merge branch 'master' of github.com:MISP/PyMISP 2017-03-09 16:33:29 +01:00
Raphaël Vinot 1da447abf2 Reorganisation, make add attribute more flexible 2017-03-09 16:32:51 +01:00
rmarsollier 8ae32703e8 example using tag() function instead of add_tag() 2017-03-08 10:51:47 +01:00
Déborah Servili a4f90a7ac1 add legend 2017-02-03 16:34:50 +01:00
Déborah Servili 03089ea7da Merge branch 'master' of https://github.com/MISP/PyMISP 2017-02-03 16:16:58 +01:00
Déborah Servili 910cfda4bc restore file deleted by mistake 2017-02-03 16:16:18 +01:00
Déborah Servili f8be16a905 add ta_scatter.py script & reorganise tools 2017-02-03 16:12:02 +01:00
Christophe Vandeplas ff921ec6a6 YARA dumper for all rules
This dumper also does YARA rule validation, ignores invalid rules and prevents duplicate rule names. The output is a file called misp.yara which can be used with your favorite YARA tool.
2017-02-03 10:43:57 +01:00
Alexander J 7b0e3b521a make it little more readable
guess that way it is easier to understand
2017-01-26 10:39:10 +01:00
Déborah Servili 87b5eb84bb exemple addtag (dirty) 2017-01-24 15:31:50 +01:00
Raphaël Vinot 35a4dd52bc Add signing support for MISP events 2016-11-17 17:07:29 +01:00
Déborah Servili 3cadc1a78d Improvements in the user api 2016-11-04 12:00:42 +01:00
Déborah Servili a11e26f80b Improvements in the user api 2016-11-04 11:58:21 +01:00
Alexandre Dulaunoy 55b4a0725b Neo4j stuff moved into graphdb directory 2016-11-04 09:31:52 +01:00
Alexandre Dulaunoy bbf9198787 Moving Neo4j into graphdb 2016-11-04 09:31:31 +01:00
Raphaël Vinot 75ebedae5c Merge pull request #68 from MISP/tooling
[WIP] Tooling
2016-11-03 16:04:28 -04:00
Déborah Servili 0b462404de add user management and examples 2016-11-03 11:23:48 +01:00
Raphaël Vinot bee1630e98 Add query example 2016-10-28 14:13:57 -04:00
Raphaël Vinot 2907fd18d7 Cleanup neo4j support 2016-10-27 15:58:08 -04:00
Raphaël Vinot abd836babb Add simple script to push MISP events into Neo4j 2016-10-25 17:28:55 -04:00
Déborah Servili 0de3f7459b add example add_named_argument.py 2016-10-22 14:52:17 +02:00
Déborah Servili 30cd45e94e remove test import 2016-10-13 15:28:18 +02:00
Déborah Servili 13dbb96111 Use only metadata in situational awareness tags functions 2016-10-13 13:39:44 +02:00
Déborah Servili b1e6765bb3 fix indentation 2016-10-13 10:11:18 +02:00
Déborah Servili 9cc55341f0 fix date formatting in mispevent.py + some PEP8 cleaning 2016-10-12 15:40:49 +02:00
Déborah Servili bc5df41179 fix situational-awareness examples 2016-10-12 12:33:42 +02:00
Raphaël Vinot 8a931a89f3 Fix upload function 2016-10-05 11:07:40 +02:00
Alexandre Dulaunoy e70cc7a985 Toggle flag instead of value 2016-09-12 13:45:37 +02:00
Raphaël Vinot bf5793992b Fix examples after removal of MISP XML support 2016-09-12 12:53:58 +02:00
Déborah Servili 84eb40e42b Add some examples 2016-09-12 11:32:04 +02:00
Déborah Servili fa66c77cd1 add tags_to_graphs.py in ecamples/situational-awareness 2016-09-05 14:14:29 +02:00
Déborah Servili 95654e083c Merge https://github.com/MISP/PyMISP 2016-09-05 13:50:35 +02:00
Déborah Servili d5bdb67090 update examples/situational-awareness/README.md 2016-09-05 13:41:02 +02:00
Déborah Servili eb427e89c9 update examples/situational-awareness/README.md 2016-08-30 10:42:34 +02:00
Raphaël Vinot 8dbeec3f96 Cleanup create_events 2016-08-26 18:25:39 +02:00
Richard van den Berg 7cbda22667 Speed up et2misp 2016-08-18 11:27:02 +02:00
Raphaël Vinot a4acc5d147 Add tests 2016-08-18 00:40:30 +02:00
Raphaël Vinot cdcb1cca5e Update testing 2016-08-18 00:23:49 +02:00
Richard van den Berg 17417bd826 Add et2misp example 2016-08-12 13:48:45 +02:00
Raphaël Vinot ab09c0a1dc Fix calls to __prepare_session
Fix #58
2016-08-11 17:45:32 +02:00
Raphaël Vinot 5937ef9e9b Version bump 2016-08-02 15:17:42 +02:00
Raphaël Vinot 93ef3595e5 Fix fetching method for tag_search and tags_count 2016-07-29 13:25:36 +02:00
Raphaël Vinot 90bb9f3ba4 Major refactoring of the SVG generator 2016-07-27 14:48:13 +02:00
Raphaël Vinot 24d131aa32 Initial refactoring, PEP8 and cleanup 2016-07-26 16:35:46 +02:00
Deborah Servili f8dbcde607 Update README.md 2016-07-26 11:09:00 +02:00
Déborah Servili cd046d2f7a Make printed date more consistent + update README.md 2016-07-26 11:05:20 +02:00
Déborah Servili 0f68ffc617 modify fetching method to use last 2016-07-21 10:09:10 +02:00
Déborah Servili b0a66da4de handling some NaN exceptions 2016-07-21 10:06:47 +02:00
Déborah Servili eebca6ecc6 delete some files 2016-07-13 15:46:50 +02:00
Déborah Servili caa8b963ec move files from examples/treemap to examples/situational-awareness/ 2016-07-13 15:24:36 +02:00
Déborah Servili e53f59bcbf Merge https://github.com/MISP/PyMISP into newbranch 2016-07-13 15:06:27 +02:00
Deborah Servili 12849622ef Rename examples/statistics/attribute_treemap.py to examples/treemap/treemap.py 2016-07-06 09:05:35 +02:00
Deborah Servili 8c0b5b943f Rename examples/treemap/treemap.py to examples/statistics/attribute_treemap.py 2016-07-05 16:26:57 +02:00
Raphaël Vinot 836845abde Use same variable names as testing environment 2016-07-01 14:30:13 +02:00
Raphaël Vinot 13e0cd0901 Make scripts executable 2016-07-01 13:52:31 +02:00
Déborah Servili 0bf368b281 Random names for dummy files 2016-07-01 12:06:49 +02:00
Déborah Servili 29476b6eba Add examples "create_dummy_event" and "create_massive_dummy_events" 2016-07-01 10:33:44 +02:00
Alexandre Dulaunoy 5bbd2a4bcc Merge pull request #20 from Delta-Sierra/master
Add example "create attributes distribution treemap"
2016-06-21 16:18:16 +02:00
Déborah Servili ec4b158c84 remove useless comments 2016-06-21 16:07:08 +02:00
Déborah Servili 4445652346 Add example "create attributes distribution treemap" 2016-06-21 15:46:09 +02:00
Raphaël Vinot 8241d4ce93 Fix python3 compat. Make Pep8 happy. 2016-06-16 13:48:40 +09:00
Alexandre Dulaunoy 0cc5d9c982 Comment removed 2016-06-13 17:20:40 +02:00
Raphaël Vinot 234de2f4d1 Add tag script 2016-06-13 19:15:04 +09:00
Raphaël Vinot 1da7660934 Make pep8 happy 2016-06-13 19:14:32 +09:00
ANSSI-BSO-D f93bad9564 form 2016-05-12 17:35:05 +02:00
Tristan METAYER acf8881658 init for ioc-2-misp 2016-05-12 17:33:13 +02:00
Alexandre Dulaunoy f9ce3999e9 Merge pull request #17 from Delta-Sierra/master
Add function for sighting using attribute id, uuid or a json file
2016-04-30 14:52:09 +02:00
Déborah Servili 3cd9ede99f Add function for sighting using attribute id, uuid or a json file 2016-04-29 16:35:27 +02:00
Alexandre Dulaunoy 7dc9e20997 More stats example 2016-04-28 15:05:31 +02:00
Alexandre Dulaunoy f490898a5c Statistics test script added 2016-04-28 14:44:01 +02:00
Iglocska 423757530b Added the option to filter out attributes based on distribution level 2016-04-14 10:35:03 +02:00
Déborah Servili b573daf86d Correct module help 2016-04-08 10:06:35 +02:00
Raphaël Vinot dca755ef08 Improve examples 2016-03-21 21:24:15 +01:00
Koen Van Impe f495c23303 Check if objectType exists in event
Prevent failing when f.e. an event does not have a tag.
2016-03-11 14:51:38 +01:00
Iglocska 8d0c26288e Updated the feed generator
- only save fields that are actually necessary and don't reveal too much of unneeded information (such as correlation)
- add contextual fields to the manifest
2016-03-07 03:29:34 +01:00
Alexandre Dulaunoy ec7337cbc2 Merge branch 'master' of github.com:CIRCL/PyMISP 2016-03-01 15:54:17 +01:00
Iglocska be242152e7 Script for the upcoming feed generator
- also some minor modifications to the get_index api
2016-03-01 15:32:58 +01:00
Alexander J f99329a8b0 Update yara.py 2016-02-12 11:34:02 +01:00
Alexander J 11b1403e4c Update upload.py 2016-02-12 11:33:45 +01:00
Alexander J fb355d3acb Update searchall.py 2016-02-12 11:33:25 +01:00
Alexander J ca45771171 Update get.py 2016-02-12 11:31:48 +01:00
Alexander J b5764c8660 Update last.py 2016-02-12 11:30:56 +01:00
Alexandre Dulaunoy 654f238e5c Merge branch 'master' of github.com:MISP/PyMISP 2015-11-24 17:17:13 +01:00
Alexandre Dulaunoy 3403a57391 Threat level id is from 1 to 4 (not from 0 to 3)
https://github.com/MISP/MISP/issues/729
2015-11-24 17:16:47 +01:00
Will Urbanski b5acf414e9 Fix KeyError when no results in time period
Fix a KeyError when no results were found for the specified time period.
2015-11-22 17:52:12 -05:00
Alexandre Dulaunoy 2cdf1aa88d Authentication parameters updated 2015-11-06 11:40:27 +01:00
Alexandre Dulaunoy 55ca454b92 Auth parameters updated 2015-11-06 11:22:38 +01:00
Alexandre Dulaunoy 4c0ed124f5 Updated auth parameters 2015-11-06 11:16:32 +01:00
Alexandre Dulaunoy 006103d2fe Normalized auth parameters 2015-11-06 11:12:31 +01:00
Alexandre Dulaunoy 761b831708 Normalized auth parameters 2015-11-06 11:11:22 +01:00
Alexandre Dulaunoy 424c1671aa Sample keys file added 2015-11-06 11:09:17 +01:00
Alexandre Dulaunoy dbd45f0f01 Normalized auth parameters 2015-11-06 11:06:37 +01:00
Alexandre Dulaunoy fc0873bed1 Normalized auth parameters 2015-11-06 10:17:20 +01:00
Alexandre Dulaunoy db1221dd52 Auth parameters normalized 2015-11-06 10:14:45 +01:00
Raphaël Vinot 08e1c40987 Add helpers to update events with specific attributes. 2015-09-01 18:46:10 +02:00
Raphaël Vinot c7b03640fb Multiple updates, cleanup
* Remove attribute (Fix #4)
* Deprecate pure XML API
* Cleanups and fixes in the upload file functionality
2015-08-28 17:07:30 +02:00
Raphaël Vinot 218ffcd915 Add Yara rules download support (by event) 2015-08-19 10:43:03 +02:00
iglocska 4436383624 Dump the entire event including the 'Event' container element 2015-08-12 10:09:23 +02:00
Raphaël Vinot d8a6ae28ff Add test scripts to get and update an event. 2015-08-11 17:14:16 +02:00
Raphaël Vinot 2ad737e7b7 Add search all 2015-08-06 17:43:12 +02:00
Raphaël Vinot 208091dc47 Fix last commit 2015-08-06 09:49:44 +02:00
Raphaël Vinot cc13a779b3 Merge branch 'master' of github.com:CIRCL/PyMISP 2015-08-05 17:30:20 +02:00
Raphaël Vinot 97dfe2a4f6 Add last param to restSearch + example script 2015-08-05 17:20:59 +02:00
Raphaël Vinot effd8084a7 Cleanup of the upload API 2015-08-05 16:01:57 +02:00
Koen Van Impe 2e1da3b777 Add netflow filter output
- get event data for event with “—event X”
- get netflow filter with “—netflow”
   simple host X or host X
2015-08-04 23:25:15 +02:00
Raphaël Vinot bef354ac44 Preliminary version of the file uploader 2015-08-04 16:24:55 +02:00
Raphaël Vinot 58bfd30a23 Add test script to add attachement to event 2015-08-03 18:09:39 +02:00
Raphaël Vinot 4e942fa33b Merge remote-tracking branch 'origin/master'
Conflicts:
	examples/get_network_activity.py  (python 2.7 & 3 compatible print)
2015-08-03 16:44:52 +02:00
Iglocska 507c5a5446 API made a bit more flexible with input data
- input for add_event() and update_event() can now be a JSON object, JSON string, XML
2015-07-30 15:53:34 +02:00
Iglocska c315ecaef6 Fix to an issue with using XML as input for add_event() and update_event()
- also a change to the copy_list.py script to account for the change
2015-07-30 15:26:05 +02:00
Raphaël Vinot b6ff8746bc Make the code python3 friendly 2015-05-03 02:47:47 +02:00
Raphaël Vinot 33597f97ef Make PEP8 Happy 2015-02-24 14:31:01 +01:00
Koen Van Impe 4c7e0731cd Example script to download MISP network activity 2014-11-16 17:02:23 +01:00
Raphaël Vinot 81fd66d612 support update events 2014-05-02 17:10:53 +02:00
Raphaël Vinot 31eb0d4c96 Add the following options:
- possibility to copy in one direction or the other between instance
- add loop to simply put event ids to copy
2014-05-02 11:46:04 +02:00
Raphaël Vinot f2c7bbe5c9 Add installer, proper copy script 2014-04-16 15:14:58 +02:00