MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
14,607 Commits
35 Branches
366 Tags
190 MiB
Commit Graph

534 Commits (d48a013f3a06c740a03ac16cdd4e256f2f0763c7)

Author SHA1 Message Date
iglocska 5b62965e98 fix: fixed invalid refanging 2018-04-18 11:23:57 +02:00
iglocska d04f263c95 chg: Added [:] to the refanging options 2018-04-18 09:50:04 +02:00
Sami Mokaddem 303ff41ea1 feature: Better support of extended event in event graph - Added a 
colored region for each event extending the current event scope
 2018-04-17 15:23:28 +00:00
Sami Mokaddem f4ae1d4740 fix: bug when plotting event without attribute or object 2018-04-16 13:39:43 +00:00
Sami Mokaddem 368aa2f128 feature: Added support of extended event in event graph 2018-04-16 12:02:43 +00:00
Sami Mokaddem cd0c00384e Replaced scope rotation key typeahead by selector + removed trailling 
spaces
 2018-04-16 09:17:19 +00:00
Sami Mokaddem e2f4aade02 Added filtering based on authorized JSON key + JSON key is displayed in the header scope badge 2018-04-09 12:07:53 +00:00
Sami Mokaddem 09127a24c5 Support of graph per JSON key (using typeahead) 2018-04-09 11:39:45 +00:00
Sami Mokaddem 6ee5419297 feature: Draft of generic graphing from any key 2018-04-09 09:12:26 +00:00
Sami Mokaddem f3b2741843 feature: Support of Tags in the event graph 2018-04-06 14:44:40 +00:00
Sami Mokaddem 921224ed40 Merge branch 'quick-fix-metacategory-graph' into ref_graph 2018-04-06 07:50:27 +00:00
Sami Mokaddem fc168c5a35 Draft of filtering per attribute value 2018-04-05 14:21:40 +00:00
Sami Mokaddem 0ecccee108 Moved reference logique server-side + First draft of filtering capabilities 2018-04-05 10:31:26 +00:00
Sami Mokaddem 3933baf9c9 Compute graph serverside 2018-04-04 13:12:16 +00:00
iglocska 9485dfe5e2 chg: Refactor of the complex type tool 
- makes it more readable
 2018-04-03 22:25:52 +02:00
Sami Mokaddem 5e83caf8fb Added retreiving of object templates in order to let the user choose the field we want to see in the event graph 2018-03-29 16:05:19 +00:00
Sami Mokaddem 6a0abcce22 Renamed script again 2018-03-22 16:56:43 +00:00
Sami Mokaddem c78fca0ede Added possibility to edit references on the fly + edit objects on their dedicated webpage 2018-03-22 15:53:53 +00:00
root a3a6a77611 Initial references graphs commit 2018-03-19 08:44:25 +00:00
iglocska c6fe2db137 fix: Added sightings to object attributes in the JSON output, fixes #3007 2018-03-07 13:03:01 +01:00
Andras Iklody 353611e708
Merge pull request #2997 from 0xmilkmix/validate_suricata_rules 
Validate suricata rules
 2018-03-03 23:12:54 +01:00
milkmix 05eac2bfe5 removed tests from class 2018-03-02 19:09:55 +01:00
milkmix ff103277ad finished http validation function using sticky and modifiers 2018-03-02 19:08:59 +01:00
Émilio Gonzalez bb8d4fa634 Fixed a bug regarding filename|ssdeep attributes importing using FreeTextImport. See Issue #2971 2018-02-28 18:34:46 -05:00
iglocska 9fd8a1c14f chg: Use <> as delimiters for the freetext import too, fixes #2978 2018-02-27 18:21:08 +01:00
iglocska 501b933a56 fix: Don't try to refang filepaths, fixes #2926 2018-02-25 23:24:54 +01:00
iglocska 10bd1f69c4 new: Allow requesting of misp standard format for the export modules 
- just set the `require_standard_format` to true in the moduleinfo disctionary
 2018-02-21 11:42:30 +01:00
iglocska 6a29d06566 new: Tie tags into PubSub channel 
- Reset the catastrophic @ilmoka enrage timer for another 5 days
 2018-01-26 19:27:27 +01:00
iglocska 9858d63712 fix: Suricata export URL encodes an IPv6 between [], fixes #2872 2018-01-24 00:27:12 +01:00
milkmix f6d4839123 wrote dns validation func, checking modifier after dns_query keyword 2018-01-19 18:45:18 +01:00
milkmix b25bfac4ab added options extraction function 2018-01-19 18:31:30 +01:00
iglocska 57197f092b fix: Add alternative x509 fingerprint hashes to the freetext import tool, fixes #2821 2018-01-17 10:16:33 +01:00
iglocska 58c97d8263 chg: Tuned the freetext import tool, fixes #2822 
- refang e-mail addresses
- add [@] refanging
 2018-01-16 15:01:21 +01:00
milkmix 940916d034 added validation function for global syntax 2018-01-12 18:22:58 +01:00
milkmix ddf5f82f4c initial regexp to match rule pattern 2018-01-12 17:37:36 +01:00
iglocska 811ea4a6d8 fix: Removed the https url rule for now 2018-01-12 15:17:43 +01:00
iglocska cdffeafbf7 fix: Broken Suricata rules due to removed https branch 
- possible fix, mimicing contents of https://[ip]
 2018-01-12 14:59:17 +01:00
Andras Iklody 9d6c20709e
chg: Add hybrid analysis to the freetext import tool, fixes #2797 2018-01-09 22:43:12 +01:00
dewiestr 90bdb37174
Update NidsSuricataExport.php 
Removed the ':' from the suricata msg as it removes the message after it in squert.
 2018-01-07 12:11:51 +04:00
iglocska 3a45410e10 fix: Naive fix for an issue with tab separated feeds being broken by the switch to str_getcsv 2017-12-29 10:40:03 +01:00
iglocska 0df15f03e1 fix: Fixed the invalid default TLDs if no warninglist is loaded 2017-12-08 12:28:28 +01:00
iglocska 4f6dba5f35 new: various improvements 
- use the feed uuid caches to link directly to affected MISP events
- various UI improvements
- Feed preview pagination / POSTed event ID filters added
 2017-12-05 00:05:11 +01:00
iglocska 67f0acb6c6 fix: Made CSV parser for freetext import tool / feed ingestion compatible with escaped CSVs 
- "" now handled correctly
 2017-11-30 16:52:22 +01:00
iglocska 6135468c41 new: Added full audit logging to ZMQ and Syslog, fixes #2635 
- syslog now includes all audit log entries and it's separated into proper severity levels
- ZMQ logging and syslog logging are both optional features
 2017-11-24 12:01:53 +01:00
Thomas Gardner 93160d69c1 added target-email to FreeText Import types 2017-11-22 11:07:42 -07:00
Milan Pikula 22fbe12762 fix: don't verify peer name on self signed certs; don't verify self signed peer if cert is missing 2017-11-22 16:19:41 +01:00
iglocska 45a2d1a09b new: Added phone number recognition to the freetext import tool 
- also, changed the massaging of phone number type attributes to replace 00 with +
 2017-11-16 16:25:46 +01:00
iglocska 298269fe29 fix: minor tuning of suricata rules 2017-11-07 16:54:07 +01:00
iglocska 68f4833893 new: First version of the zmq reimplementation 2017-10-27 09:10:46 +02:00
iglocska fa7d3fdb36 new: First round of updates to the correlation engine ready 
- node deletion temporarily disabled until a bug is resolved
 2017-10-08 19:50:28 +02:00
iglocska a399ef1186 new: Further work on the graphing engine 2017-10-07 16:18:39 +02:00
iglocska 5290214c9b new: First iteration of the graphing engine rework 2017-10-06 10:05:00 +02:00
iglocska 416ff3f095 fix: Sanitise all the things for XML, fixes #2522 
- Sanitise all the things!

─────────────────────────────▄██▄
─────────────────────────────▀███
────────────────────────────────█
───────────────▄▄▄▄▄────────────█
──────────────▀▄────▀▄──────────█
──────────▄▀▀▀▄─█▄▄▄▄█▄▄─▄▀▀▀▄──█
─────────█──▄──█────────█───▄─█─█
─────────▀▄───▄▀────────▀▄───▄▀─█
──────────█▀▀▀────────────▀▀▀─█─█
──────────█───────────────────█─█
▄▀▄▄▀▄────█──▄█▀█▀█▀█▀█▀█▄────█─█
█▒▒▒▒█────█──█████████████▄───█─█
█▒▒▒▒█────█──██████████████▄──█─█
█▒▒▒▒█────█───██████████████▄─█─█
█▒▒▒▒█────█────██████████████─█─█
█▒▒▒▒█────█───██████████████▀─█─█
█▒▒▒▒█───██───██████████████──█─█
▀████▀──██▀█──█████████████▀──█▄█
──██───██──▀█──█▄█▄█▄█▄█▄█▀──▄█▀
──██──██────▀█─────────────▄▀▓█
──██─██──────▀█▀▄▄▄▄▄▄▄▄▄▀▀▓▓▓█
──████────────█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──███─────────█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──██──────────█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──██──────────█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──██─────────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──██────────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█
──██───────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▌
──██──────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▌
──██─────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▌
──██────▐█▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█▌
 2017-09-29 12:21:52 +02:00
iglocska fd45eed6c4 chg: Added .onion to the TLD list for the complext type tool 2017-09-26 09:14:00 +02:00
iglocska 3b6a6f6e5f Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2017-09-25 12:37:11 +02:00
iglocska 3f76fd6ea7 new: Rework of the attachment uploader 
- add attachments and upload_sample now share code
- allow the same features via upload_sample (object creation / use of advanced add attachments)
- new flag: advanced

- example:

  POST to mymisp/events/upload_sample
  BODY:
{"request":{"files": [{"filename": "bla.exe", "data": "U3RhckNyYWZ0IElJIGZvcmV2ZXI="}], "distribution": 1, "advanced":1, "info":"bla"}}

- this commit was brought to you by CEF and

MMMH$= -  .,   ,,.          %H++  ,= %%$$$$X+ ;=== .=  :+HHHMMMHMMM####MMH@@@@@@HHH$=      HHH@HHHHH+XXX$$$$$$$$XXXXXXX+
MMH = -.  . ,-,,-,.         :H@H  =;;++$HH+XX$%+X%+$++=:=.XH@@@HMMMMMMMMH@@@@@@@HHX$   ,X@@@@@@@HHHHHHHHHHXXXXXXXXXXXXXX
  . ---,  -    ,,,            +@ .. ;++$HH+HHH++$+++HH+++, .+%HHMHHHHHHHHH+%%%++++$+   +++HHHHHHH+++++++++HHHHHHHHHHHHHH
- -- ,,,  --,. -                 , ,; +$XHH@@@@HHH@@@HHHH+$+$X+HH+$$+ ;  ;=  .    %   +  ,+$X+++XXXXXXXXXXXXX++HH+++++++
---==,,--,-,-., :     .          -,,:/ $XHH@HMMMMMMMMMM@HHX$H@MHHHHX+H%%$%+H/:.%. $. @,,,. $$XXXXXXXXXXXXXXXXXXXXXXXXXX+
  =  - --,,   , --   ..             =/ +$+H@@HMMMMMMMMH+H+++HHHHHHHH@+++++H+X++X+$$  = ,,, - $$XXXXX$$$$X$$$$$$$$$$$$$$X
====== --,,,, ,= =              ,==== ++$$+HHMMM####MH+$$+++HH@+HH@MHMMH@@H@@@HH+$+    ,,, ,. $$+$++$$$$$$$$$$++$$$$$$$X
 :==-===-,. ,., ==   .           :;; +++%$+H@HMMMMMMM%$%$$$+H@@+HH@MMMMMM@@@@HHH++H. .,,-,,--=/+$$%%%%%%%%$+%%$$$$$XXXXX
,  =  ==- -  .  ==             . =; ++++%++HHHHHHHHHH++%$$X+@@H+HHHMMMMMMHH@@@+X+    , ,,,,-  , ,$$$$$$$+++++$$$$XXXXX$$
,,-       ,    --=    ..       . ;/ ++++%$X+HHHHHHH  ++$++X+HH+X+H@HMMHHHHHHHH+.       ,,  ,,  , .    +$$$$+%+$$$$$$$$$$
,-----=-=--,   ,==             ..;/ +% +%$XX+HH++HH+/+$%++H@@HHXHHH@@@@@@@@HXX  .   .,,,.  ,,,,     ,-=$$$$$$$$$$$$$$$$$
 - ,- --  -,   ,-=     .         =/++%++%+++++XXXXX$$+.  +HHH@+$XHHHHHHHHH++$        -,,,  ,,      ,,,.   ,+$$$$$$$$$$$$
 ---,-----, .   ==               =/+%+++%++$$+++$X$$$$++,$$+++XXHHHHHHHH+X$+%       ,-,-,        ,,    .  .  ,+$$+++++++
== --, -- =--, ,,=          .    ./++$$++$+X$+/++$$XXXX$$$$XXXXXXH+HH+H+X$%%/     .,,,,,,    ..  ..    ,. ,,,-=+%+++ /++
+   -- -  -,,-  .,    .  . .      = +$$++++HH+.  ,+$$+++++++$XX$X$XHHH+X$$+      ..--,-    .. .        .    ,-, = ======
MH - ---- --,,,    .       .. ,      %++$$X++++ +%++++++++%++$$$$$+H++X$$+        --,    .         .   .        =  .====
MM=,-, ---,,,,,    . .     ...,,,   =/++%$$XXXX+/+++@@H@HX$+%$$+HHHHH$$$+:       ,--    .     ,. ..       .. ==::;=-:;;;
MM+ ,----,,,,              , .. ,.      +++X+HH+++++%++$++++$$+HHH+++$$          ,-          ,   .       .   : ;/ +%+.
MMH ,-,-,, ,,.        .    -,     =     = +$+H@HH++++$$X$$+++HHH+++$                       ,    ..       ,  +++++++%%+%+
MM@,--,-,,,,,. .     ,,     .    ,-,    .=+$XHHHXXHHHHHHHH@@@@HX$%+:          ,, .      ..,,  .....    ...%%%%++%%%%%%%%
M@@== ,,,  ,                               ++++XX++HHHHHH++HHH+,              ,         ,  .  ....     . +$+%%%%%%+%%%%%
H@H+=,,,  ..                                  ,,+%$+H@HHHXX++,               ,         ,,  .  ...   . ,$$$$$%%%%%+%+%%%%
@H+,-,,.....       .                          .,.;; ++$$X+%+:-              ,  .     .,,,  .  ...   . XXX$$$%%%%%%+%%%%%
+++ -, . ...                             .  .======== === ,                          ,, . .  ..   . -,XXX$X$+$+%%%%%%%%%
$+     .                                ===:; ++++ ++++-,.  ,                       ,-,          .  $X+XX+XXX$$+%++%%%%%
++: ,. .                         ,-,,-==:; %%%%%+%$$%$$X$$$+%+:==        .        . ,,           ..+X$XXXXXX$$$+%%$$%%%%
=:                              ,,,  ==   ++++++$+$$%+++$$$++$+ . ==     .        .,,,             +$$$$$$$$$$$$$$+$%%%+
 ,                          ,---, =:;/++$$XX$$$$$$X+H@H@HHH$%%%$X$++;===== .      .,            .. +%%+$++$%$$$$$$%%++%+
                               ===; +++$$$$+ +%+++%+HH@@@@HH+++ ++%+$+,  ===      ..             ,=;   +++++++++..   :;;
                      .   =:;   /++%$$++,  ,++HHMMHH@@@@HHHH@HH++++++ ,+$$+ .     ..                :=;;:;;;;;==========
                  .,,-==;;;+%  %%+$$$$ /+++@@@@@@@@@@HH@M@MH@@@HHHHH$$% /%$XXX$X  .                -=====::::=========::
                .    =;  ++++++$+++  , +%H@@@HHH@HH++HHH@MHHH@HHHHHH++++ , +%%+$                    ,, -       --- ==:=:
               ====;    ++++$$+%  ++H@HHHHHHH+X++X++@@@HHH@MMMMHHHHHH@HHHH+++++.                        ,,,,-,--- =:==;;
     .,., ==;// / ++++%+%+%+++$$+@H@@@@H@HHH+XXX$%+HHHH@@HH@HMMMMMMMMMMMMMMH@+%;                       ...,,,,,--==;;;/;
 .  ...=    .,+%$++%+$XXX$++%+++H@@@@HHH@HHH+++.   ++++H+HHHHHHHMMMMMMMMMMMM@++:                            ,,, ===;;;;;
==: .  ++++++++HH%H+++X++HH+H@HHHH@HHHHHHH+++++%++%%+%%++ . ,   = ++$H@@HMHMMH%=                                .  ..,,=
+++%$XXHHHHHH@H@@@@@H@HH@MMM@@HH@HH+HXH@HH%%+HH+XX$$$+++/;:=== ,,,,,, = ::; % :,                                   ...,,
%+++HHH@HHH@@HMHHHH@HHHMHMHHHHHH+XH+HHH++++HHHH@HHHHH++%+ -,  = ,=== ,,  ,,, .
H@HHHH#M#M#MHHHM#MMMMMMMHHHH@H@H++@H$+++HHM#MMMMHMMH@@HHHHHH%+++++%%%+++    ,  .
%%%%%%%%%%%%%%++++%%++   ..   ...  ..  .                                   +++%+++++++%++++%+++++++++%+%++%+%%++%++++++%
 2017-09-25 12:22:19 +02:00
Kyle Parrish c5d3ae7b1f RPZExport - Alternate NS 
Added option to add an alternate nameserver to RPZ export.
 2017-09-19 13:25:17 -04:00
iglocska 09dfb7aa14 fix: Reverted CakePHP version 2017-09-19 15:50:19 +02:00
iglocska 76ec7f1c10 fix: Fixed the XML view 
- please stop using XML, for your own sanity, I beg of you!
 2017-09-19 12:05:21 +02:00
iglocska 48b1679216 Merge branch '2.4' into objects_wip 2017-09-18 10:41:54 +02:00
iglocska 50911c9f85 chg: cakephp updated 2017-09-18 10:19:37 +02:00
iglocska 864b680774 fix: Updated the xml export tool to support objects 
- though why do we still support XML?...
 2017-09-13 14:25:13 +02:00
iglocska a931af7223 chg: Some tuning to the freetext import tool 2017-09-12 10:20:38 +02:00
iglocska 40ea22a272 Merge branch '2.4' into objects_wip 2017-09-04 17:38:06 +02:00
Andras Iklody ad60bddc2d fix: Removed url -> tls_cert_subject rule conversion for the suricata export, fixes #2396 2017-08-28 14:09:23 +02:00
iglocska eae062bdb6 fix: Fix to the max items displayed / page using the custom pagination tool 2017-08-25 14:39:23 +02:00
iglocska 0e7dd2eddc new: Added first iteration of object references and other changes 
- various fixes
- rework of the pagination library
 2017-08-09 17:53:25 +02:00
iglocska 3b004d5686 Merge branch '2.4' into objects_wip 2017-08-03 11:20:34 +02:00
iglocska 02464da6f2 fix: cakephp updated 2017-07-05 11:25:11 +02:00
Kevin Allix e7d3991bc3 Use a password to connect to Redis if MISP.redis_password is set in config.php 2017-07-03 12:11:26 +02:00
iglocska df5daae664 chg: Further work on the objects 
- view events with objects via the API
- Further improvements to adding objects
 2017-07-02 22:42:44 +02:00
Andras Iklody 3cd94c7e7c Revert "Use posix_getpgid to check whether a pid is running" 2017-06-26 11:07:59 +02:00
Kevin Allix bee2dc3c49 Use posix_getpgid to check whether a pid is running 2017-06-25 22:34:55 +02:00
Kevin Allix a124aef569 grepping the output of ps: the grep pattern should be ^pid_value$ 2017-06-25 12:23:30 +02:00
iglocska 98d45d2d9f fix: Fixed sanitisation of feed correlation fields 2017-06-22 23:12:06 +02:00
iglocska 926a16310c fix: meta field in galaxy cluster should be a dict even if empty in the JSON output, fixes #2280 2017-06-22 23:06:45 +02:00
iglocska 894415f82a fix: Fixed an issue in the XML export due to neglect 2017-06-19 15:30:16 +02:00
iglocska 473fc9897c fix: Further performance improvements to the zmq module 
- should make inserting data faster
 2017-06-16 10:08:36 +02:00
iglocska bcc3923e8e Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2017-06-16 08:45:16 +02:00
iglocska 57857c3a32 new: Performance improvements for the pub-sub modules 
- Only load and open connection to redis for the pub-sub connection once.
- Massive performance boost when the ZMQ functionality is enabled
 2017-06-16 08:41:12 +02:00
Hannah Ward 9ab1331bfb
new: Push new Discussion items to ZMQ 
Under the topic misp_json_conversation
 2017-06-15 15:30:43 +01:00
iglocska 859a2eb436 fix: typo fixed 2017-06-09 12:44:48 +02:00
iglocska 8b4fc61189 chg: Performance tuning: Custom pagination tool 
- changed set operation to a more performance alternative
 2017-06-09 11:44:46 +02:00
iglocska 95429723ed fix: 
- cleanup refactoring of pub sub tool
- better handling of no access to redis
 2017-06-09 11:43:53 +02:00
iglocska bce780090f new: Added User and Organisation addition/change data to the ZMQ feed 2017-05-29 16:18:37 +02:00
iglocska ab9f282a44 new: Added sightings to ZMQ pub sub system 2017-05-28 00:33:20 +10:00
iglocska 56c079642d new: Added attribute JSONs to pubsub system 
- also made mispzmq a but more generic
 2017-05-22 14:30:58 +02:00
iglocska 4c4f9a4dbb chg: Allow for \t to be used as a CSV feed delimiter 2017-05-11 14:46:20 +02:00
iglocska 468834b210 fix: Updated cakephp solving TLS 1.2 issues 2017-05-11 08:38:50 +02:00
Andras Iklody eef8b55120 Merge pull request #2128 from deloittem/2.4 
Snort attribute generation rule now contains the initial msg field
 2017-05-09 10:46:47 +02:00
Ángel González 926895733b Cosmetic changes 
Change space indents to tabs
Remove ?> at end of file
Add or remove some indentation where appropriate
 2017-05-08 00:45:57 +02:00
Pablo Panero 153926e0af BroExport types updeted 2017-05-05 16:38:38 +02:00
deloittem 5c2bc871ca Update rule generation for attribute snort: generated rule now contains the initial snort rule msg 2017-04-10 15:57:33 +02:00
iglocska 6d33845701 fix: Fixed a typo in the previous commit 2017-04-07 16:56:55 +02:00
iglocska dadd9b3c81 fix: remove sharing groups from json output if empty 2017-04-07 16:51:37 +02:00
iglocska 3b6807ef72 new: Rework of the restsearch APIs 
- allows for alternate download types (supported for now: openioc)
- major refactor of the openioc export
- refactor of the CIDR tool
 2017-03-31 19:27:34 +02:00
Mathieu Deloitte 47bcd264e2 Manage the new attributes IP-SRC|PORT and IP-DST|PORT when exporting NIDS rules 2017-03-22 16:19:32 +01:00
Andras Iklody edaeaf9194 Merge pull request #2031 from deloittem/2.4 
Suricata export update
 2017-03-09 09:43:30 +01:00
Mathieu Deloitte 59df951071 Only display the tag name if the array contains values (depending if the tag is exportable or not) 2017-03-09 08:48:22 +01:00
iglocska 717786d70a chg: cakephp updated 2017-03-08 15:08:27 +01:00
Mathieu Deloitte 27b2effffd Add the attribute tags to the msg field (Suricata rule) to sort easier the raised alerts 2017-03-08 15:04:45 +01:00
Mathieu Deloitte d8feddd47f Initialize host to empty value when the URL is formed incorrectly 2017-03-08 14:52:40 +01:00
iglocska 35cdd5eefe fix: Missing file added 2017-03-02 12:02:36 +01:00
iglocska a59aab9b23 fix: Re-added the accidentally removed code in a merge, fixes #1965 
- affects f0e1a27b7d
 2017-02-20 18:43:36 +01:00
iglocska dc8a9707c0 Merge branch '2.4' into feature/enhanced_sightings 2017-02-16 22:52:53 +01:00
Alexandre Dulaunoy 5ace946502 Merge branch '2.4' into 2.4 2017-02-15 17:44:51 +01:00
iglocska ab7aadb924 fix: Fixed a bug with the freetext import that broke the detection of IP addresses 2017-02-14 16:51:07 +01:00
Alexandre Dulaunoy 5bd06f86e5 Merge branch '2.4' into 2.4 2017-02-12 11:41:41 +01:00
iglocska e1f5463a82 fix: Added correct recognition of ip:port indicators to the freetext import tool, fixes #1919 2017-02-10 17:59:35 +01:00
iglocska ca22435831 fix: Added (dot) to the refanging 2017-02-10 10:32:43 +01:00
Mathieu Deloitte 98864fb82e NidsSuricataExport refactoring for attribute *URL* 2017-02-08 14:12:30 +01:00
iglocska a229af43ae fix: Empty delimiter for CSV feeds causing grief 2017-01-25 06:02:55 +01:00
Alexandre Dulaunoy 16d31458a8
fix: whois-registrant-email added as type when an email is detected in freetext 2017-01-18 14:13:36 +01:00
iglocska 4ad022b03c Merge branch '2.4' into feature/attribute-tagging 2017-01-16 16:15:06 +01:00
iglocska 7dcc11f0f7 fix: Copy paste fail 2017-01-01 16:29:50 +01:00
iglocska 734ff59cb4 fix: Left off changes to the complextypetool 
- oops
 2017-01-01 16:28:23 +01:00
iglocska 76e9398df9 new: Various new feed features 
- import feed descriptor json pastes to add a list of pre-defined feeds
- improvements to the feed pull (a single non validating attribute shouldn't break the process)
- altered the saving of the attributes to happen in chunks during a feed pull to avoid very large feeds from stalling the process
- split the feeds into 3 tabs: default, custom, all
 2016-12-31 09:04:46 +01:00
iglocska 7f8a81e161 new: Added caching and pagination to freetext/csv feeds 2016-12-30 16:16:56 +01:00
iglocska 7146652059 Merge branch '2.4' into feature/attribute-tagging 2016-12-26 23:30:21 +01:00
iglocska da433c3549 Merge branch '2.4' of https://github.com/MISP/MISP into feature/disable_correlation 2016-12-22 21:01:58 +01:00
iglocska 3a2e051b91 fix: Added an alternative to bcmod if it doesn't exist 
- simply threw an exception if the module wasn't loaded on the event view if it contained an IBAN number
 2016-12-22 18:06:20 +01:00
Andras Iklody ac994530e6 fix: broken bro export 
- Sanitisation issues with linebreaks in comments breaking the export
 2016-12-21 17:35:00 +01:00
iglocska 4155e32629 fix: Added additional refanging patterns to the complex type tool, fixes #470 2016-12-12 14:20:07 +01:00
iglocska 01f078344c fix: Fixed an issue with the freetext importer 
- It looks like PHP does parse single quoted strings and replaces double backslashes with a single literal backslash
 2016-12-09 08:59:59 +01:00
Armins 7ba143bcd1 Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2016-12-07 18:12:49 +02:00
Armins 4c67f0a2c8 Added fast_pattern 2016-12-07 18:07:12 +02:00
Andras Iklody 44ec75e462 Merge pull request #1726 from liviuvalsan/bro_export_improvements 
Performance improvements, bug fixes and new features for the export to Bro
 2016-12-07 16:52:15 +01:00
Liviu Valsan 4c022beafc - Performance improvements when exporting a large number of attributes into Bro format. 
- Fixed file header formatting for the export to Bro format (tabs used consistently).
- Computing the time needed for generating the export to Bro format when done using a background job.
- When generating the Bro export from the UI all the attributes are generated in one single text file similar to the CSV export instead of a zip file with different files inside.
- Changed the file extension of Bro export files from ".intel" to ".txt".
- Removed the allowNonIDS option from the Bro export as it doesn’t make sense to have it (Bro is an IDS).
- Fixed some of the API endpoints which were not accepted (ACL issues).
- Added support for a list of events that should be / should not be included in the export.
- Added a new "meta.desc" column (added in Bro 2.5, see https://www.bro.org/sphinx/frameworks/intel.html) containing the description of the event and of the attribute.
- Sanitized the exported data for Bro.
- Fixed a number of value substitutions which were imported from Snort/Suricata and which were not working for Bro. Did instead substitutions needed for Bro.
 2016-12-07 16:33:17 +01:00
Iglocska 1e7dccf272 Merge branch '2.4' into feature/galaxy 2016-12-06 16:11:59 +01:00
Iglocska 8f220378ce new: First RC of MISP galaxies 1.0 2016-12-06 15:52:20 +01:00
Iglocska 576d58462d fix: Trim strings of brackets before running the freetext detection on them 2016-12-01 12:24:42 +01:00
Iglocska 162e024eb8 fix: Temporary fix for a keyword mismatch between the import modules and the freetext import 2016-11-29 11:56:16 +01:00
Iglocska 6e52070f48 fix: Fixed an issue that prevented the feeds from working in CSV mode if no value field was set 2016-11-24 09:50:22 +01:00
Iglocska 7e75aafc22 fix: Added domain|ip to nids exports 2016-11-09 17:08:06 +01:00
Iglocska c2fc803fed chg: Use the TLD lists from the warninglists, fixes #1149 
- simply load any enable warninglist entries from the pre-defined TLD warninglists
- Pass the resulting array to the complex type tool
- during domain type heuristics, if the TLD list is not empty use the supplied list
- alternatively generate a list based on the old TLD rules
- does not alter any functionality otherwise
 2016-10-25 22:23:01 +02:00
Iglocska 6ffa949657 fix: Invalid bro export generation due to invalid syntax on the intel field 2016-10-25 12:48:51 +02:00
Iglocska 9891234662 new: CSV feeds and various fixes 
- Added the CSV feed format
  - users can specify which fields in the CSV should be parsed
  - comment lines are automatically omitted
  - new settings system added to feeds, currently only used for the value fields

- Slight rework of the correlation lookup for the feeds
  - got the Speed Force treatment
  - correctly checks against value1 and value2 instead of value

- Various freetext import fixes
 2016-10-08 14:36:24 +02:00
Iglocska 721cfd8d98 fix: Fixes to the ssdeep detection as it was way too loose 2016-10-07 20:20:53 +02:00
Iglocska 503661a240 new: First implementation of the freetext feed pull 2016-10-07 17:33:54 +02:00
Andreas Ziegler 0e3fc2192e fix: export attributetags as Tag elements (like eventtags) 2016-09-29 16:53:04 +02:00
Cristian Bell 5be1e17bce Revert "fix: missing new TLDs in free text import, solves #1149 (#1574)" 
This reverts commit e3bb9d3a42.
 2016-09-27 16:38:35 +02:00
Cristian Bell e3bb9d3a42 fix: missing new TLDs in free text import, solves #1149 (#1574) 
* fix: missing new TLDs in free text import, solves #1149
 2016-09-27 15:53:43 +02:00
Iglocska 9b7191f878 fix: Don't show the org restriction of a tag in the event view JSON 2016-09-27 09:38:32 +02:00
Andreas Ziegler a6e93d6020 chg: update cakephp to 2.8.9 (#1560) 2016-09-23 04:36:26 +02:00
iglocska f6187f8fa5 fix: Fallback to insecure random for php 5.x if the random_compat submodule isn't loaded 2016-09-18 16:11:33 +02:00
iglocska 62a2211a23 Merge branch '2.4' into 1457 2016-09-18 13:06:03 +02:00
iglocska a599ec24f7 Merge branch '2.4' into 1501 2016-09-18 11:07:10 +02:00
Iglocska 6d822ee45e fix: Refactor of the bro export to always create a zip archive with separate files if "all" types are queried 2016-09-16 16:49:54 +02:00
Iglocska 1991f7a208 fix: Some changes to the bro export 
- moved the whitelisting out of the plugin
- source now contains the instance host org name (if applicable), the event UUID and the creator org name
 2016-09-16 14:55:25 +02:00
Iglocska 2cede15e68 Merge branch '2.4' into feature/bro-export 
Conflicts:
	app/Model/Event.php
 2016-09-15 18:00:25 +02:00
Iglocska 40626963cc chg: Cleanup of removed Hids and Nids BroExport libraries that got merged into BroExport.php 2016-09-15 17:45:51 +02:00
Iglocska 59ecf40f42 chg: Refactor of the Bro export 2016-09-15 17:44:59 +02:00
Andreas Ziegler 25e52a6786 chg: remove some references to variables 2016-09-15 17:08:58 +02:00
Andreas Ziegler 72730e54ef new: add Tool for random string generation 2016-09-15 17:07:12 +02:00
Andreas Ziegler b3c5e56b38 new: add compatibility Lib for random_int 2016-09-15 17:07:12 +02:00
Iglocska 85879e735c chg: Reverted the changes to the NIDS export 2016-09-15 16:29:30 +02:00
Andreas Ziegler 8d8227690e chg: update cakephp to 2.8.7 2016-09-13 01:46:03 +02:00
Iglocska 01695e326a new: Added the metadata flag to the event restsearch API 
- allows fetching metadata only without including attributes/proposals
 2016-09-12 12:09:19 +02:00
Andreas Ziegler 4b8a82098d chg: replace 4 spaces after tab by double tab 2016-09-05 00:45:51 +02:00
iglocska d85fd0d813 fix: Fixed a newly introduced bug that breaks the NIDS exports, as referenced in #1489 2016-09-01 14:44:03 +02:00
ppanero 131e2f760a bro export funtionality 2016-08-29 17:26:14 +02:00
iglocska 5a72f84c22 Merge branch '2.4' into 2.4.51 2016-08-28 21:08:02 +02:00
iglocska 8f528ae881 fix: Removed incorrect uses of pass by reference, fixes #1472 2016-08-24 09:50:19 +02:00
iglocska 37297c2e15 Merge branch '2.4' into 2.4.51 2016-08-23 00:26:25 +02:00
Andreas Ziegler 30fb4e2b2e chg: remove whitespace at end of line 2016-08-22 02:54:51 +02:00
iglocska 3c0f3fb8bb Merge branch '2.4' into 2.4.51 2016-08-21 22:59:30 +02:00
Andreas Ziegler f0905dc536 chg: rename FileAccess to FileAccessTool 
every other tool classes name in the Lib/Tools/ folder also ends with "Tool"
 2016-08-19 19:25:32 +02:00
Andreas Ziegler a2ff5424e1 chg: change FileAccess from static to instantiable class 2016-08-19 19:22:15 +02:00
iglocska 444171bd2d Merge branch '2.4' into sslclientsync 2016-08-18 09:58:52 +02:00
Andreas Ziegler 9f952b595c chg: update cakephp to 2.8.6 2016-08-17 19:24:23 +02:00
iglocska b9f5297b3a fix: Fixed some issues with the misp export importer and added better logging. 2016-08-10 12:06:36 +02:00
iglocska 05fea819c2 fix: Some cleanup in the freetext tool 2016-08-08 17:32:01 +02:00
Richard van den Berg 81a5838131 Add support for sync server SSL client certificates 2016-08-01 16:30:22 +02:00
Iglocska fc9c0dcfe5 fix: Aligned freetext import with the changes to the attribute resolution 2016-07-26 11:42:38 +02:00
Iglocska 09ea3ab828 fix: Fix virustotal detection for the freetext import tool, fixes #1373 
- regex currently looks for https://www.virustotal.com, but https://virustotal.com is also valid
 2016-07-19 11:29:23 +02:00
Andreas Ziegler 78e8371608 chg: coding conventions in FileAccess.php 2016-07-04 22:54:35 +02:00
Iglocska b80cc56ec9 Merge branch '2.4' into write 2016-07-04 19:33:45 +02:00
Raphaël Vinot 1fa1777e89 Update testing 2016-07-01 14:32:04 +02:00
Iglocska a129c34de0 fix: removed some useless loops, fixes #1231 2016-06-10 16:17:31 +02:00
Andreas Ziegler aec73ed50a chg: improve file access using new Lib 2016-06-07 00:21:14 +02:00
Andras Iklody 5bd341b450 Merge pull request #1230 from rotanid/bugfix2 
fix: brace ordering
 2016-06-06 17:32:31 +02:00
Andras Iklody 25833a48fb Merge pull request #1233 from rotanid/cleanup-variables 
chg: remove obsolete variables
 2016-06-06 17:31:45 +02:00
Andras Iklody 54f9415e48 Merge pull request #1229 from rotanid/bugfix1 
fix: dont override type variable
 2016-06-06 17:28:10 +02:00
Andreas Ziegler 44b7e93df6 chg: remove obsolete variables 2016-06-06 17:19:46 +02:00
Andreas Ziegler cadda1ae45 chg: remove obsolete files 2016-06-06 17:18:26 +02:00
Andreas Ziegler 81709b4395 fix: brace ordering 2016-06-06 16:43:25 +02:00
Andreas Ziegler f1c79ed4a6 fix: dont override type variable 2016-06-06 16:42:16 +02:00
Andreas Ziegler 57c1a71066 fix: case-sensitive functions calls 2016-06-06 16:32:56 +02:00
Andreas Ziegler 1d06f25b38 chg: add newline character before EOF to non-minified (text-)files 2016-06-06 10:09:55 +02:00
Andreas Ziegler 958aa7c414 use consistent spacing around else if 2016-06-04 15:49:54 +02:00
Andreas Ziegler 7cadf8340c remove space after unset before opening brace 2016-06-04 15:45:57 +02:00
Andreas Ziegler 985451642e add space after keywords if/for/foreach/while/switch/catch 2016-06-04 15:45:39 +02:00
Andreas Ziegler 8f9e152d8c add space before opening curly brackets 2016-06-04 15:45:11 +02:00
Andreas Ziegler c1eda1e04b remove single spaces after tabs 2016-06-04 01:54:19 +02:00
Andreas Ziegler 0fe692c56a remove whitespace at end of line 2016-06-04 01:10:45 +02:00
Andreas Ziegler 898ea1d97c remove whitespace (space/tab) from empty lines 2016-06-04 01:08:16 +02:00
Andreas Ziegler dc0974a55b misc cleanup 2016-05-21 05:10:49 +02:00
Andreas Ziegler 7ae4c37f0b progressive removal of commented out if-statements 2016-05-20 00:48:54 +02:00
Iglocska 2b1d352073 fix: resolved commented out request type checks, fixes #1141 2016-05-19 08:33:33 +02:00
Iglocska f4b7c101e5 chg: Attribute search download also offered as JSON, fixes #1035 
- also added some convenience functions for JSON/XML collections in the appropriate export tools
- can start reusing them in other functionalities
 2016-05-02 10:31:40 +02:00
Iglocska 55f20d5381 chg: Small fix to the headers sent for SMIME 2016-04-27 11:05:00 +02:00
Iglocska 8db889ce7e SMIME changes 
- tied into auto upgrade system
- tied into server settings
- some cleanup of overly verbose debug
- Enforcing enable/disable everywhere
- Changed temporary file structure
 2016-04-26 16:40:12 +02:00
Iglocska 163f86ce35 Fix to an invalid check 2016-04-22 21:50:05 +02:00
Iglocska 1accaa2fee Small tune to the freetext import 
- url vs filename differentiation still being a headache
- will need a more thorough look
 2016-04-22 16:40:13 +02:00
Iglocska 681e8b5f72 Fixed the IDS flag default setting for freetext-imported virus total links 2016-04-20 17:02:46 +02:00
Iglocska 47e5d382ff Fixed several invalid detections in the freetext import tool 
- Composite filename|hash types were incorrectly detected as hash types
 2016-04-20 10:55:56 +02:00
Iglocska 4d57149e75 Freetext import tuning 
- refanging of various . notations
 2016-04-20 10:04:36 +02:00
Iglocska 24c7fa61fe Merge branch 'permissionfix' into 2.4 2016-04-18 17:41:59 +02:00
Iglocska 968fb75165 Pretty print event JSONs 2016-04-18 10:06:16 +02:00
Iglocska e826f98163 Fix to an issue with the freetext import tool 
- Due to a typo 64 character long hashes could not be correctly added via the freetext import tool
- Should be fixed now.
 2016-04-12 08:55:32 +02:00
Iglocska 3c98d3fa9b JSON structure inconsistencies and bug, fixes #1065 2016-04-08 16:47:22 +02:00
Iglocska f4a1a3283d CakePHP update 2016-04-08 08:55:14 +02:00
Iglocska bb372c5f7e Better sanitisation of the XML exports 2016-04-07 14:30:00 +02:00
Iglocska 0c316fd2e2 Reworked the Tag add/remove APIs 
- new syntax
- old syntax still accepted

- new tool for rearranging request data to allow the APIs to automatically catch and correct typical rearrange errors
 2016-03-30 11:05:06 +02:00
Iglocska b3af1d0463 Some refactoring of the freetext tool 2016-03-29 23:03:01 +02:00
Iglocska 18ce6872d4 Handling of the "freetext" return format via the enrichment modules, and error handling fixed 
- freetext is now a valid return format, it will allow module developers to return an unparsed text blob which MISP will try to loop through the freetext import's detection mechanism
- still a lot of improvements to be done for the detection mechanism

- error handling for modules, instead of discarding errors they are now shown as a flash message on the freetext import result screen
 2016-03-29 20:05:50 +02:00
Iglocska 9f5cb88aca Correctly detect e-mail addresses in the freetext import tool 2016-03-11 16:02:38 +01:00
Iglocska 255c65942e Further progress on the feeds 2016-02-29 22:32:04 +01:00
Iglocska 75a8b1adc4 Better detection of the proxy settings not being set 2016-02-23 16:44:07 +01:00
William Robinet 4fea371c4b Fix permissions 2016-02-11 17:03:51 +01:00
Iglocska c2c41b04d3 Fixed an issue with the freetext import 
- url detection would detect any word with a trailing "." as a valid url
  - google. was detected as a url
- this also caused training "."s to be included in valid urls
  - http://www.google.com.
 2016-01-23 20:19:44 +01:00
Iglocska 427da7d579 Removed lowercasing of parsed strings in the freetext import 
- case sensitive values also got lower-cased
 2016-01-18 15:24:48 +01:00
Iglocska 176ad85b88 Rework of the scheduled caching jobs 
- fixed a series of issues with the exports
 2016-01-10 19:45:33 +01:00
Raphaël Vinot 7b401141f2 Update travis for 2.4 2016-01-04 18:11:56 +01:00
Iglocska 5d7aa73e27 First version of the quick filters for the event view 2016-01-04 10:23:07 +01:00
iglocska 627f9abbd6 Fix to several issues with the sync and and an issue preventing the editing of events, fixes #788, fixes #784 2015-12-24 15:22:05 +01:00
Raphaël Vinot dcc182fa74 Update cakephp 2.7 to HEAD 
Fix #740
 2015-12-17 17:18:07 +01:00
Iglocska b40e0fdc7c Merge branch 'master' into 2.4-syncrework 
Conflicts:
	VERSION.json
	app/Controller/AttributesController.php
	app/Controller/ShadowAttributesController.php
	app/Lib/Tools/ComplexTypeTool.php
	app/Model/Attribute.php
	app/View/Pages/administration.ctp
 2015-12-09 02:00:23 +01:00
iglocska fbd97df3ac Several fixes, among others fixes #748 
- Double sanitisation when edditing an attribute/proposal comment removed
- Fixed an issue where an ip/resource was recognised as a CIDR notation IP range instead of a url
- Changed the flash message for publishing without e-mails to something less scary
 2015-12-08 15:12:13 +01:00
iglocska 65faeb48d9 Further tweaks 
- fixed some corner cases
- added support for the same defanging to the freetext import tool
 2015-12-04 11:33:14 +01:00
iglocska 744cf50fb9 Update to attribute validation and the freetext import tool, fixes #742 
- defanged URL type attributes are refanged on input
- admin script to do the same for all existing attributes

- admin tool doesn't recognise a word followed by a . as a url
 2015-12-04 10:43:38 +01:00
Iglocska d433618c71 Also, enabled the filtering on pull 
Merge branch 'master' into 2.4-beta

Conflicts:
	VERSION.json
	app/Controller/EventsController.php
	app/Lib/Tools/XMLConverterTool.php
	app/Model/Event.php
	app/Model/Server.php
 2015-12-03 00:27:56 +01:00
iglocska 268c7683a0 Rework of the event add/edit 
- allows for saving an event even if an attribute fails
  - logs attributes that fail validation

- same for edit

- add_misp_export updated with the above in mind
 2015-12-01 15:39:12 +01:00
Iglocska 5dbbe84069 Merge branch 'master' into 2.4-beta 
Conflicts:
	VERSION.json
	app/Controller/EventsController.php
	app/Controller/ShadowAttributesController.php
	app/Model/Event.php
	app/View/Elements/side_menu.ctp
 2015-11-30 09:37:22 +01:00
iglocska 9eb5680ee8 Reimplementation of the Add XML feature 
- called Add MISP export now
- can be an XML / JSON file
- result browser with explanations of failures

- REST XML/JSON add/edit of events returns errors instead of the partially succeeding event
 2015-11-30 02:28:07 +01:00
iglocska 8cac54fd66 Updated CakePHP version to 2.7.7 2015-11-26 18:36:25 +01:00
Iglocska 0572b2030e Further work on the taxonomies 
- colour coding
- filters on the index
- mass tag creation
 2015-11-26 04:31:24 +01:00
iglocska b5857696e9 Update to the Taxonomies 2015-11-24 12:02:39 +01:00
iglocska 97f6da18da Added file as an option when a url like google.com is recognised 2015-11-22 17:50:22 +01:00
iglocska c71c8f968d Fix to a bug in the financial tool's validation router 
- it didn't use the validation type -> validation method array to call the validation function
- resulted in CC validation not being called as expected
 2015-11-17 22:25:37 +01:00
iglocska db359170f6 some left over merging issues among other things 2015-11-17 22:01:22 +01:00
iglocska 485c007b39 Merge branch 'master' into 2.4-beta 
Conflicts:
	VERSION.json
	app/Lib/Tools/XMLConverterTool.php
	app/Model/Event.php
	app/Model/EventTag.php
	app/Model/TemplateElementAttribute.php
	app/Model/TemplateElementFile.php
	app/Model/TemplateElementText.php
	app/Model/ThreatLevel.php
	app/View/Attributes/index.ctp
	app/View/Elements/eventattribute.ctp
	app/View/Elements/eventattributerow.ctp
	app/View/Elements/global_menu.ctp
	app/View/Elements/side_menu.ctp
	app/View/Events/automation.ctp
	app/View/Events/index.ctp
	app/View/Pages/administration.ctp
	app/View/ShadowAttributes/index.ctp
	app/View/Tags/index.ctp
 2015-11-17 01:14:51 +01:00
iglocska 053c27ae9a Removed a crappy solution to an issue with attributes being overwritten that was fixed a long time ago correctly on data entry 2015-11-16 19:51:38 +01:00
iglocska 45f0e04738 Warning icon if a financial indicator fails the validation 2015-11-09 13:54:38 +01:00
iglocska 1d3ec3afa3 Merge branch 'feature/sg' of https://github.com/MISP/MISP into feature/sg 2015-10-22 10:01:25 +02:00
iglocska ef1d3949e7 Merge branch 'master' into feature/sg 
Conflicts:
	VERSION.json
	app/Controller/ShadowAttributesController.php
	app/Lib/Tools/JSONConverterTool.php
	app/Lib/Tools/XMLConverterTool.php
	app/Model/User.php
	app/View/Elements/eventattribute.ctp
 2015-10-22 09:59:00 +02:00
iglocska 61e865956b Fixes to several issues, fixes #693 
- Fixed a critical bug in the XML export
  - As of recently XML exports include relations as they were missing before
  - the sanitisation of the event info field in related attributes was incorrectly sanitized of unicode characters
  - this can lead to the XML export breaking and also for affected events to be blocked from synchronisation

- Proposal fixes
  - fixed an invalid uuid generation that lead to an exception
  - fixed the attachments for proposals still using the old attachment system that disallows most filenames
  - added the automatic creation of hashes for attachment proposals
 2015-10-21 23:44:07 +02:00
iglocska ae4d0af532 Fix to the cc validator 2015-10-18 22:51:40 +02:00
iglocska 38f5c443d3 Debug removed 2015-10-18 22:25:54 +02:00
iglocska 9ea162aece Fix to the financial tool (incorrect CC validation) 2015-10-18 22:24:05 +02:00