Commit Graph

369 Commits (f84f3042545a901c558cfe53add287630b42f4ed)

Author SHA1 Message Date
mokaddem 09646bf913
chg: [eventGraph] Renamed `rotation key` to `pivot key` and do not
collaspe when adding/removing edges.
Fix #3683
2019-10-09 15:59:16 +02:00
mokaddem 015ec7d989
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-10-08 14:12:38 +02:00
garanews 85c28ce36e Fix some typo
Fix some typo
2019-10-04 13:02:59 +02:00
iglocska c28314076b
fix: [API] massive performance boost for large events with many correlations
- the logic of the JSON converter was heavy and unnecesary
2019-10-04 12:26:49 +02:00
mokaddem b44b369eec
Merge remote-tracking branch 'origin/2.4' into zoidberg-timeline 2019-10-02 14:35:00 +02:00
mokaddem bb3cf85776
Merge branch '2.4' into zoidberg-timeline 2019-10-02 11:23:04 +02:00
Jakub Onderka bc8aa0e2b4
new: [internal] Allow to use custom CA 2019-09-26 14:26:58 +02:00
Jakub Onderka a9f6af9fcb chg: [user] GPG key fetching by server 2019-09-23 22:09:02 +02:00
Andras Iklody d10495f789
Relaxed defanging rules, fixes #5203
Removed multiple dot implosion for links/urls
2019-09-23 14:26:42 +02:00
iglocska 58148c0aa3
new: [Internal] Index generator refactor
- loads of new features added
- bunch of helpers updated
- mainly a back/forward port from the frozen feed-rework branch with some custom changes
2019-08-28 17:02:43 +02:00
Bechkalo Evgeny f2e299ba19 fix: error during creating and deleting Attributes on PostgreSQL 2019-07-23 13:46:06 +03:00
Bechkalo Evgeny 672601f7b8 fix: MariaDB error for quoting tablename 2019-07-23 11:58:18 +03:00
Bechkalo Evgeny 76d58e6e53 fix: some PostgreSQL issues
Closes: #3066, #3067
Fixes issues:
- wrong boolean and smallint conversion;
- postgresql table and field naming (field 1_event_id is wrong name for
field for example);
- postgresql grouping (you cannot select columns without grouping them);
- wrong checkbox rendering without keyword.
2019-07-23 11:17:56 +03:00
mokaddem e7f3d0d9df new: [timeline/*-seen] Initial import of the timeline code from the
zoidberg branch
2019-06-13 09:16:34 +02:00
mokaddem 93673b4d4c chg: [distributionNetwork] Filter out organisations not being marked as
local. Fix #4568
2019-05-03 15:32:02 +02:00
iglocska 69c6562888 fix: [freetext] Also trim out no-break spaces
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
░░░░░░░░███████████████░░░░░░░░░░
░░░░░░███████████████████░░░░░░░░
░░░░░███░░░░░░░░░░░░░░████░░░░░░░
░░░░██░░░░░░░░░░░░░░░░░░░███░░░░░
░░░██░░░░░░░░░░░░░░░░░░░░░███░░░░
░░██░░███████░░░░░░██████░░██░░░░
░██░░██─────██░░░░██────██░░██░░░
░██░░█▄▄▄▄▄▄▄██░░░█▄▄▄▄▄▄██░░██░░
░██░░████─────█░░░████────█░░░██░
░██░░█────────█░░░█───────█░░░██░
██░░░██──────██░░░██─────██░░░░██
██░░░░████████░░░░░███████░░░░░██
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░███████████████░░░░░░░░░░░█
█░░░████░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██
░██░░░░░░░░░░░░░░░░░░░░░░░░░░░░█░
░░███░░░░░░░░░░░░░░░░░░░░░░░░░██░
░░░░██░░░░░░░░░░░░░░░░░░░░░░░██░░
2019-04-26 09:39:10 +02:00
iglocska 8076dbfad1 fix: [refanging] Removed invalid pattern 2019-04-09 15:51:13 +02:00
iglocska 5eecd75e5b fix: [bug] Typo causing "\" to be stripped from attributes where it shouldn't be stripped 2019-04-09 15:48:29 +02:00
iglocska a3381b8196 new: [refanging] Attributes automatically refanged in beforeValidate, fixes #4442 2019-04-09 14:53:39 +02:00
iglocska 510b781762 fix: [freetext] Stop parsing dates as phone numbers 2019-03-20 13:10:53 +01:00
Nikos Filippakis 9d59b10368 Publish events to Kafka
Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2019-03-18 15:53:22 +01:00
mokaddem b8334521d3 chg: Updated comments 2019-03-15 11:52:37 +01:00
mokaddem d67af3c4dd fix: [distributionGraph] Transform associative array into regular array 2019-03-12 10:59:51 +01:00
mokaddem 8a5cce8cb3 chg: [distributionNetwork] Improved consistency in event index and
improved UX - WiP
2019-03-12 10:47:16 +01:00
mokaddem ead0b96e13 chg: [distributionGraphNetwork] Adjusted gravitationalConstant and mass 2019-03-01 14:55:51 +01:00
mokaddem 24d8f197ef new: [DistributionGraph] Added pie chart on sharing group. fix #4101 2019-03-01 10:56:33 +01:00
mokaddem ef045e01b3 chg: [distributionGraph] Added support of sharing group - WiP 2019-02-27 15:34:20 +01:00
mokaddem 431529c81c chg: [attackMatrix] UI: improved color scale - WiP 2019-02-11 17:54:29 +01:00
iglocska 4dd53eb8e3 fix: [freetext import] Handle cases where a value can be both a hash and a btc address better 2019-01-22 08:58:02 +01:00
Daniel Roethlisberger 5b4079637a new: [attributes] Add cdhash attribute, 40+ digit hash, default Payload delivery, ids=1 (#3965) 2018-12-19 20:19:49 +01:00
iglocska a28909d366 new: [freetext] Added BTC recognition, fixes #3864 2018-11-13 12:06:00 +01:00
Hannah Ward 608ddaa969
fix: aws would error if asked to del non-existing 2018-10-26 11:01:44 +01:00
Steve Clement a699c5fcd4 Merge branch '2.4' into py-virtualenv 2018-10-02 20:55:46 +08:00
iglocska 58b9a3c50b fix: [graph] Made the correlation graph aware of the new correlation loading 2018-10-02 07:35:34 +02:00
Steve Clement df3d87b5c8 Merge branch '2.4' into py-virtualenv 2018-09-28 15:00:46 +02:00
Steve Clement 05e0c412ef chg: [fix] Some fixed to the python virtualenv tweaks 2018-09-28 14:58:55 +02:00
Sami Mokaddem b66eeefffa fix: [eventGraph] prevents bug if object has no attributes 2018-09-28 12:05:46 +02:00
iglocska cefab3e01c new: [freetext import] Added detection for AS 2018-09-23 11:43:55 +02:00
iglocska 48c6150257 new: [Complex type tool] Detection of [1] style refanging 2018-09-21 15:08:33 +02:00
iglocska bc9524c712 fix: [feeds] Feed caching generates a lot of notices 2018-09-07 13:29:26 +02:00
iglocska 57a6460d81 chg: [internal] JSONConverterTool's support for the deprecated showorg flag removed 2018-09-03 17:53:51 +02:00
iglocska 5f1edc9bad Merge branch 'feature/api_rework2' into 2.4 2018-08-31 13:37:41 +02:00
iglocska f3558fb18a Merge branch 'feature/api_rework' into 2.4 2018-08-31 12:58:44 +02:00
iglocska 7ad08d9553 new: [API] Added the new XML converter 2018-08-30 21:54:47 +02:00
Daniel Roethlisberger b19e405b32 Fix broken timestamps by using 24 hour clock and ISO 8601 date format
The event view shows a wrong "Last change", e.g. "2018/08/23 06:01:45"
for "2018/08/23 18:01:45".  The same problem affects the timestamp in
the XML generated by IOCExportTool.php.  Fix by correcting the PHP
date() code "h" to "H".

While here, also switch to a clearer ISO 8601 date representation for
"Last change", using dashes instead of slashes for separation of year,
month and day.
2018-08-23 18:39:01 +02:00
iglocska f675fb8b29 Merge branch '2.4' into feature/api_rework 2018-08-17 14:49:09 +02:00
Hannah Ward c883a7b6d6
new: Add upload/download for attachments 2018-08-15 14:07:44 +01:00
Hannah Ward 597802501f
new: Add S3 client class 2018-08-15 11:14:03 +01:00
iglocska 32f79d2eab fix: [cleanup] Fixed an assignment in a comparison 2018-08-05 19:22:07 +02:00
iglocska a81894f14c chg: [CS] Changed to PSR-2
- to make contributions easier, adopted PSR-2
- used php-cs-fixer to rework the style
- *sniff sniff* Goodbye tab indentation
2018-07-19 11:48:22 +02:00
Andras Iklody f0964c11f6
Merge pull request #3479 from FloatingGhost/feature-send-logs-to-elasticsearch
Feature: send logs to elasticsearch
2018-07-12 12:09:53 +02:00
iglocska 53f974895e fix: [zmq] Fixed execution of the ZMQ start/stop commands still being python 2 2018-07-12 08:01:41 +02:00
iglocska 90a8e9110b fix: [freetext] parser was detecting any number as a phone number, fixes #3469
- new requirement: must start with + or contain a -
2018-07-11 15:02:30 +02:00
Hannah Ward a62b23088e
fix: Use spaces entirely 2018-07-10 17:06:56 +01:00
Hannah Ward 09aacabe26
fix: Indentation on ES client 2018-07-10 17:05:42 +01:00
Hannah Ward a70f8e45d8
new: Add ability to log to elasticsearch 2018-07-10 17:01:57 +01:00
iglocska 898aef2e10 new: [data model] Added support for monero - new type xmr
- soft validation
- secondary validation with warnings for malformed addresses
- supporting epic facial hair styles
2018-07-04 10:22:58 +02:00
iglocska 59b17b5af6 new: [sync] Added flag to avoid using the proxy
- in some cases you have internal sync between instances in which case going through the proxy is silly
2018-07-02 16:56:50 +02:00
Sami Mokaddem e3988c73ad new: [attackMatrix] Also consider attack galaxy at event level in the
heatmap
fix: [attackMatrix] Typo in ATT&CK + division by 0 in gradiendTool
2018-06-18 14:51:29 +00:00
Sami Mokaddem 8d145086f0 new: [attackMatrix] statistic about attack tags used in the instance
chg: [attackMatrix] moved functions in to model and matrix view into elements
2018-06-18 09:58:20 +00:00
iglocska ea389ef3ce fix: Edge case with empty objects caused *barf* 2018-05-08 15:02:22 +02:00
Sami Mokaddem f1cefb3503 fix: [DistributionGraph] include metadata for all distribution level
When fetching distribution graph data, returns information about all
distribution level (even not concerned).
2018-05-08 07:19:51 +00:00
Sami Mokaddem 9c5b05a679 fix: Directly take the sharing group name from the event
Do not fetch the sharing group name as it is already included in the
event.
+ fixed a css glitch
2018-05-04 12:40:35 +00:00
Sami Mokaddem d6f02fe1b6 fix: [DistributionGraph] incorrect number in the sg progressbar tooltip
Set the correct number of involved sharing instead of the sum of sharing
group in the sg progressbar tooltip
2018-05-04 12:00:55 +00:00
Sami Mokaddem 37f0281d59 fix: [DistributionGraph] sharing group search and uniqueness of results
fix a bug where filtering per sharing group was not inlcuding inherit
attributes.
Enforce uniqueness of involved entities.
2018-05-04 09:33:14 +00:00
Sami Mokaddem 507cd0ee85 chg: Trying not to break the MVC pattern
Server model is not passed to the constructor anymore, as well as the
Organisation model.
2018-05-04 06:27:54 +00:00
Sami Mokaddem 6d476814b0 Merge remote-tracking branch 'upstream/2.4' into distributionGraphDonut 2018-05-03 13:52:40 +00:00
Sami Mokaddem 3db21f0396 chg: [distributionGraph] support of the sharing group event distribution
chg: [distributionGraph] code cleanup
2018-05-03 12:55:00 +00:00
Sami Mokaddem 73e432f16d chg: sanitization of data for distribution graph 2018-05-03 08:42:53 +00:00
Sami Mokaddem 668488b8c9 chg: add additional distribution info about to whom we are sharing even
if we don't have element on this distribution level
2018-05-03 08:27:34 +00:00
Sami Mokaddem f4e4c7b335 chg: moved sharing group outside of the distribution progressbar (as it is a special case), distribution range is displayed when clicking on the pb labels and lots of minor improvements. 2018-05-02 13:37:48 +00:00
iglocska 8cc1d86068 fix: Allow filename as an alternative for parsed domains/hostnames 2018-04-25 15:41:58 +02:00
Sami Mokaddem 7a94612161 new: Possibility to view connected communities and concerned sharing groups in distribution graph's tooltip 2018-04-25 09:48:03 +00:00
Sami Mokaddem 0c593728de removed useless codes 2018-04-24 10:13:28 +00:00
Sami Mokaddem f9414871b8 Initial version of the distribution graph 2018-04-23 12:51:15 +00:00
Andras Iklody 607d203c04
Merge pull request #3170 from mokaddem/ref_graph
Extended event support and tag filtergin in the event graph
2018-04-20 13:40:48 +02:00
Sami Mokaddem 7ee1717628 Added confirmation box to draw the network based on a threshold 2018-04-20 11:20:52 +00:00
Sami Mokaddem 677f466c4e perf: unset filtered data instead of adding them to a new array (thus,
reducing memory consumption by a factor of 2)
2018-04-20 09:10:51 +00:00
Sami Mokaddem ee1e39360e Being consistent with indentation + removed useless comment. 2018-04-20 08:46:11 +00:00
Sami Mokaddem 2ca3515f10 Feature: Possibility to filter on tags 2018-04-20 08:35:38 +00:00
iglocska 8c2dd6d00a fix: Fix to the invalid refanging (Third time's the charm) 2018-04-18 11:25:58 +02:00
iglocska 5b62965e98 fix: fixed invalid refanging 2018-04-18 11:23:57 +02:00
iglocska d04f263c95 chg: Added [:] to the refanging options 2018-04-18 09:50:04 +02:00
Sami Mokaddem 303ff41ea1 feature: Better support of extended event in event graph - Added a
colored region for each event extending the current event scope
2018-04-17 15:23:28 +00:00
Sami Mokaddem f4ae1d4740 fix: bug when plotting event without attribute or object 2018-04-16 13:39:43 +00:00
Sami Mokaddem 368aa2f128 feature: Added support of extended event in event graph 2018-04-16 12:02:43 +00:00
Sami Mokaddem cd0c00384e Replaced scope rotation key typeahead by selector + removed trailling
spaces
2018-04-16 09:17:19 +00:00
Sami Mokaddem e2f4aade02 Added filtering based on authorized JSON key + JSON key is displayed in the header scope badge 2018-04-09 12:07:53 +00:00
Sami Mokaddem 09127a24c5 Support of graph per JSON key (using typeahead) 2018-04-09 11:39:45 +00:00
Sami Mokaddem 6ee5419297 feature: Draft of generic graphing from any key 2018-04-09 09:12:26 +00:00
Sami Mokaddem f3b2741843 feature: Support of Tags in the event graph 2018-04-06 14:44:40 +00:00
Sami Mokaddem 921224ed40 Merge branch 'quick-fix-metacategory-graph' into ref_graph 2018-04-06 07:50:27 +00:00
Sami Mokaddem fc168c5a35 Draft of filtering per attribute value 2018-04-05 14:21:40 +00:00
Sami Mokaddem 0ecccee108 Moved reference logique server-side + First draft of filtering capabilities 2018-04-05 10:31:26 +00:00
Sami Mokaddem 3933baf9c9 Compute graph serverside 2018-04-04 13:12:16 +00:00
iglocska 9485dfe5e2 chg: Refactor of the complex type tool
- makes it more readable
2018-04-03 22:25:52 +02:00
Sami Mokaddem 5e83caf8fb Added retreiving of object templates in order to let the user choose the field we want to see in the event graph 2018-03-29 16:05:19 +00:00
Sami Mokaddem 6a0abcce22 Renamed script again 2018-03-22 16:56:43 +00:00
Sami Mokaddem c78fca0ede Added possibility to edit references on the fly + edit objects on their dedicated webpage 2018-03-22 15:53:53 +00:00
root a3a6a77611 Initial references graphs commit 2018-03-19 08:44:25 +00:00