Commit Graph

81 Commits (2.4)

Author SHA1 Message Date
Jakub Onderka 3ccf211bae fix: [correlation] MISP.enable_advanced_correlations config option works again 2022-05-15 09:21:00 +02:00
Jakub Onderka a82de04e8c chg: [correlation] Reduce memory usage when generating all correlations 2022-05-15 09:20:43 +02:00
Jakub Onderka 6e4e4ab850 chg: [correlation] Code cleanup 2022-05-15 09:20:43 +02:00
Jakub Onderka 2aa2b7c79b fix: [correlation] Avoid duplicate correlation 2022-05-15 09:20:43 +02:00
Jakub Onderka 14ff781b5b chg: [correlation exclusion] Check both part of attribute 2022-05-15 09:20:43 +02:00
Jakub Onderka fc17263401 fix: [internal] Better check which value correlated 2022-05-15 09:20:43 +02:00
Jakub Onderka 2f7c671adb new: [internal] Simplify checking if connection is MySQL/MariaDB 2022-05-14 10:17:06 +02:00
Jakub Onderka 316df3a96c fix: [internal] Generate top correlations for all values 2022-05-09 14:59:07 +02:00
Jakub Onderka f2580dda7a chg: [internal] Top correlations cleanup 2022-05-09 14:34:38 +02:00
Luciano Righetti cce5e581d4 fix: add missing jobId param to enqueue() calls 2021-11-02 16:25:43 +01:00
Luciano Righetti 4d43789311 chg: use new bg jobs tool in correlation model 2021-11-02 15:35:23 +01:00
Jakub Onderka 497910b941 fix: [internal] Remove ssdeep data when deleting attribute 2021-09-03 16:38:49 +02:00
Jakub Onderka e55f8171d9 chg: [internal] Faster calls 2021-09-03 16:20:14 +02:00
Jakub Onderka cd0a2bdf4b chg: [internal] Faster deleting correlation when deleting attribute 2021-09-03 12:22:16 +02:00
Jakub Onderka 2c2d879996 chg: [internal] Use object variable and not Configure again and again 2021-09-03 11:58:02 +02:00
Jakub Onderka 645fb52b64 chg: [internal] Do not fetch 'Event.disable_correlation' field 2021-09-03 11:54:28 +02:00
Jakub Onderka 6642d8bf7b chg: [correlation] Allow to drop Correlation.{date,info} columns 2021-08-30 09:43:47 +02:00
Jakub Onderka 14b78904ba chg: [internal] Do not load exclusion list from Redis again and again 2021-08-12 15:05:39 +02:00
Jakub Onderka d4c2d10fe3 chg: [internal] Convert array to const 2021-07-27 16:43:27 +02:00
Jakub Onderka 6440133989 chg: [internal] Convert variable to const 2021-07-21 09:32:51 +02:00
Jakub Onderka 99070b73ef fix: [correlations] Correctly handle exclusion 2021-05-10 08:30:56 +02:00
Jakub Onderka 3b4a876c6a chg: [correlation] Cleanup Correlation model code 2021-05-10 08:30:42 +02:00
iglocska a0b1d0964e
chg: [correlations] reverted the division by 2 for the correlation counts
- there are legitimate cases where we get one way correlations
  - we use the value field to aggregate the count, which leads to it being incorrect when using advanced correlations (the reverse correlation will use the value of the remote side)
2021-04-27 08:41:41 +02:00
iglocska 81c2d01bcd
fix: [top correlations] Divide the count by 2
- Each correlation has 2 entries in the DB (A->B and B->A)
- this doesn't mean that we should count each of those entries, but rather divide by 2 to get the actual correlation count
2021-04-27 00:45:51 +02:00
iglocska 0d348bfd01
new: [Correlations] Added cached toplist
- stored via zset in redis
- very fast, but needs to be generated
- generation background processed
2021-04-27 00:40:40 +02:00
iglocska 3cbe5280d8
fix: [correlations] added fix for invalid function call
- introduced by the refactor, looking up Attribute object variables such as noncorrelatingTypes
2021-04-25 23:28:17 +02:00
iglocska 5b67cd147a
fix: [correlations] Don't barf when trying to add data with no correlations 2021-04-25 23:18:28 +02:00
iglocska 4ba1a1ab13
fix: [Correlations] fixed advanced correlations for ssdeep and separated into own function 2021-04-25 18:09:37 +02:00
iglocska 823a870ca0
Merge branch 'develop' of github.com:MISP/MISP into develop 2021-04-25 17:54:53 +02:00
iglocska 15fc60ebbf
new: [Correlations] refactor / rework
- moved to own controller and model
- refactored several long incomprehensible functions

- extracted reused tasks from functions and made them reusable
- added a way to correlate individual values as opposed to attributes

- Added top correlations index
2021-04-25 17:36:29 +02:00
iglocska 9e1b9f9b6e
new: [Correlations] added dedicated controller/model/views 2021-04-22 09:46:10 +02:00