MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
23,820 Commits
34 Branches
364 Tags
187 MiB
Commit Graph

710 Commits (e6001bc9fa81938e68042105f43a4002ae9455a4)

Author SHA1 Message Date
Sami Mokaddem 962754dd3b
chg: [overCorrelatingValue] Truncated the `value` column 
- We keep the unique constraint on the table
- Correlating values over the max. allowed size are truncated to fit the size requirement. That means large correlating values might be marked as over-correlating despite the fact they are not (as only the starting portion of the value is evaluated).
 2022-08-29 10:50:59 +02:00
Sami Mokaddem 8db8cbd398
new: [user:periodicNotification] Started development of system allowing users to receive period notifications by email 2022-08-29 09:03:42 +02:00
Jakub Onderka ec209a98b9 chg: [internal] Drop unnecessary indexes from default_correlations table 2022-08-11 12:54:49 +02:00
iglocska c764bb0e8f
Merge branch 'log_last_api' into develop 2022-08-11 09:36:30 +02:00
iglocska 097ad78d54
chg: [correlations] performance tuning 
- added a new constraint to check for correlation uniqueness (a_id, 1_a_id, value_id)
- stopped dropping correlations on a full recorrelation
- only correlate "upwards" towards higher IDs
 2022-08-09 14:41:59 +02:00
iglocska 9dc233bc9e
fix: [proposal] correlations are deprecated 2022-08-05 12:55:04 +02:00
Tom King 64d508b4e3 fix: Rollback change to DB upgrade 86 2022-08-05 09:41:05 +01:00
Tom King d2a7b4b3c8 fix: Use correct field for altering table to include last_api_access 2022-08-05 09:35:15 +01:00
iglocska ea0932b7b1
chg: [correlations] value field changed to varchar 2022-08-04 13:49:30 +02:00
Sami Mokaddem 79d182dc79
fix: [appModel:update] Fixed forgotten schema update for workflows 2022-08-04 11:08:19 +02:00
Sami Mokaddem 5074528d91
fix: [appModel:update] Set update success to true for case 89 and 90 2022-08-04 10:58:44 +02:00
Sami Mokaddem 49575533ad
Merge remote-tracking branch 'origin/develop' into feature-workflows-2 2022-08-04 10:08:56 +02:00
iglocska bce4508716
chg: [grammar] minor fix 
- fat finger induced typo
 2022-08-04 08:20:20 +02:00
iglocska a04f3964e7
new: [correlation rework] round 2 
- long list of fixes
- update scripts
- correlation engine management interface
- recorrelation/truncation tools
- various performance tunings and bug fixes
 2022-08-03 15:44:27 +02:00
Sami Mokaddem 44b2a1c3cd
new: [workflow:debugging] Added debugging capability by POSTing workflow exec to an URL 2022-08-03 11:45:17 +02:00
Tom King de351faaac new: [internal] Add option to log last API request 2022-08-01 15:02:49 +01:00
Sami Mokaddem d83039f520
fix: [appmodel] Fixed merged conflict and typo in sql query 2022-08-01 10:19:38 +02:00
Sami Mokaddem 151d23a8c2
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows-2 2022-08-01 09:59:44 +02:00
iglocska 3ce754d595
Merge branch 'develop' into new_correlations 2022-07-31 23:58:40 +02:00
iglocska 3a4289d631
new: [correlation] engine rewrite 
- allow for multiple concurrent engines
  - default: similar behaviour as before, ACL enforced
  - No ACL: for endpoint misps, disable the enforcement of ACL for correlations altogether

- rework:
  - correlation entries are fully indexed reference tables
  - values are now stored separately
  - built in protection against overcorrelating values (defaults to 20 max)
  - 1 way correlations to cut the size in half
  - unsigned IDs to double the ID space
  - loads of performance improvements
  - fix to the broken event index with correlation counts enabled

- UI improvements
  - search for values from the correlation column directly (in case there are non-correlating versions of the same value)
  - added correlations to the attribute search/index

- TODO:
  - upgrade scripts
 2022-07-31 23:48:38 +02:00
Sami Mokaddem a6f676a66c
chg: [workflow] Simplified call to executeTrigger 2022-07-27 10:43:09 +02:00
Sami Mokaddem 5e29f975d1
chg: [workflow] Enable some module by default when updating 2022-07-26 11:16:08 +02:00
Sami Mokaddem 2be9d07502
chg: [workflowBlueprint] Added new column `default` to recognize default BP 2022-07-19 11:48:12 +02:00
Sami Mokaddem 87b92109d2
chg: [workflow] Various fixes, improved enrichment support and new logic module 2022-07-15 11:47:08 +02:00
Sami Mokaddem 2a31408555
fix: [workflow] Create table sql statement updated 2022-07-14 14:07:35 +02:00
Sami Mokaddem 99a76812bc
Merge remote-tracking branch 'origin/develop' into feature-workflows-2 2022-07-07 09:09:39 +02:00
Sami Mokaddem dbcedbc505
chg: [workflow] Various improvement and added support of `enrichment_before_query` trigger 2022-07-05 11:19:18 +02:00
Jakub Onderka 640a732c29
Merge pull request #8358 from JakubOnderka/memory-leak-fix 
fix: [internal] PHP memory leak
 2022-06-29 13:32:26 +02:00
Sami Mokaddem 8b8055a625
chg: [WorkflowBlueprint] Renamed WorkflowPart into WorkflowBlueprint 2022-06-24 17:09:49 +02:00
Sami Mokaddem 351a3cfd4e
chg: [workflow] non-blocking workflows are run by background workers 2022-06-24 12:20:03 +02:00
Sami Mokaddem a13c1a39e9
chg: [workflow] Improved logging capabilities and stop aborting execution if non-blocking module return false 2022-06-24 11:22:26 +02:00
Sami Mokaddem df8f0f9ed0
new: [workflow_module:webhook] Added new webhook module 2022-06-22 13:38:19 +02:00
Sami Mokaddem 4e47782a04
new: [workflowPart] Started integration of workflow parts 2022-06-22 09:46:57 +02:00
Sami Mokaddem 07b99a1a5a
chg: [workflow] Added more logging while executing WF 2022-06-13 14:36:44 +02:00
Jakub Onderka 2a2b52f64e chg: [internal] Convert to const 2022-06-08 10:44:33 +02:00
Sami Mokaddem d8f8225b9e
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows 2022-05-30 14:37:36 +02:00
Sami Mokaddem 9e29830dfa
chg: [workflow] Better module loading and execution errors get propagated to the caller for blocking path 2022-05-30 14:34:45 +02:00
Jakub Onderka 6a4facf8ed chg: [internal] Faster Model::isUnique and Model::exists method 2022-05-29 16:21:25 +02:00
Jakub Onderka 715ca6d1e2 fix: [internal] PHP memory leak 2022-05-22 18:19:45 +02:00
Sami Mokaddem ce9b823cbb
chg: [workflows:trigger] Added support of execution order for blocking triggers 2022-05-19 12:02:35 +02:00
Luciano Righetti 217be89e00 new: add MysqlExtended DboSource to support index query hints 2022-05-19 11:08:31 +02:00
Sami Mokaddem 62e53f81f6
chg: [workflow] Started working on priority_level and trigger list - WiP 2022-05-16 14:44:54 +02:00
Sami Mokaddem c5af331bc5
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows 2022-05-16 10:50:47 +02:00
Sami Mokaddem 62107f14b9
chg: [workflows] Added enabled state 2022-05-16 10:47:55 +02:00
Jakub Onderka 2f7c671adb new: [internal] Simplify checking if connection is MySQL/MariaDB 2022-05-14 10:17:06 +02:00
Sami Mokaddem c477935752
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows 2022-05-11 13:03:03 +02:00
Luciano Righetti 53b0e4918b
Merge pull request #8331 from righel/add-attr-timestamp-index 
chg: add index for attributes.timestamp, show index diagnostics in di…
 2022-05-10 10:33:06 +02:00
Jakub Onderka 2294232442 fix: [internal] Strict types 2022-05-07 10:40:41 +02:00
Luciano Righetti a6e71bd649 fix: add missing break 2022-05-04 09:22:01 +02:00
Luciano Righetti 5fbb80d04d chg: add index for attributes.timestamp, show index diagnostics in diagnostics page 2022-05-04 09:04:30 +02:00
Sami Mokaddem 6670333f10
chg: [workflow] Added database migration 2022-05-04 00:00:19 +02:00
Tom King 5de04ecffa chg: Merge from develop 2022-04-20 08:07:36 +01:00
iglocska be9fb9e802
Merge branch 'develop' of github.com:MISP/MISP into develop 2022-04-19 15:18:40 +02:00
iglocska 2688961cc3
Merge branch 'develop' of github.com:MISP/MISP into develop 2022-04-18 17:59:34 +02:00
iglocska ccbd119417
chg: [internal] setupSyncRequest made public 
- to be able to access it via shell scripts
 2022-04-18 17:57:38 +02:00
iglocska 93821c0de6
fix: [security] Sanitise paths for several file interactions 
- remove :// anywhere we don't expect a protocol to be supplied
- remove phar:// in certauth plugin's fetcher

- as reported by Dawid Czarnecki of Zigrin Security
 2022-04-17 18:25:51 +02:00
iglocska 0108f1bde2
fix: [security] unregister phar from stream wrappers globally for all Model code 
- blanket protection against phar deserialization vulnerabilities

- as reported by Dawid Czarnecki of Zigrin Security
 2022-04-17 16:30:14 +02:00
Jakub Onderka 9e8608b3d3 chg: [jsonTool] Properly handle invalid JSON for PHP 7.2 and older 2022-04-10 09:38:25 +02:00
Jakub Onderka 9d8fc81678 chg: [internal] Throw exception if Redis class not found 2022-03-19 12:31:55 +01:00
Sami Mokaddem d65ef9c966
chg: [cryptographicKeys] Indexed more column and bumped db_schema 2022-03-17 14:43:01 +01:00
iglocska b80a7af2c3
new: [cryptographic keys] model and controllers added 
- sets MISP up for information signing
- sign data during synchronisation
 2022-03-13 12:35:12 +01:00
iglocska 870b9d761b
fix: [db schema] fixed 2022-03-04 16:10:12 +01:00
iglocska 7834ec3760
fix: [organisations] made meta fields default to '' and not allow null values 
- fixes a filtering issue with sharing group blueprints leading to sharing groups that are more restrictive than expected
 2022-03-02 17:32:35 +01:00
iglocska 639a4929e3
new: [sharing group blueprints] 
- create a rule based blueprint that is used to create and update a sharing group
- nest sharing groups
- filter organisations by metadata fields
- nested via boolean operators
- CLI exposed
- API exposed
- Lightweight ownership model (only blueprint owner can see and edit the blueprint)
 2022-03-02 02:09:20 +01:00
Tom King 5f618ed0ca chg: Update from upstream develop 2022-02-17 10:41:58 +00:00
Jakub Onderka bd9cecbc2c chg: [internal] Use ProcessTool in Sighting 2021-11-22 09:57:39 +01:00
Jakub Onderka 0cc7804219 chg: [internal] Current branch and commit checking 2021-11-14 19:09:38 +01:00
Jakub Onderka a424c01844 chg: [internal] Small optim 2021-11-14 19:09:38 +01:00
Jakub Onderka db3183ae54 chg: [internal] Faster way how to get current commit 2021-11-14 19:09:38 +01:00
Jakub Onderka 553036de94
Merge pull request #7955 from JakubOnderka/code-style-background-jobs 
chg: [internal] Code style
 2021-11-12 11:48:28 +01:00
Jakub Onderka fb99021ab9 chg: [internal] Code style 2021-11-12 09:44:21 +01:00
Jakub Onderka 6806cdf574 new: [user] Add sub field for user 2021-11-11 16:34:27 +01:00
Luciano Righetti ba0399a6fb chg: remove MISP.use_simple_background_jobs setting in favor of SimpleBackgroundJobs.enabled 2021-11-10 16:44:02 +01:00
Luciano Righetti 9e23a2744f chg: merge develop, fix conflicts 2021-11-09 14:23:51 +01:00
Luciano Righetti 4218787934 chg: move initTool() logic to constructor 2021-11-09 08:43:39 +01:00
Jakub Onderka 5c5f2b54ca
Merge pull request #7938 from JakubOnderka/authkeys-encrypted-vol2 
Authkeys encrypted vol2
 2021-11-09 08:36:44 +01:00
Luciano Righetti 96a46b6899 chg: merge develop, fix conflicts. 2021-11-08 14:37:27 +01:00
iglocska dc925a321f
Merge branch 'develop' of github.com:MISP/MISP into develop 2021-11-08 14:17:01 +01:00
iglocska 952a98dba8
Merge branch 'local_only' into develop 2021-11-08 14:15:18 +01:00
Luciano Righetti f2c9d12eae chg: merge develop, fix conflicts. 2021-11-08 11:35:20 +01:00
Jakub Onderka 5569d7d2bf new: [security] Store authkeys for servers encrypted 2021-11-07 17:51:04 +01:00
Jakub Onderka 3459a09943 chg: [internal] App model cleanup 2021-11-07 10:40:01 +01:00
Jakub Onderka c9b8564c0a
Merge pull request #7927 from JakubOnderka/system-setting 
System setting in database
 2021-11-06 16:15:23 +01:00
Luciano Righetti f80fcec2db chg: rename settings 2021-11-05 15:46:42 +01:00
Jakub Onderka c9597ba3e9 new: Store system settings in database 2021-11-05 09:19:56 +01:00
Luciano Righetti 309f7ed50e fix: wrong update query 2021-11-04 16:51:56 +01:00
Luciano Righetti 5242d3204d chg: rename conf name 2021-11-04 11:39:24 +01:00
iglocska a0e8acc45f
Merge branch 'develop' of github.com:MISP/MISP into develop 2021-11-04 11:05:32 +01:00
iglocska eddf95355a
fix: [database] upgrade script using mb4 defaulted to 255 key length 
- default should be 191
 2021-11-04 11:04:56 +01:00
Luciano Righetti 946bdb68e6 chg: add db update 2021-11-03 17:09:42 +01:00
Luciano Righetti b1eaed2b1b chg: merge develop, fix conflicts 2021-11-03 13:54:20 +01:00
Luciano Righetti 453418d6a5 fix: use class registry to get job class 2021-11-02 16:53:42 +01:00
Luciano Righetti cce5e581d4 fix: add missing jobId param to enqueue() calls 2021-11-02 16:25:43 +01:00
Luciano Righetti ae3300d5d0 chg: use new bg jobs tool in AppModel 2021-11-02 11:55:12 +01:00
Jakub Onderka f1283996ad chg: [internal] Optimise AuditLog 2021-10-31 10:52:47 +01:00
Jakub Onderka 9ae7d88f23 chg: [internal] Optimise Tag::findTagIdsByTagNames 2021-10-30 19:43:17 +02:00
Jakub Onderka 94808488ce
Merge pull request #7852 from JakubOnderka/optimise-sighting 
chg: [internal] Optimise sightings
 2021-10-29 21:25:42 +02:00
Jakub Onderka 0d8292289d chg: [internal] Optimise afterFind and simplify ISODatetimeToUTC 2021-10-28 13:56:27 +02:00
Jakub Onderka 6ade27a27c chg: [internal] Move attribute validation to different tool 2021-10-28 09:42:04 +02:00
Loïc Fortemps dc71f6e451
Merge branch 'develop' into local_tags 2021-10-27 17:06:02 +02:00