MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
25,602 Commits
42 Branches
370 Tags
181 MiB
Commit Graph

768 Commits (3c163d0c12b233ae0d095b71c00bcb7fab6f7099)

Author SHA1 Message Date
iglocska 7d93f74b7b
new: [role permission] for viewing feed correlations 
- added additional role permission
- allows hiding feed correlations from users
  - main purpose is with very large instances, to reduce the load on redis
 2023-06-09 08:22:45 +02:00
iglocska 06973cf7cc
chg: [taxii] added collection field to taxii servers 2023-06-07 14:26:14 +02:00
Christophe Vandeplas cb74ad507f chg: [security] OTP support for HOTP 2023-05-25 23:28:14 +02:00
Christophe Vandeplas 6311f7d3e6 Merge branch 'develop' into feature/totp 2023-05-25 20:53:06 +02:00
Jakub Onderka a9f407bc10
Merge pull request #9057 from JakubOnderka/sentry 
new: [internal] Send exceptions to Sentry if enabled
 2023-05-24 10:40:09 +02:00
iglocska 2252d16c91
fix: [over-correlations] weren't truly case insensitive, causing potential issues matching and entering values 
- wrapped adding a new value in a try catch, no need to make synchronisations fail over this
- added case insensitive change to values on entry (table should be all lower-case)
- added update script to lowercase existing values
 2023-05-22 11:26:05 +02:00
Christophe Vandeplas 6caccac94d new: [security] TOTP authentication 2023-05-19 06:57:16 +02:00
Jakub Onderka 8606222352 new: [internal] Send exceptions to Sentry if enabled 2023-05-18 19:01:04 +02:00
Sami Mokaddem 8ec8f4b4f2
fix: [db:workflows] Changed workflows.data from TEXT to LONGTEXT 
Should fix issue  #8979
 2023-03-30 10:42:22 +02:00
Christophe Vandeplas 16a2713403 chg: [AuthKey] db change 2023-03-19 11:52:52 +01:00
iglocska 738fd95db2
fix: [taxii servers] invalid baseurl field type 
- copy pasta strikes again
 2023-02-24 02:00:16 +01:00
Jakub Onderka 2305c4d15e fix: [internal] Migration 105 2022-12-22 10:08:38 +01:00
Christophe Vandeplas bdda94529d chg: [roles] set default role to User if none is set 2022-12-06 10:39:53 +01:00
Jakub Onderka 097275ec96 fix: [log] Fetching remote IP address 
Fixes #8795 and #8788
 2022-12-02 09:50:18 +01:00
Jakub Onderka b3fd267105 new: [log] Add ability to log sql queries for access log 2022-12-02 09:39:05 +01:00
Jakub Onderka 8ea28a1385 fix: [db] Duplicate migration 2022-12-01 17:19:54 +01:00
Jakub Onderka 52e37972b1
Merge pull request #8796 from JakubOnderka/fix-cluster-attach 
fix: [internal] Attaching clusters
 2022-12-01 14:36:54 +00:00
Luciano Righetti 1d919354d2
Merge pull request #8794 from righel/highlighted-tags 
new: highlighted tags
 2022-12-01 15:32:25 +01:00
Jakub Onderka 4a76fae0ea chg: [test] Show application logs 2022-12-01 15:03:02 +01:00
Christophe Vandeplas bc4b1e6f49 fix: [logs] only allow for perm_audit & promote the perm to all 2022-12-01 10:49:50 +01:00
Luciano Righetti 2e7d1d30c1
fix: conflics and update db_schema.json 2022-11-30 16:00:17 +01:00
Christophe Vandeplas 192ed311b9 fix: [remote_ip] respect MISP.log_client_ip_header everywhere fixes #8781 2022-11-28 14:08:11 +01:00
iglocska dbc18f2ca7
fix: [logs] reverted the removal of api logs from the /logs/ logging system unless confirmed 
- breaks logging with existing configurations
 2022-11-28 13:27:40 +01:00
iglocska 65665928d5
fix: [updates] fixed invalid numbering 2022-11-28 12:41:50 +01:00
iglocska 24f656ac3f
new: [restsearch] added optional ordering 
- available on event/attribute restsearch
- uses the new findOrder() internal function to have consistent filtering
 2022-11-27 11:15:47 +01:00
iglocska db24a9bbf4
Merge branch 'develop' into taxii 2022-11-21 13:51:37 +01:00
iglocska f4ad815b26
Merge branch 'taxii' of github.com:MISP/MISP into taxii 2022-11-21 13:29:35 +01:00
Jakub Onderka f247571542 chg: [logs] Add SQL queries count to access log 2022-11-13 17:42:40 +01:00
Jakub Onderka b508674f2c fix: [logs] Remove support for elastic logging for auditlog, as it was broken and didnt work 2022-11-12 20:42:27 +01:00
Jakub Onderka 4aabc2d097 new: [logging] Access log 2022-11-12 13:45:21 +01:00
Jakub Onderka 4fe5a73386 chg: [internal] Use subquery to sighting fetching 2022-11-01 10:56:14 +01:00
Jakub Onderka 446cf12799 chg: [internal] Cleanup code for JSON decoding 2022-10-23 13:17:39 +02:00
Jakub Onderka 9770ee17a9 new: [correlation] Do not correlate over correlating value again for full correlation 
Should help with #8685
 2022-10-20 09:55:39 +02:00
Jakub Onderka aae65c42c6
Merge pull request #8653 from JakubOnderka/workflow-fixes 
fix: [workflow] Basic cleanup
 2022-10-18 21:20:40 +02:00
Jakub Onderka 21857cbfd1 new: [internal] Add support for simdjson extension 2022-10-18 18:39:08 +02:00
Jakub Onderka c5e2b39a9a chg: [api] Better specify what `last` attribute means 2022-10-17 16:59:23 +02:00
Jakub Onderka 47ab2bb72e chg: [internal] Speedup saving attributes when workflow is disabled 2022-10-10 22:12:32 +02:00
iglocska 684ab0537b
fix: [relationship_type] field made nullable 2022-10-03 12:40:40 +02:00
iglocska 1ba1db596d
Merge branch 'develop' into tag_relationships 2022-10-03 12:28:10 +02:00
iglocska 91cab11f7b
new: [tag] relationships added 
- add a relationship to any attributeTag / eventTag relationship
- works for both clusters and tags
- displayed on the event index/view
- included in the API

- new endpoint to modify the relationship via /tags/modifyTagRelationship/[scope]/[id]
  - scope is attribute/event
  - id is the id of the EventTag / AttributeTag object
 2022-10-03 12:14:46 +02:00
Jakub Onderka 51a399248c chg: [schema] Add missing index for object_template_elements:object_template_id column 2022-09-25 16:17:44 +02:00
Jakub Onderka fc0ceb8206 chg: [internal] Mark AppModel::convert_to_memory_limit_to_mb method as protected 2022-09-25 16:17:44 +02:00
Jakub Onderka c300b1aad2 chg: [internal] Mark some AppModel methods as private 2022-09-22 14:52:21 +02:00
Jakub Onderka 17981be04d new: [internal] RedisTool 2022-09-11 10:59:14 +02:00
Jakub Onderka f8b5b22aea
Merge pull request #8572 from JakubOnderka/correlation-value-transaction 
chg: [correlation] Faster saving correlations
 2022-09-10 20:44:29 +02:00
Sami Mokaddem 782bb69a1b
Merge remote-tracking branch 'origin/develop' into feature-periodic-notification 2022-09-09 14:06:16 +02:00
Jakub Onderka a52b8bba7f chg: [internal] Code cleanup 2022-09-08 15:37:36 +02:00
Sami Mokaddem f4262c081b
Merge branch 'develop' of github.com:MISP/MISP into feature-periodic-notification 2022-09-06 09:39:23 +02:00
Sami Mokaddem c0f96852e0
chg: [appModel:removeDuplicateAttributes] Provide additional log info if needed 2022-09-06 09:15:26 +02:00
Sami Mokaddem 902dc4713a
chg: [appModel:removeDuplicateUUID] Added support of sort_by 2022-09-06 09:14:36 +02:00
Sami Mokaddem 67d76d9b7a
Merge branch 'develop' into fix-duplicated-uuids 2022-09-05 12:40:29 +02:00
Sami Mokaddem 0799ff8c6c
chg: [appModel] Removed unused functions 2022-09-05 12:37:53 +02:00
Sami Mokaddem fd6c824172
Merge branch 'develop' of github.com:MISP/MISP into fix-duplicated-uuids 2022-09-05 12:33:38 +02:00
Sami Mokaddem bc6d4a8305
chg: [appModel] Usage of `update` worker instead of `prio` 2022-09-01 15:55:37 +02:00
Sami Mokaddem 646d921d67
Merge remote-tracking branch 'origin/develop' into tomking2_feature/propagate_tag_deletion 2022-09-01 11:07:35 +02:00
Sami Mokaddem 7b504fd133
Merge branch 'develop' of github.com:MISP/MISP into feature-periodic-notification 2022-09-01 09:58:41 +02:00
Sami Mokaddem c9327174b9
Merge branch 'fix-truncate-overcorrelating-value' into 2.4 2022-08-29 15:10:48 +02:00
Sami Mokaddem 05ad105750
fix: [update-91] Remove duplicates before creating the constraint 2022-08-29 12:12:39 +02:00
Sami Mokaddem 962754dd3b
chg: [overCorrelatingValue] Truncated the `value` column 
- We keep the unique constraint on the table
- Correlating values over the max. allowed size are truncated to fit the size requirement. That means large correlating values might be marked as over-correlating despite the fact they are not (as only the starting portion of the value is evaluated).
 2022-08-29 10:50:59 +02:00
Sami Mokaddem 8db8cbd398
new: [user:periodicNotification] Started development of system allowing users to receive period notifications by email 2022-08-29 09:03:42 +02:00
Jakub Onderka ec209a98b9 chg: [internal] Drop unnecessary indexes from default_correlations table 2022-08-11 12:54:49 +02:00
iglocska c764bb0e8f
Merge branch 'log_last_api' into develop 2022-08-11 09:36:30 +02:00
iglocska 097ad78d54
chg: [correlations] performance tuning 
- added a new constraint to check for correlation uniqueness (a_id, 1_a_id, value_id)
- stopped dropping correlations on a full recorrelation
- only correlate "upwards" towards higher IDs
 2022-08-09 14:41:59 +02:00
iglocska 9dc233bc9e
fix: [proposal] correlations are deprecated 2022-08-05 12:55:04 +02:00
Tom King 64d508b4e3 fix: Rollback change to DB upgrade 86 2022-08-05 09:41:05 +01:00
Tom King d2a7b4b3c8 fix: Use correct field for altering table to include last_api_access 2022-08-05 09:35:15 +01:00
iglocska ea0932b7b1
chg: [correlations] value field changed to varchar 2022-08-04 13:49:30 +02:00
Sami Mokaddem 79d182dc79
fix: [appModel:update] Fixed forgotten schema update for workflows 2022-08-04 11:08:19 +02:00
Sami Mokaddem 5074528d91
fix: [appModel:update] Set update success to true for case 89 and 90 2022-08-04 10:58:44 +02:00
Sami Mokaddem 49575533ad
Merge remote-tracking branch 'origin/develop' into feature-workflows-2 2022-08-04 10:08:56 +02:00
iglocska bce4508716
chg: [grammar] minor fix 
- fat finger induced typo
 2022-08-04 08:20:20 +02:00
iglocska a04f3964e7
new: [correlation rework] round 2 
- long list of fixes
- update scripts
- correlation engine management interface
- recorrelation/truncation tools
- various performance tunings and bug fixes
 2022-08-03 15:44:27 +02:00
Sami Mokaddem 44b2a1c3cd
new: [workflow:debugging] Added debugging capability by POSTing workflow exec to an URL 2022-08-03 11:45:17 +02:00
Tom King de351faaac new: [internal] Add option to log last API request 2022-08-01 15:02:49 +01:00
Sami Mokaddem d83039f520
fix: [appmodel] Fixed merged conflict and typo in sql query 2022-08-01 10:19:38 +02:00
Sami Mokaddem 151d23a8c2
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows-2 2022-08-01 09:59:44 +02:00
iglocska 3ce754d595
Merge branch 'develop' into new_correlations 2022-07-31 23:58:40 +02:00
iglocska 3a4289d631
new: [correlation] engine rewrite 
- allow for multiple concurrent engines
  - default: similar behaviour as before, ACL enforced
  - No ACL: for endpoint misps, disable the enforcement of ACL for correlations altogether

- rework:
  - correlation entries are fully indexed reference tables
  - values are now stored separately
  - built in protection against overcorrelating values (defaults to 20 max)
  - 1 way correlations to cut the size in half
  - unsigned IDs to double the ID space
  - loads of performance improvements
  - fix to the broken event index with correlation counts enabled

- UI improvements
  - search for values from the correlation column directly (in case there are non-correlating versions of the same value)
  - added correlations to the attribute search/index

- TODO:
  - upgrade scripts
 2022-07-31 23:48:38 +02:00
Sami Mokaddem a6f676a66c
chg: [workflow] Simplified call to executeTrigger 2022-07-27 10:43:09 +02:00
Sami Mokaddem 5e29f975d1
chg: [workflow] Enable some module by default when updating 2022-07-26 11:16:08 +02:00
Sami Mokaddem 2be9d07502
chg: [workflowBlueprint] Added new column `default` to recognize default BP 2022-07-19 11:48:12 +02:00
Sami Mokaddem 87b92109d2
chg: [workflow] Various fixes, improved enrichment support and new logic module 2022-07-15 11:47:08 +02:00
Sami Mokaddem 2a31408555
fix: [workflow] Create table sql statement updated 2022-07-14 14:07:35 +02:00
Sami Mokaddem 99a76812bc
Merge remote-tracking branch 'origin/develop' into feature-workflows-2 2022-07-07 09:09:39 +02:00
Sami Mokaddem dbcedbc505
chg: [workflow] Various improvement and added support of `enrichment_before_query` trigger 2022-07-05 11:19:18 +02:00
Jakub Onderka 640a732c29
Merge pull request #8358 from JakubOnderka/memory-leak-fix 
fix: [internal] PHP memory leak
 2022-06-29 13:32:26 +02:00
Sami Mokaddem 8b8055a625
chg: [WorkflowBlueprint] Renamed WorkflowPart into WorkflowBlueprint 2022-06-24 17:09:49 +02:00
Sami Mokaddem 351a3cfd4e
chg: [workflow] non-blocking workflows are run by background workers 2022-06-24 12:20:03 +02:00
Sami Mokaddem a13c1a39e9
chg: [workflow] Improved logging capabilities and stop aborting execution if non-blocking module return false 2022-06-24 11:22:26 +02:00
Sami Mokaddem df8f0f9ed0
new: [workflow_module:webhook] Added new webhook module 2022-06-22 13:38:19 +02:00
Sami Mokaddem 4e47782a04
new: [workflowPart] Started integration of workflow parts 2022-06-22 09:46:57 +02:00
Sami Mokaddem 07b99a1a5a
chg: [workflow] Added more logging while executing WF 2022-06-13 14:36:44 +02:00
Jakub Onderka 2a2b52f64e chg: [internal] Convert to const 2022-06-08 10:44:33 +02:00
Sami Mokaddem d8f8225b9e
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows 2022-05-30 14:37:36 +02:00
Sami Mokaddem 9e29830dfa
chg: [workflow] Better module loading and execution errors get propagated to the caller for blocking path 2022-05-30 14:34:45 +02:00
Jakub Onderka 6a4facf8ed chg: [internal] Faster Model::isUnique and Model::exists method 2022-05-29 16:21:25 +02:00
Jakub Onderka 715ca6d1e2 fix: [internal] PHP memory leak 2022-05-22 18:19:45 +02:00
Sami Mokaddem ce9b823cbb
chg: [workflows:trigger] Added support of execution order for blocking triggers 2022-05-19 12:02:35 +02:00
Luciano Righetti 217be89e00 new: add MysqlExtended DboSource to support index query hints 2022-05-19 11:08:31 +02:00
Sami Mokaddem 62e53f81f6
chg: [workflow] Started working on priority_level and trigger list - WiP 2022-05-16 14:44:54 +02:00