Sami Mokaddem
962754dd3b
chg: [overCorrelatingValue] Truncated the `value` column
...
- We keep the unique constraint on the table
- Correlating values over the max. allowed size are truncated to fit the size requirement. That means large correlating values might be marked as over-correlating despite the fact they are not (as only the starting portion of the value is evaluated).
2022-08-29 10:50:59 +02:00
Sami Mokaddem
8db8cbd398
new: [user:periodicNotification] Started development of system allowing users to receive period notifications by email
2022-08-29 09:03:42 +02:00
Jakub Onderka
ec209a98b9
chg: [internal] Drop unnecessary indexes from default_correlations table
2022-08-11 12:54:49 +02:00
iglocska
c764bb0e8f
Merge branch 'log_last_api' into develop
2022-08-11 09:36:30 +02:00
iglocska
097ad78d54
chg: [correlations] performance tuning
...
- added a new constraint to check for correlation uniqueness (a_id, 1_a_id, value_id)
- stopped dropping correlations on a full recorrelation
- only correlate "upwards" towards higher IDs
2022-08-09 14:41:59 +02:00
iglocska
9dc233bc9e
fix: [proposal] correlations are deprecated
2022-08-05 12:55:04 +02:00
Tom King
64d508b4e3
fix: Rollback change to DB upgrade 86
2022-08-05 09:41:05 +01:00
Tom King
d2a7b4b3c8
fix: Use correct field for altering table to include last_api_access
2022-08-05 09:35:15 +01:00
iglocska
ea0932b7b1
chg: [correlations] value field changed to varchar
2022-08-04 13:49:30 +02:00
Sami Mokaddem
79d182dc79
fix: [appModel:update] Fixed forgotten schema update for workflows
2022-08-04 11:08:19 +02:00
Sami Mokaddem
5074528d91
fix: [appModel:update] Set update success to true for case 89 and 90
2022-08-04 10:58:44 +02:00
Sami Mokaddem
49575533ad
Merge remote-tracking branch 'origin/develop' into feature-workflows-2
2022-08-04 10:08:56 +02:00
iglocska
bce4508716
chg: [grammar] minor fix
...
- fat finger induced typo
2022-08-04 08:20:20 +02:00
iglocska
a04f3964e7
new: [correlation rework] round 2
...
- long list of fixes
- update scripts
- correlation engine management interface
- recorrelation/truncation tools
- various performance tunings and bug fixes
2022-08-03 15:44:27 +02:00
Sami Mokaddem
44b2a1c3cd
new: [workflow:debugging] Added debugging capability by POSTing workflow exec to an URL
2022-08-03 11:45:17 +02:00
Tom King
de351faaac
new: [internal] Add option to log last API request
2022-08-01 15:02:49 +01:00
Sami Mokaddem
d83039f520
fix: [appmodel] Fixed merged conflict and typo in sql query
2022-08-01 10:19:38 +02:00
Sami Mokaddem
151d23a8c2
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows-2
2022-08-01 09:59:44 +02:00
iglocska
3ce754d595
Merge branch 'develop' into new_correlations
2022-07-31 23:58:40 +02:00
iglocska
3a4289d631
new: [correlation] engine rewrite
...
- allow for multiple concurrent engines
- default: similar behaviour as before, ACL enforced
- No ACL: for endpoint misps, disable the enforcement of ACL for correlations altogether
- rework:
- correlation entries are fully indexed reference tables
- values are now stored separately
- built in protection against overcorrelating values (defaults to 20 max)
- 1 way correlations to cut the size in half
- unsigned IDs to double the ID space
- loads of performance improvements
- fix to the broken event index with correlation counts enabled
- UI improvements
- search for values from the correlation column directly (in case there are non-correlating versions of the same value)
- added correlations to the attribute search/index
- TODO:
- upgrade scripts
2022-07-31 23:48:38 +02:00
Sami Mokaddem
a6f676a66c
chg: [workflow] Simplified call to executeTrigger
2022-07-27 10:43:09 +02:00
Sami Mokaddem
5e29f975d1
chg: [workflow] Enable some module by default when updating
2022-07-26 11:16:08 +02:00
Sami Mokaddem
2be9d07502
chg: [workflowBlueprint] Added new column `default` to recognize default BP
2022-07-19 11:48:12 +02:00
Sami Mokaddem
87b92109d2
chg: [workflow] Various fixes, improved enrichment support and new logic module
2022-07-15 11:47:08 +02:00
Sami Mokaddem
2a31408555
fix: [workflow] Create table sql statement updated
2022-07-14 14:07:35 +02:00
Sami Mokaddem
99a76812bc
Merge remote-tracking branch 'origin/develop' into feature-workflows-2
2022-07-07 09:09:39 +02:00
Sami Mokaddem
dbcedbc505
chg: [workflow] Various improvement and added support of `enrichment_before_query` trigger
2022-07-05 11:19:18 +02:00
Jakub Onderka
640a732c29
Merge pull request #8358 from JakubOnderka/memory-leak-fix
...
fix: [internal] PHP memory leak
2022-06-29 13:32:26 +02:00
Sami Mokaddem
8b8055a625
chg: [WorkflowBlueprint] Renamed WorkflowPart into WorkflowBlueprint
2022-06-24 17:09:49 +02:00
Sami Mokaddem
351a3cfd4e
chg: [workflow] non-blocking workflows are run by background workers
2022-06-24 12:20:03 +02:00
Sami Mokaddem
a13c1a39e9
chg: [workflow] Improved logging capabilities and stop aborting execution if non-blocking module return false
2022-06-24 11:22:26 +02:00
Sami Mokaddem
df8f0f9ed0
new: [workflow_module:webhook] Added new webhook module
2022-06-22 13:38:19 +02:00
Sami Mokaddem
4e47782a04
new: [workflowPart] Started integration of workflow parts
2022-06-22 09:46:57 +02:00
Sami Mokaddem
07b99a1a5a
chg: [workflow] Added more logging while executing WF
2022-06-13 14:36:44 +02:00
Jakub Onderka
2a2b52f64e
chg: [internal] Convert to const
2022-06-08 10:44:33 +02:00
Sami Mokaddem
d8f8225b9e
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows
2022-05-30 14:37:36 +02:00
Sami Mokaddem
9e29830dfa
chg: [workflow] Better module loading and execution errors get propagated to the caller for blocking path
2022-05-30 14:34:45 +02:00
Jakub Onderka
6a4facf8ed
chg: [internal] Faster Model::isUnique and Model::exists method
2022-05-29 16:21:25 +02:00
Jakub Onderka
715ca6d1e2
fix: [internal] PHP memory leak
2022-05-22 18:19:45 +02:00
Sami Mokaddem
ce9b823cbb
chg: [workflows:trigger] Added support of execution order for blocking triggers
2022-05-19 12:02:35 +02:00
Luciano Righetti
217be89e00
new: add MysqlExtended DboSource to support index query hints
2022-05-19 11:08:31 +02:00
Sami Mokaddem
62e53f81f6
chg: [workflow] Started working on priority_level and trigger list - WiP
2022-05-16 14:44:54 +02:00
Sami Mokaddem
c5af331bc5
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows
2022-05-16 10:50:47 +02:00
Sami Mokaddem
62107f14b9
chg: [workflows] Added enabled state
2022-05-16 10:47:55 +02:00
Jakub Onderka
2f7c671adb
new: [internal] Simplify checking if connection is MySQL/MariaDB
2022-05-14 10:17:06 +02:00
Sami Mokaddem
c477935752
Merge branch 'develop' of github.com:MISP/MISP into feature-workflows
2022-05-11 13:03:03 +02:00
Luciano Righetti
53b0e4918b
Merge pull request #8331 from righel/add-attr-timestamp-index
...
chg: add index for attributes.timestamp, show index diagnostics in di…
2022-05-10 10:33:06 +02:00
Jakub Onderka
2294232442
fix: [internal] Strict types
2022-05-07 10:40:41 +02:00
Luciano Righetti
a6e71bd649
fix: add missing break
2022-05-04 09:22:01 +02:00
Luciano Righetti
5fbb80d04d
chg: add index for attributes.timestamp, show index diagnostics in diagnostics page
2022-05-04 09:04:30 +02:00
Sami Mokaddem
6670333f10
chg: [workflow] Added database migration
2022-05-04 00:00:19 +02:00
Tom King
5de04ecffa
chg: Merge from develop
2022-04-20 08:07:36 +01:00
iglocska
be9fb9e802
Merge branch 'develop' of github.com:MISP/MISP into develop
2022-04-19 15:18:40 +02:00
iglocska
2688961cc3
Merge branch 'develop' of github.com:MISP/MISP into develop
2022-04-18 17:59:34 +02:00
iglocska
ccbd119417
chg: [internal] setupSyncRequest made public
...
- to be able to access it via shell scripts
2022-04-18 17:57:38 +02:00
iglocska
93821c0de6
fix: [security] Sanitise paths for several file interactions
...
- remove :// anywhere we don't expect a protocol to be supplied
- remove phar:// in certauth plugin's fetcher
- as reported by Dawid Czarnecki of Zigrin Security
2022-04-17 18:25:51 +02:00
iglocska
0108f1bde2
fix: [security] unregister phar from stream wrappers globally for all Model code
...
- blanket protection against phar deserialization vulnerabilities
- as reported by Dawid Czarnecki of Zigrin Security
2022-04-17 16:30:14 +02:00
Jakub Onderka
9e8608b3d3
chg: [jsonTool] Properly handle invalid JSON for PHP 7.2 and older
2022-04-10 09:38:25 +02:00
Jakub Onderka
9d8fc81678
chg: [internal] Throw exception if Redis class not found
2022-03-19 12:31:55 +01:00
Sami Mokaddem
d65ef9c966
chg: [cryptographicKeys] Indexed more column and bumped db_schema
2022-03-17 14:43:01 +01:00
iglocska
b80a7af2c3
new: [cryptographic keys] model and controllers added
...
- sets MISP up for information signing
- sign data during synchronisation
2022-03-13 12:35:12 +01:00
iglocska
870b9d761b
fix: [db schema] fixed
2022-03-04 16:10:12 +01:00
iglocska
7834ec3760
fix: [organisations] made meta fields default to '' and not allow null values
...
- fixes a filtering issue with sharing group blueprints leading to sharing groups that are more restrictive than expected
2022-03-02 17:32:35 +01:00
iglocska
639a4929e3
new: [sharing group blueprints]
...
- create a rule based blueprint that is used to create and update a sharing group
- nest sharing groups
- filter organisations by metadata fields
- nested via boolean operators
- CLI exposed
- API exposed
- Lightweight ownership model (only blueprint owner can see and edit the blueprint)
2022-03-02 02:09:20 +01:00
Tom King
5f618ed0ca
chg: Update from upstream develop
2022-02-17 10:41:58 +00:00
Jakub Onderka
bd9cecbc2c
chg: [internal] Use ProcessTool in Sighting
2021-11-22 09:57:39 +01:00
Jakub Onderka
0cc7804219
chg: [internal] Current branch and commit checking
2021-11-14 19:09:38 +01:00
Jakub Onderka
a424c01844
chg: [internal] Small optim
2021-11-14 19:09:38 +01:00
Jakub Onderka
db3183ae54
chg: [internal] Faster way how to get current commit
2021-11-14 19:09:38 +01:00
Jakub Onderka
553036de94
Merge pull request #7955 from JakubOnderka/code-style-background-jobs
...
chg: [internal] Code style
2021-11-12 11:48:28 +01:00
Jakub Onderka
fb99021ab9
chg: [internal] Code style
2021-11-12 09:44:21 +01:00
Jakub Onderka
6806cdf574
new: [user] Add sub field for user
2021-11-11 16:34:27 +01:00
Luciano Righetti
ba0399a6fb
chg: remove MISP.use_simple_background_jobs setting in favor of SimpleBackgroundJobs.enabled
2021-11-10 16:44:02 +01:00
Luciano Righetti
9e23a2744f
chg: merge develop, fix conflicts
2021-11-09 14:23:51 +01:00
Luciano Righetti
4218787934
chg: move initTool() logic to constructor
2021-11-09 08:43:39 +01:00
Jakub Onderka
5c5f2b54ca
Merge pull request #7938 from JakubOnderka/authkeys-encrypted-vol2
...
Authkeys encrypted vol2
2021-11-09 08:36:44 +01:00
Luciano Righetti
96a46b6899
chg: merge develop, fix conflicts.
2021-11-08 14:37:27 +01:00
iglocska
dc925a321f
Merge branch 'develop' of github.com:MISP/MISP into develop
2021-11-08 14:17:01 +01:00
iglocska
952a98dba8
Merge branch 'local_only' into develop
2021-11-08 14:15:18 +01:00
Luciano Righetti
f2c9d12eae
chg: merge develop, fix conflicts.
2021-11-08 11:35:20 +01:00
Jakub Onderka
5569d7d2bf
new: [security] Store authkeys for servers encrypted
2021-11-07 17:51:04 +01:00
Jakub Onderka
3459a09943
chg: [internal] App model cleanup
2021-11-07 10:40:01 +01:00
Jakub Onderka
c9b8564c0a
Merge pull request #7927 from JakubOnderka/system-setting
...
System setting in database
2021-11-06 16:15:23 +01:00
Luciano Righetti
f80fcec2db
chg: rename settings
2021-11-05 15:46:42 +01:00
Jakub Onderka
c9597ba3e9
new: Store system settings in database
2021-11-05 09:19:56 +01:00
Luciano Righetti
309f7ed50e
fix: wrong update query
2021-11-04 16:51:56 +01:00
Luciano Righetti
5242d3204d
chg: rename conf name
2021-11-04 11:39:24 +01:00
iglocska
a0e8acc45f
Merge branch 'develop' of github.com:MISP/MISP into develop
2021-11-04 11:05:32 +01:00
iglocska
eddf95355a
fix: [database] upgrade script using mb4 defaulted to 255 key length
...
- default should be 191
2021-11-04 11:04:56 +01:00
Luciano Righetti
946bdb68e6
chg: add db update
2021-11-03 17:09:42 +01:00
Luciano Righetti
b1eaed2b1b
chg: merge develop, fix conflicts
2021-11-03 13:54:20 +01:00
Luciano Righetti
453418d6a5
fix: use class registry to get job class
2021-11-02 16:53:42 +01:00
Luciano Righetti
cce5e581d4
fix: add missing jobId param to enqueue() calls
2021-11-02 16:25:43 +01:00
Luciano Righetti
ae3300d5d0
chg: use new bg jobs tool in AppModel
2021-11-02 11:55:12 +01:00
Jakub Onderka
f1283996ad
chg: [internal] Optimise AuditLog
2021-10-31 10:52:47 +01:00
Jakub Onderka
9ae7d88f23
chg: [internal] Optimise Tag::findTagIdsByTagNames
2021-10-30 19:43:17 +02:00
Jakub Onderka
94808488ce
Merge pull request #7852 from JakubOnderka/optimise-sighting
...
chg: [internal] Optimise sightings
2021-10-29 21:25:42 +02:00
Jakub Onderka
0d8292289d
chg: [internal] Optimise afterFind and simplify ISODatetimeToUTC
2021-10-28 13:56:27 +02:00
Jakub Onderka
6ade27a27c
chg: [internal] Move attribute validation to different tool
2021-10-28 09:42:04 +02:00
Loïc Fortemps
dc71f6e451
Merge branch 'develop' into local_tags
2021-10-27 17:06:02 +02:00