MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
23,820 Commits
53 Branches
373 Tags
190 MiB
Commit Graph

710 Commits (e6001bc9fa81938e68042105f43a4002ae9455a4)

Author SHA1 Message Date
Jakub Onderka 062390ed83
Merge pull request #7155 from JakubOnderka/push-optim 
Push optim
 2021-03-04 19:12:31 +01:00
Jakub Onderka 921583f24d chg: [sync] Code cleanup 2021-03-04 18:30:28 +01:00
Jakub Onderka 599819f7f9 new: [authkeys] Allowed IPs 2021-03-03 09:23:07 +01:00
Jakub Onderka e3b2a0a40c chg: [schema] Add index for EventReport.event_id 2021-03-03 08:17:18 +01:00
Jakub Onderka 874ec66c9b chg: [schema] Convert GalaxyCluster tag name to case insensitive 2021-03-03 08:17:16 +01:00
iglocska b4e84726f1
Merge branch '2.4' into develop 2021-02-25 11:58:11 +01:00
iglocska 64879b836e
new: [CLI] check if updates are done yet or not 
usage:

- /var/www/MISP/app/Console/cake Admin updatesDone [blocking]
- returns True or False based on whether it is done
- When the blocking parameter is set, it will not return until all updates are done
 2021-02-25 11:37:37 +01:00
mokaddem 7222197245
new: [galaxy] Support of enabled/disabled state at galaxy level 
Fix #7019
 2021-02-19 13:33:24 +01:00
Jakub Onderka 1ac21e04c6 chg: [internal] Make Redis connection static 2021-02-16 09:50:46 +01:00
Jakub Onderka f910d8538b
Merge pull request #6809 from JakubOnderka/optimise-sightings-saving 
chg: [internal] Optimise sightings saving
 2021-01-09 13:23:36 +01:00
iglocska b8823b86e2
new: [correlation] added system to exclude certain values from the correlation engine 
- simply add values at /exclude_correlations
- new values coming in will not correlate if they trip over the values listed there
- to remove existing correlations run the cleaner tool on the above endpoint

- values can be 1:1 matches, or substring searches (denoted with a leading, ending, or both '%')
  - https://www.google.com/%  will match anything starting with https://www.google.com/
  - %google.com% will match anything that contains google.com
 2021-01-07 09:31:38 +01:00
Jakub Onderka 2bbe36c0ed chg: [internal] Optimise sightings saving 2021-01-01 15:45:05 +01:00
Jakub Onderka 4d4b306b60 new: [internal] New model method find('column') 2020-12-23 10:42:05 +01:00
Jakub Onderka a585466dff new: [security] Setting to hide orgs form sharing group view 2020-12-22 23:43:30 +01:00
Jakub Onderka 8df77748b0 chg: [internal] Small optimisations 2020-12-17 13:50:25 +01:00
Jakub Onderka d7c027fe91 chg: [internal] Force to update session data after database update 2020-12-17 13:49:32 +01:00
iglocska a332e1379c
Merge branch '2.4' into cerebrate 2020-11-30 23:49:40 +01:00
iglocska e65b255a62
new: [Cerebrate] db update added 2020-11-30 23:38:38 +01:00
Jakub Onderka 2c7d6e4466 new: [auth] Allow to enforce auth plugin authentication 2020-11-30 14:46:36 +01:00
Jakub Onderka 3b791aea7e fix: [internal] Do not try to fetch empty job 2020-11-19 19:16:12 +01:00
mokaddem eb2fa7b393 Merge branch '2.4' into galaxy-cluster2.0 2020-11-18 11:45:01 +01:00
iglocska 360dadc35f
fix: [authkey] default value incorrect 2020-11-18 11:37:23 +01:00
mokaddem dc65c79130
Merge branch '2.4' of github.com:MISP/MISP into galaxy-cluster2.0 2020-11-13 16:26:35 +01:00
mokaddem 410e3fbeee
chg: [appModel] Logout users after update 2020-11-12 11:08:08 +01:00
iglocska dbffebe503
Merge branch '2.4' into CRUD 2020-11-11 11:19:23 +01:00
mokaddem 150b4cb7d1
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0 2020-11-09 10:07:43 +01:00
mokaddem 0f69022a4e
fix: [appmodel] Make sure parameter is a string before accessing string 
index.
Fix #6544
 2020-11-05 10:13:05 +01:00
Jakub Onderka 49660255fe new: [av] Malware protection for uploaded files 2020-10-22 16:38:07 +02:00
iglocska 0b6da917d4
new: [advanced authkey] API key copy to the new system added to diagnostics 2020-10-20 08:35:21 +02:00
Jakub Onderka 46ba51a32e new: [sync] Show client certificate info in connection test 2020-10-19 09:57:06 +02:00
Jakub Onderka ac6761d7a5 fix: [internal] Remove unused AppModel::checkVersionRequirements method 2020-10-18 22:19:07 +02:00
mokaddem b628e15eb9
Merge remote-tracking branch 'origin/2.4' into feature-event-report 2020-10-09 14:54:47 +02:00
Jakub Onderka cf070bd7dd fix: [internal] Array to string conversion when constructing request 2020-10-06 17:27:53 +02:00
mokaddem 28a05576a4
chg: [appmodel] Set default value for warninglist_entry_count 2020-10-06 10:53:29 +02:00
Jakub Onderka c2f6908002 chg: [optimisation] Load MISP version and commit just once 2020-10-03 20:02:55 +02:00
mokaddem 02725982cc
Merge branch '2.4' of github.com:MISP/MISP into feature-event-report 2020-10-01 10:38:28 +02:00
mokaddem 2fdcede6b6
chg: [appModel] Removed prio worker from the list of available workers 
to perform an update
 2020-10-01 09:28:02 +02:00
iglocska 6b0252d7e4
new: [user agent] string changed for MISP -> MISP synchronisation 2020-09-29 14:56:19 +02:00
mokaddem 6aab30c871
chg: [appmodel] Make sure to trigger the event report db update 2020-09-28 15:53:12 +02:00
mokaddem eb84b3344f
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0 2020-09-22 12:08:12 +02:00
mokaddem 4408a17dff
Merge remote-tracking branch 'origin/2.4' into feature-event-report 2020-09-22 10:15:22 +02:00
mokaddem 8e6b5ecb44
chg: [appmodel] New entry to create an upper bound for the unwanted 
action through login

This will be used by recovery scripts
 2020-09-15 12:00:26 +02:00
mokaddem 32dde67374
Merge branch '2.4' of github.com:MISP/MISP into feature-event-report 2020-09-03 13:38:41 +02:00
mokaddem fb5c26997a
chg: [appModel] Create UUID unique index for other tables 2020-09-03 13:29:54 +02:00
mokaddem b374e67a15
chg: [appModel] Delete duplicated sightings uuid 2020-09-03 13:15:24 +02:00
mokaddem 6f092dfeb4
chg: [appModel] Optimized deduplication, log removed elements and 
regenerate unique indexes on update
 2020-09-03 10:55:14 +02:00
mokaddem 1729de3995
Merge branch '2.4' into fix-duplicated-uuids 2020-09-03 10:21:56 +02:00
Golbark 3a0bedb104 chg: [internal] Using Allowedlist instead of Whitelist 2020-09-01 16:28:20 +02:00
Golbark 3fb47d1cce chg: [internal] Using blocklist instead of blacklist 2020-09-01 16:27:36 +02:00
mokaddem 355861daa7
fix: [appmodel] Create indexes after the column has been added 2020-09-01 08:16:17 +02:00
iglocska 8fc66eda9f
new: [internal] cache tags instead of loading them over and over via the event fetcher, fixes #6201 
- should speed things up for exports of datasets that have a lot of recurring tags
- moved the caching of some internals to the appmodel level to make it more generic
 2020-08-18 09:52:48 +02:00
iglocska 0e898ce4f4
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2020-08-18 09:26:56 +02:00
iglocska 24a5a9dc5e
chg: [internal] stub for a simple caching mechanism for recurring queries 2020-08-18 09:26:25 +02:00
Jakub Onderka cac497158e chg: [internal] Reuse AttachmentTool instance 2020-08-14 12:30:12 +02:00
Jakub Onderka a411b36666 chg: [internal] Move attachment handling to one place 2020-08-13 15:34:32 +02:00
Jakub Onderka 6fc8e4d119 fix: [internal] SendEmail exceptions message and logging 2020-08-12 19:33:15 +02:00
mokaddem 8d6f2eab03
chg: [eventReport] Added markdown-it dependency and started integration 
- WiP
 2020-08-04 10:08:18 +02:00
mokaddem dfaafd8054
Merge branch '2.4' of github.com:MISP/MISP into feature-event-report 2020-08-03 14:38:06 +02:00
mokaddem 8da680c60a
chg: [db_schema] Add constraint on UUID for Attribute, Object and Event 
tables
 2020-08-03 12:37:42 +02:00
mokaddem 3d224cfdb8
Merge branch '2.4' of github.com:MISP/MISP into fix-mass-edit-form-with-post 2020-07-30 15:29:20 +02:00
iglocska a2c0010d48
fix: [unicode] Temporarily escape 4 byte characters until we move the attribute value fields to mb4, fixes #5123 
- fixes sync/feed issues related to 4 byte unicode characters
 2020-07-30 09:17:27 +02:00
mokaddem 8ee87fb754
chg: [attributes:massEditForm] Pass attributes ids to be edited via POST 
Fix #5500
 2020-07-27 12:10:53 +02:00
Jakub Onderka f3d11f8d55 chg: [internal] Better error handling for JSON decoding 2020-07-13 15:59:32 +02:00
mokaddem 2051880b3f
new: [GalaxyCluster] Added soft and hard deletion 2020-07-13 12:34:58 +02:00
mokaddem 724de8a759
new: [clusterBlocklist] Added initial blocklist similar to the event one 2020-07-13 11:14:56 +02:00
mokaddem f7a3ed1db8
fix: [appModel:db_changes] Added new entry 2020-07-07 09:47:13 +02:00
mokaddem f3a9481c61
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0 2020-07-01 16:22:55 +02:00
mokaddem d4b1ecb088
new: [tag] Added 2 new columns to fetch tags from galaxies faster 2020-06-18 11:02:08 +02:00
mokaddem 864ecbdcce
new: [galaxyCluster] Added `published` flag to clusters 2020-06-16 16:36:49 +02:00
mokaddem 6e850f9455
fix: [clusterRelations] Corrected conditions allowing the 
creation/update of relations. Plus, get rid of not-used relation's value
 2020-06-15 08:31:40 +02:00
iglocska 82425123ab
fix: [internal] object level restsearch issues resolved when querying via filters on the attribute scope, fixes #6016 
- use subqueries instead of trying to query on the current scope
- associated find queries don't work on habtm relationships
 2020-06-14 20:23:48 +02:00
mokaddem 9b630ab146
new: [server] Added `pull_galaxy_cluster` option in the server config 2020-06-10 09:36:34 +02:00
Jakub Onderka aab0dc75a9 chg: [internal] Drop correlations.{org_id,sharing_group_id,a_sharing_group_id} indexes 2020-06-07 19:16:33 +02:00
Jakub Onderka ad9a729ffe chg: [internal] Drop correlations.value index 2020-06-07 18:58:04 +02:00
mokaddem a3bfd89f3e
chg: [appModel:update-55] Added indexes and removed incorrect sql 
commands
 2020-05-28 14:17:23 +02:00
mokaddem 5c04b9a8c1
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0 2020-05-28 14:06:30 +02:00
mokaddem 176e29c94f
chg: [server:push] Drafty version of galaxyCluster push 2020-05-26 15:08:24 +02:00
mokaddem 9ba5bc186f
chg: [galaxyCluster] Usage of both ID and UUID for relation with drafty working 
version of import/export
 2020-05-25 10:04:07 +02:00
mokaddem 4fa9642879
fix: [sightingdbs:model] Added default value for `timestamp`. Fix #5887 2020-05-14 15:23:28 +02:00
Jakub Onderka 1e07dfc9ae chg: [pubsub] Refactored PubSub tool 2020-05-11 08:22:37 +02:00
mokaddem d61e864a1e
chg: [clusterRelations:view_relations] Attached referencing relations 2020-05-08 09:17:37 +02:00
mokaddem 23d24c3c9e
chg: [clusterRelations] Model linking and basic index 2020-05-07 11:03:18 +02:00
mokaddem 01ec581a86
new: [clusterRelation] Early work on Galaxy Cluster Relations 2020-05-06 07:59:57 +02:00
iglocska 49499f1df8
Merge branch '5834' into 2.4 2020-04-30 07:07:25 +02:00
iglocska 2de54d59e5
fix: [user settings] corrected field name 2020-04-30 06:46:46 +02:00
iglocska d2ed0c8956
fix: [internal] Edge case where due to an old invalid update script an instance could end up with the wrong key in user settings 
- this should resolve the issue for affected users
- no change for everyone else
 2020-04-29 16:13:54 +02:00
mokaddem b86db16697
chg: [galaxyClusters] Added column `extends_version` 2020-04-29 10:00:22 +02:00
Jakub Onderka 4dec19dedf new: [feed] Support for compressed feeds 2020-04-28 16:16:34 +02:00
iglocska 1c3dc3fe51
Merge branch '2.4' into 5272 2020-04-28 15:31:17 +02:00
iglocska 2d63f68fc1
new: [feeds] index refactor and new features 
- added the ability to select an orgc ID for CSV/freetext feeds
  - all events created from this feed will carry the selected orgc_id

- Refactored the index fully
  - using the factories
  - better warnings against the dangerous new feed each pull setting
  - event index search added
  - several settings cleaned up / made more clear

- auto reload of default feed configuration disabled, fixes #2542, fixes #5789
  - added a button / endpoint to handle that instead to allow for the deleted default feeds to stay deleted
 2020-04-17 14:17:54 +02:00
iglocska 4ded5a73c4
new: [database] New MySQL data source added for debugging 
- MySQLObserver datasource added - prepends all queries with the requested controller/action and user ID for better debugging
 2020-04-14 15:04:33 +02:00
mokaddem dd1be03597
Merge branch '2.4' of github.com:MISP/MISP into galaxy-cluster2.0 2020-04-09 14:26:48 +02:00
iglocska 4ebc0a7988
new: [inbox] system added 
- user self-registration is the first use-case
- if the feature is enabled, users can unauthenticated send a registration request to MISP
  - request includes information on desired org and some privileges (sync / org admin / publisher)
- requests land in the inbox, admins can inspect the registration requests
  - they can accept/discard them individually or en masse
  - users will be notified of their credentials automatically
  - quick user creation if the user asks for an org that doesn't exist yet
 2020-04-07 13:21:01 +02:00
mokaddem 04dcdebb1f
new: [galaxyCluster] Initial import of Galaxy2.0 codebase - WiP 2020-03-12 10:26:09 +01:00
iglocska 8d02332b31
new: [dashboard] persistence package 
- export dashboard state
- import dashboard state
- save dashboard state
  - make it available to others on the instance on demand
  - admins can set a default password for users that don't have anything configured yet
  - load another template based on what the community has shared
- added Whoami widget which was an outcome of the ESDC training
- various improvements, new fields for genericElements, etc
 2020-03-08 23:36:27 +01:00
iglocska 03dc9a8206
fix: [synctool] tests improved 2020-03-02 23:09:47 +01:00
mokaddem 0831a3804a
chg: [database] Added db entry to re-correlate Attributes 2020-02-26 14:37:09 +01:00
mokaddem 745e42dc01
chg: [eventReport] Continuation of implementation - WiP 2020-02-18 15:02:35 +01:00
mokaddem 5408bb14f8
fix: [indexes] Added SQL index for tag numerical_value 2020-02-07 12:56:27 +01:00
Jakub Onderka e609ad6b77 chg: [internal] Log also previous exception 2020-01-26 18:56:41 +01:00
mokaddem 9d9ee3dbe7
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2020-01-20 13:25:51 +01:00
mokaddem 92ebdfaba8
fix: [updateProgress] Display errors in between updates 
Return empty progress when the current progress can't be fetched
 2020-01-20 13:24:51 +01:00
iglocska 2ac7ea62da
fix: [internal] upgrade issues fixed 2020-01-20 11:56:50 +01:00
iglocska 379c6c3443
fix: [upgrade] removed test change 2020-01-20 10:25:00 +01:00
iglocska 22b2e82ace
fix: [upgrade] Added a safety net for launching superfluous updates 2020-01-20 10:23:51 +01:00
mokaddem 73b3acda94
fix: [model:comment] Do not index the `comment` field anymore for 
shadowAttribute and Objects
 2020-01-17 10:20:13 +01:00
mokaddem 4525f263c9
fix: [Attribute:comment] Do not index the `comment` field anymore 2020-01-17 09:23:18 +01:00
mokaddem a2127bea83
fix: [update:index] Correctly log index addition errors 2019-12-16 14:59:53 +01:00
mokaddem 5ff9c29fb8
fix: [update] Function name conflict introduced by the merge + UI 
Improvements
 2019-12-09 09:53:18 +01:00
mokaddem a1fdea754d
Merge branch 'zoidberg-timeline' into zoidberg-final 2019-12-06 15:40:51 +01:00
iglocska 7a8574851e
fix: [internal] potential fix to uninitialised AdminSetting model errors when calling changeSetting() in the upgrade process 2019-11-27 09:47:25 +01:00
Richard van den Berg dd963c2e21 Sync sightings on push, pull and push on add 2019-11-22 21:53:51 +01:00
iglocska 288df9d8e7
chg: [internal] Renamed log action name for db worker issues to be <= 20 characters in length 
- it was a restriction based on the db schema of the log table from before
 2019-11-18 11:35:10 +01:00
iglocska fc5c9d0db6
Revert "Revert "Merge pull request #5304 from JakubOnderka/version-loading"" 
This reverts commit 623bb20cb0.
 2019-11-12 13:12:42 +01:00
Raphaël Vinot 623bb20cb0 Revert "Merge pull request #5304 from JakubOnderka/version-loading" 
This reverts commit 71fb7fcbd7, reversing
changes made to 11ee95aeb3.

Note: broke freetext import.
 2019-11-12 10:16:36 +01:00
Andras Iklody 71fb7fcbd7
Merge pull request #5304 from JakubOnderka/version-loading 
fix: [internal] Load MISP version just once in AppController
 2019-11-11 15:15:20 +01:00
mokaddem 3babd222ac
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-11-08 11:42:27 +01:00
iglocska 5f25f451df
chg: [sightingdb] Added support for bulk lookups and namespacing 
- aligned with the latest version of the sightingdb (support for the /rb endpoint)
- added namespacing as an option / sightingdb connection, defaults to "all" if left empty
 2019-11-08 08:24:04 +01:00
iglocska d24f23b5bd
Merge branch '2.4' into sightingdb 2019-11-06 21:22:00 +01:00
iglocska c4f1d4d15e
new: [SightingDB] Added integration with SightingDB 
- Added configuration tool
- Added lookups from the event view
- Added includeSightingdb flag for the restSearch searches
- Added SightingDB search tool
- Added SightingDB connection test tool
 2019-11-06 21:20:04 +01:00
mokaddem f58770a90a
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-11-05 13:51:03 +01:00
mokaddem b78d7825b9
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-11-05 13:48:39 +01:00
Koen Van Impe 75f993e524 Force botvrij.eu feed to use HTTPS 2019-10-31 08:48:00 +01:00
mokaddem 0087a49fcc
chg: Usage of camelCase instead of snake_case 2019-10-29 09:57:25 +01:00
mokaddem 5e8720c8ee
chg: [update] Added special log action for workers performing the DB 
update
 2019-10-15 12:07:29 +02:00
mokaddem b87ee19146
chg: [update] Actually reset `UpdateFailNumber` when manually unlocking 2019-10-15 11:44:34 +02:00
mokaddem 13173e8926
chg: [update] Added postgresql allowed DB update errors 2019-10-14 12:37:19 +02:00
mokaddem 5d4142f62a
chg: [update] Parametrized ignore_disabled in ondemand_action and 
support of string update in update_progress
 2019-10-14 10:49:41 +02:00
Jakub Onderka 9314e9c968 fix: [internal] Load MISP version just once in AppController 2019-10-13 11:29:43 +02:00
mokaddem 0e2205c061
chg: [update] Added endpoint to release lock and integration with UI 2019-10-10 12:02:23 +02:00
mokaddem 571b4887f0
chg: [update] Do not release the lock if update fails more than 3 times 2019-10-10 10:14:22 +02:00
mokaddem 6652b922e6
chg: [appModel] Fixed merge conflict 2019-10-08 14:14:24 +02:00
mokaddem 015ec7d989
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-10-08 14:12:38 +02:00
mokaddem 8760c98c7e
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-08 12:02:09 +02:00
iglocska a8c57a8316
new: [API] Added rate limiting option to the API 
- / role setting
- can be enabled/disabled and if enabled a limit can be set
- limit counter / 15 minutes starting from the first query
- x-headers inform the user about their limit/remaining queries/reset in seconds
 2019-10-08 11:43:56 +02:00
mokaddem 035d3be878
chg: [updateProgress] Usage of GPU for animation, deleted sleeps and 
improved task
 2019-10-07 15:12:47 +02:00
mokaddem 831e970e37
Merge branch 'revisedUpdateProcess' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-07 11:52:26 +02:00
mokaddem c936d4ab51
chg: [updateProgress] Started taking into account stack of updates - WiP 2019-10-03 13:50:55 +02:00
mokaddem 2a6288c30b fix: [updateProgress] Fixed return message to better handle translation 2019-10-02 15:07:37 +02:00
mokaddem b44b369eec
Merge remote-tracking branch 'origin/2.4' into zoidberg-timeline 2019-10-02 14:35:00 +02:00
mokaddem bb3cf85776
Merge branch '2.4' into zoidberg-timeline 2019-10-02 11:23:04 +02:00
mokaddem a95aa1f5a2
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-01 15:51:18 +02:00
mokaddem 25b1faceef
chg: [update] Usage of sprintf instead of string concatenation 2019-10-01 15:46:45 +02:00
mokaddem 0f6f5d6291
chg: [dbSchemaDiagnostic] show remaining time before update unlock and 
columns that should not be there
 2019-10-01 15:22:29 +02:00
mokaddem 6fd52393b7
chg: [update] Added new worker type `update` to perform updates 2019-10-01 13:36:37 +02:00
mokaddem 3b09a4bb16
chg: [update] Correctly terminate parallels workers doing updates - WiP 2019-10-01 13:08:09 +02:00
Andras Iklody 20311ae3e6
Merge pull request #5217 from JakubOnderka/patch-36 
new: [internal] New AppModel::logException method
 2019-09-30 21:26:09 +02:00
mokaddem cc6f0dcba7
chg: [update] Moved locking system from `updateDatase` to `runUpdates` - WiP 
So that `updateMISP` is also locked and workers benefits of more context
 2019-09-30 16:28:52 +02:00
mokaddem ae5d900b40
chg: [update] Keep track of update number in job - WiP 2019-09-30 15:31:49 +02:00
mokaddem 900317d5fe
Merge remote-tracking branch 'origin/2.4' into revisedUpdateProcess 2019-09-30 10:40:01 +02:00
mokaddem 006279f91f
fix: [update] Fixed bad merge and added more comments 2019-09-30 10:30:41 +02:00
iglocska cee439dc80
fix: [performance] notifications lookup on each UI page load was slow 
- introduced a major bottleneck on large instances
- massively reduced the load times for pages that warranted none
 2019-09-30 08:36:13 +02:00
iglocska a18e49f07e
fix: [user_settings] added timestamp field 2019-09-29 22:20:46 +02:00
iglocska 480e3b2969
Merge branch 'dev_session' into 2.4 2019-09-29 20:23:00 +02:00
iglocska 9ac6552ffa
new: [user settings] Finalised first revision 
More to come in the future
 2019-09-29 20:20:39 +02:00
mokaddem c72f04dd90
Merge branch '2.4' into revisedUpdateProcess 2019-09-26 12:13:49 +02:00
iglocska 946602a696
new: [User settings] Added user settings system 
- set settings / user
- settings can be set by user themselves or their org admin / site admin
- added first setting: publish_alert_filter
- accepts boolean branched filter options
  - supports deep logical trees
  - OR/NOT/AND
  - currently supports filtering on tags and the creator organisation
 2019-09-25 11:50:54 +02:00
Jakub Onderka acd90d9675
new: [internal] New AppModel::logException method 2019-09-24 20:52:36 +02:00
Jakub Onderka 84d100e982 new: [internal] Redis diagnostic 2019-09-21 07:43:35 +02:00
Jakub Onderka 0924de0440 fix: [internal] Clear also cake core and model caches 2019-09-16 19:41:27 +02:00
iglocska 850159627a
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-09-13 11:51:05 +02:00
iglocska ffc9147018
new: [sync] Added sync priority system to prioritise the order of instances to push to 2019-09-13 11:49:12 +02:00
mokaddem 1b133062e6
chg: [decaying] Changed `name` column type from `text` to `varchar(255)` 2019-09-12 14:44:09 +02:00
mokaddem ead4f46cf4
fix: [decaying] Correctly add database indexes 2019-09-12 14:28:49 +02:00
mokaddem 60edb198a7
chg: [database] Added indexes to decaying models and related tables 2019-09-12 10:22:13 +02:00
mokaddem 91768dc881
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-09-09 11:42:03 +02:00
mokaddem 059b25f262
chg: [decaying] Added `default` column in decayingModels table, code 
path for `rest` and improved ACL
 2019-08-30 09:18:00 +02:00
mokaddem c39c168962
fix: [appModel] SQL query syntax fixed 2019-08-29 12:22:35 +02:00
mokaddem 0c88b99a96
chg: [roles] Initially sets perm_decaying permission to mirror sighting 
permission
 2019-08-27 09:19:44 +02:00
mokaddem 2c788b1863
fix: [appmodel] Added db_change number for the decaying update 2019-08-27 09:01:27 +02:00
mokaddem ed98d73be7
Merge branch '2.4' of github.com:MISP/MISP into decaying 2019-08-22 15:33:09 +02:00
mokaddem 3a64405bd9
chg: [decayingMapping] Refacto - Comments and code optimization 2019-08-22 11:31:50 +02:00
mokaddem 26afe1765d
chg: [roles] Added `perm_decaying` role 2019-08-20 16:37:06 +02:00
Jakub Onderka b50ffc9885 chg: [cache] Do not check if class exists when cache is already connected 2019-08-16 19:26:12 +02:00
mokaddem e421c0684e
chg: [decaying:edit] Notice if editing a default model 2019-08-16 11:59:12 +02:00
mokaddem 3a356a51e4
chg: [decaying] Added restricted edition and `all_orgs`` flag - WiP 2019-08-16 10:33:58 +02:00
mokaddem dbeaefcfff
fix: [update] Correctly sets liveOff and do not consider MySQL index addition 
as an error anymore
 2019-08-14 16:45:58 +02:00
mokaddem 7ad60f7562
fix: [update] Init class before using it.. 2019-08-14 15:22:29 +02:00
mokaddem 5c3f2a8c80
chg: [update] Update function name are more explicit 2019-08-14 12:26:54 +02:00
mokaddem 0fe9b490b8
Merge branch 'workerForDBUpdate' into revisedUpdateProcess 2019-08-14 12:14:39 +02:00
mokaddem ae49c0766a
chg: [update] `updateDatabase` returns the result of the update 
(duplicated column are nbot counted as an error)
 2019-08-14 12:13:45 +02:00
mokaddem c2cd6a3459
chg: [decaying:index] Added support of enable/disable on the index 2019-07-24 14:36:13 +02:00
mokaddem 821785273a
Merge remote-tracking branch 'origin/2.4' into decaying 2019-07-22 10:03:57 +02:00
Raphaël Vinot a8e66b50aa
fix: [debug] Remove debug call. 2019-07-15 10:23:18 +02:00
mokaddem 13efab98a1
fix: [database] bumped db update number for fs/ls update 2019-07-11 10:10:48 +02:00
mokaddem 2c57c1138e
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-07-11 09:59:57 +02:00
iglocska 2a059f7b31
new: [tags] Local tags feature added 
- Create tags locally if you are a host org user that allows in-place tagging for sync / export filtering
- Events are NOT modified using these tags
- Local tags always get stripped before syncing
- Local tags allow host org users to violate the ownership model of MISP
- galaxies, attack matries also included

- some sync fixes
 2019-07-08 11:39:41 +02:00
mokaddem b8b6a170fe
chg: [attribute:restSearch] Search support for first_seen and last_seen 2019-07-04 15:08:05 +02:00
mokaddem 76cddeb264
Merge branch '2.4' of github.com:MISP/MISP into decaying 2019-07-03 13:45:49 +02:00
mokaddem 31c44d78d9
fix: [sql] updated MySQL.sql and modified comment column type 2019-07-02 16:26:05 +02:00
mokaddem f6d360bcfe
chg: [index] Index Attribute.comment 2019-07-02 16:12:53 +02:00
mokaddem 6305b681ed
chg: [update:seen] Switch back to the usage of worker for the update 2019-07-02 15:29:49 +02:00
mokaddem 91b70e5c82
fix: [update] liveOff recognition and logs when updates are locked 2019-07-02 15:22:21 +02:00
mokaddem 1bef45a6f0
chg: [update] DO not execute pre-update test for the timeline update 
anymore (pre-update feature not fully supported yet)
 2019-07-01 16:27:09 +02:00
mokaddem 51ea04d91d
chg: [update] Usage of `indexArray` instead of raw sql 2019-07-01 14:35:56 +02:00
mokaddem 6af9719a47 new: [shadowAttribute] first_seen and last_seen on shadowAttributes 2019-06-24 10:28:55 +02:00
mokaddem 8e40114997 chg: [app] Improved and integrated *-seen database update 2019-06-13 09:38:39 +02:00
mokaddem e7f3d0d9df new: [timeline/*-seen] Initial import of the timeline code from the 
zoidberg branch
 2019-06-13 09:16:34 +02:00
iglocska 06d155d203
new: [emailing] Server admins can get a threshold for per org e-mail alerts, fixes #4714 2019-06-04 11:30:01 +02:00
mokaddem f8ac2563f9 fix: [update] do not longer start update job if no update available 2019-05-08 15:37:39 +02:00
mokaddem 420e01c4e2 fix: [update] Switched to default usage of worker for the update 2019-05-08 15:33:58 +02:00
mokaddem 8418e89c7f fix: [update] Started changing worker place in the update process 2019-05-08 15:07:26 +02:00
iglocska 61bcf4a671 fix: [update] Disabled background processing until it's fixed 2019-05-08 10:28:03 +02:00
iglocska 8e3e4d27c1 fix: [bug] Fixed a bug in the update process that caused updates to fail due to an invalid value assigned as default for org_id 2019-05-07 11:56:29 +02:00
mokaddem 3794dad160 chg: [update] Avoid executing pre-update test multiple times 2019-04-29 11:26:55 +02:00
mokaddem 47e13c8369 chg: [updates] Implented changes requested by the PR's review #4534. 2019-04-29 11:09:04 +02:00
mokaddem ee735f00d4 new: [update] Injected update-related files/changes from zoidberg 2019-04-26 09:45:03 +02:00
iglocska 82b5b37b8f fix: [API] Invalid negative lookup fixed 2019-04-23 14:02:27 +02:00
mokaddem e4bc67463b Merge branch '2.4' of github.com:MISP/MISP into decaying 2019-04-10 08:39:13 +02:00
mokaddem cdec345c4c chg: [decayingTool] Improved related type retreival and improved UI - 
WiP
 2019-04-04 09:37:51 +02:00
mokaddem 38a7f511d7 chg: [DecayingTool] Added more fields 2019-04-03 11:51:29 +02:00
mokaddem 03015aa341 chg: [decayingModel] Added update mechanism from local files and started `decaying_model_mappings` 2019-04-03 10:05:45 +02:00
iglocska 96951afc47 fix: [sync] Adding a new server caused the pull/push rules to be incorrectly set to an empty string over '[]' causing sync issues, fixes #4369 
- this fix resolves the issue - new servers added should be fine
- it also retroactively fixes broken server connections
 2019-03-25 17:36:01 +01:00
iglocska 7141f70b20 Merge branch 'kafka' into 2.4 2019-03-19 17:23:05 +01:00
iglocska 49be0f8b7a chg: [db script] version bump 2019-03-19 17:20:46 +01:00
iglocska 6efd909bfe new: [Required taxonomies] 
- Flip taxonomies into required mode to ensure that events cannot be published without at least one tag from each required taxonomy to be set
 2019-03-19 15:59:31 +01:00
iglocska 7fbc4dc34c new: [REST client] Added history/bookmarks 2019-03-19 10:55:27 +01:00
Nikos Filippakis 9d59b10368 Publish events to Kafka 
Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
 2019-03-18 15:53:22 +01:00
mokaddem 785d83607a chg: [decayingTool] Added model and controller 2019-03-08 09:22:37 +01:00
iglocska 3718a89b9a fix: [DB] removed not null constraint of the kill chain order field, fixing 4198 2019-02-22 08:04:23 +01:00
mokaddem 1f73a655cf new: [galaxies] Added support of `kill_chain_order` in galaxies 2019-02-14 10:24:42 +01:00
iglocska 09cbbe3b93 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-02-10 21:36:03 +01:00
iglocska c371c95d33 new: [CLI] verbose mode added to runUpdates 2019-02-10 21:35:23 +01:00
Christophe Vandeplas 67efc70bf5 fix: [style] consistent space indentation 2019-02-10 13:08:55 +01:00
iglocska db88592970 chg: [internal] Moved time resolution to appmodel 2019-02-06 17:46:36 +01:00
iglocska 9c5fbdd7e9 new: [server caching] Initial version WIP 2019-01-18 09:06:23 +01:00
iglocska 2e38015e8e fix: [data model] fixed a bug that caused org_id fields to be tinyint(1) for org_ids on tags 2019-01-01 16:37:15 +01:00
iglocska 7cebaef6c8 fix: [db update] Fixed update script for tag collection tags 2018-12-27 05:24:30 +01:00
iglocska 54bb5f3d01 new: [tag_collections] Added db upgrade 2018-12-15 10:02:38 +01:00
iglocska af0f40ab97 new: [server settings] Added automatic backup system for the server settings 2018-11-26 09:12:01 +01:00
iglocska 2d0259ce13 fix: [CS] coding standards script re-run 2018-11-23 14:11:33 +01:00
iglocska 39ff6f4733 fix: [api] Invalid handling of empty parameters in the built in parameter builder 2018-11-02 04:26:17 +01:00
iglocska ae597c8358 new: [galaxy] Several changes 
- moved the current uuid field on cluster level to a new "collection_uuid" field to better represent the actual purpose
- added new uuid field that actually captures the cluster's uuid
- upgrade script is multi-execution safe
- added /galaxy_clusters/view to the API
- /galaxy_clusters/view can now be queried via the uuid instead of just the ID
 2018-10-22 10:36:13 +02:00
iglocska 4f4737d22a new: [upgrade] Preparing the data for recovery after the object reference sync fix 
- update the timestamps of all events / objcts that are affected and are locked = 0
 2018-10-19 11:25:55 +02:00
iglocska c96be93e3c fix: [internal] Fix of wonky model function calls across the application for getting default attachment directories 2018-10-18 09:51:14 +02:00
www-data f9183dee3b Merge branch '2.4' into py-virtualenv 2018-10-15 17:09:18 +09:00
iglocska 09ec9aecd6 new: [API] Added the log index/search to the API 
- described in the templates / rest client page
 2018-10-11 20:20:27 +02:00
iglocska 43558a9610 new: [API] rework of the searchall/quickFilter parameters 
Now it correctly works as intended on both attribute and event contexts
 2018-10-08 10:17:57 +02:00
Steve Clement df3d87b5c8 Merge branch '2.4' into py-virtualenv 2018-09-28 15:00:46 +02:00
iglocska 0216c9ea33 new: [API] Rework of the restSearch APIs 
- peformance tuning
  - removed some redundant looping
  - internal memory profiling for attributes/restSearch
  - saving the intermediary results to file instead of keeping it all in memory to reduce the memory footprint
- added the searchall parameter
- fixed the ignore parameter
- added the event_timestamp parameter
- added manual pagination to the attribute level restsearch (limit, page)
 2018-09-28 09:21:29 +02:00
www-data 80fcca5f4c chg: [python] Added initial python virtualenv support, STIX Tests only. 2018-09-25 21:29:34 +02:00
iglocska 3b65fce2a8 fix: [db] Fixed an invalid DB field 2018-09-12 19:17:32 +02:00
iglocska 8bc7e9b91f fix: [merge issue] resolved merge issue 2018-09-10 22:23:55 +02:00
iglocska bef577ae07 Merge branch 'feature/variable_tag_value' into 2.4 2018-09-10 22:02:07 +02:00
iglocska 07376b1a9e new: [variable tags] Added the ability to load and display variable tags 
- as requested by Siemens
 2018-09-10 21:57:57 +02:00
iglocska c680f0aff5 fix: [cleanup] Cleanup of removed upgrade scripts 2018-09-10 07:53:22 +02:00
iglocska 6a333f7756 fix: [upgrade] replay potentially missed updates 2018-09-10 07:52:27 +02:00
iglocska 00155a8775 new: [API] fixed two cases where the new filter parameter builder was being naughty 
- copy-pasta fail induced skipping of parameters with only NOT parameters fixed
- OR/AND/NOT formatted parameters with singular values (such as '{"OR": "foo"}' now handled correctly
 2018-09-03 17:54:37 +02:00
iglocska b407aba746 fix: [CS] Updated recent changes 2018-08-20 10:50:09 +02:00
iglocska f675fb8b29 Merge branch '2.4' into feature/api_rework 2018-08-17 14:49:09 +02:00