Commit Graph

3397 Commits (c63d63b6eea45057d114c26c9cae1e71fbc7a8ad)

Author SHA1 Message Date
iglocska b6346ad473
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-12-18 15:09:24 +01:00
iglocska eb50b68708
fix: [UI] adding attack-like galaxy clusters to tag collections was slow
- was generating a heatmap out of all event metadata each time for no good reason
2019-12-18 15:07:41 +01:00
Jakub Onderka b0274d1cbb chg: [internal] Much faster GalaxyCluster::attachClustersToEventIndex 2019-12-18 11:52:13 +01:00
mokaddem 0400fa0370
new: [diagnostic:DBIndex] Show table columns having indexes 2019-12-17 10:54:45 +01:00
mokaddem be8d4048a0
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-final 2019-12-16 15:05:05 +01:00
mokaddem aabc568b90
fix: [Object:DeltaMerge] Gracefully catch if *_seen field is not present
in the pushed Object
2019-12-16 15:00:30 +01:00
mokaddem a2127bea83
fix: [update:index] Correctly log index addition errors 2019-12-16 14:59:53 +01:00
Andras Iklody 9454b458ee
Merge pull request #5459 from mokaddem/schemaDiagnosticImprovement
Schema diagnostic improvement
2019-12-16 14:14:41 +01:00
mokaddem 7797aeed94
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-final
Not a simple merge. Needed to fix forms and simplified how
form_seen_input works
2019-12-16 13:36:01 +01:00
mokaddem 092348ffcd
chg: [Attribute:seen] Moved conversion iso<->utc of fs/ls in dedicated
functions
2019-12-16 10:47:07 +01:00
iglocska c574329e06
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-12-12 10:08:13 +01:00
mokaddem 730541e21b
fix: [diagnostic:DBSchema] Select tables schema with UPPER case 2019-12-12 10:07:50 +01:00
iglocska f12cd3b826
fix: [taxonomies] Fixed invalid tags being shown on the taxonomy index if mistyped versions exist 2019-12-12 10:07:50 +01:00
iglocska 56902c265e
Merge branch '2.4' of github.com:MISP/MISP into add_attribute 2019-12-12 10:05:01 +01:00
iglocska 0d1e73665c
fix: [tags] Fixed the taxonomy view incorrectly fetching mistyped tags over the actual taxonomy tags 2019-12-12 10:04:16 +01:00
Andras Iklody 91a045c13f
Merge pull request #5208 from JakubOnderka/patch-34
Simplify user profile logging
2019-12-11 19:28:32 +01:00
iglocska 208e9e9cfc
fix: [API] include the local flag in tags when using restSearch 2019-12-11 10:29:45 +01:00
iglocska b6bb2bb576
new: [UI] first implementation of the modal forms 2019-12-11 10:03:43 +01:00
mokaddem 8fb705079c
fix: [diagnostic:DBSchema] Force column `table_name` to be lower case 2019-12-10 13:18:59 +01:00
mokaddem 883473eed7
chg: [dianostic:DBSchema] Consider quoted default_value as non-critical 2019-12-10 10:49:44 +01:00
mokaddem 8934e7dafb
chg: [diagnostic:DBSchemaDiagnostic] Added datasource 2019-12-10 09:21:32 +01:00
mokaddem 5ff9c29fb8
fix: [update] Function name conflict introduced by the merge + UI
Improvements
2019-12-09 09:53:18 +01:00
mokaddem a1fdea754d
Merge branch 'zoidberg-timeline' into zoidberg-final 2019-12-06 15:40:51 +01:00
Alexandre Dulaunoy a740d14593
chg: [types] new eppn attribute type added - EduPersonPricincipalName
Fix #5448
2019-12-05 19:22:38 +01:00
iglocska bde75e9443
fix: [internal] attribute restsearch - handle empty lines returned from the module better
- no more empty lines hopefully in some exports
2019-12-05 09:42:49 +01:00
iglocska 4f646d41c0
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-12-04 16:12:18 +01:00
iglocska 5768ff9485
fix: [internal] hard delete of attributes after a soft delete fixed 2019-12-04 16:11:38 +01:00
mokaddem 9747e1aad9
fix: removed useless spaces 2019-12-04 12:14:51 +01:00
mokaddem 78b0d5b734
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-12-04 12:13:09 +01:00
mokaddem da4e9e45d8
fix: [decaying:basescoreComputation] Correctly support 2-tag and 3-tag 2019-12-04 12:12:10 +01:00
Richard van den Berg f1221f886d Set rate_limit_count to 0 when not defined 2019-12-04 08:47:54 +01:00
mokaddem f4c767a273
fix: [event:publishRouter] Fixed parenthesis issue 2019-12-03 15:13:49 +01:00
Richard van den Berg 4f1ca7ef1b Push sightings regardsless of distribution level 2019-12-03 14:28:22 +01:00
iglocska bed34757f8
fix: [internal] When capturing an object, avoid throwig notice errors if no attributes are set, fixes #5439 2019-12-02 08:47:07 +01:00
iglocska c2d59dd3a5
Merge branch 'db_fix' into 2.4 2019-11-29 15:26:43 +01:00
Andras Iklody 687a91cba0
Merge pull request #5435 from RichieB2B/ncsc-nl/fix-sightings-push
Select right servers for pushing sightings
2019-11-29 11:57:44 +01:00
Richard van den Berg 1c6278d354 Select right servers for pushing sightings 2019-11-29 11:54:15 +01:00
iglocska b8e583f727
fix: [internal] taxonomy exclusive flag now handles the key not existing in the JSON format 2019-11-29 11:07:39 +01:00
iglocska bd2772bf82
fix: [internal] sighting restSearch
- some small fixes
2019-11-29 10:55:47 +01:00
mokaddem 386b910361
fix: deleted useless comments 2019-11-29 10:55:38 +01:00
mokaddem d5e8be036c
chg: [diagnostic:DBSchema] Added warning for `missing_table` errors 2019-11-29 10:48:59 +01:00
iglocska 1c5afa49ed
new: [refactor] Massive internal refactor and cleanup of deprecated APIs
- new centralised restSearch function in AppController as entry point via all controllers
- new component handling restSearch related support functions, such as parameter mapping
- hollowed out all deprecated export functions on the event/attribute controller
  - replaced with a new functionality that remaps them to restSearch
  - all functionality should be maintained with all additional advantages introduced with restsearch
- additional cleanup (some unused functions removed)
2019-11-29 10:11:30 +01:00
iglocska 21c25f60c2
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-27 09:48:14 +01:00
iglocska 7a8574851e
fix: [internal] potential fix to uninitialised AdminSetting model errors when calling changeSetting() in the upgrade process 2019-11-27 09:47:25 +01:00
Andras Iklody 0cbc96a7aa
Merge pull request #5430 from RichieB2B/ncsc-nl/perm-sighting
Allow pushing of sightings only for perm_sighting
2019-11-26 19:57:27 +01:00
Richard van den Berg 6458015708 Allow pushing of sightings only for perm_sighting 2019-11-26 19:40:49 +01:00
iglocska 26459f1b63
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-26 19:04:34 +01:00
iglocska f1c6465050
fix: [internal] potential fix to the sighting_timestamp missing issue when syncing with older instances 2019-11-26 17:12:55 +01:00
iglocska 9e74259bdb
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-26 17:11:56 +01:00
iglocska cbbe2b3a30
chg: [CSRF] disable CSRF if you absolutely feel like setting yourself up for failure 2019-11-26 17:11:33 +01:00
Andras Iklody fb7ec15940
Merge pull request #5248 from JakubOnderka/patch-44
new: [internal] Log exact error for GPG diag in error log
2019-11-26 13:45:25 +01:00
Andras Iklody ae5c228c05
Merge pull request #5273 from JakubOnderka/patch-54
fix: [internal] Remove unused function
2019-11-26 13:42:56 +01:00
Andras Iklody 1f3d7b5286
Merge pull request #5342 from JakubOnderka/patch-69
chg: [feed] Use precomputed hashes to speedup attaching correlation
2019-11-26 13:02:31 +01:00
Andras Iklody 76656e8ed4
Merge pull request #5404 from MISP/feature-OrgsStats
Added more Organisation statistics
2019-11-26 13:00:13 +01:00
iglocska bdfe59766a
chg: [cleanup] debug() removed 2019-11-25 16:02:54 +01:00
iglocska 8ee304eff9
fix: [sync] Set org_id to 0 on proposal push if the sighting is anonymised
- correctly prevents the remote side from misattributing the sighting to the sync user's org
2019-11-25 15:45:40 +01:00
iglocska 95f17d6acd
fix: [sync] Some minor changes to the sighting push
- correctly handle anonymisation
- only push sightings, not rest of the event (decide on sender side)
- handle receiving sanitised sightings
2019-11-25 14:32:22 +01:00
Richard van den Berg dd963c2e21 Sync sightings on push, pull and push on add 2019-11-22 21:53:51 +01:00
mokaddem d653a1cf86
chg: [diagnostic:fixDBSchema] Support of missing table + support of
non-critical warnings
2019-11-19 16:14:51 -05:00
mokaddem b8b5880ef6
chg: [diagnostic:db_schema] Added support of default_value and quick fix 2019-11-19 12:07:30 -05:00
mokaddem bcd5ce2362
Merge branch '2.4' of github.com:MISP/MISP into feature-fix-db-inconsistencies 2019-11-19 09:20:42 -05:00
mokaddem 3a8d283e46
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-18 19:08:51 -05:00
mokaddem 4c58602e3a
fix: [attribute:massEdit] Allow removal of non exportable tags. Fix #5408 2019-11-18 19:08:16 -05:00
mokaddem 69746c8d4f
new: [server:fixDBSchema] Preliminary work to fix database schema 2019-11-18 19:06:41 -05:00
iglocska 8008bbbd26
new: [sql diagnostics] Started work on a system to automatically generate scripts to fix issues
- currently somewhat limited
- requires additional input to generate correct queries, needs an update for the default schemas
- generated, but not exposed for now
2019-11-18 11:36:31 +01:00
iglocska 288df9d8e7
chg: [internal] Renamed log action name for db worker issues to be <= 20 characters in length
- it was a restriction based on the db schema of the log table from before
2019-11-18 11:35:10 +01:00
mokaddem 806f443764
new: [statistics] Added organisation activity over time 2019-11-16 15:40:02 -05:00
iglocska 1cc6a67335
fix: [internal] site admins should not have to be host org users to see server correlations 2019-11-16 14:06:46 +01:00
iglocska e4c82eb9ff
fix: [API] adding objects now has better validation errors
- instead of silently dropping attributes in certain cases
2019-11-15 14:11:24 +01:00
iglocska 382b57102b
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-13 19:27:02 +01:00
iglocska 0c15043cfa
new: [sync] view remote user tool added to the server index
- should help with debugging what user is being used
2019-11-13 19:09:37 +01:00
mokaddem 2f3c4778f4
chg: [diagnostic:dbSchema] Whitelist columns to ignore and highlight
critical differences
2019-11-13 16:38:30 +01:00
mokaddem 1234e9b03e
chg: [dbDiagnostic] Removed datefield precision as it's only available
on MySQL 5.6+
2019-11-13 12:28:15 +01:00
mokaddem 28abffd6a2
chg: [dbDiagnostic] Diagnostic result is stored in a keyed array instead
of indexed array
2019-11-13 12:21:19 +01:00
iglocska a7aad6f042
fix: [temporary] Dirty fix for the diagnostic page failing on MySQL < 5.6 2019-11-13 10:05:25 +01:00
iglocska fc5c9d0db6
Revert "Revert "Merge pull request #5304 from JakubOnderka/version-loading""
This reverts commit 623bb20cb0.
2019-11-12 13:12:42 +01:00
iglocska b3bb039faf
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-11-12 13:11:04 +01:00
iglocska 1b13734483
fix: [internal] Removed duplicate loading of configuration
- lazy-loading the event model after an on-the-fly config change would purge the change otherwise
- config already loaded in bootstrap anyway
2019-11-12 13:10:17 +01:00
Raphaël Vinot 623bb20cb0 Revert "Merge pull request #5304 from JakubOnderka/version-loading"
This reverts commit 71fb7fcbd7, reversing
changes made to 11ee95aeb3.

Note: broke freetext import.
2019-11-12 10:16:36 +01:00
iglocska 0f40cef0f1
fix: [internal] Attribute/Event connectors for attribute_timestamp added 2019-11-11 16:09:54 +01:00
Andras Iklody 71fb7fcbd7
Merge pull request #5304 from JakubOnderka/version-loading
fix: [internal] Load MISP version just once in AppController
2019-11-11 15:15:20 +01:00
iglocska 78f3f68127
chg: [internal] Sharing group loader was grabbing organisations one by one, refactored
- simply fetch all org objects for the ACL checks in one shot instead of doing it on demand
  - has no real performance impact even on large sharing instances
  - reduces the number of queries greatly making debugging easier
2019-11-11 08:11:36 +01:00
mokaddem e396941dcf
fix: [attributes:restSearch] Fixed typo 2019-11-08 15:25:53 +01:00
Sami Mokaddem b141122bc0
Merge pull request #5381 from MISP/feature-meta-search
Feature meta search
2019-11-08 13:46:25 +01:00
mokaddem 3babd222ac
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-11-08 11:42:27 +01:00
mokaddem 437490872b
chg: [restSearch] Improved meta-search code
- Correctly returns nothing if search on metas does not return anything
- Renamed `orgc.sector` into `org.sector` while still being `orgc`
behind the hood
- Removed duplicated code
2019-11-08 11:37:43 +01:00
mokaddem 8e60c3d8d4
Merge branch '2.4' of github.com:MISP/MISP into feature-meta-search 2019-11-08 11:15:34 +01:00
iglocska 5f25f451df
chg: [sightingdb] Added support for bulk lookups and namespacing
- aligned with the latest version of the sightingdb (support for the /rb endpoint)
- added namespacing as an option / sightingdb connection, defaults to "all" if left empty
2019-11-08 08:24:04 +01:00
iglocska d24f23b5bd
Merge branch '2.4' into sightingdb 2019-11-06 21:22:00 +01:00
iglocska c4f1d4d15e
new: [SightingDB] Added integration with SightingDB
- Added configuration tool
- Added lookups from the event view
- Added includeSightingdb flag for the restSearch searches
- Added SightingDB search tool
- Added SightingDB connection test tool
2019-11-06 21:20:04 +01:00
iglocska 48b30c5095
fix: [Tagging] Fixed the user/org restrictions not being adhered to when tagging 2019-11-06 21:17:53 +01:00
mokaddem a53a06d080
new: [attribute:restSearch] Support of Orgc and GalaxyElement meta searches 2019-11-06 11:12:30 +01:00
mokaddem 1b1a3f61f3
new: [event:restSearch] Support of Orgc meta searches 2019-11-06 11:04:16 +01:00
mokaddem 7bb01d9b76
new: [event:restSearch] Initial work for GalaxyElement searches 2019-11-06 10:38:34 +01:00
mokaddem f58770a90a
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-11-05 13:51:03 +01:00
mokaddem b78d7825b9
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-11-05 13:48:39 +01:00
mokaddem e4564b587c
Merge branch '2.4' of github.com:MISP/MISP into feature-exclusive-taxonomy 2019-11-04 16:02:57 +01:00
mokaddem be8f6f0bb0
fix: [Tag:exclusive] Code clean up 2019-11-04 15:46:53 +01:00
mokaddem 21fac1d165
chg: [event:view] Exclusive notice UI improvements 2019-11-04 15:19:15 +01:00
mokaddem 0222b8c090
chg: [tag:exclusive] Added support of local while checking for
exclusivity
2019-11-04 14:55:43 +01:00
mokaddem 1aaa590e30
chg: [event:view] Added notice and improved inconsistency text message 2019-11-04 13:41:53 +01:00
Alexandre Dulaunoy 983a58afba
chg: [default] old default 'TLP Amber' is now 'tlp:amber' to be consistent and use MISP taxonomy naming 2019-11-04 12:41:52 +01:00
mokaddem 213e8b5049
new: [tags] Preminilary support of exclusive tags based on taxonomy data 2019-11-04 11:54:22 +01:00
Koen Van Impe 75f993e524 Force botvrij.eu feed to use HTTPS 2019-10-31 08:48:00 +01:00
mokaddem d63f81076b
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-30 13:58:43 +01:00
mokaddem 0087a49fcc
chg: Usage of camelCase instead of snake_case 2019-10-29 09:57:25 +01:00
Andras Iklody 77ac6e6525
Merge pull request #5348 from JanSkalny/2.4
new: [cli] server connectivity test
2019-10-29 09:32:33 +01:00
mokaddem 6eda00f701
chg: [timeline] Synchronize *-seen at Object and ObjectAttribute level,
few fixes and Improved UI
2019-10-28 15:45:33 +01:00
Jan Skalny 0ef0cf49cb new: [cli] server connectivity test 2019-10-22 20:13:11 +02:00
chrisr3d 103355c67a
fix: Bumped latest stix2 python version + Updated expected version in the diagnostic 2019-10-22 13:40:25 +02:00
Jakub Onderka 7539cbff2c
chg: [feed] Use precomputed hashes to speedup attaching correlation 2019-10-20 19:08:01 +02:00
mokaddem 5e8720c8ee
chg: [update] Added special log action for workers performing the DB
update
2019-10-15 12:07:29 +02:00
mokaddem b87ee19146
chg: [update] Actually reset `UpdateFailNumber` when manually unlocking 2019-10-15 11:44:34 +02:00
mokaddem e058c7a302
chg: [update] Improved feedback for update lock type in the UI 2019-10-15 11:31:22 +02:00
chrisr3d 086965347c
fix: [stix diagnostic] Updated the latest stix2 version 2019-10-15 10:39:10 +02:00
chrisr3d bab16edab7 Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2019-10-15 10:08:15 +02:00
chrisr3d b140d6be09
add: [restSearch] Support of stix1 json export 2019-10-15 09:54:48 +02:00
iglocska 6d8b76ae4f
fix: [user] Include user settings in /users/view 2019-10-15 07:24:43 +02:00
Jakub Onderka 4260216ad8
fix: [internal] Load Regexp just when they are requested 2019-10-14 20:17:14 +02:00
mokaddem 13173e8926
chg: [update] Added postgresql allowed DB update errors 2019-10-14 12:37:19 +02:00
mokaddem 5d4142f62a
chg: [update] Parametrized ignore_disabled in ondemand_action and
support of string update in update_progress
2019-10-14 10:49:41 +02:00
Jakub Onderka 9314e9c968 fix: [internal] Load MISP version just once in AppController 2019-10-13 11:29:43 +02:00
Jakub Onderka caa62220ff new: [internal] Attribute::isImage method 2019-10-12 09:34:49 +02:00
Jakub Onderka 688bab2778 chg: [internal] Simplify UserController::admin_edit 2019-10-11 20:35:27 +02:00
Jakub Onderka 899f53dd5a chg: [internal] Simplify User::extralog method 2019-10-11 20:35:26 +02:00
Jakub Onderka e4071f205e new: [internal] Log exact error for GPG diag in error log 2019-10-11 18:53:09 +02:00
Andras Iklody 9f7ac04ddb
Merge pull request #5263 from JakubOnderka/patch-51
chg: [sync] Code cleanup
2019-10-10 23:50:21 +02:00
Jakub Onderka 8e197d463d chg: [internal] Much better error handling for feed preview 2019-10-10 19:10:52 +02:00
mokaddem 0e2205c061
chg: [update] Added endpoint to release lock and integration with UI 2019-10-10 12:02:23 +02:00
iglocska e5ed126cc5
Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-10-10 11:59:28 +02:00
iglocska 15b10bbcf7
new: [user settings] Added restrictions for certain settings
- based on role permissions
- enforce the checks on set/delete
- add it to the UI elements
- /users/view /admin/users/view now include the user settings in a simplified format
2019-10-10 11:58:26 +02:00
mokaddem 571b4887f0
chg: [update] Do not release the lock if update fails more than 3 times 2019-10-10 10:14:22 +02:00
mokaddem 80b1061639
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-10 09:54:51 +02:00
Jakub Onderka d41951cf89 fix: [internal] Remove closing PHP tag 2019-10-09 19:34:54 +02:00
Richard van den Berg 7417f9d6a0 Check if Organisation index exists, fixes #4809 2019-10-09 02:22:18 +02:00
mokaddem 6652b922e6
chg: [appModel] Fixed merge conflict 2019-10-08 14:14:24 +02:00
mokaddem 015ec7d989
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline 2019-10-08 14:12:38 +02:00
mokaddem 8760c98c7e
Merge branch '2.4' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-08 12:02:09 +02:00
iglocska a8c57a8316
new: [API] Added rate limiting option to the API
- / role setting
- can be enabled/disabled and if enabled a limit can be set
- limit counter / 15 minutes starting from the first query
- x-headers inform the user about their limit/remaining queries/reset in seconds
2019-10-08 11:43:56 +02:00
iglocska 173054ccd3
fix: [internal] Set attribute restsearch page to 1 if limit is set without setting the page number 2019-10-08 08:27:21 +02:00
mokaddem 035d3be878
chg: [updateProgress] Usage of GPU for animation, deleted sleeps and
improved task
2019-10-07 15:12:47 +02:00
mokaddem 831e970e37
Merge branch 'revisedUpdateProcess' of github.com:MISP/MISP into revisedUpdateProcess 2019-10-07 11:52:26 +02:00
Jakub Onderka 87b568526f
fix: [internal] Remove unused function 2019-10-05 23:17:19 +02:00
garanews 85c28ce36e Fix some typo
Fix some typo
2019-10-04 13:02:59 +02:00
iglocska 5f8ff97616
fix: [internal] massive performance boost when loading events with a lot of objects 2019-10-04 09:40:04 +02:00
Jakub Onderka 3dbc6c0221
chg: [sync] Code cleanup 2019-10-03 19:58:15 +02:00
mokaddem c936d4ab51
chg: [updateProgress] Started taking into account stack of updates - WiP 2019-10-03 13:50:55 +02:00
Jakub Onderka 1d15722195
fix: [internal] user_count variable is already number 2019-10-03 12:02:21 +02:00
Andras Iklody 6bcd8b200c
Merge branch '2.4' into fix/import_empty_feed 2019-10-02 22:55:14 +02:00
Pierre-Jean Grenier 8415cea5db fix: [feed] fix REST problems 2019-10-02 16:53:44 +02:00