Commit Graph

128 Commits (ad1b373766c79d8ad9c17b88cc6343043607d27a)

Author SHA1 Message Date
iglocska c5fe480745
chg: [logging] Truncate description lengths that would be longer than what the DB can store with the default setup 2019-08-13 12:03:30 +02:00
iglocska 122ff89f2f
new: [setting] Disable DB logging completely, fixes #4921
- Not recommended, but for certain use-cases it might be desirable
2019-07-31 09:52:05 +02:00
iglocska 36b43f1306
fix: [security] Org admins could reset credentials for site admins
- org admins have the inherent ability to reset passwords for all of their org's users
- this however could be abused if for some reason the host org of an instance would create org admins
  - the org admin could set a password manually for the site admin or simply use the API key of the site admin to impersonate them
- the potential for abuse is very circumstancial as it requires the host org to create lower privilege org admins instead of the usual site admins
- only org admins of the same organisation as the site admin could abuse this

- as reported by Raymond Schippers
2019-06-11 11:05:34 +02:00
iglocska e89b4525ad
new: [logging] Added paranoid logging mode
- will log ANY query's (UI/API):
  - http method
  - requested URL

- optionally disable DB logging for paranoid log entries
2019-05-17 11:45:20 +02:00
Nikos Filippakis 9d59b10368 Publish events to Kafka
Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2019-03-18 15:53:22 +01:00
Nikos Filippakis b783b5e393 Fix checking of incorrect flags when publishing to ZMQ
* Check Plugin.ZeroMQ_event_notifications_enable instead of Plugin.ZeroMQ_attribute_notifications_enable in Event.php
* Check Plugin.ZeroMQ_audit_notifications_enable instead of Plugin.ZeroMQ_user_notifications_enable in Log.php
* Check Plugin.ZeroMQ_object_notifications_enable instead of Plugin.ZeroMQ_attribute_notifications_enable in MispObject.php

Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2019-03-13 11:51:40 +01:00
Christophe Vandeplas 67efc70bf5 fix: [style] consistent space indentation 2019-02-10 13:08:55 +01:00
iglocska a81894f14c chg: [CS] Changed to PSR-2
- to make contributions easier, adopted PSR-2
- used php-cs-fixer to rework the style
- *sniff sniff* Goodbye tab indentation
2018-07-19 11:48:22 +02:00
Hannah Ward f7f5296de6
fix: Make indentation line up 2018-07-12 11:03:01 +01:00
Hannah Ward a70f8e45d8
new: Add ability to log to elasticsearch 2018-07-10 17:01:57 +01:00
StefanKelm 9951590ef8
Update Log.php 2018-04-23 15:06:56 +02:00
StefanKelm 739210c58f
Update Log.php
Alphabetically sort list of Actions pull-down menu within "Search Logs"
2018-04-23 14:59:01 +02:00
iglocska 6135468c41 new: Added full audit logging to ZMQ and Syslog, fixes #2635
- syslog now includes all audit log entries and it's separated into proper severity levels
- ZMQ logging and syslog logging are both optional features
2017-11-24 12:01:53 +01:00
iglocska d6b6767b0d fix: Add galaxy to valid log action list 2017-11-16 12:16:03 +01:00
iglocska bb4be714ca fix: Truncate log descriptions that are over 65532 character long 2017-06-09 11:46:00 +02:00
iglocska 3cccbb9c5c fix: Truncate the change field in log entries if it becomes humongous
- solves a rare situation with massive PGP keys breaking user additions / edits
2017-05-29 07:59:44 +02:00
iglocska b4b52a77e3 fix: Truncating the title of a log entry at 65KB for some pretty rare edge cases 2017-02-27 18:04:28 +01:00
iglocska 47075721e0 chg: Added default log org entry 2017-02-02 18:09:26 +01:00
iglocska 5def49a028 fix: Remove the admin setting changes too using the prune job 2017-01-30 15:44:56 +01:00
iglocska 48843b60ec fix: Fix and cleanup script for a specific bug
- rare occurance, but some MISP servers enter an upgrade loop causing massive amounts of log entries
- this patch cleans up the bug preventing further upgrade loops as well as offers a script to clean up the fallout
2017-01-30 09:16:43 +01:00
iglocska 341d23892b fix: org field missing in log entry causing proposal sync to fail
- Added SYSTEM as the default value
2016-12-29 16:06:11 +01:00
Andreas Ziegler 9bf0e16ac6 new: add basic experimental support for PostgreSQL 2016-08-31 04:11:49 +02:00
iglocska 5a72f84c22 Merge branch '2.4' into 2.4.51 2016-08-28 21:08:02 +02:00
iglocska 822b0bf8fa chg: Cleanup of the controllers and models
- removed incorrect, useless boiler plate comments
- kept useful comments intact
- added some missing line breaks to make the codebase a bit more uniform
- removed some obviously obsolete TODO comments
2016-08-25 11:38:37 +02:00
iglocska 734df04000 new: New piece by piece stix export allowing large datasets to be exported 2016-08-23 00:24:54 +02:00
iglocska 5d412b426d fix: fixed an issue with the histogram on newer MySQL versions 2016-08-17 16:25:39 +02:00
iglocska b9f5297b3a fix: Fixed some issues with the misp export importer and added better logging. 2016-08-10 12:06:36 +02:00
Steve Fossen b6fb900ab6 log created is datetime not timestamp. 2016-07-12 10:14:53 -07:00
Andreas Ziegler 1d06f25b38 chg: add newline character before EOF to non-minified (text-)files 2016-06-06 10:09:55 +02:00
Andreas Ziegler 0fe692c56a remove whitespace at end of line 2016-06-04 01:10:45 +02:00
Andreas Ziegler 898ea1d97c remove whitespace (space/tab) from empty lines 2016-06-04 01:08:16 +02:00
Iglocska 3edf8157ca Quick filters for the logs 2016-05-15 19:50:12 +02:00
Iglocska 4020628df4 chg: Further work on the soft deletes 2016-05-10 10:31:56 +02:00
Iglocska 24c7fa61fe Merge branch 'permissionfix' into 2.4 2016-04-18 17:41:59 +02:00
Iglocska 03cec95ef1 Some cleanup for the sync
- fixed some issues with the error detection on synced events
- pre-filtering of events based on sync filters before pushing them should improve performance a great deal
2016-04-06 10:30:25 +02:00
Iglocska 4c14d3a859 Merge branch '2.4' into features/delegation
Conflicts:
	app/Controller/AppController.php
	app/Model/AppModel.php
	app/Model/Event.php
	app/Model/Log.php
	app/Model/Server.php
	app/View/Elements/footer.ctp
	app/webroot/css/main.css
2016-02-12 05:56:32 +01:00
Iglocska a1ffdc7790 First finished version 2016-02-12 05:47:06 +01:00
William Robinet 4fea371c4b Fix permissions 2016-02-11 17:03:51 +01:00
Iglocska 1362cf40b7 Added org merge tool
- allows a site admin to merge all objects belonging to an organisation into another
  - this can be useful if duplicate organisations exist for example
  - the tool overrides the built in mechanism and should only be used if absolutely required
  - at the end of the process the original organisation is removed

- the tool generates 2 files that are dropped in the log directory of MISP
  - 1 contains a JSON with all the changed fields and the IDs
  - 1 contains an SQL script that allows an admin to revert the changes
2016-01-27 23:59:34 +01:00
iglocska fea225c409 Rework of the contributor field, some MYSQL.sql tweaks
- added indeces to the MYSQL.sql file
- contributors now looks for shadow attributes instead of log entries (should make the event view much faster and resolve some timeout issues on sync when the log is massive)
2015-12-27 19:00:35 +01:00
iglocska dd53f7bfa2 Added default values to the log entry creation to avoid empty fields giving notices, fixes #769 2015-12-17 11:01:54 +01:00
iglocska 3449291533 Fixed issue with the headmmap, fixes #759 and fixes #760 2015-12-16 23:18:05 +01:00
Iglocska 714ed198e5 Various fixes throughout the application
- org field still used in some places other than the legitimate use-cases
2015-11-28 23:49:52 +01:00
Iglocska d20946f5ed Fix to logging causing certain functions to fail on migrated installations 2015-11-27 14:10:25 +01:00
iglocska 3784209cf5 First bash at Taxonomies
What works:
- added submodules for taxonomies
- added import tool for taxonomies
- added models and convenience functions for taxonomies

- site admins can update taxonomy libraries
- list taxonomies / view indvidual ones (with all resolved tags)
- create tags manually if a taxonomy is enabled
- view related tags / events quickly from the Taxonomy view

What doesn't work:
- Users still cannot choose a tag from taxonomy lists (this will be the main functionality)
- Feature cannot be disabled
2015-11-24 03:27:14 +01:00
iglocska 485c007b39 Merge branch 'master' into 2.4-beta
Conflicts:
	VERSION.json
	app/Lib/Tools/XMLConverterTool.php
	app/Model/Event.php
	app/Model/EventTag.php
	app/Model/TemplateElementAttribute.php
	app/Model/TemplateElementFile.php
	app/Model/TemplateElementText.php
	app/Model/ThreatLevel.php
	app/View/Attributes/index.ctp
	app/View/Elements/eventattribute.ctp
	app/View/Elements/eventattributerow.ctp
	app/View/Elements/global_menu.ctp
	app/View/Elements/side_menu.ctp
	app/View/Events/automation.ctp
	app/View/Events/index.ctp
	app/View/Pages/administration.ctp
	app/View/ShadowAttributes/index.ctp
	app/View/Tags/index.ctp
2015-11-17 01:14:51 +01:00
iglocska da5fac5873 Added logging of auth key changes, fixes #715
- Changing the auth key now creates a log entry that inclues the user's ID, e-mail address old and new autkeys
- Also removed the logging of the hashed password for newly created users
2015-11-16 00:22:58 +01:00
iglocska 324418dcf9 Merge branch 'master' into 2.4-beta
Merge and upgrade of several new features

Conflicts:
	VERSION.json
	app/Controller/ShadowAttributesController.php
	app/Controller/TagsController.php
	app/Model/AppModel.php
	app/Model/Event.php
	app/Plugin/SysLogLogable/Model/Behavior/SysLogLogableBehavior.php
2015-11-08 23:26:19 +01:00
iglocska 16e7974fc0 Improved logging, fixes #695
- Added logging of failed login attempts
- Added (optional) logging of successful authentications
  - admin setting that has to be enabled
  - will log all API calls (both HTTP method and target url)

- optional logging of user IP address for all logs
  - each log entry created while this setting is enabled will log the IP address of the client
  - disabling it also hides the IPs from the interface
  - added new IP field for the log search (only if enabled)
2015-11-08 22:35:46 +01:00
Iglocska 328c666e3a Merge branch 'master' into feature/sg
Merging all the new changes from master

Conflicts:
	VERSION.json
	app/Console/Command/AdminShell.php
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Log.php
	app/Model/Server.php
	app/Model/User.php
	app/View/Elements/side_menu.ctp
	app/View/Pages/administration.ctp
	app/View/Users/admin_index.ctp
2015-10-12 09:41:20 +02:00
Iglocska dc638adac2 Upgrade to CakePHP 2.7, fixes #684
- cakephp submodule updated to 2.7
- make sure that you update your instance!

- not updating will not break compatibility
2015-10-09 15:59:25 +02:00
Iglocska e242c9add2 Set of changes to the sync
- finished preview feature
  - can now view events and attributes remotely
  - can copy over new event to local instance

- new sync mode (update)
  - allows to only pull changes to events that exist locally already
  - works well with the manual pull of events, no need to pull events that we didn't manually confirm, but can still update all events that we pulled over

- Fixed an issue with background tasks causing the logging to fail

- reworked connection test showing version numbers of both instances
  - also telling the admin whether the sync is compatible or not

- Further refactoring / tweaking of the vent view
2015-10-06 01:16:48 +02:00
Iglocska 3fb7764c00 First revision of the upgrade scripts
- .sql file to add all the new fields / tables
- admin tool to convert the old organisation fields to the new objects
- still missing a cleanup method (to remove the old organisation fields once the conversion is done)
2015-09-02 01:34:52 +02:00
Iglocska 704880ce59 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Server.php
2015-08-30 13:29:05 +02:00
Iglocska 80dcdcf9d8 Database update admin-actions are now logged and if they fail the errors are logged 2015-08-18 09:44:59 +02:00
Iglocska 7e10eb7b46 Some fixes to the upload malware API
- Threat level ID options correctly set
- Threat level ID validation tightened to reject anything but the existing threat levels
- The upload malware API now logs validation issues during the failed creation of attributes / events
2015-08-05 00:11:16 +02:00
Iglocska 1bf2995f4e Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
2015-07-22 17:19:13 +02:00
Iglocska 1645ee1e3b Rework of the diagnostics for background workers
- shows dead background workers
- allows site admins to add workers to any queue on the fly
- allows site admins to kill workers on the fly
2015-07-12 21:45:11 +02:00
Iglocska c73f71f243 Merge branch 'master' into feature/sg
The merging is complete

Conflicts:
	VERSION.json
	app/Console/Command/ServerShell.php
	app/Controller/AppController.php
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Controller/PostsController.php
	app/Controller/UsersController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Log.php
	app/Model/Server.php
	app/Model/User.php
	app/View/Elements/side_menu.ctp
	app/View/Users/admin_index.ctp
	app/webroot/js/ajaxification.js
2015-06-10 22:54:20 +02:00
Iglocska a81f4a19ed Server setting changes logged, fixes #531 2015-06-09 09:55:22 +02:00
iglocska ecf9facf07 Further progress 2015-05-25 20:54:10 +02:00
Iglocska 3e28f80f66 checkVersionCompatibility tool finished
- compares the local to the remote version
- creates log entries for mismatches / connection issues
- should be used for any server to server action
2015-04-26 11:40:54 +02:00
Iglocska c24704a6c2 Lots of progress
- further work on implementing the SG changes everywhere
- reworked the alert e-mails
- reworked a lot of the logging
- several convenience methods
2015-04-20 11:46:55 +02:00
iglocska 3723119d83 Logging of admin emails and auth refresh
- admin emails now generate log entries
- authentication is refreshed on activity
2014-12-12 10:42:12 +01:00
iglocska 16f1072120 Statistics changes
- remove actions such as login, logout, changepw
- fixed range so that a addinga a massive event doesn't make every other day seem less active
2014-02-12 17:17:15 +01:00
iglocska 949f0fd009 Fix to the statistics page
- heatmap now fed the correct data
2014-02-12 16:49:14 +01:00
iglocska c8dfdff1e7 Changes to the installation instructions
- some changes also to the scripts
- replaced old scripts with newer versions (jquery, d3)
- Some updates to the manual (still needs more work)
2014-02-11 15:54:46 +01:00
iglocska 62a5b259c2 Bruteforce logging
- if a user becomes blacklisted, the system will log it. Fixes #206
2014-02-05 14:30:59 +01:00
iglocska f51d61f8f9 Various changes
- contributors shown on the event view (list of the organisation logos of users that have contributed through proposals)
- these link to the event history containing only entries from their organisation

- changes to the activity heatmap
- heatmap now dynamically changes the range on the graph based on the obtained values
- performance improved
- buttons to move back or forward in time on the calendar

- Attributes:
- warning for the user if he/she has selected the attribute category "targeting-data" or "attribution" as these could contain classified information
- UI improvements across most attribute and shadowattribute input views

- Updated cal-heatmap to the newest version
2014-02-05 13:45:18 +01:00
iglocska 70c7f650f6 Changes to the logging and scheduling
- Scheduled tasks for pull / push now working as intended
- Rescheduling of all tasks fixed
- protection against the rescheduled task ending up in the past

- further event history fixes
- fixed lots of erroneous logging
- performance improvement with logging (no longer loading controllers for no reason)
- logging extra actions that weren't logged before (proposal accept / discard, server pull / push)
2014-01-29 15:52:09 +01:00
iglocska 2740da9094 Changes to the log system
- View Event history now shows the logo of the org whose action triggered the log entry
- View Event History now shows different fields than before
- Proposals now logged
- Accepting / Discarding a proposal now doesn't create junk edit / delete entries as before.
- Creators of an event can now see all of the log entries altering an event in the event history log. This includes deleted events.
2014-01-28 16:27:58 +01:00
Noud de Brouwer a9a1bc91a1 AdminCrud and coding standard
more AdminCrud and coding standard clean up.
2013-01-04 15:48:46 +00:00
Andras Iklody 1ceadab700 Added features from branch analysis_levels
-Analaysis levels setable for events as per milestone item 94
-Password change forced as per milestone item 109
-Added feedback on entered search terms for search attributes
-fixed the authentication issue
-some minor fixes
2012-12-17 15:51:30 +01:00
noud 0232148631 dropdowns
better optgroup support in dropdowns where 'ALL' or '' is used
in Search Attributes and Search Logs.
2012-10-24 10:42:32 +02:00
noud 8f3d624c1a Merge branch 'master' into develop
Conflicts:
	app/Controller/AppController.php
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Controller/ServersController.php
	app/Controller/UsersController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Server.php
	app/Model/User.php
	app/View/Attributes/edit.ctp
	app/View/Attributes/index.ctp
	app/View/Elements/actions_menu.ctp
	app/View/Events/add.ctp
	app/View/Events/index.ctp
	app/View/Events/view.ctp
	app/View/Events/xml/view.ctp
	app/View/Servers/index.ctp
	app/View/Users/admin_index.ctp
2012-09-24 16:02:01 +02:00
noud 66c5312ea6 DataBase migrate, Audit and Access Control granulation. 2012-06-28 17:24:12 +02:00
root b4558887ce Revert "Audit and ACL first cut."
This reverts commit 5818231f48.
2012-06-26 09:40:52 +02:00
noud 5818231f48 Audit and ACL first cut. 2012-06-25 15:54:52 +02:00