- refactor revealed that the sync user access on the remote was never correctly determined
- fallback method that has since been removed for 2+ year old instances was always used due to the above issue
- corrected list of parameters
- added sane defaults so that only the minimum list of fields is actually required
- fixed a bunch of stuff that was just plain broken with this API
- a server setting allowing the override of the path variable for esoteric RHEL systems allowed site admins to inject arbitrary commands
- impact was limited by the setting being only accessible to the site administrator
- as reported by Michael Grolimund from Swiss Post (@grolinet)
- CVE-2018-6926
Usage:
Viewing current setting value:
GET /servers/serverSettingsEdit/[mysetting]
Accept: application/json
Content-type: application/json
Authorization: [mykey]
Altering setting value:
POST /servers/serverSettingsEdit/[mysetting]
Accept: application/json
Content-type: application/json
Authorization: [mykey]
Body: {"value":"My new value"}
As a reminder, get all settings and diagnostics via:
GET /servers/serverSettings/download
Accept: application/json
Content-type: application/json
Authorization: [mykey]
- Encryption passwords as well as redis password are now redacted from the server settings
- Also includes the JSON dump of the server settings
- Thanks to cert.govt.nz for the security report.
- right now it's pretty dumb, it simply pulls the same branch that the current user is on
- Any failure is shown but not acted upon, if the git pull fails the user will see it but it needs to be resolved via the command line
- Diagnosing not loaded extensions was a nightmare
- New system checks the loaded extensions via php and php-cli (could help with un****ing some RHEL/CentOS issues)
- Version check for the php-cli php version added
- only one extension is checked currently, to be updated at a later point in time (remember to also update the web and the cli extension list!)
- Only allow enabling internal mode if the host organisation is set and it is chosen as the remote organisation when adding the server sync
- This ensures that internal sync only happens when the same organisation owns both instances
- removed incorrect, useless boiler plate comments
- kept useful comments intact
- added some missing line breaks to make the codebase a bit more uniform
- removed some obviously obsolete TODO comments
- tied into auto upgrade system
- tied into server settings
- some cleanup of overly verbose debug
- Enforcing enable/disable everywhere
- Changed temporary file structure
- sync users that have not accepted the terms / have had a password reset initiated were redirected to the login page
- fixes to the issue
- if a user with automation/sync access uses the API and gets blocked because the terms weren't accepted or there is a pending password change they will be notified in a JSON/XML response
- the sync test now takes this into consideration starting with this version and will report the cause of the failure
- Both instances have to be 2.4.24+ for this to be reported correctly
- Added a way to remove the certificate file when editing the server connection
- Also, it shows the currently selected certificate file as it caused some confusion before
- red warning on the settings page if the config.php file is not writeable
- failed changes in settings due to the config.php file not being writeable logged
- made the PHP settings check look a bit more clear and changed it from failures to recommendations
- added a file permission check for config.php (can add more in the future such as the background worker log files which can prevent the workers from starting)
- as discovered and reported by Egidio Romano of Minded Security
- Lacking checks of HTTP methods in some functionality could lead to a site admin uploading and executing malicious scripts
- Tightened HTTP method verification across the board for actions that modify data
- Turned some administrative tasks to POST only actions
Merging all the new changes from master
Conflicts:
VERSION.json
app/Console/Command/AdminShell.php
app/Controller/AttributesController.php
app/Controller/EventsController.php
app/Model/Attribute.php
app/Model/Event.php
app/Model/Log.php
app/Model/Server.php
app/Model/User.php
app/View/Elements/side_menu.ctp
app/View/Pages/administration.ctp
app/View/Users/admin_index.ctp
- finished preview feature
- can now view events and attributes remotely
- can copy over new event to local instance
- new sync mode (update)
- allows to only pull changes to events that exist locally already
- works well with the manual pull of events, no need to pull events that we didn't manually confirm, but can still update all events that we pulled over
- Fixed an issue with background tasks causing the logging to fail
- reworked connection test showing version numbers of both instances
- also telling the admin whether the sync is compatible or not
- Further refactoring / tweaking of the vent view
- implemented a custom pagination tool for data sets that are not directly taken from teh db
- currently creates a pagination object that mocks CakePHP pagination
- supports the CakePHP pagination view helper
- supports: pagination, sorting, custom filters
- implemented first step of the remote instance browser for admins
- view an index of events on another instance
- filter the events
- uses the new pagination
- still missing:
- remote event view
- fetch event from remote instance
- reworked the event view
- separated API and UI code path
- major speedup for the API!
- cleaner code as there was almost 0 overlap
- discussions and attributes are now loaded separately from the event view
- added after the event view loads via ajax
- cleaner pagination
- attribute pagination now finally allows for sorting
- future improvement (coming soon): Show proposals only filter
- filtering on the attributes in general
- diagnostic tool would throw exceptions because the db session tables are still missing in some older instances
- if a different session handler is used, the test is skipped
- fixed an issue where pushing a single event would fail
- both event and attribute edits via the API work without providing a timestamp. The current timestamp is instead attached
- both event and attribute edits fill the required fields from the data in the database if not supplied (as long as the uuid is found)
- new functionality: Event blacklisting by UUID
- site admins cna enable this feature in the server settings
- enabling the feature will make the required db changes
- any deleted event will automatically get blacklisted
- this prevents deleted events from flowing back from a synced instance
- site admins can manually add UUIDs to the list and remove entries
- fix to UUID duplication issues for attributes
- simply run the admin script and it will regenerate the UUID of attributes that are duplicates, if any such exist
- timestamps/event published status will not be affected
- config.core.php now includes a change that prevents from 404 exceptions being logged
- the sync uses 404s to signal that an event with a given uuid does not exist when negotiating proposal synchronisation
- this causes a dangerously high amount of noise in the logs
- python server running in the background doing the publishing
- MISP -> python script communication via redis
- configurable / controllable via the admin UI
- by installing the requirements described in the update and the install instructions (ubuntu only for now, centos/red-hat versions to be tested and described), administrators can enable the pub/sub feature
- assign a port to the service via the interface
- each time an event is published, MISP will use ZMQ's PUB feature to push out a MISP JSON package using the "misp_json" prefix
- users can specify an alternate gnupg executable
- Since GnuPG2 is not compatible with the last stable CryptGPG version, there are 3 options for CentOS / Red Hat users:
1. Don't use a passphrase for the server's PGP key
2. Install the beta version of CryptGPG (1.4.0b4)
3. Install GnuPG classic and point MISP to the executable
- This patch enables option 3, administrators can point MISP to the alternate executable in the server settings
- server settings and validation work
- configurable template via settings
- configurable via API as well
- Also trying to define the structure for future Plugin settings
- The idea is to have them in a separate tab all prepended with the plugin name
- since this is not yet part of the future flexible plugin system, it is still kept in the main codebase, but the idea is to get the naming conventions ready for the future version
- add the remote organisation while adding a server
- remote organisation can be chosen from the list of local or known remote organisations. Alternatively a new remote org can be created on the fly
- Several UI changes
- Scheduled pulls should work correctly now
- Scheduled pushes and pulls correctly display in the logs
- Scheduled caching correctly sets the next date of execution
- Important! Logo images have now moved to a different location! Make sure that you update your settings!
- Site admins can now manage the uploaded image files and the terms of use file via the server settings interface
- add, link, delete files directly from the interface
- Scheduled tasks for pull / push now working as intended
- Rescheduling of all tasks fixed
- protection against the rescheduled task ending up in the past
- further event history fixes
- fixed lots of erroneous logging
- performance improvement with logging (no longer loading controllers for no reason)
- logging extra actions that weren't logged before (proposal accept / discard, server pull / push)
- you can now upload a certificate file and allow a server link to use a provided self signed certificate. This should solve the issues that some organisations are having when trying to connect their instances
- Corrected some weak notifications on background jobs
- Changed the view slightly to view background jobs
- fixed an issue where editing a sync server setting would cause an error due to the id not being passed to the logging plugin
- reworked almost all of the side menues to be centralised
- Some fixes for the IOC export not handling two new-ish types correctly
- Some changes to the menues (including a few options that didn't exist before)
- rework of the popovers in some forms
- ADMIN org removed.
- Siteadmins are now identified by the perm_site_admin flag
- Siteadmins can now be of any organisation
- editing the regexp / whitelist rules can now be done by a special user with the perm_regexp_access in his/her role
- Executing a mass replace of attribute values based on the regexp rules cannot be initiated by a regexp/whitelist user, only by a site admin
- If the login page is reached without any users / roles defined they are automatically created (perviously it was only the user that was created)
- Org admins are restricted from assigning perm_site_admin, perm_sync and perm_regexp_access roles to users. This can only be done by a site admin.
- until now checkAction was used to check permissions of a user
- but since all of the role permissions are checked beforefilter in
appcontroller and saved into a public array, doing a lookup of the
array saves an SQL call for each permission check.
- there was a bug that pushed the data entered into the "published"
filter field to the date fields -> fixed
- Also a bug in the serverscontroller, pulling threw an undefined
warning from the log controller because a single saveField was used and
the logController couldn't save the url data for the action
- some changes to the access control
- re-renabled regexp and blacklists, will need a closer look though
- editing a role should update ACL
- some other minor things
- Fixed issues with the sync
- Secondary publishes on remote servers failed
- Introduced new fields in events to stop backward traverse of
edit information that lead to low performance and eroneous
distribution information updates when more than 2 servers were
linked
- Deletion of an attribute now deletes on remote servers
- Changes to the event ownership
- Original creator org now noted in the event itself
- Only original creator org can change distribution
- Events will show up with the original creator org for users
(admins can see both that and the owner of the event on the
local instance)
- Server.organization now used in junction with the connecting
user's org and the instance's org (from the bootstrap) to
determine distribution flow control and access rights
- Lots of minor changes
- perm_auth new toggle, can disable auth key usage for a role
- prevents sync / rest with a perm_auth == false key
- some changes to sync to provide better feedback on why it failed
- rewording of distribution options
Steve Clement
iglocska