Merging all the new changes from master
Conflicts:
VERSION.json
app/Console/Command/AdminShell.php
app/Controller/AttributesController.php
app/Controller/EventsController.php
app/Model/Attribute.php
app/Model/Event.php
app/Model/Log.php
app/Model/Server.php
app/Model/User.php
app/View/Elements/side_menu.ctp
app/View/Pages/administration.ctp
app/View/Users/admin_index.ctp
- .sql file to add all the new fields / tables
- admin tool to convert the old organisation fields to the new objects
- still missing a cleanup method (to remove the old organisation fields once the conversion is done)
- CSV export ignored the tag parameters
- tagging events didn't work as expected in some cases
- timing out and clicking on an admin action results in being redirected to a non-existing admin login page
- distribution setting ignored when uploading attachments
- UUID uniqueness was previously not enforced
- changed the MYSQL.sql file to reflect the changes
- Added upgrade admin tool to remove duplicate events and make the database changes required
- Tweaked the tool for the attribute uuid fix so that it cannot created duplicate keys
- some minor fixes, such as automatically removing eventTag objects on event deletion
- new functionality: Event blacklisting by UUID
- site admins cna enable this feature in the server settings
- enabling the feature will make the required db changes
- any deleted event will automatically get blacklisted
- this prevents deleted events from flowing back from a synced instance
- site admins can manually add UUIDs to the list and remove entries
- fix to UUID duplication issues for attributes
- simply run the admin script and it will regenerate the UUID of attributes that are duplicates, if any such exist
- timestamps/event published status will not be affected
- config.core.php now includes a change that prevents from 404 exceptions being logged
- the sync uses 404s to signal that an event with a given uuid does not exist when negotiating proposal synchronisation
- this causes a dangerously high amount of noise in the logs
- simply pass an MD5 hash along and receive a sample if available zipped and base64 encoded in a response object
- pass any hash along with a flag set and receive any samples from events that have the passed hash
- Also, fix for an issue with the freetext import not using semi-colons as separators
- added a new entry to the admin tools (Administartion -> Administrative tools)
- converts title and change columns in the logs table to text from varchar(255)
- user will get an explanation of the csrf error and that going back and refreshing the form will fix it
- also, there is a link that will take the user to the baseurl (which will redirect to the login page if the csrf issue occured on the login page)
- Users being logged on would not be able to use the actions that are also used for automation
- Those actions trigger a check of the authorization header, which in certain use cases can be set with values that is outside of the scope of MISP
- MISP will now try to only detect MISP auth keys in the headers and if it detects something else it ignores it
- The event export buttons have been unified into a single download as... button
- clicking it loads a popup with all of the export formats
- added snort, suricata, text dump to the export options
- added the option for an extra setting for some exports (such as including non IDS flagged attributes, encoding attachments)
- easily extendable system
- moved the hidden popup divs into the general layout, can be easily reused anywhere
- removed the auth refresh option that was re-enabled recently as it seems to sometimes cause issues
- text exports now allow "all" to be specified as type, which will dump all attribute values that the user can see
- text exports now allow restricting the results based on event id
MYSQL.sql and upgrade_2.3.sql updated
Fixed incorrect proposal counts showing up due to attributes that are flagged for deletion also being counted
Added some extra fields to the view proposal view to make it more useful
- Templates can now be created and populated
- Users can populate an event using a template (still needs work)
- File type elements are not yet implemented
- Authentication via headers was only allowed if _isRest() returned true
- this only happened for pages returning JSON or XML content
- a new check, _isAutomation() was added that allows authentication via headers for certain methods used by the automation system
- xml version now included in the xml exports
- MISP will now check the xml version on all imports related to sync / add MISP XML and try to update the incoming info if it detects an older version
- exports now take tag names as a parameter (affected exports: XML, text, HIDS, NIDS)
- eventtags now correctly get removed when an event is deleted
- authorization key should be sent through headers.
- passing it in the url is deprecated
- updated automation page to reflect the changes
- csv export now has headers
- Contextual comments for proposals
- shows proposal count in the top bar
- new view showing all of the events of the user's organisation with an active proposal
Iglocska