- eventid a new parameter for both event and attribute restsearch
- these APIs now accept arrays in both json and xml format (you can send "eventid": ["15", "16"] instead of "eventid": "15&&16" in addition to the old functionality
- enter a UUID in the event ID field of the attribute search to find attributes belonging to a certain event
- use event IDs / UUIDs to filter events on the event index
- new functionality: Event blacklisting by UUID
- site admins cna enable this feature in the server settings
- enabling the feature will make the required db changes
- any deleted event will automatically get blacklisted
- this prevents deleted events from flowing back from a synced instance
- site admins can manually add UUIDs to the list and remove entries
- fix to UUID duplication issues for attributes
- simply run the admin script and it will regenerate the UUID of attributes that are duplicates, if any such exist
- timestamps/event published status will not be affected
- config.core.php now includes a change that prevents from 404 exceptions being logged
- the sync uses 404s to signal that an event with a given uuid does not exist when negotiating proposal synchronisation
- this causes a dangerously high amount of noise in the logs
- as explained on the automation page
- also, better error handling
- all API calls that fail during authentication will now return a JSON/XML error message instead of redirecting to the login page
- simply pass an MD5 hash along and receive a sample if available zipped and base64 encoded in a response object
- pass any hash along with a flag set and receive any samples from events that have the passed hash
- Also, fix for an issue with the freetext import not using semi-colons as separators
- new API for uploading malware samples
- allows the upload of several files
- can be used to populate a pre-existing event, or create a new event
- expects a JSON or an XML object with the samples base64 encoded
- new way of storing malware samples
- original filename not used any longer
- samples are renamed to their md5 hashes
- original filename preserved in a secondary txt file
- removed filename validation as it is no longer used for the command line execution
- this allows unicode name files to be uploaded!
- changed the UI attachment upload to reflect these changes
- code more centralised and extendible
- added the new flag "last" to the list of parameters
- exports affected: XML, CSV, NIDS, HIDS, STIX, Text, RestSearch
- Valid values: number + format where format can be d, m, h for day, minute, hour (examples: 5d or 12h or 30m)
- documented in automation view
- right now it follows the simple rule of user > admin settings > default values when generating the export
- Parameters can be passed via url / JSON object / XML object
- filters include filter on event ID, date range, tags
TODO:
- buttons for a per event download via the UI
- introduce new export option for normal users (via background workers and the old style export)
- server settings and validation work
- configurable template via settings
- configurable via API as well
- Also trying to define the structure for future Plugin settings
- The idea is to have them in a separate tab all prepended with the plugin name
- since this is not yet part of the future flexible plugin system, it is still kept in the main codebase, but the idea is to get the naming conventions ready for the future version
- Unified the way exports accept negated parameters
- Fixed the documentation
- Most exports are now restrictable by the event date (From/To parameters)
- none cached XML export now writes to file after converting each event, clearing the memory and resolving any potential memory issues
- Deleting attributes on connected MISP instances can cause serious performance issues on multiple interconnected instnaces, temporarily removed
- Version number incremented
- The event export buttons have been unified into a single download as... button
- clicking it loads a popup with all of the export formats
- added snort, suricata, text dump to the export options
- added the option for an extra setting for some exports (such as including non IDS flagged attributes, encoding attachments)
- easily extendable system
- moved the hidden popup divs into the general layout, can be easily reused anywhere
- removed the auth refresh option that was re-enabled recently as it seems to sometimes cause issues
- text exports now allow "all" to be specified as type, which will dump all attribute values that the user can see
- text exports now allow restricting the results based on event id
- UI improvements, events appear unpublished after ajax queries that alter attributes
- Events get unpublished by the attribute replace tool and template population as they should
- currently to_xml() has performance issues, if it's not resolved fast, it would be a good idea to move the export to the background workers
- some UI changes
- first version of templating system complete
- first version of freetext importer complete
- first version of mass attribute replace tool complete
- some UI changes
- resolved bugs with permissions
- fixed the broken mass delete tool
- Fixed an issue with the type not being chosen correctly for file type attributes when created through the templating tool
- malware samples / attachments couldn't be downloaded
- links weren't actually links
- deleting an attribute / shadowattribute now opens a custom confirmation dialogue. This is also where the CSRF tokens are generated for the post request to execute the delete, resulting in a faster event view load
- AJAX requests now also respond with a small message at the bottom of the page, notifying the user of the result
- The following actions work now on the event page via ajax:
1. Add / remove tags
2. quick edit any attribute field if eligible
3. quickly create a proposal of any attribute field if not eligible to edit
4. popover attribute creation (also works with batch add)
5. popover proposal creation (also works with batch add)
6. delete attributes
7. accept/discard proposals
8. mass edit / delete attributes
Also, replaced the old memberslist, with a small lightweight css/js based one.
- forms are now dynamically pulled onclick
- performance greatly enhanced
- solves the issues with the CSRF protection kicking in if the user edits a field after using the back button
- Exporting a JSON object erroneously included related objects which prevented the exported event from being added back to MISP via the API
- Downloading search results as XML / CSV now correctly includes all of the search results instead of just the 60 visible ones on the UI (cut off by the pagination)
- The tags parameter in the exports now correctly accepts null as a valid value even if it is the last parameter
- users can now edit all fields in an attribute whilst on the event page
- issues left to fix:
- tag changes after an attribute change run into CSRF protection
- batch add not handled gracefully yet
- going back to the event view and editing a field gives users an error message over the CSRF protection - instead, silently check if the page is loaded in a dirty way and refresh the ajax fields silently
- quickadd of attributes still missing
- events/restSearch, attributes/restSearch, events/xml, attributes/returnAttributes
- users can now POST a search array in XML / json instead of sending the parameters in the url
Iglocska