- import feed descriptor json pastes to add a list of pre-defined feeds
- improvements to the feed pull (a single non validating attribute shouldn't break the process)
- altered the saving of the attributes to happen in chunks during a feed pull to avoid very large feeds from stalling the process
- split the feeds into 3 tabs: default, custom, all
- simply load any enable warninglist entries from the pre-defined TLD warninglists
- Pass the resulting array to the complex type tool
- during domain type heuristics, if the TLD list is not empty use the supplied list
- alternatively generate a list based on the old TLD rules
- does not alter any functionality otherwise
- Added the CSV feed format
- users can specify which fields in the CSV should be parsed
- comment lines are automatically omitted
- new settings system added to feeds, currently only used for the value fields
- Slight rework of the correlation lookup for the feeds
- got the Speed Force treatment
- correctly checks against value1 and value2 instead of value
- Various freetext import fixes
- new syntax
- old syntax still accepted
- new tool for rearranging request data to allow the APIs to automatically catch and correct typical rearrange errors
- freetext is now a valid return format, it will allow module developers to return an unparsed text blob which MISP will try to loop through the freetext import's detection mechanism
- still a lot of improvements to be done for the detection mechanism
- error handling for modules, instead of discarding errors they are now shown as a flash message on the freetext import result screen
- url detection would detect any word with a trailing "." as a valid url
- google. was detected as a url
- this also caused training "."s to be included in valid urls
- http://www.google.com.
- Double sanitisation when edditing an attribute/proposal comment removed
- Fixed an issue where an ip/resource was recognised as a CIDR notation IP range instead of a url
- Changed the flash message for publishing without e-mails to something less scary
- defanged URL type attributes are refanged on input
- admin script to do the same for all existing attributes
- admin tool doesn't recognise a word followed by a . as a url
- allows for saving an event even if an attribute fails
- logs attributes that fail validation
- same for edit
- add_misp_export updated with the above in mind
- called Add MISP export now
- can be an XML / JSON file
- result browser with explanations of failures
- REST XML/JSON add/edit of events returns errors instead of the partially succeeding event
- it didn't use the validation type -> validation method array to call the validation function
- resulted in CC validation not being called as expected
- Fixed a critical bug in the XML export
- As of recently XML exports include relations as they were missing before
- the sanitisation of the event info field in related attributes was incorrectly sanitized of unicode characters
- this can lead to the XML export breaking and also for affected events to be blocked from synchronisation
- Proposal fixes
- fixed an invalid uuid generation that lead to an exception
- fixed the attachments for proposals still using the old attachment system that disallows most filenames
- added the automatic creation of hashes for attachment proposals
- implemented a custom pagination tool for data sets that are not directly taken from teh db
- currently creates a pagination object that mocks CakePHP pagination
- supports the CakePHP pagination view helper
- supports: pagination, sorting, custom filters
- implemented first step of the remote instance browser for admins
- view an index of events on another instance
- filter the events
- uses the new pagination
- still missing:
- remote event view
- fetch event from remote instance
- reworked the event view
- separated API and UI code path
- major speedup for the API!
- cleaner code as there was almost 0 overlap
- discussions and attributes are now loaded separately from the event view
- added after the event view loads via ajax
- cleaner pagination
- attribute pagination now finally allows for sorting
- future improvement (coming soon): Show proposals only filter
- filtering on the attributes in general
- simply pass an MD5 hash along and receive a sample if available zipped and base64 encoded in a response object
- pass any hash along with a flag set and receive any samples from events that have the passed hash
- Also, fix for an issue with the freetext import not using semi-colons as separators
- python server running in the background doing the publishing
- MISP -> python script communication via redis
- configurable / controllable via the admin UI
- by installing the requirements described in the update and the install instructions (ubuntu only for now, centos/red-hat versions to be tested and described), administrators can enable the pub/sub feature
- assign a port to the service via the interface
- each time an event is published, MISP will use ZMQ's PUB feature to push out a MISP JSON package using the "misp_json" prefix
- contact reporter first tries to contact orgc users on the instance, if they don't exist, it will contact the owner (instead of going straight to the owner)
- hostname / domain name validation change broke validation of hostnames/domain names / email addresses with a "-"
- Some documentation changes for the REST API (more coming)
- some tuning of the freetext import
- comma separated values now correctly parsed
- Ports in IP/url/link/domain/hostname now added as a comment
- virustotal now automatically recognised as external analysis / link
- decision to be revised: exports don't expose Sharing groups / org uuids to users unless they are admin (for the future: at least sync users have to be added for the new sync)
- Performance improvements for the event search exports
- JSON view code moved to Lib
- Fixed an issue that didn't restrict the dates correctly with the from / to parameters
- resolved bugs with permissions
- fixed the broken mass delete tool
- Fixed an issue with the type not being chosen correctly for file type attributes when created through the templating tool
- Templates can now be created and populated
- Users can populate an event using a template (still needs work)
- File type elements are not yet implemented
- AJAX requests now also respond with a small message at the bottom of the page, notifying the user of the result
- The following actions work now on the event page via ajax:
1. Add / remove tags
2. quick edit any attribute field if eligible
3. quickly create a proposal of any attribute field if not eligible to edit
4. popover attribute creation (also works with batch add)
5. popover proposal creation (also works with batch add)
6. delete attributes
7. accept/discard proposals
8. mass edit / delete attributes
Also, replaced the old memberslist, with a small lightweight css/js based one.
- cleaned up the methods, they all now return results without debug mode enabled
- Added a verification method for all user GPG keys (as an expired key for example would send out empty messages)
- you can now upload a certificate file and allow a server link to use a provided self signed certificate. This should solve the issues that some organisations are having when trying to connect their instances
iglocska
Alexandre Dulaunoy