- The event export buttons have been unified into a single download as... button
- clicking it loads a popup with all of the export formats
- added snort, suricata, text dump to the export options
- added the option for an extra setting for some exports (such as including non IDS flagged attributes, encoding attachments)
- easily extendable system
- moved the hidden popup divs into the general layout, can be easily reused anywhere
- removed the auth refresh option that was re-enabled recently as it seems to sometimes cause issues
- text exports now allow "all" to be specified as type, which will dump all attribute values that the user can see
- text exports now allow restricting the results based on event id
- UI improvements, events appear unpublished after ajax queries that alter attributes
- Events get unpublished by the attribute replace tool and template population as they should
- currently to_xml() has performance issues, if it's not resolved fast, it would be a good idea to move the export to the background workers
- some UI changes
- first version of templating system complete
- first version of freetext importer complete
- first version of mass attribute replace tool complete
- some UI changes
- resolved bugs with permissions
- fixed the broken mass delete tool
- Fixed an issue with the type not being chosen correctly for file type attributes when created through the templating tool
- malware samples / attachments couldn't be downloaded
- links weren't actually links
- deleting an attribute / shadowattribute now opens a custom confirmation dialogue. This is also where the CSRF tokens are generated for the post request to execute the delete, resulting in a faster event view load
- AJAX requests now also respond with a small message at the bottom of the page, notifying the user of the result
- The following actions work now on the event page via ajax:
1. Add / remove tags
2. quick edit any attribute field if eligible
3. quickly create a proposal of any attribute field if not eligible to edit
4. popover attribute creation (also works with batch add)
5. popover proposal creation (also works with batch add)
6. delete attributes
7. accept/discard proposals
8. mass edit / delete attributes
Also, replaced the old memberslist, with a small lightweight css/js based one.
- forms are now dynamically pulled onclick
- performance greatly enhanced
- solves the issues with the CSRF protection kicking in if the user edits a field after using the back button
- Exporting a JSON object erroneously included related objects which prevented the exported event from being added back to MISP via the API
- Downloading search results as XML / CSV now correctly includes all of the search results instead of just the 60 visible ones on the UI (cut off by the pagination)
- The tags parameter in the exports now correctly accepts null as a valid value even if it is the last parameter
- users can now edit all fields in an attribute whilst on the event page
- issues left to fix:
- tag changes after an attribute change run into CSRF protection
- batch add not handled gracefully yet
- going back to the event view and editing a field gives users an error message over the CSRF protection - instead, silently check if the page is loaded in a dirty way and refresh the ajax fields silently
- quickadd of attributes still missing
- events/restSearch, attributes/restSearch, events/xml, attributes/returnAttributes
- users can now POST a search array in XML / json instead of sending the parameters in the url
- Users can now elect to receive their attribute search results in the new alternative view
- instead of receiving a list of attributes matching the search options, users are presented with a list of events that contain matching attributes
- number of matches and a percentage of those matches being marked as indicators for IDSes are shown
- the events are ordered by the percentage of IDS worthy attribute
- A colon in the tag search tag will render the tag search invalid. Since colons are commonly used in tag names, this poses an issue - users should use a semi-colon instead, which gets automatically converted to a colon.
- Threatconnect import now allows any valid threatconnect csv file to be imported as long as type, value, confidence, description and source are included
- xml version now included in the xml exports
- MISP will now check the xml version on all imports related to sync / add MISP XML and try to update the incoming info if it detects an older version
- exports now take tag names as a parameter (affected exports: XML, text, HIDS, NIDS)
- eventtags now correctly get removed when an event is deleted
- cleaned up the methods, they all now return results without debug mode enabled
- Added a verification method for all user GPG keys (as an expired key for example would send out empty messages)
- authorization key should be sent through headers.
- passing it in the url is deprecated
- updated automation page to reflect the changes
- csv export now has headers
- you can now upload a certificate file and allow a server link to use a provided self signed certificate. This should solve the issues that some organisations are having when trying to connect their instances
Also, more work on the background jobs
- started work on publishing
- started making the background jobs an optional setting in bootstrap
Conflicts:
app/Controller/AppController.php
app/Controller/EventsController.php
Some small travins changes too.
FYI there's an automated travis build available at
https://travis-ci.org/MISP/MISP
We don't have unit testing and travis setup is subpar so everything will fail
for now.
- users can now download attachments using the APIkey
- security issue fixed where a user could download attachments that he/she can't even see by navigating to attributes/download/<attribute_id>
- users can search RESTfully for attributes based on various filtering mechanisms and get either an event that includes the located attribute(s) or just an array of attributes returned.
- users can also request all attributes of a (or several) types and get them returned as an XML
- GFI import issue fixed with attribute ID 1 not existing causing the
import to fail for several attributes
- GFI import change: registry keys with binary value are now artifacts
dropped instead of persistance mechanism
- GFI import change: files with size of 0 will be omitted
- file attachment download change: moved away from the deprecated media
view in favour of cakeresponse->file()
- until now checkAction was used to check permissions of a user
- but since all of the role permissions are checked beforefilter in
appcontroller and saved into a public array, doing a lookup of the
array saves an SQL call for each permission check.
- CSV export for individual events, all events, search results
- Whitelists are now preg_matches instead of simple string matches
- whitelist checks are to be applied on almost all exports
(implementation in progress)
- the exception will be the search result exports, if the (to be
implemented) to_ids only checkbox isn't checked
- Use ! to exclude terms in the value/id/org fields
- org search works the same way as value / id now, you can enter several
terms separated by a newline. Also, adding ! infront of a term will
exclude the organisation from the results
- sub string search for organisations
- New field for events, locking an event from sending out a contact
e-mail when a proposal is made to it
- Default setting for the new field is 0, if a shadow attribute is
added an e-mail is sent to all subscribing members of the orgc and the
new field is set to 1
- Accepting a change resets the field to 0
- Event correctly changes timestamp when attribute edited in the UI
- Attribute correctly changes timestamp when edited in the UI
- Still very much work in progress, several parts are not supposed to
work yet
- timestamp field added to events and attributes (int length 11 called
timestamp, default value 0)
- timestamps created on add / edit when apprioriate
- during an add, if an event/attribute is not being pushed through a
sync with an existing timestamp, create a timestamp
- on edit, check whether the timestamp is newer than the old one and
only add the attribute or event then
checks
- actAs acl removed from role and user models together with some extra
code related to the ACL
- Fix of the filename regex as pointed out by cvandeplas.
iglocska