- enter a UUID in the event ID field of the attribute search to find attributes belonging to a certain event
- use event IDs / UUIDs to filter events on the event index
- new functionality: Event blacklisting by UUID
- site admins cna enable this feature in the server settings
- enabling the feature will make the required db changes
- any deleted event will automatically get blacklisted
- this prevents deleted events from flowing back from a synced instance
- site admins can manually add UUIDs to the list and remove entries
- fix to UUID duplication issues for attributes
- simply run the admin script and it will regenerate the UUID of attributes that are duplicates, if any such exist
- timestamps/event published status will not be affected
- config.core.php now includes a change that prevents from 404 exceptions being logged
- the sync uses 404s to signal that an event with a given uuid does not exist when negotiating proposal synchronisation
- this causes a dangerously high amount of noise in the logs
- as explained on the automation page
- also, better error handling
- all API calls that fail during authentication will now return a JSON/XML error message instead of redirecting to the login page
- simply pass an MD5 hash along and receive a sample if available zipped and base64 encoded in a response object
- pass any hash along with a flag set and receive any samples from events that have the passed hash
- Also, fix for an issue with the freetext import not using semi-colons as separators
- Threat level ID options correctly set
- Threat level ID validation tightened to reject anything but the existing threat levels
- The upload malware API now logs validation issues during the failed creation of attributes / events
- new API for uploading malware samples
- allows the upload of several files
- can be used to populate a pre-existing event, or create a new event
- expects a JSON or an XML object with the samples base64 encoded
- new way of storing malware samples
- original filename not used any longer
- samples are renamed to their md5 hashes
- original filename preserved in a secondary txt file
- removed filename validation as it is no longer used for the command line execution
- this allows unicode name files to be uploaded!
- changed the UI attachment upload to reflect these changes
- code more centralised and extendible
- Instead of fetching all events at once for the export, events are fetched one by one
- Greatly reduces memory footprint (It mostly depends on the event with the most eligible attributes now, instead of the combined list of all events)
- Because of the lower memory usage, the time taken for the export is also slashed to a fragment of what it was before
- fixed some issues with unset variables (from, to, last) when triggered by the background workers
- reduced memory usage of the hids exports (removed storing the hashes twice in memory, drastically removed the data retrieved from the db when preparing the export)
- some errors in the format (wrong comment character used, rpz-ip not appended to IP addresses, missing semi-colon)
- removed hostnames that are on domains blocked by the rules based on domain attributes
- python server running in the background doing the publishing
- MISP -> python script communication via redis
- configurable / controllable via the admin UI
- added multi edit to freetext import comments
- added a missing file from hotfix-2.3.87 (pgp key import view)
- updated gitignore to ignore some items that are outside of the scope of the git package
- by installing the requirements described in the update and the install instructions (ubuntu only for now, centos/red-hat versions to be tested and described), administrators can enable the pub/sub feature
- assign a port to the service via the interface
- each time an event is published, MISP will use ZMQ's PUB feature to push out a MISP JSON package using the "misp_json" prefix
- MISP will now fetch a list of all keys matching the e-mail address from the MIT server from the user edit view
- A popup will present all the matching keys (with the creation date, key ID, email addresses associated - and the fingerprint when hovering over them)
- Once the admin clicks on one, it will fetch the desired key
- future enhancement possibility: move the second stage (the actual key fetch) to the server side instead of a direct ajax query from the user's browser
- contact reporter first tries to contact orgc users on the instance, if they don't exist, it will contact the owner (instead of going straight to the owner)
- hostname / domain name validation change broke validation of hostnames/domain names / email addresses with a "-"
- Some documentation changes for the REST API (more coming)
- some tuning of the freetext import
- HIDS exports did not include filename|hash types
- Sending a password reset / welcome message picked the opposite subject line
- line breaks were sent as literals.
- users can specify an alternate gnupg executable
- Since GnuPG2 is not compatible with the last stable CryptGPG version, there are 3 options for CentOS / Red Hat users:
1. Don't use a passphrase for the server's PGP key
2. Install the beta version of CryptGPG (1.4.0b4)
3. Install GnuPG classic and point MISP to the executable
- This patch enables option 3, administrators can point MISP to the alternate executable in the server settings
- added the new flag "last" to the list of parameters
- exports affected: XML, CSV, NIDS, HIDS, STIX, Text, RestSearch
- Valid values: number + format where format can be d, m, h for day, minute, hour (examples: 5d or 12h or 30m)
- added a new entry to the admin tools (Administartion -> Administrative tools)
- converts title and change columns in the logs table to text from varchar(255)
- Popover_form purged after the form has been submitted
- a duplicate hidden div was created for confirmation popups within the attribute creation popup and clicking publish populated the wrong div
- comma separated values now correctly parsed
- Ports in IP/url/link/domain/hostname now added as a comment
- virustotal now automatically recognised as external analysis / link
- Events published / pushed will now refuse to sync if the situation arises where no attributes would be eligible to be synced
- Events pulled that contain no attributes will be thrown away
- this commit is mostly here to capture what was changed in hotfix 2.3.69
- e-mailing completely reworked, all e-mails now flow through the same method
- that method will handle all encryption and the decisions whether to send e-mails unencrypted to users without an encryption key, whether to keep the body of the e-mail untruncated, etc
- all e-mails are now also logged here (including the reason of a potential failure)
- new server settings for default template messages for password resets / new user welcome messages
- admin e-mail interface reworked and org admins now also have access to the features
- password resets / new user for site and org admins (where applicable) - quickly reset the password of a user and alert them using the pre-defined reset template
=====
- Tuned the freetext import to really accept free-text. Let me know if you have any tips for tuning the detection further!
- it now breaks the passed string on whitespace and line-break and tries to resolve the rest. Filename resolution tightened to exclude anthing that starts or ends with a .
- user will get an explanation of the csrf error and that going back and refreshing the form will fix it
- also, there is a link that will take the user to the baseurl (which will redirect to the login page if the csrf issue occured on the login page)
- Reworking the way e-mails are sent - all of it goes through a centralised e-mail method
- just pass the recipient, recipient encryption key collection, body, alternate body if the message cannot be encrypted, subject, reply to address and pgp key for reply to along and the method will do the rest
- encrypt if possible, check if sending without encryption is allowed, signing, adding attachment for reply to encryption key, using alternate sanitised body if it is enforced for accounts that cannot use encryption is all done in one place
- easy to maintain and expand with future changes (such as the S/MIME pull request on github)
- Users being logged on would not be able to use the actions that are also used for automation
- Those actions trigger a check of the authorization header, which in certain use cases can be set with values that is outside of the scope of MISP
- MISP will now try to only detect MISP auth keys in the headers and if it detects something else it ignores it
Iglocska