- Instead of fetching all events at once for the export, events are fetched one by one
- Greatly reduces memory footprint (It mostly depends on the event with the most eligible attributes now, instead of the combined list of all events)
- Because of the lower memory usage, the time taken for the export is also slashed to a fragment of what it was before
- fixed some issues with unset variables (from, to, last) when triggered by the background workers
- reduced memory usage of the hids exports (removed storing the hashes twice in memory, drastically removed the data retrieved from the db when preparing the export)
- some errors in the format (wrong comment character used, rpz-ip not appended to IP addresses, missing semi-colon)
- removed hostnames that are on domains blocked by the rules based on domain attributes
- python server running in the background doing the publishing
- MISP -> python script communication via redis
- configurable / controllable via the admin UI
- by installing the requirements described in the update and the install instructions (ubuntu only for now, centos/red-hat versions to be tested and described), administrators can enable the pub/sub feature
- assign a port to the service via the interface
- each time an event is published, MISP will use ZMQ's PUB feature to push out a MISP JSON package using the "misp_json" prefix
- MISP will now fetch a list of all keys matching the e-mail address from the MIT server from the user edit view
- A popup will present all the matching keys (with the creation date, key ID, email addresses associated - and the fingerprint when hovering over them)
- Once the admin clicks on one, it will fetch the desired key
- future enhancement possibility: move the second stage (the actual key fetch) to the server side instead of a direct ajax query from the user's browser
- contact reporter first tries to contact orgc users on the instance, if they don't exist, it will contact the owner (instead of going straight to the owner)
- hostname / domain name validation change broke validation of hostnames/domain names / email addresses with a "-"
- Some documentation changes for the REST API (more coming)
- some tuning of the freetext import
- HIDS exports did not include filename|hash types
- Sending a password reset / welcome message picked the opposite subject line
- line breaks were sent as literals.
- users can specify an alternate gnupg executable
- Since GnuPG2 is not compatible with the last stable CryptGPG version, there are 3 options for CentOS / Red Hat users:
1. Don't use a passphrase for the server's PGP key
2. Install the beta version of CryptGPG (1.4.0b4)
3. Install GnuPG classic and point MISP to the executable
- This patch enables option 3, administrators can point MISP to the alternate executable in the server settings
- added the new flag "last" to the list of parameters
- exports affected: XML, CSV, NIDS, HIDS, STIX, Text, RestSearch
- Valid values: number + format where format can be d, m, h for day, minute, hour (examples: 5d or 12h or 30m)
- Events published / pushed will now refuse to sync if the situation arises where no attributes would be eligible to be synced
- Events pulled that contain no attributes will be thrown away
- this commit is mostly here to capture what was changed in hotfix 2.3.69
- e-mailing completely reworked, all e-mails now flow through the same method
- that method will handle all encryption and the decisions whether to send e-mails unencrypted to users without an encryption key, whether to keep the body of the e-mail untruncated, etc
- all e-mails are now also logged here (including the reason of a potential failure)
- new server settings for default template messages for password resets / new user welcome messages
- admin e-mail interface reworked and org admins now also have access to the features
- password resets / new user for site and org admins (where applicable) - quickly reset the password of a user and alert them using the pre-defined reset template
=====
- Tuned the freetext import to really accept free-text. Let me know if you have any tips for tuning the detection further!
- it now breaks the passed string on whitespace and line-break and tries to resolve the rest. Filename resolution tightened to exclude anthing that starts or ends with a .
- Reworking the way e-mails are sent - all of it goes through a centralised e-mail method
- just pass the recipient, recipient encryption key collection, body, alternate body if the message cannot be encrypted, subject, reply to address and pgp key for reply to along and the method will do the rest
- encrypt if possible, check if sending without encryption is allowed, signing, adding attachment for reply to encryption key, using alternate sanitised body if it is enforced for accounts that cannot use encryption is all done in one place
- easy to maintain and expand with future changes (such as the S/MIME pull request on github)
- documented in automation view
- right now it follows the simple rule of user > admin settings > default values when generating the export
- Parameters can be passed via url / JSON object / XML object
- filters include filter on event ID, date range, tags
TODO:
- buttons for a per event download via the UI
- introduce new export option for normal users (via background workers and the old style export)
Iglocska