- Reworking the way e-mails are sent - all of it goes through a centralised e-mail method
- just pass the recipient, recipient encryption key collection, body, alternate body if the message cannot be encrypted, subject, reply to address and pgp key for reply to along and the method will do the rest
- encrypt if possible, check if sending without encryption is allowed, signing, adding attachment for reply to encryption key, using alternate sanitised body if it is enforced for accounts that cannot use encryption is all done in one place
- easy to maintain and expand with future changes (such as the S/MIME pull request on github)
- sharing groups are now correctly checked in restfullEventToServer
- The rules are very simple, the event has to:
- be of distribution value 2 or 3
- or 4 given that the attached sharing group meets the following requirements:
- The sync user is in the sharing group's org list (otherwise he can't transfer it / become the owner)
- Or the instance that is being synced to has to be set to "all_orgs"
- The SG has to either not include any instances
- Or include the instance that is being synced to
- decision to be revised: exports don't expose Sharing groups / org uuids to users unless they are admin (for the future: at least sync users have to be added for the new sync)
- Unified the way exports accept negated parameters
- Fixed the documentation
- Most exports are now restrictable by the event date (From/To parameters)
- none cached XML export now writes to file after converting each event, clearing the memory and resolving any potential memory issues
- MYSQL.sql file now correctly includes the task entries
- GenerateCorrelation admin task is now a background job
- Organisation of events pulled now get the org in the server object as the owner instead of the one who initiates the pull
- Small fix to wrapping text in the pivot graph
- send uuids of events to be pushed together with timestamps to the other instance
- other instance removes events that are already up to date or locally created from the array
- sends the remaining uuids back
- first instance initiates the push of events that were not filtered out
- Start-up script could only be started from the script's location
- Division by zero in e-mail alerts when calculating the progress of the background job
- AJAX requests now also respond with a small message at the bottom of the page, notifying the user of the result
- The following actions work now on the event page via ajax:
1. Add / remove tags
2. quick edit any attribute field if eligible
3. quickly create a proposal of any attribute field if not eligible to edit
4. popover attribute creation (also works with batch add)
5. popover proposal creation (also works with batch add)
6. delete attributes
7. accept/discard proposals
8. mass edit / delete attributes
Also, replaced the old memberslist, with a small lightweight css/js based one.
- upgrade script broke adding events via the rest interface if they had an xml_version included
- fixed, also, add now more flexible with directly adding events from an export encapsulated in a response tag
- event level exports from the event view now export all attributes regardless of to_ids value
- to_ids value now has its own column in the csv exports
Small tweaks to email formatting to sync up with UI Changes.. also added event title to Subject (questionable if this is something desired globally as it would not be encrypted).
- It is now possible to restrict the CSV automation export by type / category
- updated the automation page to describe how the syntax works
- fixed an issue with line breaks not being sanitized for the CSV export
- during the event id pull, the local server already checks the timestamps, removing the ids of events that are not newer than the local version
- this results in only the event metadata being pulled for all events, and the attributes of only those events that need to be updated are pulled resulting in much quicker pulls
- Fixed an issue with proposals that got pulled not finding the attribute that they are proposals to (for proposals that belong to an attribute)
- regexp structural changes added to the upgrade script (type)
- Added publish / alert to the background jobs
- fixed a misalignment with the statistics
- xml version now included in the xml exports
- MISP will now check the xml version on all imports related to sync / add MISP XML and try to update the incoming info if it detects an older version
- exports now take tag names as a parameter (affected exports: XML, text, HIDS, NIDS)
- eventtags now correctly get removed when an event is deleted
- Scheduled tasks for pull / push now working as intended
- Rescheduling of all tasks fixed
- protection against the rescheduled task ending up in the past
- further event history fixes
- fixed lots of erroneous logging
- performance improvement with logging (no longer loading controllers for no reason)
- logging extra actions that weren't logged before (proposal accept / discard, server pull / push)
- new special role for tagging
- can create tags with a name + colour combination (using a colour picker plugin)
- users can assign tags to events
- can filter events by tags on the index
- upgrade script that populates threat level from the old risk field for every event that doesn't have a threat level set.
- threat levels in an event (from a sync for example) that are unknown to the local instance now show the numeric value of the threat level
- cleaned up the methods, they all now return results without debug mode enabled
- Added a verification method for all user GPG keys (as an expired key for example would send out empty messages)
- you can now upload a certificate file and allow a server link to use a provided self signed certificate. This should solve the issues that some organisations are having when trying to connect their instances
- Valid renamed to Published on the event index
- Attributes that are flagged as IDS signatures are now shown with a (IDS) notation at the end of the line in the alert e-mail
- fix to the creator of a proposal being able to also accept it
- new attributes are now shown in the e-mail denoted by a * when an event is republished
- the date of an attribute's creation is shown
- attributes in the event view now show the date when they were added / modified
- the alert e-mail now shows which attributes are new since the last commit
- Corrected some weak notifications on background jobs
- Changed the view slightly to view background jobs
- fixed an issue where editing a sync server setting would cause an error due to the id not being passed to the logging plugin
Also, more work on the background jobs
- started work on publishing
- started making the background jobs an optional setting in bootstrap
Conflicts:
app/Controller/AppController.php
app/Controller/EventsController.php
- Event.risk has been replaced by Event.threat_level_id.
all functionality remains the same and users should not see
any difference.
ENUM() used for Event.risk is vendor specific and requires
too many hacks to play nicely with bake.
- Added default schema file, SQL dumps should be avoided since
they make updating/upgrading a pain.
- Removed old unused schemas
- ADMIN org removed.
- Siteadmins are now identified by the perm_site_admin flag
- Siteadmins can now be of any organisation
- editing the regexp / whitelist rules can now be done by a special user with the perm_regexp_access in his/her role
- Executing a mass replace of attribute values based on the regexp rules cannot be initiated by a regexp/whitelist user, only by a site admin
- If the login page is reached without any users / roles defined they are automatically created (perviously it was only the user that was created)
- Org admins are restricted from assigning perm_site_admin, perm_sync and perm_regexp_access roles to users. This can only be done by a site admin.
- Regular expressions are now only checked for attributes
- Regular expressions are now defined and checked on a type by type
basis, with the setting "ALL" affecting all attributes
- creation / deletion of several attributes in one edit to accommodate
for several checked type options
- perform on all admin option now only saves attributes that actually
get changed by the regexp, making the function usable again for larger
databases
- Some feedback on what got changed during a perform on all
- UI changes in the index / regexp add / edit views to reflect the type
sensitivity changes
- Since regexp can be used to blacklist things, there's no need to have
two separate features that accomplish the same thing
- Add a regexp named /1.1.1.1/ with nothing as replacement and it will
behave the same as adding a blacklist for 1.1.1.1 in the old system.
Found a bug where an instance that has a lower attribute count pushing to
another would cause the attributes with equal attribute ID to get
overwritten with the pushed ones. Unsetting the attribute ID before the
push fixes this.
- Fixed issues with the sync
- Secondary publishes on remote servers failed
- Introduced new fields in events to stop backward traverse of
edit information that lead to low performance and eroneous
distribution information updates when more than 2 servers were
linked
- Deletion of an attribute now deletes on remote servers
- Changes to the event ownership
- Original creator org now noted in the event itself
- Only original creator org can change distribution
- Events will show up with the original creator org for users
(admins can see both that and the owner of the event on the
local instance)
- Server.organization now used in junction with the connecting
user's org and the instance's org (from the bootstrap) to
determine distribution flow control and access rights
- Lots of minor changes
- perm_auth new toggle, can disable auth key usage for a role
- prevents sync / rest with a perm_auth == false key
- some changes to sync to provide better feedback on why it failed
- rewording of distribution options
- Admins cannot manually change anyone's authkey, they need to generate a
new one via the reset link
- Some pages could be accessed by changing the url - fixed (though needs
further testing)
- Edited a change in the manual that may have been confusing
- Some changes to the way ACL is set up - still needs more work
-Analaysis levels setable for events as per milestone item 94
-Password change forced as per milestone item 109
-Added feedback on entered search terms for search attributes
-fixed the authentication issue
-some minor fixes
this is in responce to the email
From: <User1088@QET.BE>
To: <ndebrouwer@hotmail.com>, <andrzej.dereszowski@ncirc.nato.int>
Subject: Re: sync/REST
Date: Fri, 7 Dec 2012 13:30:10 +0000
in this there is a complaint about the RESTfull sync workings.
the email hints about 2 possible options:
i) RESTfull add event without attributes (conform the web interface)
ii) RESTfull add event with attributes (more conform the code)
both are implemented and can be choisen in bootstrap.php by
Configure::write('CyDefSIG.rest', 'ii') or 'i'.
iglocska