- Start-up script could only be started from the script's location
- Division by zero in e-mail alerts when calculating the progress of the background job
- AJAX requests now also respond with a small message at the bottom of the page, notifying the user of the result
- The following actions work now on the event page via ajax:
1. Add / remove tags
2. quick edit any attribute field if eligible
3. quickly create a proposal of any attribute field if not eligible to edit
4. popover attribute creation (also works with batch add)
5. popover proposal creation (also works with batch add)
6. delete attributes
7. accept/discard proposals
8. mass edit / delete attributes
Also, replaced the old memberslist, with a small lightweight css/js based one.
- upgrade script broke adding events via the rest interface if they had an xml_version included
- fixed, also, add now more flexible with directly adding events from an export encapsulated in a response tag
- event level exports from the event view now export all attributes regardless of to_ids value
- to_ids value now has its own column in the csv exports
Small tweaks to email formatting to sync up with UI Changes.. also added event title to Subject (questionable if this is something desired globally as it would not be encrypted).
- It is now possible to restrict the CSV automation export by type / category
- updated the automation page to describe how the syntax works
- fixed an issue with line breaks not being sanitized for the CSV export
- during the event id pull, the local server already checks the timestamps, removing the ids of events that are not newer than the local version
- this results in only the event metadata being pulled for all events, and the attributes of only those events that need to be updated are pulled resulting in much quicker pulls
- Fixed an issue with proposals that got pulled not finding the attribute that they are proposals to (for proposals that belong to an attribute)
- regexp structural changes added to the upgrade script (type)
- Added publish / alert to the background jobs
- fixed a misalignment with the statistics
- xml version now included in the xml exports
- MISP will now check the xml version on all imports related to sync / add MISP XML and try to update the incoming info if it detects an older version
- exports now take tag names as a parameter (affected exports: XML, text, HIDS, NIDS)
- eventtags now correctly get removed when an event is deleted
- Scheduled tasks for pull / push now working as intended
- Rescheduling of all tasks fixed
- protection against the rescheduled task ending up in the past
- further event history fixes
- fixed lots of erroneous logging
- performance improvement with logging (no longer loading controllers for no reason)
- logging extra actions that weren't logged before (proposal accept / discard, server pull / push)
- new special role for tagging
- can create tags with a name + colour combination (using a colour picker plugin)
- users can assign tags to events
- can filter events by tags on the index
- upgrade script that populates threat level from the old risk field for every event that doesn't have a threat level set.
- threat levels in an event (from a sync for example) that are unknown to the local instance now show the numeric value of the threat level
- cleaned up the methods, they all now return results without debug mode enabled
- Added a verification method for all user GPG keys (as an expired key for example would send out empty messages)
- you can now upload a certificate file and allow a server link to use a provided self signed certificate. This should solve the issues that some organisations are having when trying to connect their instances
- Valid renamed to Published on the event index
- Attributes that are flagged as IDS signatures are now shown with a (IDS) notation at the end of the line in the alert e-mail
- fix to the creator of a proposal being able to also accept it
- new attributes are now shown in the e-mail denoted by a * when an event is republished
- the date of an attribute's creation is shown
- attributes in the event view now show the date when they were added / modified
- the alert e-mail now shows which attributes are new since the last commit
- Corrected some weak notifications on background jobs
- Changed the view slightly to view background jobs
- fixed an issue where editing a sync server setting would cause an error due to the id not being passed to the logging plugin
Also, more work on the background jobs
- started work on publishing
- started making the background jobs an optional setting in bootstrap
Conflicts:
app/Controller/AppController.php
app/Controller/EventsController.php
- Event.risk has been replaced by Event.threat_level_id.
all functionality remains the same and users should not see
any difference.
ENUM() used for Event.risk is vendor specific and requires
too many hacks to play nicely with bake.
- Added default schema file, SQL dumps should be avoided since
they make updating/upgrading a pain.
- Removed old unused schemas
- ADMIN org removed.
- Siteadmins are now identified by the perm_site_admin flag
- Siteadmins can now be of any organisation
- editing the regexp / whitelist rules can now be done by a special user with the perm_regexp_access in his/her role
- Executing a mass replace of attribute values based on the regexp rules cannot be initiated by a regexp/whitelist user, only by a site admin
- If the login page is reached without any users / roles defined they are automatically created (perviously it was only the user that was created)
- Org admins are restricted from assigning perm_site_admin, perm_sync and perm_regexp_access roles to users. This can only be done by a site admin.
- Regular expressions are now only checked for attributes
- Regular expressions are now defined and checked on a type by type
basis, with the setting "ALL" affecting all attributes
- creation / deletion of several attributes in one edit to accommodate
for several checked type options
- perform on all admin option now only saves attributes that actually
get changed by the regexp, making the function usable again for larger
databases
- Some feedback on what got changed during a perform on all
- UI changes in the index / regexp add / edit views to reflect the type
sensitivity changes
- Since regexp can be used to blacklist things, there's no need to have
two separate features that accomplish the same thing
- Add a regexp named /1.1.1.1/ with nothing as replacement and it will
behave the same as adding a blacklist for 1.1.1.1 in the old system.
Found a bug where an instance that has a lower attribute count pushing to
another would cause the attributes with equal attribute ID to get
overwritten with the pushed ones. Unsetting the attribute ID before the
push fixes this.
- Fixed issues with the sync
- Secondary publishes on remote servers failed
- Introduced new fields in events to stop backward traverse of
edit information that lead to low performance and eroneous
distribution information updates when more than 2 servers were
linked
- Deletion of an attribute now deletes on remote servers
- Changes to the event ownership
- Original creator org now noted in the event itself
- Only original creator org can change distribution
- Events will show up with the original creator org for users
(admins can see both that and the owner of the event on the
local instance)
- Server.organization now used in junction with the connecting
user's org and the instance's org (from the bootstrap) to
determine distribution flow control and access rights
- Lots of minor changes
- perm_auth new toggle, can disable auth key usage for a role
- prevents sync / rest with a perm_auth == false key
- some changes to sync to provide better feedback on why it failed
- rewording of distribution options
- Admins cannot manually change anyone's authkey, they need to generate a
new one via the reset link
- Some pages could be accessed by changing the url - fixed (though needs
further testing)
- Edited a change in the manual that may have been confusing
- Some changes to the way ACL is set up - still needs more work
-Analaysis levels setable for events as per milestone item 94
-Password change forced as per milestone item 109
-Added feedback on entered search terms for search attributes
-fixed the authentication issue
-some minor fixes
this is in responce to the email
From: <User1088@QET.BE>
To: <ndebrouwer@hotmail.com>, <andrzej.dereszowski@ncirc.nato.int>
Subject: Re: sync/REST
Date: Fri, 7 Dec 2012 13:30:10 +0000
in this there is a complaint about the RESTfull sync workings.
the email hints about 2 possible options:
i) RESTfull add event without attributes (conform the web interface)
ii) RESTfull add event with attributes (more conform the code)
both are implemented and can be choisen in bootstrap.php by
Configure::write('CyDefSIG.rest', 'ii') or 'i'.
Let RESTfull only work conform the web pages (to Christophes wish),
so add/edit event apart from add/edit attribute.
(there is annotation in the code to revert back to full RESTfull and
add/edit the attribute(s) alongside add/edit the event.)
redid the sync, so if add and exist, send HTTP 302 and different
Location, and do edit there.
Still, the final result has to compare the attributes and if needed
RESTfull delete.
conform latest, having:
- Your organization only
- This server-only
- This Community-only
- Connected communities
- All communities
Push is tested, pull not yet.
Add "Pull only" as a sharing state where,
everybody does see an event, is pullable,
but will never be pushed.
Has a generatePrivate for db conversion now.
Private events are true private and
running a server in 2 modes (private and sync),
so real private (red) or private to server (amber)
or full distributable (green).
Mind this needs a change to tables events, attributes and correlation.
These are in MYSQL.private.sql.
If it's just an existing behavior or lib,
place it in a plugin directory structure in <cydefsig>/plugins.
If there is a need to change an extern existing plugin,
extend the existing plugin by a new plugin in <cydefsig>/app/Plugin.
This way there is a very clean devision between own and external code.
The external code can be updated without touching own nor changed code.
Sync worked, but we did not know what to do with user_id and org.
Now, on sync, anonymize the user_id, get the Server.organization and put
that into Event.org.
And, display owning flag if Event.user_id or get the Server.logo
belonging to Event.org (=Server.organization) when Event.user_id is
empty (=0).
To this there is organization name and logo in bootstrap and
other organizations names and logos in Servers.
Previous, upon delete only on the local server the event or attribute
was deleted.
Now, if delete, look for same event or attribute (using it's uuid)
and delete on remote servers as well.
Also look and delete if not published, so no dangling/zombie copies
remain on remote servers.
Event.user_id was re-added but we still missed some,
so an added event would get user_id set to zero.
Now Event gets the correct user_id again from
the person logged in and adding.
(lateron this must not be used during sync.)
- a server containing no MISP gives "XML cannot be read." on publish.
- a server having a non-existing internet name gives
"php_network_getaddresses: getaddrinfo failed: Name or service not
known" on publish.
iglocska