Commit Graph

218 Commits (982727819bef8017b70581cbbcf8ea6d2d5fe8c0)

Author SHA1 Message Date
iglocska c48edd6732 First release of the new API features 2013-09-20 11:40:26 +02:00
iglocska 8e75f0826f Security fix and new download attachment feature
- users can now download attachments using the APIkey

- security issue fixed where a user could download attachments that he/she can't even see by navigating to attributes/download/<attribute_id>
2013-09-19 17:28:55 +02:00
iglocska 6d1dc6df95 First round of implementations for the new API searches
- users can search RESTfully for attributes based on various filtering mechanisms and get either an event that includes the located attribute(s) or just an array of attributes returned.

- users can also request all attributes of a (or several) types and get them returned as an XML
2013-09-19 12:05:08 +02:00
iglocska 807cc4c090 Deleting attributes deletes associated shadow attributes
There was a bug causing "zombie" shadowattributes to stay in events if the attribute has been deleted
2013-09-04 08:52:30 +02:00
iglocska 20efca4d79 A previous change reverted by accident in the previous commit 2013-08-21 13:08:28 +02:00
iglocska 285ff481a5 Further updates to the sync 2013-08-12 17:23:32 +02:00
iglocska 49f90b59f2 Fix to the distribution changes breaking threatconnect imports 2013-08-08 13:59:10 +02:00
iglocska 87a03b756e Changes to the initial distribution settings
- The initial attribute distribution level now allows the option for 'event', inheriting the event's distribution level
2013-08-08 12:16:03 +02:00
iglocska 0ee45af902 Default distribution level flags in bootstrap.php
- Each instance can now have its own default event and attribute distribution level set
2013-08-06 11:53:12 +02:00
iglocska 3f0eaa73a4 Changes to the filename validation
- . allowed in filenames to allow for names such as test-1.0.ext
2013-07-29 17:19:08 +02:00
iglocska b3f6032857 Fixes an issue with the upload of malware samples not generating an md5
hash if the file is too large
2013-07-26 10:13:44 +02:00
Christophe Vandeplas ca30cf006f Import ThreatConnect attributes into event, see issue #119 2013-07-15 09:10:18 +02:00
Christophe Vandeplas 130f470c90 Revert "fix bug in removing remote attributes if push is not enabled"
This reverts commit c4d5344153.
2013-07-13 08:22:37 +02:00
Christophe Vandeplas c4d5344153 fix bug in removing remote attributes if push is not enabled 2013-07-13 08:21:42 +02:00
iglocska 3e9a967188 Removal of some references to the old private flag 2013-07-11 17:25:47 +02:00
iglocska 23017fd9a8 Change to the GFI import and the attachment downloads
- GFI import issue fixed with attribute ID 1 not existing causing the
import to fail for several attributes

- GFI import change: registry keys with binary value are now artifacts
dropped instead of persistance mechanism

- GFI import change: files with size of 0 will be omitted

- file attachment download change: moved away from the deprecated media
view in favour of cakeresponse->file()
2013-07-10 17:31:18 +02:00
iglocska eeac31bee8 Regexp changes, UI changes
- first cleanup of regexp

- some changes left off from the UI changes that were not in the views
themselves
2013-07-04 15:45:11 +02:00
iglocska abde2a47af Several fixes
- Fixed the search pagination beyond the first page

- Hard coded routing of the menues in the global actions area
2013-06-28 14:28:58 +02:00
iglocska 5948a7c329 Several copy paste failures fixed in the previous commit
- /facepalm
2013-06-27 17:57:33 +02:00
iglocska 8cdf97ff66 ACL checks changed
- until now checkAction was used to check permissions of a user

- but since all of the role permissions are checked beforefilter in
appcontroller and saved into a public array, doing a lookup of the
array saves an SQL call for each permission check.
2013-06-27 17:53:36 +02:00
iglocska 7cd742dce4 Fix to users not being able to edit attributes 2013-06-27 16:43:03 +02:00
iglocska c604241487 Bugfix for the creation of several attributes with the same UUID
- SHA256 and SHA1 hash attributes that get auto-generated on malware
sample upload had the same hash as the filename|md5. Fixed.
2013-06-26 17:25:38 +02:00
iglocska 39e99554c9 Views updated to include CSV in the menues
- CSV and also IOC downloads on events are now hidden if the event is
not published
2013-06-26 17:20:56 +02:00
iglocska e87c28863e Firther work on the exports
- Some refactoring of the whitelist checks
- tighter rules for published / to_ids on certain exports
- attribute search now has the IOC checkbox
2013-06-26 15:31:28 +02:00
iglocska 776ef3ae07 Changes to export validation, CSV export, Whitelist redesign
- CSV export for individual events, all events, search results
- Whitelists are now preg_matches instead of simple string matches
- whitelist checks are to be applied on almost all exports
(implementation in progress)
- the exception will be the search result exports, if the (to be
implemented) to_ids only checkbox isn't checked
2013-06-26 14:48:25 +02:00
iglocska 24ebbcca5c Update to the attribute search
- Use ! to exclude terms in the value/id/org fields

- org search works the same way as value / id now, you can enter several
terms separated by a newline. Also, adding ! infront of a term will
exclude the organisation from the results

- sub string search for organisations
2013-06-24 13:24:08 +02:00
iglocska 55f47d3166 ShadowAttribute notifications, and some minor fixes
- New field for events, locking an event from sending out a contact
e-mail when a proposal is made to it
- Default setting for the new field is 0, if a shadow attribute is
added an e-mail is sent to all subscribing members of the orgc and the
new field is set to 1
- Accepting a change resets the field to 0
2013-06-20 16:21:55 +02:00
iglocska d2fcda7cc6 Added 2 new type of attributes
- sha256 / filename|sha256
- uploading a malware sample now automatically creates a filename|sha1
and a filename|sha256 in addition to the sample|md5
2013-06-12 16:50:21 +02:00
Christophe Vandeplas 45cfafca98 fix file download missing extension 2013-06-11 14:52:54 +02:00
Iglocska a3edc6a4ee UI changes and more work on the sync
- updated the side menu
2013-06-11 01:20:27 +02:00
Iglocska 040227d66a Some more fixes to the sync 2013-06-10 23:34:47 +02:00
Iglocska b74179bc84 Further work on the distribution 2013-06-10 21:38:04 +02:00
Iglocska 06ca5ba0cd Further changes to the distribution
- changed to use the new int field
2013-06-10 20:49:31 +02:00
iglocska 3c23ac3a86 Change to new distribution
- first stage
2013-06-10 17:33:03 +02:00
iglocska 58c00150ba First cleanup of AttributesController and EventsController after the
move to timestamps
2013-06-07 09:45:26 +02:00
iglocska 1e7665cd6d Saving over night, something still blocks the timestamp from being saved
after a push...
2013-06-06 18:43:26 +02:00
iglocska 4e2685e0c6 More work on the timestamps
- Event correctly changes timestamp when attribute edited in the UI
- Attribute correctly changes timestamp when edited in the UI

- Still very much work in progress, several parts are not supposed to
work yet
2013-06-06 16:03:28 +02:00
iglocska d52b0a6e70 First (still non-working) version of the timestamp + uuid sync
- timestamp field added to events and attributes (int length 11 called
timestamp, default value 0)
- timestamps created on add / edit when apprioriate
- during an add, if an event/attribute is not being pushed through a
sync with an existing timestamp, create a timestamp
- on edit, check whether the timestamp is newer than the old one and
only add the attribute or event then
2013-06-06 14:55:13 +02:00
Christophe Vandeplas e3ed847ba0 fixing some REST API and XML issues 2013-05-28 11:15:21 +02:00
Christophe Vandeplas 3bbd12f461 further cleanup of the REST XML output 2013-05-22 11:21:52 +02:00
Christophe Vandeplas 2776513395 moved fragmented massagedata to Model::beforeValidate() 2013-04-30 08:20:23 +02:00
Andras Iklody 6332dbf05b Removal of more remnants of the old ACL and tightening of the filename
checks

- actAs acl removed from role and user models together with some extra
code related to the ACL

- Fix of the filename regex as pointed out by cvandeplas.
2013-04-29 10:52:07 +02:00
Andras Iklody eeaa071024 Removal of the remains of the old authorization / adding new ones where
needed
2013-04-26 14:43:44 +02:00
Andras Iklody b98818ebfb Small errors with the merge corrected
- some errors managed to slip through during the merge, should be fixed
2013-04-25 15:37:49 +02:00
Andras Iklody 4396cec8ea Integrated ownership, ACL and minor fixes
- Orgs can propose new attributes or changes to existing attributes for
  events that they do not own

- publishing users of the owner organisation can see, accept or discard
  them

- Reworked the access control

- minor fixes
2013-04-25 14:04:08 +02:00
Christophe Vandeplas e9bcacfbc3 fix sanitization in Attributes #96 2013-04-24 15:20:20 +02:00
Andras Iklody ada0fd2bfb Some fixes to indeces not set
- Affecting Event creation, attribute deletion remotely and logging of
  event deletion
2013-04-17 15:40:01 +02:00
Andras Iklody 9a6733acfd Removal of deprecated code
- The flag private is deprecated, removed together with the code that was
  affected by it
2013-04-17 11:13:09 +02:00
Christophe Vandeplas 8029d7fa29 removes multiple correlation engines Fixes #83 but after testing issue
#95 comes to light
2013-04-16 16:59:12 +02:00
iglocska b134ffd940 Extensions of filenames now validate if a number is included 2013-03-19 14:33:56 +01:00
iglocska 30c4969cbe Update to the validation of file names to allow _ in the extension 2013-03-19 14:31:35 +01:00
iglocska f008eb9f07 Search for attributes by organisation
- New search functionality on request - restrict attributes by
  organisation

- Also, attributes in the list attributes and search attributes result
  pages, that belong to the user's organisation will have a red event ID
2013-03-19 11:54:14 +01:00
iglocska 32de082c88 New export feature
- To restrict the authentication key from being used by interactive users,
implemented a new export page that uses the uses cake's user
authentication

- the old export features still exist for users with perm_auth enabled
  accounts - renamed to automation

- Exporting the events that found attributes belong to in a search
  attributes result page

- exporting of individual events to file by clicking a link in event view
2013-03-18 11:48:36 +01:00
Andras Iklody 019e976783 Removed the js title bubble for related events
- Removed javascripts based title bubble showing the event info in related
  events / attributes and in the search attribute view.

- Replaced it with values provided by extra cake queries as the delay for
  fetching the info field through a js rest request was annoyingly slow

- some coding standards
2013-03-08 13:16:02 +01:00
Andras Iklody 0e18aa099b Attribute and event access
- Updated the check for authorisation to view an event and attribute as
  the system hid some valid combinations (such as a server only attribute
  in a higher distribution level event).
2013-03-07 18:16:00 +01:00
Andras Iklody 32dc28adb9 Update to the admin privileges
- Changed the requirement for a lot of functions to be site admin as
  opposed to admin.
2013-03-05 15:19:58 +01:00
Andras Iklody 4a368918eb Colouring of search terms works in links
- links now have proper colouring to make the found terms more visible
2013-03-04 18:05:17 +01:00
Andras Iklody 036308bc0e Some changes to the search
- changes to the validation of the results

- fixes an issue where the escaping of slashes showed up with a //

- made the found results more visible and case insensitive
2013-03-04 17:33:34 +01:00
Andras Iklody 581fc5e851 Slight update to the filename regex
- accept extensions from 2 to 4 characters in length
2013-03-04 15:42:16 +01:00
Andras Iklody a72503161b Fixed some regex issues and file name validation
- Fixed an issue that caused attribute values to be converted to 1 on
  save in case of an empty regexp table

- Filename validation now happens via whitelisting instead of filename
  sanitization
2013-03-04 14:53:52 +01:00
Andras Iklody 8e6852e037 Export distribution
- Export didn't take into account distribution rules, should be fixed

- Fixed a bug with editing attributes
2013-02-28 17:16:52 +01:00
Andras Iklody a0616558e9 Still issues with the attribute search
- should be ok now
2013-02-28 10:16:42 +01:00
Andras Iklody 5ab2de4824 Fix to the updated search attributes
- issue on the live server with the search field left empty, fixed
2013-02-28 09:57:43 +01:00
Andras Iklody 01649046c0 Several things (search, migration)
- Changes to the default setting for non private events after migration

- search attribute update to be able to exclude events
2013-02-27 17:15:09 +01:00
Andras Iklody a735822ca1 Updated the migration script (SQL)
- Script updated based on the issues during testing

- Changed the file upload/downoad mechanism.
2013-02-27 15:24:38 +01:00
Andras Iklody 6dc73314bf Changes to the distribution handling of attributes
- Only the creating org of the event can change the distribution of
  attributes

- Attribute distribution setting are only pushed on edits if they were
  manually changed (so that the distribution level of events on the
  creating server doesn't get degraded by an edit and push of the event at
  a synced server when using connected community settings).

- slight change to the batch attribute search, the search terms are only
  echoed up to 9 terms to prevent the mass echoing of a long list
2013-02-25 16:38:04 +01:00
Andras Iklody 0969752dc5 Attribute edit fixed
- Editing attributes caused an error because the uuid was not passed back
  from the form (and it is used to find the attribute locally for rest)

- UUID is now used from the read attribute for non rest users. In the long
  run it would be cleaner to not allow non rest users to reach that part
  of the code.
2013-02-22 11:29:22 +01:00
Andras Iklody 0f947085cb Reworked the sync / release control
- Fixed issues with the sync
	- Secondary publishes on remote servers failed
	- Introduced new fields in events to stop backward traverse of
	  edit information that lead to low performance and eroneous
	  distribution information updates when more than 2 servers were
	  linked
	- Deletion of an attribute now deletes on remote servers

- Changes to the event ownership
	- Original creator org now noted in the event itself
	- Only original creator org can change distribution
	- Events will show up with the original creator org for users
	  (admins can see both that and the owner of the event on the
	  local instance)
	- Server.organization now used in junction with the connecting
	  user's org and the instance's org (from the bootstrap) to
	  determine distribution flow control and access rights

- Lots of minor changes
2013-02-19 15:37:35 +01:00
Andras Iklody f754eec840 Minor change to the validation
- Some types didn't have any validation info, defaulting in an incorrect
input - fixed

- re-enabled the sanitization of file names
2013-02-11 17:23:07 +01:00
Andras Iklody e17228490b Minor changes to the validation 2013-02-11 15:56:10 +01:00
Andras Iklody afed0f2046 Changes to link validation and minor fixes
- Links get validated now to filter malicios code

- removed a double edit button in the case of an admin editing himself

- fixed an error with adding new attributes
2013-02-11 11:26:34 +01:00
Andras Iklody e976242878 Reworked aros_acos creation
- moved and fixed the aros_acos creation on the new role creation

- new method in appController that sets all the aros_acos from scratch
  (for example for a new instance, or a changed acos / aros table)

- some minor changes, redirects to the terms page on invalid events
  removed, etc.
2013-02-05 17:22:37 +01:00
Andras Iklody 7f6f166838 Fixes to access rights, some sanitization, etc
- Admins cannot manually change anyone's authkey, they need to generate a
  new one via the reset link

- Some pages could be accessed by changing the url - fixed (though needs
  further testing)

- Edited a change in the manual that may have been confusing

- Some changes to the way ACL is set up - still needs more work
2013-02-04 17:55:35 +01:00
Andras Iklody 879a5fb282 Temporary fix for file-uploads under windows
Added an alternate file-upload/download path creation for PHP_OS ==
'WINNT'

Also removed autofill for the login field
2013-01-31 10:25:03 +01:00
Andras Iklody cd78baeb14 Issue with uploading attachments fixed
Uploading an attachment would fail while trying to set the event to
unpublished. Fixed.
2013-01-28 15:51:54 +01:00
Noud de Brouwer 4c83ad3cfe coding standards
Coding Standards.
2013-01-28 08:42:20 +00:00
Noud de Brouwer a6371f5ad8 coding standards
Coding Standards.
2013-01-28 08:32:01 +00:00
Andras Iklody 6afc1e993f Attribute distributions
Added feature to block distribution levels that would get overruled by the
event distribution. The distribution of the event will be the currently
selected distribution when creating an attribute.
2013-01-25 13:44:43 +01:00
Noud de Brouwer 3b07348849 distribution
attributes inherit distribution from event.
2013-01-25 11:25:18 +00:00
Noud de Brouwer 8526459173 Merge branch 'develop' of ssh://misp.ncirc.nato.int/home/git/cydefsig into develop 2013-01-22 10:06:31 +00:00
Andras Iklody 3e0bc0a4fd Small change to batch searches
An empty new line caused every attribute to be displayed. Fixed.
2013-01-18 11:31:06 +01:00
Andras Iklody f544ac2e08 Batch search for attributes
Implementation of request to be able to do batch attribute searches
2013-01-17 16:03:04 +01:00
Noud de Brouwer 4c4a2bcb89 error
behavior error or just plain wrong on our side.
2013-01-16 15:31:52 +00:00
Noud de Brouwer 755f19c560 error
behavior error or just plain wrong on our side.
2013-01-16 15:27:15 +00:00
Noud de Brouwer 4073bd9b65 error
behavior error or just plain wrong on our side.
2013-01-16 14:11:02 +00:00
Noud de Brouwer 804a0cf0f6 PHP practice
array-content.
2013-01-14 16:32:09 +00:00
Noud de Brouwer 89845933a4 CakePHP
odity, if i add "tes\ntestt\ntes", blacklist the testt,
i get "tes\ntestt" as content. (other behaviors?)
2013-01-14 16:23:57 +00:00
Andras Iklody 879154eab2 Fixed deprecated errors
Removed cause of deprecated errors (Pass by reference)
2012-12-20 14:48:23 +01:00
Noud de Brouwer 547a80ba7d Sanitize
Sanitize countermeasures.
2012-12-19 15:28:31 +00:00
Noud de Brouwer 95158d2ef3 Sanitize
Sanitize countermeasures.
2012-12-19 12:42:38 +00:00
Noud de Brouwer d89ab91dee coding standards
Coding Standards.
2012-12-18 16:44:07 +00:00
Noud de Brouwer 68617350e8 Sanitize
Sanitize::clean() but redo the info and value fields.
2012-12-18 03:18:48 +00:00
Noud de Brouwer 9211e4d405 search
After added feedback on entered search terms for search attributes
and search logs, this now also works for LogsController::index()
and next and previous page.

Signed-off-by: Noud de Brouwer <noud4@home.nl>
2012-12-18 03:01:02 +00:00
Noud de Brouwer 8c7f8921a7 Sanitize
do not Sanitize::clean() $this->request->data.
2012-12-17 17:33:21 +00:00
Andras Iklody 099e5d92be Fix for the Attributes 2012-12-17 17:21:57 +01:00
Andras Iklody 1ceadab700 Added features from branch analysis_levels
-Analaysis levels setable for events as per milestone item 94
-Password change forced as per milestone item 109
-Added feedback on entered search terms for search attributes
-fixed the authentication issue
-some minor fixes
2012-12-17 15:51:30 +01:00
noud 2903493205 Merge branch 'master' into develop
Conflicts:
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
2012-12-13 16:03:35 +01:00
noud 276cb5df10 RESTfull sync
this is in responce to the email
From: <User1088@QET.BE>
To: <ndebrouwer@hotmail.com>, <andrzej.dereszowski@ncirc.nato.int>
Subject: Re: sync/REST
Date: Fri, 7 Dec 2012 13:30:10 +0000
in this there is a complaint about the RESTfull sync workings.
the email hints about 2 possible options:
i) RESTfull add event without attributes (conform the web interface)
ii) RESTfull add event with attributes (more conform the code)

both are implemented and can be choisen in bootstrap.php by
Configure::write('CyDefSIG.rest', 'ii') or 'i'.
2012-12-13 15:52:00 +01:00
noud 094719fa01 Merge branch 'master' into develop
Conflicts:
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Controller/ServersController.php
	app/Model/Event.php
2012-12-12 18:01:39 +01:00
noud 26c8ad57ee Role
renamed everything group to role (i.s.o. renaming just the visable).
2012-12-12 16:15:01 +01:00