Commit Graph

136 Commits (181566bafb77f93a6af0d63422f6634ba70ee57d)

Author SHA1 Message Date
iglocska 78d58b6cbd Changes to the generation recreation 2015-12-17 01:25:23 +01:00
Iglocska bc2c14c815 Disable e-mailing globally for an instance 2015-12-15 10:52:31 +01:00
Iglocska 3a2df829a0 Default settings for roles altered 2015-12-15 10:07:11 +01:00
Iglocska b0e8214cbf Added some fixes to corner cases
- publishing an event when push is enabled to a 2.3 instance failed with an error instead of blocking
- publishing an event wth the remote instance blocking it due to a sync user sharing group conflict resulted in an exception, handled gracefully now

- Added mangle-sync towards 2.3
  - gracefully push non sharing group events in a 2.3 format
  - timestamps downgraded by 1 second - upgrading the 2.3 instance should automatically allow a resync of mangled events
2015-12-14 03:11:45 +01:00
Iglocska 5debff6b0d Visual fixes 2015-12-09 01:50:50 +01:00
Iglocska af92e29e3c Further progress on the sync
- also, added maintenance mode
- various fixes
2015-12-09 01:43:54 +01:00
Iglocska 3831ee2e90 Further work on the sync 2015-12-07 01:32:51 +01:00
Iglocska fa7fa322cb First stab at the push filters influencing the pull of a remote instance 2015-12-06 19:41:29 +01:00
Iglocska ab26eafd63 Added the possibility to enable debug for site admins
- new option in server settings
- enable debug (equal to normal debug level 1) for site admins only

- regular users will be unaffected
2015-12-03 01:45:38 +01:00
Iglocska d433618c71 Also, enabled the filtering on pull
Merge branch 'master' into 2.4-beta

Conflicts:
	VERSION.json
	app/Controller/EventsController.php
	app/Lib/Tools/XMLConverterTool.php
	app/Model/Event.php
	app/Model/Server.php
2015-12-03 00:27:56 +01:00
iglocska 268c7683a0 Rework of the event add/edit
- allows for saving an event even if an attribute fails
  - logs attributes that fail validation

- same for edit

- add_misp_export updated with the above in mind
2015-12-01 15:39:12 +01:00
iglocska ec566d6225 Small fix to the upgrade script 2015-11-20 01:09:02 +01:00
iglocska 0fb9a70449 Merge branch 'master' into 2.4-beta
Conflicts:
	VERSION.json
2015-11-20 00:15:17 +01:00
iglocska 56adab6122 Bugfix pack, fixes #724, fixes #721
- Fixed an issue with the new UUID generation method call in OpenIOC
- Fixed an invalid validation check on the salt key

- Added a note on the server page to make it more obvious that values can be changed by double clicking them
2015-11-19 16:50:14 +01:00
iglocska 8cc0996c3c Merge branch 'master' into 2.4-beta
Conflicts:
	VERSION.json
	app/View/Elements/side_menu.ctp
	app/View/Pages/administration.ctp
2015-11-14 17:16:38 +01:00
iglocska a380458d2e Fixed a security issue with the site admin file uploader
- as discovered and reported by Egidio Romano of Minded Security

- The site admin file upload tool allowed for unrestricted file upload that could lead to RCE
- Fixed the file uploader to be much more restrictive
- removed the interactive terms file upload
2015-11-13 23:48:29 +01:00
iglocska 324418dcf9 Merge branch 'master' into 2.4-beta
Merge and upgrade of several new features

Conflicts:
	VERSION.json
	app/Controller/ShadowAttributesController.php
	app/Controller/TagsController.php
	app/Model/AppModel.php
	app/Model/Event.php
	app/Plugin/SysLogLogable/Model/Behavior/SysLogLogableBehavior.php
2015-11-08 23:26:19 +01:00
iglocska 16e7974fc0 Improved logging, fixes #695
- Added logging of failed login attempts
- Added (optional) logging of successful authentications
  - admin setting that has to be enabled
  - will log all API calls (both HTTP method and target url)

- optional logging of user IP address for all logs
  - each log entry created while this setting is enabled will log the IP address of the client
  - disabling it also hides the IPs from the interface
  - added new IP field for the log search (only if enabled)
2015-11-08 22:35:46 +01:00
iglocska 1d3ec3afa3 Merge branch 'feature/sg' of https://github.com/MISP/MISP into feature/sg 2015-10-22 10:01:25 +02:00
iglocska ef1d3949e7 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
	app/Controller/ShadowAttributesController.php
	app/Lib/Tools/JSONConverterTool.php
	app/Lib/Tools/XMLConverterTool.php
	app/Model/User.php
	app/View/Elements/eventattribute.ctp
2015-10-22 09:59:00 +02:00
iglocska e191a25317 Updated the server preview to work between 2.4 instances 2015-10-18 21:31:01 +02:00
iglocska 37faddff62 Further progress
- rework of the push mechanism
- rework of the object capture on add
- rework of the sync filter UI
2015-10-18 18:03:48 +02:00
Iglocska 3dc83778c2 Fixed the current user check while removing dead workers, fixes #685
- as pointed out by RichieB2B
2015-10-14 15:07:29 +02:00
Iglocska 888042dbd6 Added the first version of the correlation graphing
Conflicts:
	VERSION.json
2015-10-14 03:45:34 +02:00
Iglocska c9bf8105e6 Replaced get_current_user for the process owner identification, fixes #685
- As RichieB2B noted, get_current_user() gets the owner of the script in CentOS / RHEL not the user executing the script (as in Ubuntu)

- Current solution uses posix_getpwuid and posix_geteuid if the php-posix package is installed
- if not, it uses whoami
2015-10-13 10:49:55 +02:00
Iglocska f9f9eed6fd Issue fixed with open_basedir preventing the worker diagnostics from working, fixes #685
- for some users the workers appeared to be dead even though the worker processes were functional and started by the correct user
- this was due to access to /proc being blocked by open_basedir directive settings
- added a check and the corresponding view changes to this being the case
2015-10-12 16:48:43 +02:00
Iglocska 328c666e3a Merge branch 'master' into feature/sg
Merging all the new changes from master

Conflicts:
	VERSION.json
	app/Console/Command/AdminShell.php
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Log.php
	app/Model/Server.php
	app/Model/User.php
	app/View/Elements/side_menu.ctp
	app/View/Pages/administration.ctp
	app/View/Users/admin_index.ctp
2015-10-12 09:41:20 +02:00
Iglocska f018c60c01 Fixed the worker diagnostics showing incorrect data under Red Hat / CentOS, fixes #685
- Under these distros, php is blocked from seeing concurrently running php processes even under the same user
- instead of running ps, the diagnostic now checks the existance of the pid file in /proc/
2015-10-09 17:09:33 +02:00
Iglocska dc638adac2 Upgrade to CakePHP 2.7, fixes #684
- cakephp submodule updated to 2.7
- make sure that you update your instance!

- not updating will not break compatibility
2015-10-09 15:59:25 +02:00
Iglocska e242c9add2 Set of changes to the sync
- finished preview feature
  - can now view events and attributes remotely
  - can copy over new event to local instance

- new sync mode (update)
  - allows to only pull changes to events that exist locally already
  - works well with the manual pull of events, no need to pull events that we didn't manually confirm, but can still update all events that we pulled over

- Fixed an issue with background tasks causing the logging to fail

- reworked connection test showing version numbers of both instances
  - also telling the admin whether the sync is compatible or not

- Further refactoring / tweaking of the vent view
2015-10-06 01:16:48 +02:00
Iglocska 6bcf104724 Progress on several features
- implemented a custom pagination tool for data sets that are not directly taken from teh db
  - currently creates a pagination object that mocks CakePHP pagination
  - supports the CakePHP pagination view helper
  - supports: pagination, sorting, custom filters

- implemented first step of the remote instance browser for admins
  - view an index of events on another instance
  - filter the events
  - uses the new pagination

- still missing:
  - remote event view
  - fetch event from remote instance

- reworked the event view
  - separated API and UI code path
    - major speedup for the API!
    - cleaner code as there was almost 0 overlap
  - discussions and attributes are now loaded separately from the event view
    - added after the event view loads via ajax
    - cleaner pagination
  - attribute pagination now finally allows for sorting
    - future improvement (coming soon): Show proposals only filter
    - filtering on the attributes in general
2015-09-29 02:54:25 +02:00
Iglocska a55e38ffda Fix to the previous commit 2015-09-23 12:32:00 +02:00
Iglocska 6ef7772e0b Fixed an issue with old upgraded instances that didn't use the db session handler
- diagnostic tool would throw exceptions because the db session tables are still missing in some older instances
- if a different session handler is used, the test is skipped
2015-09-23 12:14:32 +02:00
iglocska 656a391223 Progress on the sync
- pull from 2.3 -> 2.4 should work correctly now
2015-09-21 14:55:40 +02:00
iglocska 1079c4a1ea Added a diagnostic to check and purge overgrown session tables 2015-09-16 18:55:28 +02:00
iglocska 26e640e19c Further progress on several issues 2015-09-16 12:15:43 +02:00
Iglocska df59748719 progress on several issues
- switching workstations, this is all WiP
2015-09-15 15:11:08 +02:00
Iglocska e1a95c62b0 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
	app/Controller/EventsController.php
	app/Controller/ServersController.php
	app/Model/Attribute.php
	app/View/Users/statistics.ctp
2015-09-10 15:02:44 +02:00
Iglocska 2142bf87c3 Several issues resolved
- fixed an issue where pushing a single event would fail

- both event and attribute edits via the API work without providing a timestamp. The current timestamp is instead attached

- both event and attribute edits fill the required fields from the data in the database if not supplied (as long as the uuid is found)
2015-09-07 14:25:24 +02:00
Iglocska 3fb7764c00 First revision of the upgrade scripts
- .sql file to add all the new fields / tables
- admin tool to convert the old organisation fields to the new objects
- still missing a cleanup method (to remove the old organisation fields once the conversion is done)
2015-09-02 01:34:52 +02:00
Iglocska 704880ce59 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
	app/Controller/AttributesController.php
	app/Controller/EventsController.php
	app/Model/Attribute.php
	app/Model/Event.php
	app/Model/Server.php
2015-08-30 13:29:05 +02:00
Iglocska bc4bc46fca Fix to the previous hotfix
- indexes were not created if they already existed
- this was an issue if a non unique index was present

- also made the process more verbose and added a generic method that deals with index removal
2015-08-25 11:56:29 +02:00
Iglocska 556688069a Resolved an issue that can lead to duplicate events showing up in MISP
- UUID uniqueness was previously not enforced
- changed the MYSQL.sql file to reflect the changes
- Added upgrade admin tool to remove duplicate events and make the database changes required
- Tweaked the tool for the attribute uuid fix so that it cannot created duplicate keys

- some minor fixes, such as automatically removing eventTag objects on event deletion
2015-08-25 10:32:26 +02:00
Iglocska 80dcdcf9d8 Database update admin-actions are now logged and if they fail the errors are logged 2015-08-18 09:44:59 +02:00
iglocska a93807439d Several bigger changes
- new functionality: Event blacklisting by UUID
  - site admins cna enable this feature in the server settings
  - enabling the feature will make the required db changes
  - any deleted event will automatically get blacklisted
  - this prevents deleted events from flowing back from a synced instance
  - site admins can manually add UUIDs to the list and remove entries

- fix to UUID duplication issues for attributes
  - simply run the admin script and it will regenerate the UUID of attributes that are duplicates, if any such exist
  - timestamps/event published status will not be affected

- config.core.php now includes a change that prevents from 404 exceptions being logged
  - the sync uses 404s to signal that an event with a given uuid does not exist when negotiating proposal synchronisation
  - this causes a dangerously high amount of noise in the logs
2015-08-17 16:10:10 +02:00
Iglocska 67bd44f5dc Relaxed visibility of org UUIDs and sharing groups (the latter for sync users) 2015-08-03 16:49:03 +02:00
Iglocska ad21d5c35a Further work on the Sharing Groups 2015-08-03 14:12:20 +02:00
Iglocska a181f3f2a8 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
2015-07-24 16:02:31 +02:00
Iglocska 47cc0e4d18 Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
	app/Model/Attribute.php
	app/Model/Event.php
2015-07-22 18:04:02 +02:00
Iglocska 1bf2995f4e Merge branch 'master' into feature/sg
Conflicts:
	VERSION.json
2015-07-22 17:19:13 +02:00