Commit Graph

67 Commits (c3ebc18afa81a68fa3fa0722cb7a1d0fa8ea8eb4)

Author SHA1 Message Date
iglocska 61e865956b Fixes to several issues, fixes #693
- Fixed a critical bug in the XML export
  - As of recently XML exports include relations as they were missing before
  - the sanitisation of the event info field in related attributes was incorrectly sanitized of unicode characters
  - this can lead to the XML export breaking and also for affected events to be blocked from synchronisation

- Proposal fixes
  - fixed an invalid uuid generation that lead to an exception
  - fixed the attachments for proposals still using the old attachment system that disallows most filenames
  - added the automatic creation of hashes for attachment proposals
2015-10-21 23:44:07 +02:00
Iglocska 4a358c9fee Removed junk 2015-10-15 17:44:33 +02:00
Iglocska 7fab2ce2be Added the attribute relations to the XML / JSON output, fixes #687 2015-10-15 17:39:17 +02:00
Iglocska dc638adac2 Upgrade to CakePHP 2.7, fixes #684
- cakephp submodule updated to 2.7
- make sure that you update your instance!

- not updating will not break compatibility
2015-10-09 15:59:25 +02:00
Raphaël Vinot b2876c6959 Update default version for cakephp, make sure PyMISP follows master 2015-09-22 17:36:41 +02:00
Iglocska 5273c2c5aa New functionality: API to download sample by hash
- simply pass an MD5 hash along and receive a sample if available zipped and base64 encoded in a response object
- pass any hash along with a flag set and receive any samples from events that have the passed hash

- Also, fix for an issue with the freetext import not using semi-colons as separators
2015-08-07 12:04:44 +02:00
Iglocska 684be69ed0 Incremental export generation for HIDS and NIDS exports
- Instead of fetching all events at once for the export, events are fetched one by one
- Greatly reduces memory footprint (It mostly depends on the event with the most eligible attributes now, instead of the combined list of all events)
- Because of the lower memory usage, the time taken for the export is also slashed to a fragment of what it was before
2015-07-20 16:25:16 +02:00
Iglocska 93f9a01c87 Changes to the hids exports
- fixed some issues with unset variables (from, to, last) when triggered by the background workers
- reduced memory usage of the hids exports (removed storing the hashes twice in memory, drastically removed the data retrieved from the db when preparing the export)
2015-07-17 10:50:50 +02:00
Iglocska 2438ec2f35 Some tuning to the hostname / url type recognition in the freetext import tool, fixes #562 2015-07-09 10:56:17 +02:00
Iglocska 6a25471ea4 Fix to an error with very large strings in an array causing a failure in the XML conversion of simpleXML, fixes #500
Moved the XML conversion in restfullEventToServer() to MISP's own xml conversion tool
2015-07-08 10:37:20 +02:00
Iglocska e706562cd5 Fixes to the RPZ export based on the testing of elhoim
- some errors in the format (wrong comment character used, rpz-ip not appended to IP addresses, missing semi-colon)
- removed hostnames that are on domains blocked by the rules based on domain attributes
2015-07-07 14:42:28 +02:00
Iglocska 3f215743f0 Complete rework of the ZeroMQ implementation
- python server running in the background doing the publishing
- MISP -> python script communication via redis
- configurable / controllable via the admin UI
2015-06-29 08:56:45 +02:00
Iglocska 3d2b8c3c10 Added pub/sub feature using ZeroMQ, fixes #540 and fixes #526
- by installing the requirements described in the update and the install instructions (ubuntu only for now, centos/red-hat versions to be tested and described), administrators can enable the pub/sub feature
- assign a port to the service via the interface
- each time an event is published, MISP will use ZMQ's PUB feature to push out a MISP JSON package using the "misp_json" prefix
2015-06-24 17:38:15 +02:00
Iglocska ae0e1aeacb Merge branch 'feature/rpz' into hotfix-2.3.86
Conflicts:
	app/Console/Command/EventShell.php
	app/Model/Server.php
2015-06-23 17:19:08 +02:00
Iglocska 0e09319eae Tuning of the complex type tool 2015-06-22 14:11:43 +02:00
iglocska a0afab66a7 Various changes and bug fixes
- contact reporter first tries to contact orgc users on the instance, if they don't exist, it will contact the owner (instead of going straight to the owner)
- hostname / domain name validation change broke validation of hostnames/domain names / email addresses with a "-"
- Some documentation changes for the REST API (more coming)
- some tuning of the freetext import
2015-06-18 14:49:25 +02:00
Iglocska 822172d425 Fixes to the e-mailer and the HIDS export
- HIDS exports did not include filename|hash types
- Sending a password reset / welcome message picked the opposite subject line
- line breaks were sent as literals.
2015-06-10 09:50:38 +02:00
Iglocska 197e6a69cc Free text import tool tuning, fixes #510
- comma separated values now correctly parsed
- Ports in IP/url/link/domain/hostname now added as a comment
- virustotal now automatically recognised as external analysis / link
2015-06-02 12:02:07 +02:00
iglocska 2fd600b44a Left of tuning of complex type tool in previous commit
- also, appcontroller now loads the security component, so that the blackhole override doesn't produce errors
2015-05-27 19:10:14 +02:00
iglocska 5559536bf1 Freetext import tool now splits the input by line break and whitespace, fixes #502 2015-05-24 10:09:41 +02:00
iglocska dedc9f5ef2 First revision of the RPZ export complete
- documented in automation view
- right now it follows the simple rule of user > admin settings > default values when generating the export
- Parameters can be passed via url / JSON object / XML object
- filters include filter on event ID, date range, tags

TODO:
- buttons for a per event download via the UI
- introduce new export option for normal users (via background workers and the old style export)
2015-05-20 15:04:41 +02:00
iglocska 81bd84af11 Further progress, still rough around the edges
- server settings and validation work
- configurable template via settings
- configurable via API as well

- Also trying to define the structure for future Plugin settings
- The idea is to have them in a separate tab all prepended with the plugin name
- since this is not yet part of the future flexible plugin system, it is still kept in the main codebase, but the idea is to get the naming conventions ready for the future version
2015-05-19 15:25:45 +02:00
iglocska 99f79ec318 First version of the RPZ export
- still undocumented
- very naive policy settings
- limit per event / tags / date range
2015-05-15 14:58:53 +02:00
iglocska 7fb6e2f95e Fixed issue with proxy settings attempted to be added in synctool, even if not set 2015-05-15 09:25:07 +02:00
Iglocska 35976ac847 Merge branch 'cakephp-update' into hotfix-2.3.59 2015-04-08 22:38:18 +02:00
Richard van den Berg 056b21ff6f Update cakephp to latest 2.6 branch 2015-03-24 10:03:17 +01:00
Richard van den Berg 106b6cb06b Allow SyncTool with empty $server 2015-03-19 13:03:16 +01:00
Richard van den Berg 05bc4c4389 Allow SyncTool to be used for generic HTTP(S) connections 2015-03-19 12:57:46 +01:00
Richard van den Berg 0d242d8549 configProxy() checks for empty arguments, no need to do it twice 2015-03-19 10:03:40 +01:00
Richard van den Berg 9e7ea8fe74 Add proxy support to SyncTool 2015-03-19 10:03:25 +01:00
iglocska 0dd8318fb3 Further work on the exports
- Performance improvements for the event search exports
- JSON view code moved to Lib
- Fixed an issue that didn't restrict the dates correctly with the from / to parameters
2015-02-16 17:31:32 +01:00
iglocska 2aa902cd4b Removed the old documentation, fixes #378 and some small fixes
- resolved an issue of warnings being generated when an event without attributes / relations gets XML exported.
- added new dump of the documentation
2015-02-05 10:11:29 +01:00
iglocska 2401cedfdc Left off file in previous hotfix added
- added a file that was not pushed during the last hotfix
- some improvements to the XML export to lower memory usage
2015-02-04 11:06:41 +01:00
iglocska 6e7554a26f Freetext import tool enhancement
- mass edit types where applicable
- ip-src/ip-dst type will create two attributes, one for each
2014-12-10 13:06:52 +01:00
iglocska 4568589044 File management added and various small changes
- Important! Logo images have now moved to a different location! Make sure that you update your settings!
- Site admins can now manage the uploaded image files and the terms of use file via the server settings interface
- add, link, delete files directly from the interface
2014-10-29 15:20:30 +01:00
iglocska 4fec5d041f Added missing empty file 2014-10-23 15:59:38 +02:00
iglocska dcc028516d Further improvements to the freetext regex to remove unprintable chars 2014-10-14 14:29:06 +02:00
iglocska 53d295868f Remove non printable characters from free text import 2014-10-14 10:59:12 +02:00
iglocska c7ec328fcf Better split on linebreaks for the freetext import 2014-10-13 17:43:31 +02:00
iglocska caeba7fb91 CIDR now recognised by freetext import 2014-10-13 09:16:09 +02:00
iglocska b178358f2a Default config.php added 2014-09-10 11:48:48 +02:00
iglocska 9813a3cfaa Added CVE to the freetext tool 2014-08-20 15:34:42 +02:00
iglocska 988ae667a1 CakePHP update 2014-08-20 14:58:53 +02:00
iglocska edc8029869 Updated cakephp 2014-08-06 10:09:04 +02:00
iglocska 3da49c964b Fixed broken AJAX queries in MISP as a result to changes in cakephp 2.4.8+
A change in cakephp version 2.4.8+ has resulted in ajax form submitions breaking. Reason for this was a change in the SecurityComponent taking the url specified in the form into account when generating the CSRF tokens.

This is now fixed by embedding the correct url in the ajax forms.
2014-08-06 10:02:52 +02:00
Christophe Vandeplas 9da90a1f0c Merge branch 'hotfix/export_suricata' into develop 2014-07-29 14:30:03 +02:00
Christophe Vandeplas f4e6e3fb12 regex bugfix in the ids export + suricata export using dns keyword 2014-07-29 14:28:34 +02:00
iglocska 893ef5a129 Freetext import tool
Added freetext import tool
2014-07-10 17:02:19 +02:00
iglocska 029ef252a2 Some fixes to the templating
- resolved bugs with permissions
- fixed the broken mass delete tool
- Fixed an issue with the type not being chosen correctly for file type attributes when created through the templating tool
2014-07-08 11:31:23 +02:00
iglocska a4a987e027 More work on the templates
- Templates can now be created and populated
- Users can populate an event using a template (still needs work)
- File type elements are not yet implemented
2014-06-19 08:44:35 +02:00