Commit Graph

215 Commits (7f8a81e161bf6b46e45e036b6f4831b2ed45abb3)

Author SHA1 Message Date
iglocska 7f8a81e161 new: Added caching and pagination to freetext/csv feeds 2016-12-30 16:16:56 +01:00
iglocska da433c3549 Merge branch '2.4' of https://github.com/MISP/MISP into feature/disable_correlation 2016-12-22 21:01:58 +01:00
iglocska 3a2e051b91 fix: Added an alternative to bcmod if it doesn't exist
- simply threw an exception if the module wasn't loaded on the event view if it contained an IBAN number
2016-12-22 18:06:20 +01:00
Andras Iklody ac994530e6 fix: broken bro export
- Sanitisation issues with linebreaks in comments breaking the export
2016-12-21 17:35:00 +01:00
iglocska 4155e32629 fix: Added additional refanging patterns to the complex type tool, fixes #470 2016-12-12 14:20:07 +01:00
iglocska 01f078344c fix: Fixed an issue with the freetext importer
- It looks like PHP does parse single quoted strings and replaces double backslashes with a single literal backslash
2016-12-09 08:59:59 +01:00
Armins 7ba143bcd1 Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2016-12-07 18:12:49 +02:00
Armins 4c67f0a2c8 Added fast_pattern 2016-12-07 18:07:12 +02:00
Andras Iklody 44ec75e462 Merge pull request #1726 from liviuvalsan/bro_export_improvements
Performance improvements, bug fixes and new features for the export to Bro
2016-12-07 16:52:15 +01:00
Liviu Valsan 4c022beafc - Performance improvements when exporting a large number of attributes into Bro format.
- Fixed file header formatting for the export to Bro format (tabs used consistently).
- Computing the time needed for generating the export to Bro format when done using a background job.
- When generating the Bro export from the UI all the attributes are generated in one single text file similar to the CSV export instead of a zip file with different files inside.
- Changed the file extension of Bro export files from ".intel" to ".txt".
- Removed the allowNonIDS option from the Bro export as it doesn’t make sense to have it (Bro is an IDS).
- Fixed some of the API endpoints which were not accepted (ACL issues).
- Added support for a list of events that should be / should not be included in the export.
- Added a new "meta.desc" column (added in Bro 2.5, see https://www.bro.org/sphinx/frameworks/intel.html) containing the description of the event and of the attribute.
- Sanitized the exported data for Bro.
- Fixed a number of value substitutions which were imported from Snort/Suricata and which were not working for Bro. Did instead substitutions needed for Bro.
2016-12-07 16:33:17 +01:00
Iglocska 1e7dccf272 Merge branch '2.4' into feature/galaxy 2016-12-06 16:11:59 +01:00
Iglocska 8f220378ce new: First RC of MISP galaxies 1.0 2016-12-06 15:52:20 +01:00
Iglocska 576d58462d fix: Trim strings of brackets before running the freetext detection on them 2016-12-01 12:24:42 +01:00
Iglocska 162e024eb8 fix: Temporary fix for a keyword mismatch between the import modules and the freetext import 2016-11-29 11:56:16 +01:00
Iglocska 6e52070f48 fix: Fixed an issue that prevented the feeds from working in CSV mode if no value field was set 2016-11-24 09:50:22 +01:00
Iglocska 7e75aafc22 fix: Added domain|ip to nids exports 2016-11-09 17:08:06 +01:00
Iglocska c2fc803fed chg: Use the TLD lists from the warninglists, fixes #1149
- simply load any enable warninglist entries from the pre-defined TLD warninglists
- Pass the resulting array to the complex type tool
- during domain type heuristics, if the TLD list is not empty use the supplied list
- alternatively generate a list based on the old TLD rules
- does not alter any functionality otherwise
2016-10-25 22:23:01 +02:00
Iglocska 6ffa949657 fix: Invalid bro export generation due to invalid syntax on the intel field 2016-10-25 12:48:51 +02:00
Iglocska 9891234662 new: CSV feeds and various fixes
- Added the CSV feed format
  - users can specify which fields in the CSV should be parsed
  - comment lines are automatically omitted
  - new settings system added to feeds, currently only used for the value fields

- Slight rework of the correlation lookup for the feeds
  - got the Speed Force treatment
  - correctly checks against value1 and value2 instead of value

- Various freetext import fixes
2016-10-08 14:36:24 +02:00
Iglocska 721cfd8d98 fix: Fixes to the ssdeep detection as it was way too loose 2016-10-07 20:20:53 +02:00
Iglocska 503661a240 new: First implementation of the freetext feed pull 2016-10-07 17:33:54 +02:00
Cristian Bell 5be1e17bce Revert "fix: missing new TLDs in free text import, solves #1149 (#1574)"
This reverts commit e3bb9d3a42.
2016-09-27 16:38:35 +02:00
Cristian Bell e3bb9d3a42 fix: missing new TLDs in free text import, solves #1149 (#1574)
* fix: missing new TLDs in free text import, solves #1149
2016-09-27 15:53:43 +02:00
Iglocska 9b7191f878 fix: Don't show the org restriction of a tag in the event view JSON 2016-09-27 09:38:32 +02:00
Andreas Ziegler a6e93d6020 chg: update cakephp to 2.8.9 (#1560) 2016-09-23 04:36:26 +02:00
iglocska f6187f8fa5 fix: Fallback to insecure random for php 5.x if the random_compat submodule isn't loaded 2016-09-18 16:11:33 +02:00
iglocska 62a2211a23 Merge branch '2.4' into 1457 2016-09-18 13:06:03 +02:00
iglocska a599ec24f7 Merge branch '2.4' into 1501 2016-09-18 11:07:10 +02:00
Iglocska 6d822ee45e fix: Refactor of the bro export to always create a zip archive with separate files if "all" types are queried 2016-09-16 16:49:54 +02:00
Iglocska 1991f7a208 fix: Some changes to the bro export
- moved the whitelisting out of the plugin
- source now contains the instance host org name (if applicable), the event UUID and the creator org name
2016-09-16 14:55:25 +02:00
Iglocska 2cede15e68 Merge branch '2.4' into feature/bro-export
Conflicts:
	app/Model/Event.php
2016-09-15 18:00:25 +02:00
Iglocska 40626963cc chg: Cleanup of removed Hids and Nids BroExport libraries that got merged into BroExport.php 2016-09-15 17:45:51 +02:00
Iglocska 59ecf40f42 chg: Refactor of the Bro export 2016-09-15 17:44:59 +02:00
Andreas Ziegler 25e52a6786 chg: remove some references to variables 2016-09-15 17:08:58 +02:00
Andreas Ziegler 72730e54ef new: add Tool for random string generation 2016-09-15 17:07:12 +02:00
Andreas Ziegler b3c5e56b38 new: add compatibility Lib for random_int 2016-09-15 17:07:12 +02:00
Iglocska 85879e735c chg: Reverted the changes to the NIDS export 2016-09-15 16:29:30 +02:00
Andreas Ziegler 8d8227690e chg: update cakephp to 2.8.7 2016-09-13 01:46:03 +02:00
Iglocska 01695e326a new: Added the metadata flag to the event restsearch API
- allows fetching metadata only without including attributes/proposals
2016-09-12 12:09:19 +02:00
Andreas Ziegler 4b8a82098d chg: replace 4 spaces after tab by double tab 2016-09-05 00:45:51 +02:00
iglocska d85fd0d813 fix: Fixed a newly introduced bug that breaks the NIDS exports, as referenced in #1489 2016-09-01 14:44:03 +02:00
ppanero 131e2f760a bro export funtionality 2016-08-29 17:26:14 +02:00
iglocska 5a72f84c22 Merge branch '2.4' into 2.4.51 2016-08-28 21:08:02 +02:00
iglocska 8f528ae881 fix: Removed incorrect uses of pass by reference, fixes #1472 2016-08-24 09:50:19 +02:00
iglocska 37297c2e15 Merge branch '2.4' into 2.4.51 2016-08-23 00:26:25 +02:00
Andreas Ziegler 30fb4e2b2e chg: remove whitespace at end of line 2016-08-22 02:54:51 +02:00
iglocska 3c0f3fb8bb Merge branch '2.4' into 2.4.51 2016-08-21 22:59:30 +02:00
Andreas Ziegler f0905dc536 chg: rename FileAccess to FileAccessTool
every other tool classes name in the Lib/Tools/ folder also ends with "Tool"
2016-08-19 19:25:32 +02:00
Andreas Ziegler a2ff5424e1 chg: change FileAccess from static to instantiable class 2016-08-19 19:22:15 +02:00
iglocska 444171bd2d Merge branch '2.4' into sslclientsync 2016-08-18 09:58:52 +02:00