Commit Graph

582 Commits (7a01de535928b52ebc7c219dcbba7fd82edb6b7b)

Author SHA1 Message Date
www-data f9183dee3b Merge branch '2.4' into py-virtualenv 2018-10-15 17:09:18 +09:00
iglocska 4ce81e5d0b fix: [API] Further fixes to the query builder 2018-10-10 20:38:09 +02:00
iglocska 40e7584f10 fix: [API] Further fixes to the tag handling 2018-10-10 15:17:39 +02:00
iglocska 6ead277f0d fix: [API] Handle filters with no valid tags set as filter patterns correctly 2018-10-10 15:08:05 +02:00
iglocska 9412081680 fix: [API] Fixed broken check for overriding IDS flags via proposals, fixes #3748 2018-10-09 13:07:13 +02:00
iglocska 43558a9610 new: [API] rework of the searchall/quickFilter parameters
Now it correctly works as intended on both attribute and event contexts
2018-10-08 10:17:57 +02:00
Steve Clement e26e4a2e92 Merge branch '2.4' into py-virtualenv 2018-10-08 07:45:04 +08:00
iglocska 65e16854d4 new: [export] Further changes required for the reworked export added 2018-10-05 14:48:41 +02:00
iglocska 7d7c733005 new: [internal] restsearch's bulk code moved to the model for attributes 2018-10-04 22:23:58 +02:00
Steve Clement a699c5fcd4 Merge branch '2.4' into py-virtualenv 2018-10-02 20:55:46 +08:00
Steve Clement 602fb17335 Merge branch '2.4' into py-virtualenv 2018-10-01 09:19:02 +02:00
iglocska f18f8b579a new: [API] events/restsearch rework - chunked export for performance gains 2018-09-29 23:43:52 +02:00
iglocska 6e9c61a6b4 fix: [API] handle empty value fields when running a quick search 2018-09-28 15:57:41 +02:00
iglocska c5c29b62a0 fix: [API] Fixed the quickfilter parameter 2018-09-28 15:54:08 +02:00
Steve Clement df3d87b5c8 Merge branch '2.4' into py-virtualenv 2018-09-28 15:00:46 +02:00
Steve Clement 51a3a5cdbf chg: [python] Added and amended varios places where python is called 2018-09-28 14:59:26 +02:00
iglocska 0216c9ea33 new: [API] Rework of the restSearch APIs
- peformance tuning
  - removed some redundant looping
  - internal memory profiling for attributes/restSearch
  - saving the intermediary results to file instead of keeping it all in memory to reduce the memory footprint
- added the searchall parameter
- fixed the ignore parameter
- added the event_timestamp parameter
- added manual pagination to the attribute level restsearch (limit, page)
2018-09-28 09:21:29 +02:00
iglocska 0df898f42d fix: [API] Fixed the handling of the to_ids flag 2018-09-14 10:41:39 +02:00
iglocska 45c882b420 fix: [API] various fixes to the timestamp handling 2018-09-11 23:20:00 +02:00
iglocska a3f83fac63 fix: [API] fixed an invalid dissection of the tag parameter if the parameter is not set 2018-09-10 14:31:55 +02:00
iglocska 51b3ef61dd new: [API] Added the includeEventTags parameter to the /attributes/restSearch API
- appends all event level tags to each attribute
2018-09-09 16:49:59 +02:00
iglocska 55fd24d491 fix: [RPZ] flatten attributes for the RPZ export 2018-09-08 23:46:37 +02:00
iglocska 141c7d451f fix: [API] invalid pass by reference parameter not passed as a variable
- fixes "Cannot pass parameter 1 by reference" bug
2018-09-03 10:11:08 +02:00
iglocska 5f1edc9bad Merge branch 'feature/api_rework2' into 2.4 2018-08-31 13:37:41 +02:00
iglocska 9932b73421 chg: [internal] not needed conditional cleaned up 2018-08-31 13:17:54 +02:00
iglocska f3558fb18a Merge branch 'feature/api_rework' into 2.4 2018-08-31 12:58:44 +02:00
Alexandre Dulaunoy cc437a2a07
chg: [data-model] new bro attribute type to store rule in Bro rule-format
Fixed #3584
2018-08-28 20:42:10 +02:00
iglocska 97d075f22f new: [API] reworked the attribute level restsearch
- use the new filter parameters
- use the new condition building mechanism

- no more pre-filtering
2018-08-24 14:21:50 +02:00
iglocska b407aba746 fix: [CS] Updated recent changes 2018-08-20 10:50:09 +02:00
iglocska a60b24a14a chg: [API] Fixed fetchAttributes lookup on value to be only optionally a substring search 2018-08-20 07:45:38 +02:00
iglocska f675fb8b29 Merge branch '2.4' into feature/api_rework 2018-08-17 14:49:09 +02:00
Hannah Ward c883a7b6d6
new: Add upload/download for attachments 2018-08-15 14:07:44 +01:00
iglocska 006a922e9f chg: [API] further work on the new CSV export 2018-08-14 23:38:01 +02:00
iglocska e2a268aea7 new: [API] further rework of the restsearch api
- move to the new popping filter system
2018-08-09 15:11:57 +02:00
iglocska 8d699d35a9 new: [internal] Further work on the filtering 2018-08-09 07:46:31 +02:00
iglocska 4600cdef87 fix: [internal] looplimit lowered to 50k for fetchAttributes
- maybe we should base this number off the available memory somehow...
2018-08-06 20:53:30 +02:00
iglocska 76ede22308 new: [refactor] CSV api refactor
- performance gains
- first step in unifying all APIs
- moved the CSV data lookup into fetchattributes
- internal pagination is now more clever with a watchdog flag that can prevent unneeded executions by whatever calls fetchattributes
2018-08-05 19:10:52 +02:00
iglocska 49d78bea32 chg: [cleanup] Removed duplicate capture field definitions 2018-08-03 15:03:35 +02:00
iglocska 273af898ce fix: [internal] removed massive duplicate lookup function 2018-08-03 14:58:24 +02:00
Christophe Vandeplas aba4c90e0c
fixes missing hostname|port in network activity mapping
The hostname|port has default category "Network Activity" , but was not allowed by the mapping.
2018-08-03 13:31:23 +02:00
iglocska f712efe85e chg: [internal] instead of replicating the event level distribution rules, the attribute model now inherits the event code 2018-08-03 11:34:55 +02:00
iglocska 1810c920b4 new: [API] rework of the attribute fetcher
- correctly handles attribute tags
- performance improvements due to rework of the internal pagination
- fixes to issues with too many hits on a tag search causing queries that are too long
2018-08-02 15:41:51 +02:00
iglocska 1930823474 fix: [API] set attribute distribution if it isn't set in the capture attribute call
- should have worked via the beforevalidate() but it didn't
- ah well
2018-07-20 16:32:30 +02:00
iglocska a81894f14c chg: [CS] Changed to PSR-2
- to make contributions easier, adopted PSR-2
- used php-cs-fixer to rework the style
- *sniff sniff* Goodbye tab indentation
2018-07-19 11:48:22 +02:00
Steve Clement c607729e18 chg: [i18n] More __(); 2018-07-12 23:36:47 +02:00
iglocska fc40bee6ee fix: [python version] changed generate_file_objects.py's execution to python3 2018-07-06 11:24:16 +02:00
iglocska 8d567782d9 chg: [cleanup] Removed the deprecated GFI sandbox import
- Burn the heretic. Kill the mutant. Purge the unclean.
2018-07-06 10:57:44 +02:00
iglocska 064748d87d new: [internal] setTimestampConditions unified and improved
- no more separate codepath for setPublishTimestampConditions
- accept shorthand time descriptions (1d, 5h, etc)
- always accept single values or arrays with start/end times
2018-07-04 15:33:50 +02:00
iglocska 898aef2e10 new: [data model] Added support for monero - new type xmr
- soft validation
- secondary validation with warnings for malformed addresses
- supporting epic facial hair styles
2018-07-04 10:22:58 +02:00
iglocska c67b575ba6 fix: [bug] Potential fix for SQL return size limit reached when fetching a list of attributes 2018-07-02 10:38:24 +02:00