- The @FloatingCode and @ilmoka care package
- Improved CSV performance for instances with large number of events
- Added "value" filter for CSV (use-case: I want all indicators for this value with context)
- Added attribute tags to the output of the CSV export
- use the feed uuid caches to link directly to affected MISP events
- various UI improvements
- Feed preview pagination / POSTed event ID filters added
- massive performance boost by using redis pipelining
- for events with 10k+ attributes, show truncated feed correlation lookups, informing the user about the number of correlating attributes and a boolean flag on attributes saying that they correlate
- The overall feed correlation counter also allows users to pivot to a view that loads all correlations, though it should be used with some caution as it can be somewhat heavy
- settings (csv column number, delimiter) were ignored
- skipped fields were still counted by the paginator showing some pages with fewer than the expected 60 values
- import feed descriptor json pastes to add a list of pre-defined feeds
- improvements to the feed pull (a single non validating attribute shouldn't break the process)
- altered the saving of the attributes to happen in chunks during a feed pull to avoid very large feeds from stalling the process
- split the feeds into 3 tabs: default, custom, all
- the issue was that in the case of a feed update to a fixed event without delta merge, MISP tried to insert all parsed attributes, which correctly automatically blocked duplicates
- however, since these attributes were blocked by the validator, the feed fetcher reported that the fetch didn't succeed as it contained validation errors
- this fix simply runs non-delta merge mode updates through the comparisons to the existing event, removing duplicates in advance
- simply load any enable warninglist entries from the pre-defined TLD warninglists
- Pass the resulting array to the complex type tool
- during domain type heuristics, if the TLD list is not empty use the supplied list
- alternatively generate a list based on the old TLD rules
- does not alter any functionality otherwise
- Added the CSV feed format
- users can specify which fields in the CSV should be parsed
- comment lines are automatically omitted
- new settings system added to feeds, currently only used for the value fields
- Slight rework of the correlation lookup for the feeds
- got the Speed Force treatment
- correctly checks against value1 and value2 instead of value
- Various freetext import fixes
- allow users to override the IDS flags and keep all attributes pulled from a freetext feed IDS = off
- UI changes
- fix to a bug that caused already deleted attributes to be counted as existing ones
- removed incorrect, useless boiler plate comments
- kept useful comments intact
- added some missing line breaks to make the codebase a bit more uniform
- removed some obviously obsolete TODO comments
- set the distribution and sharing group of a feed
- will set all events received to the appropriate setting
- set a tag that should be applied by default to the events received from the feed