MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
26,023 Commits
42 Branches
370 Tags
181 MiB
Commit Graph

233 Commits (2.4)

Author SHA1 Message Date
Luciano Righetti c947b5556c fix: fix pull rules legend not showing on feeds/edit load. 2021-06-24 12:07:09 +02:00
Jakub Onderka eb7473d990
Merge pull request #7395 from JakubOnderka/feed-download-error-handlig 
fix: [feed] Better error handling when downloading MISP feeds
 2021-05-05 18:00:15 +02:00
Jakub Onderka 48228171a5 fix: [feed] Better error handling when downloading MISP feeds 2021-05-05 17:24:27 +02:00
Jakub Onderka ad1b373766 new: [log] Audit log 2021-05-03 13:44:44 +02:00
iglocska 6c6f6170b0
new: [Cache] search allows bulk lookups 
- it is now possible to search for a list of values such as:

{
    "value": ["1.1.1.1", "8.8.8.8", "8.8.4.4"]
}

- this will now return a dictionary with the key being the lookup value and the value being a list of hits and their metadata

- passing a single value will revert to the old behaviour, returning a simple list with the hits and their metadata
 2021-04-20 17:21:18 +02:00
mokaddem 0eb106ef34
Merge branch '2.4' of github.com:MISP/MISP into develop 2021-03-25 16:07:30 +01:00
mokaddem e72579f5a9
fix: [sync:local-tag] Local tags converted into global after sync for internal sync 
Fix #7253
 2021-03-25 16:04:23 +01:00
Jakub Onderka 9ab5447725 fix: [feed] Convert invalid key case 2021-03-10 14:09:06 +01:00
Jakub Onderka bc4d5c35de chg: [internal] When caching feed, save progress to db less often 2021-02-24 20:28:37 +01:00
Jakub Onderka 6259404bd7
Merge pull request #7020 from JakubOnderka/feed-saving-simplified 
chg: [feed] Simplified code for saving feed
 2021-02-18 17:34:30 +01:00
Jakub Onderka 423559fd11 chg: [feed] Simplified code for loading feeds 2021-02-16 16:10:25 +01:00
iglocska 1041707638
fix: [feed] edit ignored changes to the header, fixes #6780 2021-02-16 13:08:29 +01:00
Jakub Onderka a70aa9c392 fix: [feed] Feed name is required 2021-02-09 21:36:36 +01:00
Jakub Onderka 5f78f6a2d5 chg: [internal] Use strict comparison for in_array 2021-01-10 20:12:31 +01:00
Jakub Onderka 8de4a6d731 chg: [sync] Convert connection timeout to exception 2021-01-06 20:07:55 +01:00
Jakub Onderka 8c686304a0 new: [feed] Support brotli compression 2021-01-05 17:05:58 +01:00
Jakub Onderka 06d4c73dce fix: [inernal] Remove duplicates from server correlations 2021-01-02 18:30:02 +01:00
Jakub Onderka 1a184ebbb5 new: [internal] Allow to output directly TmpFileTool 2020-12-21 21:02:37 +01:00
Jakub Onderka ffcb54edc3 chg: [feed] Optimise freetext feed caching 2020-11-19 09:53:02 +01:00
Jakub Onderka b749610c56 chg: [feed] Check also URL without protocol 2020-10-29 18:55:17 +01:00
iglocska 42aaea20f8
fix: [server] caching notice fixed 2020-10-16 14:41:45 +02:00
Jakub Onderka 0e7d3ddc5d chg: [internal] Better error messages for unzipping feed file 2020-10-07 09:07:11 +02:00
Jakub Onderka c2f6908002 chg: [optimisation] Load MISP version and commit just once 2020-10-03 20:02:55 +02:00
Jakub Onderka 95dfd75af7 chg: [correlations] Really limit number of correlations, not number of attributes 2020-09-28 19:52:19 +02:00
Jakub Onderka d1eec44d8d chg: [internal] Initialize Job class just once 2020-09-28 19:40:32 +02:00
Jakub Onderka 7f29b6b089 chg: [internal] Delete Redis key in pipeline 2020-09-28 18:33:56 +02:00
Jakub Onderka 552315555c chg: [feed] Faster feed list fetching 2020-09-28 17:19:25 +02:00
Jakub Onderka c87616e6ec fix: [correlations] Properly delete feeds caches 2020-09-28 17:00:00 +02:00
Jakub Onderka a11e27bedc chg: [correlations] Fetch just server of feed that has data in Redis 2020-09-28 15:49:54 +02:00
Jakub Onderka 0c5be866fc chg: [correlations] Refactor feed cached correlations 2020-09-28 15:02:08 +02:00
Jakub Onderka 2deb9d3c8b
Merge pull request #6156 from JakubOnderka/feed-httpsocket-optional 
fix: [feed] Make HttpSocket instance optional for local feeds
 2020-08-03 10:51:03 +02:00
iglocska b5dffc2023
fix: [internal] older PHP still not happy with the return from a generator 2020-07-30 17:32:35 +02:00
iglocska dfc743aab2
fix: [internal] make ancient PHP versions happy 2020-07-30 17:24:19 +02:00
Jakub Onderka d9f21f75f5 chg: [feed] Better exception messages for invalid JSON 2020-07-28 18:33:05 +02:00
Jakub Onderka 6c411203c0 fix: [feed] Make HttpSocket instance optional for local feeds 2020-07-28 17:59:55 +02:00
mokaddem afbf870e16
fix: [feed:importFreeText] Make sure to update the timestamp when soft-deleting after delta-merge 
Fix #6013
 2020-07-21 12:13:59 +02:00
mokaddem 3d2fd93de2
fix: [feed] Accept more text content. Fix #5969 2020-07-21 10:10:08 +02:00
Andras Iklody 8eaa16c94d
Merge pull request #6036 from JakubOnderka/cache-freetext-feed-optim 
chg: [feed] Faster freetext feed caching
 2020-07-07 12:19:41 +02:00
Andras Iklody e6995dab67
Merge pull request #6035 from JakubOnderka/rest-search-optim 
chg: [internal] Attribute REST search optimisations and error handling
 2020-07-07 12:18:29 +02:00
Jakub Onderka cb3608b009 fix: [feed] Incorrect call in Feed::__saveEvent 2020-06-26 14:56:38 +02:00
Jakub Onderka e768d5071f chg: [feed] Faster freetext feed caching 2020-06-20 10:13:01 +02:00
Jakub Onderka f46ca1634a chg: [internal] Use tmp file fro Feed::getCache 2020-06-20 09:54:32 +02:00
Jakub Onderka b2c193c984 fix: [pull] Correct progress for pull job 2020-06-07 22:25:32 +02:00
mokaddem d022b0fca2
fix: [feeds:saveFreetext] Soft-delete Attributes when performing a 
delta-merge
 2020-06-03 16:17:16 +02:00
mokaddem dcd3664bba
Merge branch '2.4' of github.com:MISP/MISP into pr-5256 2020-05-18 14:14:51 +02:00
mokaddem b531762bf9
Merge branch '2.4' into pr-5862 2020-05-18 09:42:47 +02:00
Andras Iklody 5f9197c0a8
Merge pull request #5897 from JakubOnderka/fixed_event_freetext_feed_speedup 
chg: [feed] Modify value when checking if value exists in current event
 2020-05-14 09:55:29 +02:00
Andras Iklody db154195ec
Merge pull request #5903 from JakubOnderka/correlation-speedup 
Correlation speedup
 2020-05-14 08:33:09 +02:00
Jakub Onderka d249d0cce0 fix: [correlations] Do not check all attributes when cache is empty 2020-05-13 13:03:39 +02:00
Jakub Onderka d91a42d6b9 chg: [feed] Modify value when checking if value exists in current event 2020-05-12 11:42:55 +02:00
Jakub Onderka d92f7480b5 fix: [feed] Job progressbar fix 2020-05-11 18:17:29 +02:00
Jakub Onderka 246cf22571 fix: [feed] Optimise saving freetext feeds with a lot of attributes 2020-05-11 17:06:55 +02:00
Jakub Onderka 94b36c9259 chg: [feed] Provide more info when caching feeds about failures 2020-05-11 15:00:52 +02:00
Jakub Onderka 060cf3d459 fix: [feed] Follow redirect when fetching manifest 2020-05-05 12:49:34 +02:00
iglocska 49499f1df8
Merge branch '5834' into 2.4 2020-04-30 07:07:25 +02:00
Jakub Onderka 4dec19dedf new: [feed] Support for compressed feeds 2020-04-28 16:16:34 +02:00
mokaddem 436fd184c9
Merge branch '2.4' into JakubOnderka-redis-delete-multiple 2020-04-28 09:16:13 +02:00
iglocska 2d63f68fc1
new: [feeds] index refactor and new features 
- added the ability to select an orgc ID for CSV/freetext feeds
  - all events created from this feed will carry the selected orgc_id

- Refactored the index fully
  - using the factories
  - better warnings against the dangerous new feed each pull setting
  - event index search added
  - several settings cleaned up / made more clear

- auto reload of default feed configuration disabled, fixes #2542, fixes #5789
  - added a button / endpoint to handle that instead to allow for the deleted default feeds to stay deleted
 2020-04-17 14:17:54 +02:00
Bechkalo Evgeny 4d5f1aaf00 Fixed bugs with PostgreSQL in bruteforce and feed models 2020-03-30 16:19:43 +03:00
iglocska 30ff4b6451
chg: [security] Added setting to restrict the encoding of local feeds 
- By adding local feeds, a malicious administrator could point MISP to ingest configuration files that the apache user has access to
- This includes some more sensitive files (database.php / config.php / .gnupg data)
- Whilst this is currently not leading to an exploitable vulnerability as the current implementation wouldn't trigger on the values,
  having a setting to disable this will become much more interesting once we have a system in place for custom feed parsers
- The setting can only be enabled/disabled via the CLI

- As reported by Matthias Weckbecker
 2020-03-30 14:02:14 +02:00
Jakub Onderka 7539cbff2c
chg: [feed] Use precomputed hashes to speedup attaching correlation 2019-10-20 19:08:01 +02:00
garanews 85c28ce36e Fix some typo 
Fix some typo
 2019-10-04 13:02:59 +02:00
Pierre-Jean Grenier 8415cea5db fix: [feed] fix REST problems 2019-10-02 16:53:44 +02:00
Pierre-Jean Grenier 027aa91dd4 fix: [feed] array from NULL is not NULL 2019-10-02 16:37:52 +02:00
Andras Iklody 20311ae3e6
Merge pull request #5217 from JakubOnderka/patch-36 
new: [internal] New AppModel::logException method
 2019-09-30 21:26:09 +02:00
Jakub Onderka 28e2e71668
chg: [feed] Compute md5 value just once 2019-09-26 11:55:47 +02:00
Jakub Onderka ca3e9427bb
chg: [feed] Use new AppModel::logException method 2019-09-24 20:54:11 +02:00
Andras Iklody d656c2b318
Merge pull request #5097 from JakubOnderka/patch-18 
chg: [feed] Break loop when match is found
 2019-09-16 09:15:37 +02:00
Jakub Onderka 85754ccc83 fix: [internal] Deleting multiple Redis keys 2019-09-15 13:05:55 +02:00
iglocska 3240a6cac4
new: [CLI] Added cleanup tool to purge all events related to a feed 
- Simply run /var/www/MISP/app/Console/cake Admin purgeFeedEvents [user_id] [feed_id]
- works for CSV/Freetext feeds
 2019-09-13 10:48:43 +02:00
Jakub Onderka 609998e24c
fix: [feed] When fetching feeds, accept also text/plain in HTTP 2019-09-10 12:21:42 +02:00
Jakub Onderka 7904509fde fix: [feed] Fetching event manifest 2019-09-05 19:16:26 +02:00
Jakub Onderka e20fc9ce60
chg: [feed] Break loop when match is found 2019-09-03 18:04:47 +02:00
Jakub Onderka a45575a44d chg: [feed] Error handling for saveFreetextFeedData 2019-09-03 17:24:06 +02:00
Jakub Onderka 38e2c1cfc2 fix: [feed] Use own solution how to redirect when fetching feeds 2019-09-02 19:52:41 +02:00
Jakub Onderka d5e3be18b5 chg: [feed] Throw exception if feed filter rules is not valid JSON 2019-09-02 19:52:41 +02:00
Jakub Onderka de2b143748 chg: [feed] Better error handling for cacheFeedInitiator method 2019-09-02 19:52:41 +02:00
Jakub Onderka d7809df2d0 chg: [feed] More accurate progress logging for downloadFromFeed 2019-09-02 19:52:41 +02:00
Jakub Onderka 6c878ff31c chg: [feed] Better exception logging 2019-09-02 19:52:41 +02:00
Jakub Onderka 35d67c261e chg: [feed] New method Feed::jobProgress 2019-09-02 17:51:37 +02:00
Jakub Onderka 77fe7e10fc chg: [feed] New private method Feed::isFeedLocal 2019-09-02 17:51:37 +02:00
Jakub Onderka 49df6ccd1a chg: [feed] Use Feed::feedGetUri in Feed::getFreetextFeed 2019-09-02 17:51:37 +02:00
Jakub Onderka 18eb86ff32 chg: [feed] Feed::getNewEventUuids always return array with 'add' and 'edit' fields 2019-09-02 17:51:37 +02:00
Jakub Onderka 8973cb75f7 chg: [feed] Proper error handling for fetching feed cache 2019-09-02 17:51:37 +02:00
Jakub Onderka ba1d6ed541 chg: [feed] New method Feed::feedGetUri to deduplicate code 2019-09-02 17:51:37 +02:00
Jakub Onderka 78f113c563 chg: [feed] Proper error handling for fetching feed events 2019-09-02 17:51:37 +02:00
Jakub Onderka 3c70f816c6 chg: [feed] Deduplicate preparing filter rules 2019-09-02 17:51:37 +02:00
Jakub Onderka 5b2bb6d0fa chg: [feed] Proper error handling for fetching freetext feed 2019-09-02 17:51:37 +02:00
Jakub Onderka 3adadb100e chg: [feed] Follow redirects by internal HttpSocket functionality 2019-09-02 17:51:37 +02:00
Jakub Onderka 9ee3570379 chg: [feed] Proper error handling for fetching feed manifest 2019-09-02 17:51:37 +02:00
Jakub Onderka 0ddfbc9ccf fix: [feed] shell_exec don't throw Exception 2019-09-02 17:51:37 +02:00
Jakub Onderka 0aeacde256 chg: [feed] Name variables after explore in Feed::attachFeedCorrelations 2019-08-24 17:18:10 +02:00
Jakub Onderka d346de0118 chg: [feed] Compute Redis cache key prefix just once 2019-08-21 22:16:27 +02:00
Jakub Onderka 9846635603 chg: [feed] Fetch sources just when there is correlation 2019-08-21 22:14:30 +02:00
Jakub Onderka 0cf1baf8f3 chg: [feed] Compute composite types just once 2019-08-21 22:14:30 +02:00
Jakub Onderka ada263c79d fix: [feed] Remove unused variables 2019-08-21 22:14:30 +02:00
iglocska a26b38e2f5 fix: [feeds] Stop the reset of the attribute count after a feed pull is completed, fixes #4414 2019-04-05 10:46:32 +02:00
iglocska ab858d5f17 new: [feeds] Feed/Server cache search added 
- /feeds/searchCaches added
 2019-04-01 16:09:24 +02:00
iglocska fc34510eda new: [Feeds] New overlap tool finished 
- compare a feed against a combination of feeds/servers to find if you can cover the contents with a combination of other cached feeds
 2019-03-10 18:09:46 +01:00
iglocska 947e59aec7 new: [Feeds] Implementation of the feed coverage tool (WIP) 2019-03-09 22:19:37 +01:00