Commit Graph

164 Commits (791dc9deabd2d7fdfd7c371331eb557d14782372)

Author SHA1 Message Date
Jakub Onderka b749610c56 chg: [feed] Check also URL without protocol 2020-10-29 18:55:17 +01:00
iglocska 42aaea20f8
fix: [server] caching notice fixed 2020-10-16 14:41:45 +02:00
Jakub Onderka 0e7d3ddc5d chg: [internal] Better error messages for unzipping feed file 2020-10-07 09:07:11 +02:00
Jakub Onderka c2f6908002 chg: [optimisation] Load MISP version and commit just once 2020-10-03 20:02:55 +02:00
Jakub Onderka 95dfd75af7 chg: [correlations] Really limit number of correlations, not number of attributes 2020-09-28 19:52:19 +02:00
Jakub Onderka d1eec44d8d chg: [internal] Initialize Job class just once 2020-09-28 19:40:32 +02:00
Jakub Onderka 7f29b6b089 chg: [internal] Delete Redis key in pipeline 2020-09-28 18:33:56 +02:00
Jakub Onderka 552315555c chg: [feed] Faster feed list fetching 2020-09-28 17:19:25 +02:00
Jakub Onderka c87616e6ec fix: [correlations] Properly delete feeds caches 2020-09-28 17:00:00 +02:00
Jakub Onderka a11e27bedc chg: [correlations] Fetch just server of feed that has data in Redis 2020-09-28 15:49:54 +02:00
Jakub Onderka 0c5be866fc chg: [correlations] Refactor feed cached correlations 2020-09-28 15:02:08 +02:00
Jakub Onderka 2deb9d3c8b
Merge pull request #6156 from JakubOnderka/feed-httpsocket-optional
fix: [feed] Make HttpSocket instance optional for local feeds
2020-08-03 10:51:03 +02:00
iglocska b5dffc2023
fix: [internal] older PHP still not happy with the return from a generator 2020-07-30 17:32:35 +02:00
iglocska dfc743aab2
fix: [internal] make ancient PHP versions happy 2020-07-30 17:24:19 +02:00
Jakub Onderka d9f21f75f5 chg: [feed] Better exception messages for invalid JSON 2020-07-28 18:33:05 +02:00
Jakub Onderka 6c411203c0 fix: [feed] Make HttpSocket instance optional for local feeds 2020-07-28 17:59:55 +02:00
mokaddem afbf870e16
fix: [feed:importFreeText] Make sure to update the timestamp when soft-deleting after delta-merge
Fix #6013
2020-07-21 12:13:59 +02:00
mokaddem 3d2fd93de2
fix: [feed] Accept more text content. Fix #5969 2020-07-21 10:10:08 +02:00
Andras Iklody 8eaa16c94d
Merge pull request #6036 from JakubOnderka/cache-freetext-feed-optim
chg: [feed] Faster freetext feed caching
2020-07-07 12:19:41 +02:00
Andras Iklody e6995dab67
Merge pull request #6035 from JakubOnderka/rest-search-optim
chg: [internal] Attribute REST search optimisations and error handling
2020-07-07 12:18:29 +02:00
Jakub Onderka cb3608b009 fix: [feed] Incorrect call in Feed::__saveEvent 2020-06-26 14:56:38 +02:00
Jakub Onderka e768d5071f chg: [feed] Faster freetext feed caching 2020-06-20 10:13:01 +02:00
Jakub Onderka f46ca1634a chg: [internal] Use tmp file fro Feed::getCache 2020-06-20 09:54:32 +02:00
Jakub Onderka b2c193c984 fix: [pull] Correct progress for pull job 2020-06-07 22:25:32 +02:00
mokaddem d022b0fca2
fix: [feeds:saveFreetext] Soft-delete Attributes when performing a
delta-merge
2020-06-03 16:17:16 +02:00
mokaddem dcd3664bba
Merge branch '2.4' of github.com:MISP/MISP into pr-5256 2020-05-18 14:14:51 +02:00
mokaddem b531762bf9
Merge branch '2.4' into pr-5862 2020-05-18 09:42:47 +02:00
Andras Iklody 5f9197c0a8
Merge pull request #5897 from JakubOnderka/fixed_event_freetext_feed_speedup
chg: [feed] Modify value when checking if value exists in current event
2020-05-14 09:55:29 +02:00
Andras Iklody db154195ec
Merge pull request #5903 from JakubOnderka/correlation-speedup
Correlation speedup
2020-05-14 08:33:09 +02:00
Jakub Onderka d249d0cce0 fix: [correlations] Do not check all attributes when cache is empty 2020-05-13 13:03:39 +02:00
Jakub Onderka d91a42d6b9 chg: [feed] Modify value when checking if value exists in current event 2020-05-12 11:42:55 +02:00
Jakub Onderka d92f7480b5 fix: [feed] Job progressbar fix 2020-05-11 18:17:29 +02:00
Jakub Onderka 246cf22571 fix: [feed] Optimise saving freetext feeds with a lot of attributes 2020-05-11 17:06:55 +02:00
Jakub Onderka 94b36c9259 chg: [feed] Provide more info when caching feeds about failures 2020-05-11 15:00:52 +02:00
Jakub Onderka 060cf3d459 fix: [feed] Follow redirect when fetching manifest 2020-05-05 12:49:34 +02:00
iglocska 49499f1df8
Merge branch '5834' into 2.4 2020-04-30 07:07:25 +02:00
Jakub Onderka 4dec19dedf new: [feed] Support for compressed feeds 2020-04-28 16:16:34 +02:00
mokaddem 436fd184c9
Merge branch '2.4' into JakubOnderka-redis-delete-multiple 2020-04-28 09:16:13 +02:00
iglocska 2d63f68fc1
new: [feeds] index refactor and new features
- added the ability to select an orgc ID for CSV/freetext feeds
  - all events created from this feed will carry the selected orgc_id

- Refactored the index fully
  - using the factories
  - better warnings against the dangerous new feed each pull setting
  - event index search added
  - several settings cleaned up / made more clear

- auto reload of default feed configuration disabled, fixes #2542, fixes #5789
  - added a button / endpoint to handle that instead to allow for the deleted default feeds to stay deleted
2020-04-17 14:17:54 +02:00
Bechkalo Evgeny 4d5f1aaf00 Fixed bugs with PostgreSQL in bruteforce and feed models 2020-03-30 16:19:43 +03:00
iglocska 30ff4b6451
chg: [security] Added setting to restrict the encoding of local feeds
- By adding local feeds, a malicious administrator could point MISP to ingest configuration files that the apache user has access to
- This includes some more sensitive files (database.php / config.php / .gnupg data)
- Whilst this is currently not leading to an exploitable vulnerability as the current implementation wouldn't trigger on the values,
  having a setting to disable this will become much more interesting once we have a system in place for custom feed parsers
- The setting can only be enabled/disabled via the CLI

- As reported by Matthias Weckbecker
2020-03-30 14:02:14 +02:00
Jakub Onderka 7539cbff2c
chg: [feed] Use precomputed hashes to speedup attaching correlation 2019-10-20 19:08:01 +02:00
garanews 85c28ce36e Fix some typo
Fix some typo
2019-10-04 13:02:59 +02:00
Pierre-Jean Grenier 8415cea5db fix: [feed] fix REST problems 2019-10-02 16:53:44 +02:00
Pierre-Jean Grenier 027aa91dd4 fix: [feed] array from NULL is not NULL 2019-10-02 16:37:52 +02:00
Andras Iklody 20311ae3e6
Merge pull request #5217 from JakubOnderka/patch-36
new: [internal] New AppModel::logException method
2019-09-30 21:26:09 +02:00
Jakub Onderka 28e2e71668
chg: [feed] Compute md5 value just once 2019-09-26 11:55:47 +02:00
Jakub Onderka ca3e9427bb
chg: [feed] Use new AppModel::logException method 2019-09-24 20:54:11 +02:00
Andras Iklody d656c2b318
Merge pull request #5097 from JakubOnderka/patch-18
chg: [feed] Break loop when match is found
2019-09-16 09:15:37 +02:00
Jakub Onderka 85754ccc83 fix: [internal] Deleting multiple Redis keys 2019-09-15 13:05:55 +02:00