MISP
/
MISP
mirror of https://github.com/MISP/MISP
2
2
Fork 0
Code Issues Releases Wiki Activity
25,614 Commits
34 Branches
365 Tags
188 MiB
Commit Graph

162 Commits (471840ce33233a8d9db86766d007153f66727e8d)

Author SHA1 Message Date
Jakub Onderka a322217cbd chg: [sync] Try to save memory when fetching sightings 2024-04-08 09:45:33 +02:00
Jakub Onderka 2b38de942b chg: [internal] Server sync debug messages 2024-04-08 09:45:33 +02:00
Jakub Onderka 2e32d22d2c chg: [sync] Move blocklist fetching out of ServerSyncTool and reduce sightings fetched in one fetch 2024-04-06 14:05:44 +02:00
iglocska 914ae20dd4
fix: [junk] removed 2024-04-04 12:14:03 +02:00
iglocska ef39b8959e
new: [sighting sync] blocklisting added 
- block organisations' sightings from being created / pulled
- Added a new option to the restsearch of sightings too which this feature uses if available
  - if it isn't, the system will block the insertion on the beforeValidate() level

- Outcome of the JTAN hackathon on 04.04.2024 in Luxembourg
 2024-04-04 12:08:22 +02:00
iglocska 31a2507fb4
chg: [sighting restsearch] added org negations 
- the org_id filter now allows for the use of a prepended '!' character for negations
 2024-04-04 09:42:14 +02:00
Jakub Onderka 5159a72d11
Merge pull request #9660 from JakubOnderka/duplicate-sighting-uuid 
fix: [sync] Avoid problem with duplicate sightings UUID
 2024-04-03 16:32:56 +02:00
Jakub Onderka 2f72afd59f fix: [sync] Avoid problem with duplicate sightings UUID 2024-04-03 13:42:23 +02:00
Jakub Onderka 67e2478845
Merge pull request #8760 from JakubOnderka/sightings-conditions-simplify 
chg: [internal] Speedup sighting rest search
 2024-04-03 13:09:16 +02:00
Jakub Onderka 646c58095f chg: [internal] Better error handling when fetching sightings 2024-03-23 11:30:44 +01:00
Alexandre Dulaunoy 4834fa96a4
Merge branch '2.4' into develop 2024-03-13 11:18:19 +01:00
Sami Mokaddem c797865c7c
chg: [sightings:getLastSighting] Added support of sighting policy 
Fix #8660
 2024-03-12 14:41:22 +01:00
Sami Mokaddem ec769c3f27
chg: [attribute:restSearch] Improved performance of `includeDecayScore` by a factor of 5 2024-03-12 11:32:10 +01:00
iglocska 3c79ebbc06
new: [settings] added setting to (temporarily) disable the loading of sightings via the API 
- affected endpoints: restsearch and /events/view
- temporarily skips the loading of sightings

- helps alleviate absolutely massive sighting data sets from killing server performance
- temporary measure, doesn't prevent the creation of sightings / viewing of sightings via the UI
 2024-03-12 08:24:13 +01:00
iglocska c1638e0a9c
fix: [sighting sync] speculative fix for critical sync issue 
- pulls from an instance with extremely high numbers of sightings (~300M+) can lead to the pulled instance becoming unusable
- This fix addresses multiple issues:
  - The use of last:0 as a sighting pull filter parameter lead to a search using an unindexed field
  - Internally searching for sighting IDs across 500 events in one shot can lead to massive data-sets
  - Internally searching for sighting IDs by Event.uuid on a joined table is extremely slow compared to searching on the sighting table alone
 2024-02-16 14:21:10 +01:00
Jakub Onderka d2911274b5 fix: [internal] Fix error code when fetching sightings 2024-01-13 16:41:20 +01:00
Sami Mokaddem d758516042
fix: [Attribute:editPostProcessing] Fixed sighting capture 2023-11-27 11:23:30 +01:00
Sami Mokaddem 4f524b9413
fix: [sighting:anonymisation] Anonymize sightings if and only if anonymization is set in the settings 2023-11-20 08:57:29 +01:00
Sami Mokaddem 4215c0f149
chg: [worflow-trigger:sighting_after_save] Change name to after-save and make it misp_core_format compatible 2023-10-25 15:32:39 +02:00
Sami Mokaddem ceb43a345c
Merge branch 'develop' of github.com:MISP/MISP into vincenzocaputo_add-sighting-publish-trigger 2023-10-25 14:32:01 +02:00
Sami Mokaddem 90cb068a0e
fix: [sighting:attachOrgToSightings] Stopped double unpacking 2023-10-18 16:14:52 +02:00
vincenzocaputo ea32fef68c Add sighting publish workflow trigger 2023-07-23 17:48:59 +02:00
iglocska 9a153f9a3d
fix: [sightings] don't be case insensitive on code side 2023-03-17 15:34:33 +01:00
iglocska cdf270606c
fix: [Sighting] rework of the loading via restsearch 
- the chunking and limiting by attribute IDs in the sighting restsearch caused long delays due to a select with two AND-ed in value lists causing the query optimiser to constantly run statistics on the table
- moved the filtering by attribute to PHP side via a loop, it should boost the performance of the function - and with it the sync considerably
 2023-03-02 09:49:44 +01:00
Jakub Onderka 8f3f7bc866 chg: [internal] Speedup sighting rest search 2022-12-22 12:57:54 +01:00
Jakub Onderka be32cda693 fix: [sync] Pulling sighting new way 2022-11-14 16:10:06 +01:00
Jakub Onderka a33c14f9a8 fix: [sync] Remove events without sightings from pull 2022-11-03 12:04:41 +01:00
Jakub Onderka bd0dde5e37 chg: [API] Throw exception if invalid ID provided 2022-11-01 10:56:31 +01:00
Jakub Onderka 4fe5a73386 chg: [internal] Use subquery to sighting fetching 2022-11-01 10:56:14 +01:00
Jakub Onderka 1bc02930cf fix: [sighting] Return just requested sighting 2022-11-01 10:56:14 +01:00
Jakub Onderka 338de3178c chg: [sync] Use new sighting pull for new MISP instances 2022-11-01 10:56:14 +01:00
Jakub Onderka 35b00b6c84 chg: [sighting] Include organisation in rest response 2022-11-01 10:56:14 +01:00
Jakub Onderka 77fd20a98f chg: [sightings] Optimised fetching 2022-11-01 10:56:14 +01:00
Jakub Onderka 5a1a8aace9 chg: [api] Allow to include uuids to sighting 2022-11-01 10:56:14 +01:00
Jakub Onderka 7a29e18d23 chg: [sync] New way how to pull sightings 2022-11-01 10:56:14 +01:00
Jakub Onderka 2aa4ee3097 chg: [internal] Optimise sighting rest search 2022-11-01 10:56:14 +01:00
Jakub Onderka ecb88f1c93 new: [test] Check sighting rest search ACL vol. 2 2022-11-01 10:41:52 +01:00
Jakub Onderka 6b634c7520 fix: [widgets] Fix some widgets 2022-10-20 14:54:36 +02:00
Jakub Onderka 2f7c671adb new: [internal] Simplify checking if connection is MySQL/MariaDB 2022-05-14 10:17:06 +02:00
Jakub Onderka 90cd99685f chg: [sync] Simplify code for sighting pushing 2022-03-07 17:45:06 +01:00
Jakub Onderka bd9cecbc2c chg: [internal] Use ProcessTool in Sighting 2021-11-22 09:57:39 +01:00
Jakub Onderka e1895cba81 chg: [internal] Optimise sightings 2021-10-17 16:17:17 +02:00
Jakub Onderka d9e89955bc chg: [internal] Simplify fetching Kafka topic 2021-10-11 09:41:46 +02:00
Jakub Onderka 4973c7480d chg: [sync] Examine less events for sightings pulling 2021-09-01 11:52:25 +02:00
Jakub Onderka 01e588324c fix: [sync] Fix pulling sightings 2021-08-23 13:52:35 +02:00
Jakub Onderka faf5054583
Merge pull request #7649 from JakubOnderka/pull-sightings 
chg: [sync] Pull just necessary data when pulling sightings
 2021-08-20 09:26:09 +02:00
mokaddem 3770ae6798
new: [sighting:add] Ability to provide filtering parameters when adding sightings for specific values 
Fix #7669
 2021-08-18 14:36:34 +02:00
Jakub Onderka f7d05e8e24 chg: [sync] Filter out events that do not exist locally when pulling sightings 2021-08-12 10:21:08 +02:00
Jakub Onderka 0174336156 chg: [sync] Pull just necessary data when pulling sightings 2021-08-11 18:42:22 +02:00
Jakub Onderka 683e52702d new: [sync] When saving sightings, push just new sightings 2021-03-14 09:28:15 +01:00