Commit Graph

4146 Commits (791dc9deabd2d7fdfd7c371331eb557d14782372)

Author SHA1 Message Date
Jakub Onderka 813a475f4a chg: [internal] Allow to have empty nationality 2020-10-16 09:21:26 +02:00
Jakub Onderka a38afcea2a chg: [UI] Update country names 2020-10-16 09:21:26 +02:00
Alexandre Dulaunoy fdfbcf784e
Merge pull request #6447 from JakubOnderka/as-normalization
chg: [internal] Normalize AS type to asplain notation
2020-10-16 07:03:33 +02:00
Jakub Onderka 28af05528f chg: [internal] Normalize AS type to asplain notation 2020-10-15 22:46:12 +02:00
Jakub Onderka 870021cdc0
Merge pull request #6289 from JakubOnderka/save-sighting-optim
chg: [internal] Sighting saving optimisation
2020-10-15 17:30:05 +02:00
Alexandre Dulaunoy 40e8d619cc
chg: [attribute] cpe was already present but not specified in any categories 2020-10-15 14:42:10 +02:00
Alexandre Dulaunoy cf05b94cdc
new: [attribute type] cpe Common Platform Enumeration attribute type added 2020-10-15 14:39:26 +02:00
mokaddem b8044d9c07
new: [eventReports] Attributes suggestion replacement + UI - Draft 2020-10-15 11:56:21 +02:00
Jakub Onderka ba057863c1 fix: [eventReport] Include just tags that belongs to requested event or its parent, not to other child 2020-10-14 20:55:42 +02:00
Jakub Onderka af578e3515 chg: [eventReport] Put back attribute galaxies 2020-10-14 20:55:42 +02:00
Jakub Onderka 3e984071f9 fix: [eventReport] Properly validate UUID 2020-10-14 20:55:42 +02:00
Jakub Onderka 3487302f13 fix: [eventReport] Optimize loading by UUID 2020-10-14 20:55:42 +02:00
Jakub Onderka f801bf8940 fix: [eventReport] Template loading condition 2020-10-14 20:55:42 +02:00
Jakub Onderka 1ec7a2a391 chg: [eventReport] Do not fetch event reports when it is not necessary 2020-10-14 20:55:42 +02:00
Jakub Onderka c1fccbb07e chg: [eventReport] Do not fetch attribute tags again, they are included in object 2020-10-14 20:55:42 +02:00
Jakub Onderka 9d7705fbdb chg: [eventReport] Do not fetch unnecessary event and attribute galaxies 2020-10-14 20:55:42 +02:00
Jakub Onderka 7982171247 chg: [eventReport] Do not create separate array, merge is expensive 2020-10-14 20:55:42 +02:00
Jakub Onderka f090fcceb4 chg: [eventReport] Do not attach sharing groups 2020-10-14 20:55:42 +02:00
Jakub Onderka dc437ab8f2 chg: [eventReport] Fetch object templates just when event contains objects 2020-10-14 20:55:42 +02:00
Jakub Onderka ee0dd87fb5 chg: [eventReport] Fetch parental event just when it is necessary 2020-10-14 20:55:42 +02:00
Jakub Onderka 8910a13d75 chg: [eventReport] Do not fetch sightings for events 2020-10-14 20:55:42 +02:00
Jakub Onderka 47d3293880
Merge pull request #6440 from JakubOnderka/event-ui-vol3
Event ui vol3
2020-10-14 19:45:20 +02:00
Jakub Onderka 2b24260555 fix: [internal] Missing variable 2020-10-14 18:54:59 +02:00
Jakub Onderka 0eaf0e857a
Merge pull request #6436 from JakubOnderka/attribute-validation
Attribute validation
2020-10-14 10:30:07 +02:00
Alexandre Dulaunoy f3ab1f28c7
Merge pull request #6429 from cudeso/2.4
Send a message to ZMQ when an event from a connected server is added or edited
2020-10-14 10:03:01 +02:00
Jakub Onderka e96b1afafc chg: [validation] Provide better invalid messages for ip-dst|port, ip-src|port and hostname|port 2020-10-14 08:19:49 +02:00
Jakub Onderka 3606d368d1 chg: [validation] Simplify composite validation 2020-10-14 08:19:49 +02:00
Jakub Onderka 0d055b12d7 fix: [validation] Correct validation for iban, bic, btc, dash and xmr attributes 2020-10-14 08:19:49 +02:00
Jakub Onderka cd591e684b fix: [validation] Normalize mac-address and mac-eui-64 to lowercase 2020-10-14 08:19:49 +02:00
Jakub Onderka 335814af88 fix: [validation] Do not accept floats where should be just integers 2020-10-14 08:19:49 +02:00
Jakub Onderka d9abb37c8f
Merge pull request #6438 from JakubOnderka/hostname-port-correlation
fix: [correlations] Disable correlation for port part in hostname|port
2020-10-13 23:29:17 +02:00
Jakub Onderka 5f98426f7d fix: [correlations] Disable correlation for port part in hostname|port type 2020-10-13 23:08:59 +02:00
Alexandre Dulaunoy 550f09f628
new: [attribute] telfhash attribute type added - fix #6435 2020-10-13 22:29:43 +02:00
Jakub Onderka 72d0b873c8 chg: [UI] Optimise loading contributors orgs 2020-10-13 16:08:22 +02:00
Jakub Onderka e2c65d2a51 chg: [validation] Provide more precise and faster attribute validation 2020-10-13 10:51:18 +02:00
Koen Van Impe d4e77bf7c9 Send message to ZMQ when there is event add/edit coming from a connected server 2020-10-13 09:23:07 +02:00
Jakub Onderka d1a6a94f78 chg: [internal] URL is already defang in ComplexTypeTool 2020-10-12 08:24:34 +02:00
Jakub Onderka ace025e085 fix: [internal] Properly convert `hostname|port` when delimiter is `:` 2020-10-12 08:23:49 +02:00
Jakub Onderka 8c6ad2b414 fix: [validation] Convert vulnerability attribute to uppercase 2020-10-12 08:23:49 +02:00
Jakub Onderka 46ea861576 chg: [UI] Validate object when revising 2020-10-12 08:23:46 +02:00
Jakub Onderka a8dcd9aec4 fix: [validation] Float validation 2020-10-12 08:22:56 +02:00
Jakub Onderka 2a9feb7cd3
Merge pull request #6417 from JakubOnderka/fetch-event-optimisation
Fetch event optimisation
2020-10-12 08:17:39 +02:00
Jakub Onderka 2f4d38b2f1 chg: [internal] Cleanup and simplify ShadowAttribute model code 2020-10-11 15:20:20 +02:00
Jakub Onderka 0b3f5a5eeb chg: [freetext] Process just big number of attributes in background 2020-10-11 12:35:59 +02:00
Jakub Onderka d57b559bd3 fix: [UI] Remove not used organisation landing page 2020-10-11 10:28:47 +02:00
Jakub Onderka 150ccfcd73 chg: [internal] Merge EventReports for extended view 2020-10-10 14:27:07 +02:00
Jakub Onderka c46db527f6 chg: [internal] Optimise event fetching 2020-10-10 14:23:55 +02:00
Jakub Onderka d5d8906670
Merge pull request #6395 from JakubOnderka/module-fixes
Module fixes
2020-10-10 09:02:35 +02:00
Jakub Onderka 5c16ceb9d4
Merge pull request #6300 from JakubOnderka/validate-gpg-key
Validate gpg key
2020-10-09 23:14:33 +02:00
Jakub Onderka cef7021f9d chg: [internal] More checks in Module model 2020-10-09 23:10:42 +02:00
Jakub Onderka 380fba5405 new: [GPG] Validate fetched GPG key 2020-10-09 16:58:59 +02:00
mokaddem b628e15eb9
Merge remote-tracking branch 'origin/2.4' into feature-event-report 2020-10-09 14:54:47 +02:00
mokaddem b6d5136df6
fix: [eventReport] Do not try to fetch report after successful hard
deletion
2020-10-09 11:36:34 +02:00
mokaddem daf355f564
fix: [eventReport] Improved variable name and do not crash if event is
not extending another one
2020-10-08 16:59:45 +02:00
mokaddem d891143b8c
new: [evenReport] Support of extended event 2020-10-08 09:05:44 +02:00
Jakub Onderka 34a1671e8a
Merge pull request #6397 from JakubOnderka/zip-error-messages
chg: [internal] Better error messages for unzipping feed file
2020-10-07 11:35:16 +02:00
mokaddem aafdeab9d6
chg: [eventReport:markdownEditor] Fetch MISP elements with a different
request
2020-10-07 11:13:30 +02:00
Jakub Onderka 0e7d3ddc5d chg: [internal] Better error messages for unzipping feed file 2020-10-07 09:07:11 +02:00
Jakub Onderka cf070bd7dd fix: [internal] Array to string conversion when constructing request 2020-10-06 17:27:53 +02:00
Jakub Onderka b88bffd5b5 chg: [internal] Sighting saving optimisation 2020-10-06 16:48:47 +02:00
Jakub Onderka d12173d09b
Merge pull request #6385 from JakubOnderka/sighting-fetch-optim
Sighting fetch optim
2020-10-06 16:16:39 +02:00
mokaddem 28a05576a4
chg: [appmodel] Set default value for warninglist_entry_count 2020-10-06 10:53:29 +02:00
Jakub Onderka 0225afec40 chg: [internal] Validate sighting UUID 2020-10-05 14:44:23 +02:00
Jakub Onderka 2a30620368 chg: [internal] Remove unused code 2020-10-05 14:44:23 +02:00
Jakub Onderka 7fb7e8ae39 chg: [internal] Optimise sightings fetching 2020-10-05 14:44:23 +02:00
mokaddem c50ff4b1bd
new: [eventReport:markdownEditor] Support to reference object attribute 2020-10-05 14:07:51 +02:00
mokaddem 1c4aea010b
fix: [event:push] Setup requests headers before sending request 2020-10-05 09:54:10 +02:00
mokaddem 485a1afff8
fix: [server:push] Allow pushing events only having event reports 2020-10-05 09:50:56 +02:00
Jakub Onderka 503206a13d chg: [internal] Initialize Sighting class just when necessary 2020-10-04 13:01:30 +02:00
Jakub Onderka c2f6908002 chg: [optimisation] Load MISP version and commit just once 2020-10-03 20:02:55 +02:00
Jakub Onderka 00c318b451 chg: [internal] Optimise fetching sightings for object 2020-10-03 19:50:02 +02:00
Jakub Onderka 9a5d906e25 fix: [warnings] enforceWarninglist works again 2020-10-03 15:06:32 +02:00
Jakub Onderka 2dea93375e fix: [warnings] Cache deletion 2020-10-03 12:49:33 +02:00
Jakub Onderka 83bc08201e fix: [warninglists] Include warning for merged events 2020-10-03 10:17:06 +02:00
Jakub Onderka 36a9b15211 chg: [warninglist] Use faster method for fetching data from Redis 2020-10-03 10:17:06 +02:00
Jakub Onderka 93b16280ca chg: [warninglist] Hash key in binary and store just for one hour 2020-10-03 10:17:06 +02:00
Jakub Onderka 8b5868d8f1 fix: [warnings] Attach warnings to feed and server event preview 2020-10-03 10:17:05 +02:00
Jakub Onderka 41ec29e12a chg: [internal] Fix for exact string match 2020-10-03 10:17:05 +02:00
Jakub Onderka d6c83993c1 chg: [internal] Normalize CIDR and hostname warninglists 2020-10-03 10:17:05 +02:00
Jakub Onderka 4a339773b5 fix: [internal] IPv6 CIDR warninglist 2020-10-03 10:17:05 +02:00
Jakub Onderka a77ee50845 chg: [internal] Simplified Warninglist::__checkValue 2020-10-03 10:17:05 +02:00
Jakub Onderka b031ad3a07 chg: [internal] Change method name to show that it just filter one attr 2020-10-03 10:17:05 +02:00
Jakub Onderka e54482c77e chg: [internal] Save memory when storing warninglist to cache 2020-10-03 10:17:05 +02:00
Jakub Onderka 6eb66835e4 chg: [internal] Start IPv4 checking from zero 2020-10-03 10:17:05 +02:00
Jakub Onderka c79b282134 chg: [internal] Warninglist code cleanup 2020-10-03 10:17:05 +02:00
Jakub Onderka aeef4e2946 chg: [internal] Fix event warnings without redis cache 2020-10-03 10:17:05 +02:00
Jakub Onderka 1882b60ddf new: [internal] Allow to set warning checking for all attributes, not just IDS 2020-10-03 10:17:05 +02:00
Jakub Onderka 7173663c19 chg: [internal] Move getting missing tlds list to model 2020-10-03 10:17:05 +02:00
Jakub Onderka ce3581c291 chg: [internal] Refactoring warninglist loading and saving 2020-10-03 10:17:05 +02:00
Jakub Onderka 648bb4b1db chg: [internal] Simplified Event::__prepare... methods 2020-10-03 10:17:05 +02:00
Jakub Onderka b8a9b5e87e chg: [internal] Attach warnings to proposals 2020-10-03 10:17:05 +02:00
Jakub Onderka 66b270b42e chg: [internal] Rename Warninglist::simpleCheckForWarning to checkForWarning 2020-10-03 10:17:05 +02:00
Jakub Onderka 40ab937181 new: [warninglist] Allow to check if IP in CIDR is part of another CIDR 2020-10-03 10:17:05 +02:00
Jakub Onderka bbb08e4074 new: [warninglist] Cache warninglist results 2020-10-03 10:17:05 +02:00
mokaddem 5cb58a500e
fix: [event:updateEventReportBeforeSync] Init httpSocket 2020-10-02 16:53:50 +02:00
iglocska f7dc3bcbdd
Merge branch '2.4' of github.com:MISP/MISP into fetcher_debug 2020-10-02 16:14:26 +02:00
iglocska 4dc8cf2820
fix: [tag filters] fixed ridiculously long lists for tag filters 2020-10-02 16:13:51 +02:00
mokaddem f4c846b5a1
chg: [event:updateEventReportBeforeSync] Make sure remote instance
supports event report
2020-10-02 16:13:47 +02:00
mokaddem cbf228f332
new: [eventReport:markdownEditor] Added hints for tags 2020-10-02 10:48:52 +02:00
Andras Iklody 1523fc7ff9
Merge pull request #6370 from MISP/fix-update-no-prio
Fix update no prio
2020-10-01 15:36:29 +02:00
Andras Iklody 2c2bf5f847
Update Server.php 2020-10-01 15:35:35 +02:00
Andras Iklody 22167a1e59
chg: [cleanup] removed duplicate empty queue declaration 2020-10-01 15:34:50 +02:00
Alexandre Dulaunoy 1ffddca1b2
new: [attribute-type] filename-pattern to describe a filename base on a pattern
Fix #403

There is no specific validation on the field. This allows us to have a clear
separation between filename and filename-pattern as many users were
using filename for regexp. This also helps the creation of object
template which requires a filename pattern.
2020-10-01 15:04:13 +02:00
Andras Iklody f12f86b503
Merge pull request #6373 from RichieB2B/issue-6368
Allow OS user to be set for upgrades
2020-10-01 14:40:21 +02:00
Andras Iklody 09ff03d649
Merge pull request #6375 from RichieB2B/issue-6342
Do not limit TLD to 5 characters
2020-10-01 14:39:38 +02:00
Jakub Onderka b6033a36a6
Merge pull request #6374 from JakubOnderka/test-fix
Test fix
2020-10-01 14:25:31 +02:00
Richard van den Berg 502806ff50 fix: [server] Do not limit TLD to 5 characters. Fix #6342 2020-10-01 13:58:38 +02:00
Jakub Onderka 0d62475834 fix: [internal] Variable should be defined all the time 2020-10-01 13:54:55 +02:00
Jakub Onderka 1da6cd8cd4 fix: [proposal] No return when org is not defined 2020-10-01 13:53:35 +02:00
Richard van den Berg bb8981353b chg: [Shell] Add MISP.osuser for updates. Fix #6368 2020-10-01 13:47:51 +02:00
mokaddem fc9c77b917
Merge branch '2.4' of github.com:MISP/MISP into feature-event-report 2020-10-01 13:40:06 +02:00
Andras Iklody 90dd9db25a
Merge pull request #6372 from RichieB2B/ncsc-nl/pullAll
Document "cake Server pullAll"
2020-10-01 12:51:59 +02:00
Richard van den Berg b9ff5fc7db fix: [doc] Document "cake Server pullAll" 2020-10-01 12:22:31 +02:00
Andras Iklody c4608a94eb
Merge pull request #6362 from imidoriya/2.4
Fix date filter on to / from #6239
2020-10-01 11:28:00 +02:00
mokaddem 4ef7442a9a
chg: [eventReport] Replace ID by UUID when referencing MISP Elements 2020-10-01 11:08:30 +02:00
mokaddem 02725982cc
Merge branch '2.4' of github.com:MISP/MISP into feature-event-report 2020-10-01 10:38:28 +02:00
mokaddem c4bb8e2ecb
fix: [server:workerDiagostics] Default queue status to false 2020-10-01 09:46:17 +02:00
mokaddem 2fdcede6b6
chg: [appModel] Removed prio worker from the list of available workers
to perform an update
2020-10-01 09:28:02 +02:00
iglocska 065eba4c6b
Revert "fix: [tag] Show correct count of tag attributes and events"
This reverts commit e644f4ea4c.
2020-09-30 17:47:40 +02:00
iglocska 59d08f2e54
Revert "chg: [internal] Optimise fetching sightings for object"
This reverts commit e442a394cd.
2020-09-30 17:47:29 +02:00
iglocska 6d21f1696a
fix: [hacky] readded org field to shadow attributes - just blank it out for old instances where the update failed to remove it a few years ago 2020-09-30 09:54:00 +02:00
iglocska 14400a932d
fix: [sync] better logging of error messages and handle the user ID not being set by background processes 2020-09-30 09:43:12 +02:00
deku 22cbd98aa2 Filter Event Date - convert timestamp to datetime
PyMisp sends the to / from as a timestamp.  MISP needs to convert a timestamp when comparing.
2020-09-29 15:05:49 -04:00
iglocska 6b0252d7e4
new: [user agent] string changed for MISP -> MISP synchronisation 2020-09-29 14:56:19 +02:00
mokaddem 14178a1d8c
chg: [eventReport] Added support of all galaxy matrixes 2020-09-29 12:16:50 +02:00
mokaddem 7d09f4d231
chg: [eventReport] Added rearrange function 2020-09-29 10:35:46 +02:00
mokaddem 0448f52e13
fix: [eventReport] Deleted report can be restored by non-admin users 2020-09-29 10:35:00 +02:00
Jakub Onderka 95dfd75af7 chg: [correlations] Really limit number of correlations, not number of attributes 2020-09-28 19:52:19 +02:00
Jakub Onderka d1eec44d8d chg: [internal] Initialize Job class just once 2020-09-28 19:40:32 +02:00
Jakub Onderka 7f29b6b089 chg: [internal] Delete Redis key in pipeline 2020-09-28 18:33:56 +02:00
Jakub Onderka 552315555c chg: [feed] Faster feed list fetching 2020-09-28 17:19:25 +02:00
Jakub Onderka c87616e6ec fix: [correlations] Properly delete feeds caches 2020-09-28 17:00:00 +02:00
mokaddem 6aab30c871
chg: [appmodel] Make sure to trigger the event report db update 2020-09-28 15:53:12 +02:00
Jakub Onderka a11e27bedc chg: [correlations] Fetch just server of feed that has data in Redis 2020-09-28 15:49:54 +02:00
Jakub Onderka 0c5be866fc chg: [correlations] Refactor feed cached correlations 2020-09-28 15:02:08 +02:00
mokaddem 25a58578ba
fix: [decaying] 2-tag base_score ratio. Fix #6352 2020-09-28 10:30:02 +02:00
mokaddem d0c7c44a2f
fix: [attribute] Typo in regex. Fix #6354 2020-09-28 09:34:54 +02:00
rmkml 1f08e9a27a fix #6336 vhash 2020-09-27 13:45:23 +02:00
Jakub Onderka 03b5af1f78
Merge pull request #6344 from JakubOnderka/misp-file-import
Misp file import error message
2020-09-26 19:34:55 +02:00
Jakub Onderka 731aeaaf38 chg: [internal] Initialize Log model just once 2020-09-26 13:27:59 +02:00
Jakub Onderka 804525a20e chg: [internal] Move addMISPExportFile from controller to model 2020-09-26 13:02:19 +02:00
Jakub Onderka 77776d1ff6
Merge pull request #6338 from JakubOnderka/password-reset-absolute
fix: [UI] Custom password reset link is absolute
2020-09-23 17:58:16 +02:00
Jakub Onderka 21e00a996b fix: [internal] Fix tests for missing ACL 2020-09-23 17:28:01 +02:00
Jakub Onderka 2b59792215 fix: [internal] Typo in perm name 2020-09-22 18:13:28 +02:00
Jakub Onderka 5163c98111
Merge pull request #6330 from JakubOnderka/event-creator
chg: [UI] Show event creator for users from event org
2020-09-22 17:59:29 +02:00
Jakub Onderka ff83fd00a0 fix: [internal] Do not fetch more info than necessary 2020-09-22 17:10:59 +02:00
Jakub Onderka aa613d0369 chg: [UI] Show event creator for users within event org 2020-09-22 17:10:58 +02:00
Richard van den Berg 8cfc14c3e9 Remove extraneous spaces from E-mail subject 2020-09-22 12:51:35 +02:00
mokaddem 4408a17dff
Merge remote-tracking branch 'origin/2.4' into feature-event-report 2020-09-22 10:15:22 +02:00
mokaddem 98eb9c3ceb
chg: [eventreport] Make sure the format is correct when capturing 2020-09-22 10:05:22 +02:00