Merge branch 'develop' of github.com:MISP/MISP into develop

pull/9296/head
Sami Mokaddem 2023-09-07 10:00:49 +02:00
commit ed61d9ecb4
No known key found for this signature in database
GPG Key ID: 164C473F627A06FA
32 changed files with 90 additions and 56 deletions

View File

@ -9,18 +9,18 @@ body:
Please read the [FAQ](https://www.circl.lu/doc/misp/faq/) before opening an issue.
If you would like to report a bug, please fill the template bellow:
- type: textarea
id: expected-behavior
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: actual-behavior
attributes:
label: Actual behavior
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: Expected behavior
validations:
required: true
- type: textarea
id: steps-to-reproduce
attributes:

View File

@ -2,7 +2,7 @@
MISP project is a large free software project composed of multiple sub-projects which are contributed by different contributors who are generally active users of the MISP project. MISP project fully supports the [Contributor Covenant Code of Conduct](https://github.com/MISP/MISP/blob/2.4/code_of_conduct.md) to foster an open and dynamic environment for contributing and the exchange in the threat intelligence and information exchange field.
The [MISP roadmap](/roadmap.md) is mostly based on the user communities (e.g. private communities, CSIRTs communities, security researchers, ISACs - Information Sharing and Analysis Center, security providers, governmental or military organisations) relying on MISP to perform their duties of information sharing and analysis.
The [MISP roadmap](/ROADMAP.md) is mostly based on the user communities (e.g. private communities, CSIRTs communities, security researchers, ISACs - Information Sharing and Analysis Center, security providers, governmental or military organisations) relying on MISP to perform their duties of information sharing and analysis.
Participating in the MISP project is easy and everyone can contribute following their ability.
Get familiar with [how we use GitHub at MISP Project](/GITWORKFLOW.md), then read on for details on some ways you can contribute:

View File

@ -699,8 +699,9 @@ kaliOnTheR0ckz () {
overlay=$(df -kh |grep overlay; echo $?) # if 1 overlay NOT present
if [[ ${totalRoot} -lt 3059034 ]]; then
echo "(If?) You run Kali in LiveCD mode and we need more overlay disk space."
echo "This is defined by the total memory, you have: ${totalMem}kB which is not enough."
echo "(If?) You run Kali in LiveCD mode, you need more overlay disk space."
echo "This is defined by the total memory setting in you VM config."
echo "You currently have: ${totalMem}kB which is not enough."
echo "6-8Gb should be fine. (need >3Gb overlayFS)"
exit 1
fi
@ -3602,11 +3603,12 @@ x86_64-ubuntu-bionic
x86_64-ubuntu-focal
x86_64-ubuntu-hirsute
x86_64-ubuntu-jammy
x86_64-kali-2021.4
x86_64-kali-2022.1
x86_64-kali-2022.2
x86_64-kali-2022.3
x86_64-kali-2022.4
x86_64-kali-2023.1
x86_64-kali-2023.2
armv6l-raspbian-stretch
armv7l-raspbian-stretch
armv7l-raspbian-buster

View File

@ -1,5 +1,5 @@
; Generated by RHash v1.4.2 on 2023-07-01 at 17:15.04
; Generated by RHash v1.4.4 on 2023-08-22 at 17:22.35
; Written by Kravchenko Aleksey (Akademgorodok) - http://rhash.sf.net/
;
; 160686 17:15.04 2023-07-01 INSTALL.sh
INSTALL.sh 9576C31EC5BD942E1C9B12413E6408E4623252F7 78B708FE1FC6B39BE081B9F05C6AA5E1478F8762CAF5A8A7671A12EBA4D3C1C5 27991471FB5788F42AF3BBF86FC80A95341AA17AE9487016EEC94961A48437172702EB8E2D6CB300387E87D9E8E0E3E5 C1C21FD491AEFD662C87C3EF62837D769E63E9CF2446B9BD607CCEF8AFD72528824A8F408C6892FD51109390104010EF90DA7F4828950A8671D2986A6B8E216F
; 160749 17:22.35 2023-08-22 INSTALL.sh
INSTALL.sh 06BE6B05BBAD5007BDDDB73DBA2F090A3F4552B1 A4A53EB3EC60FFAD773E8E1D76278315B40042E1B2E62971E73D3F66E9327143 98072442A60BE33F9CCF8C205E4CB2A894CB060566ED9CB835DD4B38C6EDD66B2A94ABE860EFEBD9980EE6C1EF4A5B06 EE56B1BF53930F16CCF13B9C308D55E74D52CF65C1BFB03B890E06476A84F30B2C0AF0F488E34A7A22666B3C1F49866598A35B1EB9F3ADE57427DC56E772B7C9

View File

@ -1 +1 @@
9576c31ec5bd942e1c9b12413e6408e4623252f7 INSTALL.sh
06be6b05bbad5007bdddb73dba2f090a3f4552b1 INSTALL.sh

View File

@ -1 +1 @@
78b708fe1fc6b39be081b9f05c6aa5e1478f8762caf5a8a7671a12eba4d3c1c5 INSTALL.sh
a4a53eb3ec60ffad773e8e1d76278315b40042e1b2e62971e73d3f66e9327143 INSTALL.sh

View File

@ -1 +1 @@
27991471fb5788f42af3bbf86fc80a95341aa17ae9487016eec94961a48437172702eb8e2d6cb300387e87d9e8e0e3e5 INSTALL.sh
98072442a60be33f9ccf8c205e4cb2a894cb060566ed9cb835dd4b38c6edd66b2a94abe860efebd9980ee6c1ef4a5b06 INSTALL.sh

View File

@ -1 +1 @@
c1c21fd491aefd662c87c3ef62837d769e63e9cf2446b9bd607ccef8afd72528824a8f408c6892fd51109390104010ef90da7f4828950a8671d2986a6b8e216f INSTALL.sh
ee56b1bf53930f16ccf13b9c308d55e74d52cf65c1bfb03b890e06476a84f30b2c0af0f488e34a7a22666b3c1f49866598a35b1eb9f3ade57427dc56e772b7c9 INSTALL.sh

View File

@ -850,11 +850,12 @@ x86_64-ubuntu-bionic
x86_64-ubuntu-focal
x86_64-ubuntu-hirsute
x86_64-ubuntu-jammy
x86_64-kali-2021.4
x86_64-kali-2022.1
x86_64-kali-2022.2
x86_64-kali-2022.3
x86_64-kali-2022.4
x86_64-kali-2023.1
x86_64-kali-2023.2
armv6l-raspbian-stretch
armv7l-raspbian-stretch
armv7l-raspbian-buster

2
PyMISP

@ -1 +1 @@
Subproject commit 94983c01ecced6086df28133a38a297111534142
Subproject commit a2566f0282b9f3f83b7785e9fdac3f7aa95fd88b

View File

@ -1 +1 @@
{"major":2, "minor":4, "hotfix":174}
{"major":2, "minor":4, "hotfix":175}

View File

@ -34,7 +34,7 @@ class AppController extends Controller
public $helpers = array('OrgImg', 'FontAwesome', 'UserName');
private $__queryVersion = '155';
public $pyMispVersion = '2.4.174';
public $pyMispVersion = '2.4.175';
public $phpmin = '7.2';
public $phprec = '7.4';
public $phptoonew = '8.0';

View File

@ -158,6 +158,7 @@ class AuditLogsController extends AppController
}
$this->paginate['conditions'] = $this->__createEventIndexConditions($event);
$this->set('passedArgsArray', ['eventId' => $eventId, 'org' => $org]);
$params = $this->IndexFilter->harvestParameters(['created', 'org']);
if ($org) {

View File

@ -54,7 +54,7 @@ class IndexFilterComponent extends Component
private function __massageData($data, $request, $paramArray)
{
$data = array_filter($data, function($paramName) use ($paramArray) {
return !empty($paramArray[$paramName]);
return in_array($paramName, $paramArray);
}, ARRAY_FILTER_USE_KEY);
if (!empty($paramArray)) {

View File

@ -503,7 +503,6 @@ class ServersController extends AppController
$this->Flash->error($error_msg);
}
}
if (!$fail && !empty($this->request->data['Server']['push_rules']) && !JsonTool::isValid($this->request->data['Server']['push_rules'])) {
$fail = true;
$error_msg = __('The push filter rules must be in valid JSON format.');
@ -512,17 +511,22 @@ class ServersController extends AppController
} else {
$this->Flash->error($error_msg);
}
}
$pushRules = $this->_jsonDecode($this->request->data['Server']['push_rules']);
$this->loadModel('Tag');
foreach ($pushRules['tags'] as $operator => $list) {
foreach ($list as $i => $tagName) {
if (!is_numeric($tagName)) { // tag added from freetext
$tag_id = $this->Tag->captureTag(['name' => $tagName], $this->Auth->user());
$list[$i] = $tag_id;
}
if (!$fail && !empty($this->request->data['Server']['push_rules'])) {
$pushRules = $this->_jsonDecode($this->request->data['Server']['push_rules']);
if (!empty($pushRules['tags'])) {
$this->loadModel('Tag');
foreach ($pushRules['tags'] as $operator => $list) {
foreach ($list as $i => $tagName) {
if (!is_numeric($tagName)) { // tag added from freetext
$tag_id = $this->Tag->captureTag(['name' => $tagName], $this->Auth->user());
$list[$i] = $tag_id;
}
}
}
}
}
if (!$fail) {
// say what fields are to be updated
$fieldList = array('id', 'url', 'push', 'pull', 'push_sightings', 'push_galaxy_clusters', 'pull_galaxy_clusters', 'caching_enabled', 'unpublish_event', 'publish_without_email', 'remote_org_id', 'name' ,'self_signed', 'remove_missing_tags', 'cert_file', 'client_cert_file', 'push_rules', 'pull_rules', 'internal', 'skip_proxy');

@ -1 +1 @@
Subproject commit c31bb4b4be00d2a0db22c9a038f9fad8a5950efe
Subproject commit cbd482740f9b472296e01622a3cebb34edf39623

View File

@ -3299,6 +3299,7 @@ class AppModel extends Model
$temp[] = array($key . ' NOT LIKE' => $f);
} else {
$temp[] = array($key . ' LIKE' => $f);
$temp[] = array($key => $f);
}
}
} else {

View File

@ -3573,6 +3573,9 @@ class Event extends AppModel
if (isset($dataArray['Event'])) {
$dataArray['response']['Event'] = $dataArray['Event'];
unset($dataArray['Event']);
} elseif (!isset($dataArray['response'])){
// Accept an event not containing the `Event` key
$dataArray['response']['Event'] = $dataArray;
}
if (!isset($dataArray['response']) || !isset($dataArray['response']['Event'])) {
$exception = $isXml ? __('This is not a valid MISP XML file.') : __('This is not a valid MISP JSON file.');

View File

@ -845,6 +845,10 @@ class User extends AppModel
*/
public function sendEmail(array $user, $body, $bodyNoEnc = false, $subject, $replyToUser = false)
{
if (Configure::read('MISP.disable_emailing')) {
return true;
}
if ($user['User']['disabled'] || !$this->checkIfUserIsValid($user['User'])) {
return true;
}

View File

@ -50,7 +50,8 @@ echo $this->element('genericElements/IndexTable/index_table', [
'element_path' => 'AuditLog/change'
]
],
'title' => __('Audit logs for event #%s', intval($event['Event']['id']))
'title' => __('Audit logs for event #%s', intval($event['Event']['id'])),
'persistUrlParams' => ['eventId', 'org']
]
]);
echo '</div>';

View File

@ -11,6 +11,9 @@
} else {
$attributeEvent = $event;
}
$isNew = $object['timestamp'] > $event['Event']['publish_timestamp'];
$editScope = $mayModify ? 'Attribute' : 'ShadowAttribute';
if (!empty($child)) {
if ($child === 'last' && empty($object['ShadowAttribute'])) {
@ -69,7 +72,7 @@
<td class="short context hidden">
<?php echo $this->element('/Events/View/seen_field', array('object' => $object)); ?>
</td>
<td class="short timestamp"><?= $this->Time->date($object['timestamp']) ?></td>
<td class="short timestamp <?= $isNew ? 'bold red' : '' ?>" <?= $isNew ? 'title="' . __('Element or modification to an existing element has not been published yet.') . '"' : '' ?>><?= $this->Time->date($object['timestamp']) . ($isNew ? '*' : '') ?></td>
<?php
if (!empty($extended)):
?>
@ -338,12 +341,12 @@
if ($isAclAdd && ($isSiteAdmin || !$mayModify)):
if (isset($modules) && isset($modules['types'][$object['type']])):
?>
<span class="fas fa-asterisk useCursorPointer" role="button" tabindex="0" aria-label="<?php echo __('Query enrichment');?>" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/Enrichment/ShadowAttribute');" title="<?php echo __('Propose enrichment');?>">&nbsp;</span>
<span class="fas fa-asterisk useCursorPointer" role="button" tabindex="0" aria-label="<?php echo __('Query enrichment');?>" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/0/Enrichment/ShadowAttribute');" title="<?php echo __('Propose enrichment');?>">&nbsp;</span>
<?php
endif;
if (isset($cortex_modules) && isset($cortex_modules['types'][$object['type']])):
?>
<span class="icon-eye-open useCursorPointer" role="button" tabindex="0" aria-label="<?php echo __('Query Cortex');?>" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/Enrichment/ShadowAttribute/Cortex');" title="<?php echo __('Propose enrichment through Cortex');?>"></span>
<span class="icon-eye-open useCursorPointer" role="button" tabindex="0" aria-label="<?php echo __('Query Cortex');?>" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/Enrichment/0/ShadowAttribute/Cortex');" title="<?php echo __('Propose enrichment through Cortex');?>"></span>
<?php
endif;
?>
@ -358,12 +361,12 @@
if ($isSiteAdmin || $mayModify):
if (isset($modules) && isset($modules['types'][$object['type']])):
?>
<span class="fas fa-asterisk useCursorPointer" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/Enrichment/Attribute');" title="<?php echo __('Add enrichment');?>" role="button" tabindex="0" aria-label="<?php echo __('Add enrichment');?>">&nbsp;</span>
<span class="fas fa-asterisk useCursorPointer" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/0/Enrichment/Attribute');" title="<?php echo __('Add enrichment');?>" role="button" tabindex="0" aria-label="<?php echo __('Add enrichment');?>">&nbsp;</span>
<?php
endif;
if (isset($cortex_modules) && isset($cortex_modules['types'][$object['type']])):
?>
<span class="icon-eye-open useCursorPointer" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/Enrichment/Attribute/Cortex');" title="<?php echo __('Add enrichment');?>" role="button" tabindex="0" aria-label="<?php echo __('Add enrichment via Cortex');?>"></span>
<span class="icon-eye-open useCursorPointer" onclick="simplePopup('<?php echo $baseurl;?>/events/queryEnrichment/<?= $objectId ?>/0/Enrichment/Attribute/Cortex');" title="<?php echo __('Add enrichment');?>" role="button" tabindex="0" aria-label="<?php echo __('Add enrichment via Cortex');?>"></span>
<?php
endif;
?>

View File

@ -7,6 +7,7 @@
} else {
$objectEvent = $event;
}
$isNew = $object['timestamp'] > $event['Event']['publish_timestamp'];
if ($object['deleted']) $tr_class .= ' lightBlueRow';
else $tr_class .= ' blueRow';
if (!empty($k)) {
@ -33,7 +34,7 @@ $objectId = intval($object['id']);
<td class="short context hidden">
<?php echo $this->element('/Events/View/seen_field', array('object' => $object)); ?>
</td>
<td class="short timestamp"><?= $this->Time->date($object['timestamp']) ?></td>
<td class="short timestamp <?= $isNew ? 'bold red' : '' ?>" <?= $isNew ? 'title="' . __('Element or modification to an existing element has not been published yet.') . '"' : '' ?>><?= $this->Time->date($object['timestamp']) . ($isNew ? '*' : '') ?></td>
<?php
if ($extended):
?>

@ -1 +1 @@
Subproject commit f5729ac23a7dcb8fc9dc3194a3e125484c515742
Subproject commit 34b86e4abc47d3dfbafaa813f01e22be0387168a

@ -1 +1 @@
Subproject commit 4da05293d723ad6f9db4a3e349e140daa5d2a28d
Subproject commit 8b648981573f77c9526df5322c52902ae1a81859

@ -1 +1 @@
Subproject commit 7df546216a1d2b4073714476fd92434cc166e516
Subproject commit 2337bdeb8c0aa7125db3a410fdb6a54ec50d6865

@ -1 +1 @@
Subproject commit a12d40a4ff8a19c0a6b9f4d719c9549e805108c5
Subproject commit 6771e5cd9ec22d0d24ec9f657d78d385a3c5ef80

View File

@ -33,6 +33,17 @@ from misp_stix_converter import (
from stix2.parsing import parse as stix2_parser
def _handle_return_message(traceback):
if isinstance(traceback, dict):
messages = []
for key, values in traceback.items():
messages.append(f'- {key}')
for value in values:
messages.append(f' - {value}')
return '\n '.join(messages)
return '\n - '.join(traceback)
def _process_stix_file(args: argparse.ArgumentParser):
try:
with open(args.input, 'rt', encoding='utf-8') as f:
@ -63,11 +74,10 @@ def _process_stix_file(args: argparse.ArgumentParser):
if args.debug:
for feature in ('errors', 'warnings'):
if getattr(parser, feature):
print(getattr(parser, feature), file=sys.stderr)
message = '\n - '.join(getattr(parser, feature).values())
message = _handle_return_message(getattr(parser, feature))
print(
f'{feature.title()} encountered while importing '
f'STIX {stix_version} content:\n - {message}',
f'STIX {stix_version} content:\n {message}',
file=sys.stderr
)
except Exception as e:

@ -1 +1 @@
Subproject commit 59ec473a5f7a44755a6098890a1ee290487bfc53
Subproject commit 8d8433399f0d00651238237be091fe63768a924f

@ -1 +1 @@
Subproject commit 11101527c0e55810613d3d753f2e770219895c39
Subproject commit 07a1e66092a8216574b103c650b423e816a1091a

View File

@ -16,7 +16,7 @@ Nevertheless here is a very rough **food for thoughts** bulletpoint list for you
- Do we need to encrypt the partitions where some data is stored?
- Are we redundant in case one MISP instance might fail?
- Is the database server and any other servers running on the machine bound to **localhost**? Do we need to expose because our setup is more complex?
- Do we have enough storage? What about [MISP and size estimation](https://misp-project.org/MISP-sizer/) anyways?
- Do we have enough storage? What about [MISP and size estimation](https://www.misp-project.org/sizing-your-misp-instance/) anyways?
- Do we care about BIOS updates?
- Do we care about physical access to the servers? (Disabling USB ports etc...)
- Is any fancy management engine à la [IME](https://en.wikipedia.org/wiki/Intel_Management_Engine) in use?
@ -52,7 +52,7 @@ index f9f69d4..2e8fd78 100644
## Resources
[IT Security Guidelines for TLS by NCSC.nl](https://www.ncsc.nl/english/current-topics/factsheets/it-security-guidelines-for-transport-layer-security-tls.html)
[IT Security Guidelines for TLS by NCSC.nl](https://english.ncsc.nl/publications/publications/2021/january/19/it-security-guidelines-for-transport-layer-security-2.1)
[Weak Diffie-Hellman and the Logjam Attack](https://weakdh.org/sysadmin.html)

View File

@ -519,8 +519,9 @@ kaliOnTheR0ckz () {
overlay=$(df -kh |grep overlay; echo $?) # if 1 overlay NOT present
if [[ ${totalRoot} -lt 3059034 ]]; then
echo "(If?) You run Kali in LiveCD mode and we need more overlay disk space."
echo "This is defined by the total memory, you have: ${totalMem}kB which is not enough."
echo "(If?) You run Kali in LiveCD mode, you need more overlay disk space."
echo "This is defined by the total memory setting in you VM config."
echo "You currently have: ${totalMem}kB which is not enough."
echo "6-8Gb should be fine. (need >3Gb overlayFS)"
exit 1
fi

View File

@ -6,13 +6,15 @@
# This tool is part of the MISP core project and released under the GNU Affero
# General Public License v3.0
#
# Copyright (C) 2017 Alexandre Dulaunoy
# Copyright (C) 2017-2023 Alexandre Dulaunoy
# Copyright (C) 2023 Christophe Vandeplas
import json
import requests
default_feed = '../../app/files/feed-metadata/defaults.json'
misp_website_path = '../../../misp-website-new/content/feeds.md'
verify_certificate = False
with open(default_feed) as feed_file:
feedlist = json.load(feed_file)
@ -25,7 +27,7 @@ for feed in feedlist:
items.append(output)
# try to download the feed
headers = {"Range": "bytes=0-0"}
res = requests.get(feed['Feed']['url'], headers=headers)
res = requests.get(feed['Feed']['url'], headers=headers, verify=verify_certificate)
if (res.status_code >= 200 and res.status_code < 300)\
or res.status_code == 403:
continue
@ -38,7 +40,7 @@ items = sorted(items, key=lambda s: s.casefold())
print("Updating misp-website feed.md file.")
start_header_seen = False
inserted = False
with open('../../../misp-website/content/feeds.md', 'r') as f:
with open(misp_website_path, 'r') as f:
data_new = []
for line in f:
if start_header_seen and line.startswith('- ') and not inserted: # first item
@ -56,5 +58,5 @@ with open('../../../misp-website/content/feeds.md', 'r') as f:
start_header_seen = True
with open('../../../misp-website/content/feeds.md', 'w') as f:
with open(misp_website_path, 'w') as f:
f.write('\n'.join(data_new))