MISP
/
PyMISP
mirror of https://github.com/MISP/PyMISP
2
0
Fork 0
Code Issues Releases Wiki Activity

Merge remote-tracking branch 'MISP/master'

pull/508/head
Koen Van Impe 2019-10-31 21:50:58 +01:00
parent 4b08b9baa4 87fd06a889
commit bdffbd46a6
33 changed files with 901 additions and 433 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -6,8 +6,10 @@ examples/cudeso.py
examples/feed-generator/output/*\.json examples/feed-generator/output/*\.json
examples/feed-generator/output/hashes\.csv examples/feed-generator/output/hashes\.csv
examples/feed-generator/settings\.py examples/feed-generator/settings\.py
tests/reportlab_testoutputs/*\.pdf
build/* build/*
dist/* dist/*
pymisp.egg-info/* pymisp.egg-info/*
.coverage
.idea .idea

3
.gitmodules vendored
View File

@ -4,3 +4,6 @@
[submodule "pymisp/tools/pdf_fonts"] [submodule "pymisp/tools/pdf_fonts"]
path = pymisp/tools/pdf_fonts path = pymisp/tools/pdf_fonts
url = https://github.com/MISP/pdf_fonts url = https://github.com/MISP/pdf_fonts
[submodule "tests/viper-test-files"]
path = tests/viper-test-files
url = https://github.com/viper-framework/viper-test-files.git

147
CHANGELOG.txt
View File

@ -2,6 +2,152 @@ Changelog
========= =========
v2.4.117.2 (2019-10-30)
-----------------------
Fix
~~~
- Avoid exception on legacy MISP. [Raphaël Vinot]
v2.4.117.1 (2019-10-30)
-----------------------
New
~~~
- Add support for UserSettings. [Raphaël Vinot]
Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Bump misp-objects. [Raphaël Vinot]
- Use default category from template. [Raphaël Vinot]
Fix #477
- Skip usersettings tests when emails are disabled. [Raphaël Vinot]
Fix
~~~
- [examples] typo uuid. [Jean-Louis Huynen]
give me a hoodie.
- Prevents exception when lief is not installed. [Christophe Vandeplas]
- Python <3.4 should work again.... [Raphaël Vinot]
Fix #482
- Remote_describe_types response was invalid. [Raphaël Vinot]
- Missing file in last commit. [Raphaël Vinot]
- Remove overwrite of remote_describe_types. [Raphaël Vinot]
Other
~~~~~
- Added example for checking sync servers. [wotschel]
- Corrected docstring. [Shortfinga]
- Include to_ids and replace newlines in title. [Koen Van Impe]
- Update aping.py. [ater49]
Just fixing a typo
- Remove unused MISPFileCache from PyMISP class. [Marc Hoersken]
v2.4.117 (2019-10-10)
---------------------
New
~~~
- Better handling of delete(d) attributes. [Raphaël Vinot]
* Hard delete on attribute
* Get the deleted attributes within an event
Changes
~~~~~~~
- Bump changelog. [Raphaël Vinot]
- Bump version. [Raphaël Vinot]
- Test if json exists in cached method. [Raphaël Vinot]
- Decode datetime without dateutils if possible. [Raphaël Vinot]
- Add support for rapidjson, refactoring and code cleanup. [Raphaël
Vinot]
- Cleanups. [Raphaël Vinot]
- Cleanups and improvements. [Raphaël Vinot]
- [types] updated to the latest version. [Christophe Vandeplas]
now using the gen_misp_types_categories using jq
- [describeTypes] updated to the latest version. [Alexandre Dulaunoy]
- Bump dependencies. [Raphaël Vinot]
- Add missing return formats in restsearch, bump objects. [Raphaël
Vinot]
- [misp-objects] updated to the latest version. [Alexandre Dulaunoy]
- Update search examples. [Raphaël Vinot]
- Update main notebook. [Raphaël Vinot]
- [test] remove attribute field which was not foreseen in 2.4 branch.
[Alexandre Dulaunoy]
- Fix travis tests due to sighting_timestamp. [Raphaël Vinot]
- Use default for warnings. [Raphaël Vinot]
fix: #453
- Dump dependencies, update tests. [Raphaël Vinot]
- Bump readme. [Raphaël Vinot]
- Update upload malware/attachment example script. [Raphaël Vinot]
Fix #447
Make data at attibute level more generic with getter/setter methods
Fix
~~~
- [Python2] Use LRU cache decorator, fix call to describe_types in
PyMISP. [Raphaël Vinot]
- Python2 SyntaxError... [Raphaël Vinot]
- Objects helpers were broken, do not overwrite describe_types. [Raphaël
Vinot]
- Support for legacy python versions. [Raphaël Vinot]
90 days and counting, folks.
- Cache object templates at AbstractMISP level. [Raphaël Vinot]
Related #468 and #471
- Cache describeTypes at AbstractMISP level. [Raphaël Vinot]
- Big speed improvment when loading MISPEvent. [Raphaël Vinot]
1. `properties` is a list comprehension
2. Massively reduce the amount of calls to `properties`
- Python 2.7 support. [Raphaël Vinot]
I want a cookie.
Other
~~~~~
- Use classmethod instead of staticmethod and avoid hard-coded
reference. [Marc Hoersken]
- Cache JSON definitions in memory LFU cache provided by cachetools.
[Marc Hoersken]
- Path and modified time of JSON file are used as the cache key
- Global state is hidden away inside a root-class for re-use
- Maximum size is 150 considering the number of JSON definitions
During my tests the memory usage of the test suites was halved.
- Fix mixed whitespace in the travis helper script files. [Marc
Hoersken]
- Remove explicit clonce as the viper-test-files are now a Git
submodule. [Marc Hoersken]
- Add viper-test-files repository as Git submodule. [Marc Hoersken]
- Update .gitignore to exclude files produced during tests. [Marc
Hoersken]
- Code cleanup. [Koen Van Impe]
- Update type and code cleanup. [Koen Van Impe]
- List all the sightings - show_sightings.py. [Koen Van Impe]
- Disable to_ids based on false positive sightings reporting. [Koen Van
Impe]
- Adds support to add local tags. [Antoine Cailliau]
Requires https://github.com/MISP/MISP/pull/5215 to be merged first.
- Minor grammar errors. [Miroslav Stampar]
- Make client_certs out of the box friendly. [Campbell McKenzie]
v2.4.114 (2019-08-30) v2.4.114 (2019-08-30)
--------------------- ---------------------
@ -17,6 +163,7 @@ New
Changes Changes
~~~~~~~ ~~~~~~~
- Bump Changelog. [Raphaël Vinot] - Bump Changelog. [Raphaël Vinot]
- Bump Changelog. [Raphaël Vinot]
- Temp disable tests for request_community_access. [Raphaël Vinot] - Temp disable tests for request_community_access. [Raphaël Vinot]
- Disable test for now. [Raphaël Vinot] - Disable test for now. [Raphaël Vinot]
- Bump Changelog. [Raphaël Vinot] - Bump Changelog. [Raphaël Vinot]

1
Pipfile
View File

@ -10,6 +10,7 @@ codecov = "*"
requests-mock = "*" requests-mock = "*"
pymisp = {editable = true,extras = ["fileobjects", "neo", "openioc", "virustotal", "pdfexport", "docs"],path = "."} pymisp = {editable = true,extras = ["fileobjects", "neo", "openioc", "virustotal", "pdfexport", "docs"],path = "."}
docutils = "==0.15" docutils = "==0.15"
memory-profiler = "*"
[packages] [packages]
pymisp = {editable = true,extras = ["fileobjects", "openioc", "virustotal", "pdfexport"],path = "."} pymisp = {editable = true,extras = ["fileobjects", "openioc", "virustotal", "pdfexport"],path = "."}

354
Pipfile.lock generated
View File

@ -1,7 +1,7 @@
{ {
"_meta": { "_meta": {
"hash": { "hash": {
"sha256": "4b4cf20ef3242efd0c24d7cc54ba2438dee8ba853ab3b9384ad915448ce83048" "sha256": "4be7259a433785d74e1879a4a555bb669d50c5f409d0a094652c1abc9b1227c5"
}, },
"pipfile-spec": 6, "pipfile-spec": 6,
"requires": { "requires": {
@ -18,25 +18,24 @@
"default": { "default": {
"attrs": { "attrs": {
"hashes": [ "hashes": [
"sha256:69c0dbf2ed392de1cb5ec704444b08a5ef81680a61cb899dc08127123af36a79", "sha256:ec20e7a4825331c1b5ebf261d111e16fa9612c1f7a5e1f884f12bd53a664dfd2",
"sha256:f0b870f674851ecbfbbbd364d6b5cbdff9dcedbc7f3f5e18a6891057f21fe399" "sha256:f913492e1663d3c36f502e5e9ba6cd13cf19d7fab50aa13239e420fef95e1396"
], ],
"version": "==19.1.0" "version": "==19.2.0"
}, },
"beautifulsoup4": { "beautifulsoup4": {
"hashes": [ "hashes": [
"sha256:05668158c7b85b791c5abde53e50265e16f98ad601c402ba44d70f96c4159612", "sha256:5279c36b4b2ec2cb4298d723791467e3000e5384a43ea0cdf5d45207c7e97169",
"sha256:25288c9e176f354bf277c0a10aa96c782a6a18a17122dba2e8cec4a97e03343b", "sha256:dcdef580e18a76d54002088602eba453eec38ebbcafafeaabd8cab12b6155d57"
"sha256:f040590be10520f2ea4c2ae8c3dae441c7cfff5308ec9d58a0ec0c1b8f81d469"
], ],
"version": "==4.8.0" "version": "==4.8.1"
}, },
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:046832c04d4e752f37383b628bc601a7ea7211496b4638f6514d0e5b9acc4939", "sha256:e4f3620cfea4f83eedc95b24abd9cd56f3c4b146dd0177e83a21b4eb49e21e50",
"sha256:945e3ba63a0b9f577b1395204e13c3a231f9bc0223888be653286534e5873695" "sha256:fd7c7c74727ddcf00e9acd26bba8da604ffec95bf1c2144e67aff7a8b50e6cef"
], ],
"version": "==2019.6.16" "version": "==2019.9.11"
}, },
"chardet": { "chardet": {
"hashes": [ "hashes": [
@ -92,34 +91,34 @@
}, },
"pillow": { "pillow": {
"hashes": [ "hashes": [
"sha256:0804f77cb1e9b6dbd37601cee11283bba39a8d44b9ddb053400c58e0c0d7d9de", "sha256:00fdeb23820f30e43bba78eb9abb00b7a937a655de7760b2e09101d63708b64e",
"sha256:0ab7c5b5d04691bcbd570658667dd1e21ca311c62dcfd315ad2255b1cd37f64f", "sha256:01f948e8220c85eae1aa1a7f8edddcec193918f933fb07aaebe0bfbbcffefbf1",
"sha256:0b3e6cf3ea1f8cecd625f1420b931c83ce74f00c29a0ff1ce4385f99900ac7c4", "sha256:08abf39948d4b5017a137be58f1a52b7101700431f0777bec3d897c3949f74e6",
"sha256:365c06a45712cd723ec16fa4ceb32ce46ad201eb7bbf6d3c16b063c72b61a3ed", "sha256:099a61618b145ecb50c6f279666bbc398e189b8bc97544ae32b8fcb49ad6b830",
"sha256:38301fbc0af865baa4752ddae1bb3cbb24b3d8f221bf2850aad96b243306fa03", "sha256:2c1c61546e73de62747e65807d2cc4980c395d4c5600ecb1f47a650c6fa78c79",
"sha256:3aef1af1a91798536bbab35d70d35750bd2884f0832c88aeb2499aa2d1ed4992", "sha256:2ed9c4f694861642401f27dc3cb99772be67cd190e84845c749dae0a06c3bfae",
"sha256:3fe0ab49537d9330c9bba7f16a5f8b02da615b5c809cdf7124f356a0f182eccd", "sha256:338581b30b908e111be578f0297255f6b57a51358cd16fa0e6f664c9a1f88bff",
"sha256:45a619d5c1915957449264c81c008934452e3fd3604e36809212300b2a4dab68", "sha256:38c7d48a21cd06fdeee93987147b9b1c55b73b4cfcbf83240568bfbd5adee447",
"sha256:49f90f147883a0c3778fd29d3eb169d56416f25758d0f66775db9184debc8010", "sha256:43fd026f613c8e48a25eba1a92f4d2ad7f3903c95d8c33a11611a7717d2ab654",
"sha256:571b5a758baf1cb6a04233fb23d6cf1ca60b31f9f641b1700bfaab1194020555", "sha256:4548236844327a718ce3bb182ab32a16fa2050c61e334e959f554cac052fb0df",
"sha256:5ac381e8b1259925287ccc5a87d9cf6322a2dc88ae28a97fe3e196385288413f", "sha256:5090857876c58885cfa388dc649e5db30aae98a068c26f3fd0ac9d7d9a4d9572",
"sha256:6153db744a743c0c8c91b8e3b9d40e0b13a5d31dbf8a12748c6d9bfd3ddc01ad", "sha256:5bbba34f97a26a93f5e8dec469ca4ddd712451418add43da946dbaed7f7a98d2",
"sha256:6fd63afd14a16f5d6b408f623cc2142917a1f92855f0df997e09a49f0341be8a", "sha256:65a28969a025a0eb4594637b6103201dc4ed2a9508bdab56ac33e43e3081c404",
"sha256:70acbcaba2a638923c2d337e0edea210505708d7859b87c2bd81e8f9902ae826", "sha256:892bb52b70bd5ea9dbbc3ac44f38e84f5a04e9d8b1bff48159d96cb795b81159",
"sha256:70b1594d56ed32d56ed21a7fbb2a5c6fd7446cdb7b21e749c9791eac3a64d9e4", "sha256:8a9becd5cbd5062f973bcd2e7bc79483af310222de112b6541f8af1f93a3cc42",
"sha256:76638865c83b1bb33bcac2a61ce4d13c17dba2204969dedb9ab60ef62bede686", "sha256:972a7aaeb7c4a2795b52eef52ee991ef040b31009f36deca6207a986607b55f3",
"sha256:7b2ec162c87fc496aa568258ac88631a2ce0acfe681a9af40842fc55deaedc99", "sha256:97b119c436bfa96a92ac2ca525f7025836d4d4e64b1c9f9eff8dbaf3ff1d86f3",
"sha256:7cee2cef07c8d76894ebefc54e4bb707dfc7f258ad155bd61d87f6cd487a70ff", "sha256:9ba37698e242223f8053cc158f130aee046a96feacbeab65893dbe94f5530118",
"sha256:7d16d4498f8b374fc625c4037742fbdd7f9ac383fd50b06f4df00c81ef60e829", "sha256:b1b0e1f626a0f079c0d3696db70132fb1f29aa87c66aecb6501a9b8be64ce9f7",
"sha256:b50bc1780681b127e28f0075dfb81d6135c3a293e0c1d0211133c75e2179b6c0", "sha256:c14c1224fd1a5be2733530d648a316974dbbb3c946913562c6005a76f21ca042",
"sha256:bd0582f831ad5bcad6ca001deba4568573a4675437db17c4031939156ff339fa", "sha256:c79a8546c48ae6465189e54e3245a97ddf21161e33ff7eaa42787353417bb2b6",
"sha256:cfd40d8a4b59f7567620410f966bb1f32dc555b2b19f82a91b147fac296f645c", "sha256:ceb76935ac4ebdf6d7bc845482a4450b284c6ccfb281e34da51d510658ab34d8",
"sha256:e3ae410089de680e8f84c68b755b42bc42c0ceb8c03dbea88a5099747091d38e", "sha256:e22bffaad04b4d16e1c091baed7f2733fc1ebb91e0c602abf1b6834d17158b1f",
"sha256:e9046e559c299b395b39ac7dbf16005308821c2f24a63cae2ab173bd6aa11616", "sha256:ec883b8e44d877bda6f94a36313a1c6063f8b1997aa091628ae2f34c7f97c8d5",
"sha256:ef6be704ae2bc8ad0ebc5cb850ee9139493b0fc4e81abcc240fb392a63ebc808", "sha256:f1baa54d50ec031d1a9beb89974108f8f2c0706f49798f4777df879df0e1adb6",
"sha256:f8dc19d92896558f9c4317ee365729ead9d7bbcf2052a9a19a3ef17abbb8ac5b" "sha256:f53a5385932cda1e2c862d89460992911a89768c65d176ff8c50cddca4d29bed"
], ],
"version": "==6.1.0" "version": "==6.2.0"
}, },
"pydeep": { "pydeep": {
"hashes": [ "hashes": [
@ -140,7 +139,7 @@
"pymispwarninglists": { "pymispwarninglists": {
"editable": true, "editable": true,
"git": "https://github.com/MISP/PyMISPWarningLists.git", "git": "https://github.com/MISP/PyMISPWarningLists.git",
"ref": "52b0a0f93045861330c134385f88441f212f6421" "ref": "8a47f8b7f723a268e5a6b5420fe4b873e4fd6a0b"
}, },
"pyrsistent": { "pyrsistent": {
"hashes": [ "hashes": [
@ -164,36 +163,31 @@
}, },
"reportlab": { "reportlab": {
"hashes": [ "hashes": [
"sha256:065bca611829da371df97cec255239a2972119afbab57528022df8b41881a3f6", "sha256:06b7c7436fa6d4844c7637161f3297c7a96240f35622ab2d219e4fd8387c0ab2",
"sha256:329843edd93293a96b99b2e9c226066a9ed27f0f881b4933536577e1dab898cf", "sha256:0a5acf67bd9812e38ed84be8994c07a8136b0a8f4c14a1c66c9c73a9567a9a44",
"sha256:393140710488b7ffda2762a08f63671dcccdbccfed0e4c8e8ec77e5a355080a1", "sha256:1c8ca145d03e3c620866b06febb241b179197b58fb07454fbc8e9d6184cdcc93",
"sha256:3c778843f50981a1569539120f0cfa2be0ca7a80e4c61bdfc88a74c323b90b00", "sha256:2f8d785660ee316874c86abad345633ce8c652e88e03ae8a10f1fdadc72fd23d",
"sha256:44ab0741f40899936e7cc85b0a19614a483da4b476102ac58d1ac20ef6da9fc3", "sha256:4869d342352c92a812ce40555ef2a9cfbd722390d67fe61f1d6ec770e9ca41a3",
"sha256:4582272135bd2f355a616b4ac08310947d88b0d3e4f474be16175d89fa200c0d", "sha256:493e0dcd9c085d46acf4fe3f00f941e562490a74b651409039a0dee2a0d76555",
"sha256:47612270365e21581178ebbb91edabf9b3c6b4519baf2052d3f4cbe302e3ea76", "sha256:4e606e3ee9345e68cd205022d526250ad2a1164eea8f1e29d77d6ad08631b0ba",
"sha256:4f8c5e65fcfa111be309228efca92ba17f329d3dbf3bbe055094fe907ab5d4c8", "sha256:5bf91bae8995db91650fda658129c268515358b756fd16c0261a9dd641df1856",
"sha256:4ff4942cb1ca1f70a890fd35c7e1d0657d08dbdf6bdb5bc2c0dd3e30a6301cf7", "sha256:6df0730f8f715aa12333bd6d2a72eea3a989381451861186d9b5e71889454ac7",
"sha256:5b109b347ae391963ef846e41c4c65c2bc99e81f1d4eeff687635b73ee952bf5", "sha256:7195c6ea096d10c91cc470f9f0ced3ad74470d9c0fd97923b5e764597dd13671",
"sha256:5cbd56e8dea652f73f728578cb3dbc57bd100f308012fe90596085520d2cb25a", "sha256:7431c979e2b498e8e20abf458f360a451717d76c3c1bd49d1fc5697d3504f8e5",
"sha256:5dddc51b5848a2d0a6fe47e96496220a305e7d796d4a6973cc984ab1d8160ff7", "sha256:7f7f70a8d4b573d1ff65a81415b4b6ed9545630f381dff4a69307640e09d381d",
"sha256:6c81ee26753fa09062d8404f6340eefb02849608b619e3843e0d17a7cda8798f", "sha256:9945433667a46f054d1125b4ca86fe9ee31feb254728b38242e9a6008c135efe",
"sha256:706ffb184c4cdeabcaef3b9eaba86cbf7684467c32d308ed908917fc679f86c8", "sha256:b1cdbfc1fd54ac947b9f0114e00ab94e945db679f1e03357a3c00f3a85e73eea",
"sha256:794499adc5ad419e064523f13b0782ee2860180e79c8cd02379c4c957e1f0abb", "sha256:bf149847a2fd8f24b788a8abbf97a2b9a73edc5b1bd719384b786eb84bcad15e",
"sha256:8b7fcc98b0aed3e3e4f134f4d5a498bb9c068fdce6c6b2a9f103d3a339efd8d1", "sha256:ce514bfce2bf3e302f52aba9929fe3ac7d918cfea2f5d3e30bf9dac9658bf094",
"sha256:8bc0fe11be68207866902ee96eec6645d574d82fd6abd93c8bcdcd57ac1b4040", "sha256:d243d4c8cf1a7e78b734c03628b684ec5de25df1f02ccea2e10fbd217430cb72",
"sha256:92f01e16fe65e51ffa2fe0e37da697c8b8f5d892605c05394c883a866a11efc1", "sha256:d4bee20f52b8c3c477dc780780654cafcfc0eb34d8d6960c13a34a444b431f09",
"sha256:a162484b22c52ab701b74f8c35b2a14f9ecf9694f2ab149fb38f377069743e69", "sha256:e730529bd1f62034c50f70a2b05fadbf7d1402d39ff69c9dc63db066d0ef8a46",
"sha256:a30b42d6c5ffe1ce7c677328a47386f861c3bb9057bf4de5eb0f97fe17e9b3ba", "sha256:eb54ecfbf1abe6134073b7b35fd40442c4cd81bb9a5bee1a3038b8867b721bfb",
"sha256:a7a63d35c59af1d134ec43bab75070af86e59c412289198de3788765627a611c", "sha256:f18ec70f5ee6a78b3bb4361e55f3a5ef34eb253f1e72fba76f29f0d680cd446f",
"sha256:aee6aa362cbaf9abc406944064a887a69f6f5606fa54abaecf98a78459d1d954", "sha256:f6be66f69198dcd04a79faa6052f756d35643496321858f06931c7b1ed9833ab",
"sha256:ba537b091614f3839716fb7b418e157216e213a0eab3fe7db2dfbf198fb61224", "sha256:fc5c23a53fbd97b8aab4968c8548ce5cea4a54a26b4f8c1e6835df7adb8d0fe2"
"sha256:be8f70ec622b98ef830af5591ab4c0b062a67507a19ca43327da5ff350435b43",
"sha256:c380bcb032736d45bd9a90f4208547a679b7fe2327fc1187a73a2d9b58988f1d",
"sha256:cd2fdcd1e31113878d5c5c9ae17a34368a13e1c9e12d586b66b77ff806371e23",
"sha256:f59d772b504035b1468544a11269ee27648ddb2fae1efddd45ce050da2527813",
"sha256:ff1570bf8ad010c408f72822248ad2276185d473ab9a64c70ad2ec4427dda052"
], ],
"version": "==3.5.23" "version": "==3.5.28"
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
@ -211,17 +205,17 @@
}, },
"soupsieve": { "soupsieve": {
"hashes": [ "hashes": [
"sha256:8662843366b8d8779dec4e2f921bebec9afd856a5ff2e82cd419acc5054a1a92", "sha256:605f89ad5fdbfefe30cdc293303665eff2d188865d4dbe4eb510bba1edfbfce3",
"sha256:a5a6166b4767725fd52ae55fee8c8b6137d9a51e9f1edea461a062a759160118" "sha256:b91d676b330a0ebd5b21719cb6e9b57c57d433671f65b9c28dd3461d9a1ed0b6"
], ],
"version": "==1.9.3" "version": "==1.9.4"
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:b246607a25ac80bedac05c6f282e3cdaf3afb65420fd024ac94435cabe6e18d1", "sha256:3de946ffbed6e6746608990594d08faac602528ac7015ac28d33cee6a45b7398",
"sha256:dbe59173209418ae49d485b87d1681aefa36252ee85884c31346debd19463232" "sha256:9a107b99a5393caf59c7aa3c1249c16e6879447533d0887f4336dde834c7be86"
], ],
"version": "==1.25.3" "version": "==1.25.6"
}, },
"validators": { "validators": {
"hashes": [ "hashes": [
@ -246,10 +240,10 @@
}, },
"attrs": { "attrs": {
"hashes": [ "hashes": [
"sha256:69c0dbf2ed392de1cb5ec704444b08a5ef81680a61cb899dc08127123af36a79", "sha256:ec20e7a4825331c1b5ebf261d111e16fa9612c1f7a5e1f884f12bd53a664dfd2",
"sha256:f0b870f674851ecbfbbbd364d6b5cbdff9dcedbc7f3f5e18a6891057f21fe399" "sha256:f913492e1663d3c36f502e5e9ba6cd13cf19d7fab50aa13239e420fef95e1396"
], ],
"version": "==19.1.0" "version": "==19.2.0"
}, },
"babel": { "babel": {
"hashes": [ "hashes": [
@ -260,18 +254,17 @@
}, },
"beautifulsoup4": { "beautifulsoup4": {
"hashes": [ "hashes": [
"sha256:05668158c7b85b791c5abde53e50265e16f98ad601c402ba44d70f96c4159612", "sha256:5279c36b4b2ec2cb4298d723791467e3000e5384a43ea0cdf5d45207c7e97169",
"sha256:25288c9e176f354bf277c0a10aa96c782a6a18a17122dba2e8cec4a97e03343b", "sha256:dcdef580e18a76d54002088602eba453eec38ebbcafafeaabd8cab12b6155d57"
"sha256:f040590be10520f2ea4c2ae8c3dae441c7cfff5308ec9d58a0ec0c1b8f81d469"
], ],
"version": "==4.8.0" "version": "==4.8.1"
}, },
"certifi": { "certifi": {
"hashes": [ "hashes": [
"sha256:046832c04d4e752f37383b628bc601a7ea7211496b4638f6514d0e5b9acc4939", "sha256:e4f3620cfea4f83eedc95b24abd9cd56f3c4b146dd0177e83a21b4eb49e21e50",
"sha256:945e3ba63a0b9f577b1395204e13c3a231f9bc0223888be653286534e5873695" "sha256:fd7c7c74727ddcf00e9acd26bba8da604ffec95bf1c2144e67aff7a8b50e6cef"
], ],
"version": "==2019.6.16" "version": "==2019.9.11"
}, },
"chardet": { "chardet": {
"hashes": [ "hashes": [
@ -304,10 +297,10 @@
}, },
"commonmark": { "commonmark": {
"hashes": [ "hashes": [
"sha256:14c3df31e8c9c463377e287b2a1eefaa6019ab97b22dad36e2f32be59d61d68d", "sha256:452f9dc859be7f06631ddcb328b6919c67984aca654e5fefb3914d54691aed60",
"sha256:867fc5db078ede373ab811e16b6789e9d033b15ccd7296f370ca52d1ee792ce0" "sha256:da2f38c92590f83de410ba1a3cbceafbc74fee9def35f9251ba9a971d6d66fd9"
], ],
"version": "==0.9.0" "version": "==0.9.1"
}, },
"coverage": { "coverage": {
"hashes": [ "hashes": [
@ -382,12 +375,6 @@
"index": "pypi", "index": "pypi",
"version": "==0.15" "version": "==0.15"
}, },
"future": {
"hashes": [
"sha256:67045236dcfd6816dc439556d009594abf643e5eb48992e36beac09c2ca659b8"
],
"version": "==0.17.1"
},
"idna": { "idna": {
"hashes": [ "hashes": [
"sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407", "sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407",
@ -404,10 +391,10 @@
}, },
"jinja2": { "jinja2": {
"hashes": [ "hashes": [
"sha256:065c4f02ebe7f7cf559e49ee5a95fb800a9e4528727aec6f24402a5374c65013", "sha256:74320bb91f31270f9551d46522e33af46a80c3d619f4a4bf42b3164d30b5911f",
"sha256:14dd6caf1527abb21f08f86c784eac40853ba93edb79552aa1e4b8aef1b61c7b" "sha256:9fe95f19286cfefaa917656583d020be14e7859c6b0252588391e47db34527de"
], ],
"version": "==2.10.1" "version": "==2.10.3"
}, },
"jsonschema": { "jsonschema": {
"hashes": [ "hashes": [
@ -466,11 +453,18 @@
], ],
"version": "==1.1.1" "version": "==1.1.1"
}, },
"memory-profiler": {
"hashes": [
"sha256:5fa47b274c929dd2cbcd9190afb62fec110701251d2ac2d301caaf545c81afc1"
],
"index": "pypi",
"version": "==0.55.0"
},
"neobolt": { "neobolt": {
"hashes": [ "hashes": [
"sha256:fa9efe4a4defbdc63fc3f1e552d503727049586c59d8a3acf5188a2cf1a45dce" "sha256:56b86b8b2c3facdd54589e60ecd22e0234d6f40645ab2e2cf87ef0cd79df20af"
], ],
"version": "==1.7.13" "version": "==1.7.15"
}, },
"neotime": { "neotime": {
"hashes": [ "hashes": [
@ -489,49 +483,63 @@
}, },
"packaging": { "packaging": {
"hashes": [ "hashes": [
"sha256:a7ac867b97fdc07ee80a8058fe4435ccd274ecc3b0ed61d852d7d53055528cf9", "sha256:28b924174df7a2fa32c1953825ff29c61e2f5e082343165438812f00d3a7fc47",
"sha256:c491ca87294da7cc01902edbe30a5bc6c4c28172b5138ab4e4aa1b9d7bfaeafe" "sha256:d9551545c6d761f3def1677baf08ab2a3ca17c56879e70fecba2fc4dde4ed108"
], ],
"version": "==19.1" "version": "==19.2"
}, },
"pillow": { "pillow": {
"hashes": [ "hashes": [
"sha256:0804f77cb1e9b6dbd37601cee11283bba39a8d44b9ddb053400c58e0c0d7d9de", "sha256:00fdeb23820f30e43bba78eb9abb00b7a937a655de7760b2e09101d63708b64e",
"sha256:0ab7c5b5d04691bcbd570658667dd1e21ca311c62dcfd315ad2255b1cd37f64f", "sha256:01f948e8220c85eae1aa1a7f8edddcec193918f933fb07aaebe0bfbbcffefbf1",
"sha256:0b3e6cf3ea1f8cecd625f1420b931c83ce74f00c29a0ff1ce4385f99900ac7c4", "sha256:08abf39948d4b5017a137be58f1a52b7101700431f0777bec3d897c3949f74e6",
"sha256:365c06a45712cd723ec16fa4ceb32ce46ad201eb7bbf6d3c16b063c72b61a3ed", "sha256:099a61618b145ecb50c6f279666bbc398e189b8bc97544ae32b8fcb49ad6b830",
"sha256:38301fbc0af865baa4752ddae1bb3cbb24b3d8f221bf2850aad96b243306fa03", "sha256:2c1c61546e73de62747e65807d2cc4980c395d4c5600ecb1f47a650c6fa78c79",
"sha256:3aef1af1a91798536bbab35d70d35750bd2884f0832c88aeb2499aa2d1ed4992", "sha256:2ed9c4f694861642401f27dc3cb99772be67cd190e84845c749dae0a06c3bfae",
"sha256:3fe0ab49537d9330c9bba7f16a5f8b02da615b5c809cdf7124f356a0f182eccd", "sha256:338581b30b908e111be578f0297255f6b57a51358cd16fa0e6f664c9a1f88bff",
"sha256:45a619d5c1915957449264c81c008934452e3fd3604e36809212300b2a4dab68", "sha256:38c7d48a21cd06fdeee93987147b9b1c55b73b4cfcbf83240568bfbd5adee447",
"sha256:49f90f147883a0c3778fd29d3eb169d56416f25758d0f66775db9184debc8010", "sha256:43fd026f613c8e48a25eba1a92f4d2ad7f3903c95d8c33a11611a7717d2ab654",
"sha256:571b5a758baf1cb6a04233fb23d6cf1ca60b31f9f641b1700bfaab1194020555", "sha256:4548236844327a718ce3bb182ab32a16fa2050c61e334e959f554cac052fb0df",
"sha256:5ac381e8b1259925287ccc5a87d9cf6322a2dc88ae28a97fe3e196385288413f", "sha256:5090857876c58885cfa388dc649e5db30aae98a068c26f3fd0ac9d7d9a4d9572",
"sha256:6153db744a743c0c8c91b8e3b9d40e0b13a5d31dbf8a12748c6d9bfd3ddc01ad", "sha256:5bbba34f97a26a93f5e8dec469ca4ddd712451418add43da946dbaed7f7a98d2",
"sha256:6fd63afd14a16f5d6b408f623cc2142917a1f92855f0df997e09a49f0341be8a", "sha256:65a28969a025a0eb4594637b6103201dc4ed2a9508bdab56ac33e43e3081c404",
"sha256:70acbcaba2a638923c2d337e0edea210505708d7859b87c2bd81e8f9902ae826", "sha256:892bb52b70bd5ea9dbbc3ac44f38e84f5a04e9d8b1bff48159d96cb795b81159",
"sha256:70b1594d56ed32d56ed21a7fbb2a5c6fd7446cdb7b21e749c9791eac3a64d9e4", "sha256:8a9becd5cbd5062f973bcd2e7bc79483af310222de112b6541f8af1f93a3cc42",
"sha256:76638865c83b1bb33bcac2a61ce4d13c17dba2204969dedb9ab60ef62bede686", "sha256:972a7aaeb7c4a2795b52eef52ee991ef040b31009f36deca6207a986607b55f3",
"sha256:7b2ec162c87fc496aa568258ac88631a2ce0acfe681a9af40842fc55deaedc99", "sha256:97b119c436bfa96a92ac2ca525f7025836d4d4e64b1c9f9eff8dbaf3ff1d86f3",
"sha256:7cee2cef07c8d76894ebefc54e4bb707dfc7f258ad155bd61d87f6cd487a70ff", "sha256:9ba37698e242223f8053cc158f130aee046a96feacbeab65893dbe94f5530118",
"sha256:7d16d4498f8b374fc625c4037742fbdd7f9ac383fd50b06f4df00c81ef60e829", "sha256:b1b0e1f626a0f079c0d3696db70132fb1f29aa87c66aecb6501a9b8be64ce9f7",
"sha256:b50bc1780681b127e28f0075dfb81d6135c3a293e0c1d0211133c75e2179b6c0", "sha256:c14c1224fd1a5be2733530d648a316974dbbb3c946913562c6005a76f21ca042",
"sha256:bd0582f831ad5bcad6ca001deba4568573a4675437db17c4031939156ff339fa", "sha256:c79a8546c48ae6465189e54e3245a97ddf21161e33ff7eaa42787353417bb2b6",
"sha256:cfd40d8a4b59f7567620410f966bb1f32dc555b2b19f82a91b147fac296f645c", "sha256:ceb76935ac4ebdf6d7bc845482a4450b284c6ccfb281e34da51d510658ab34d8",
"sha256:e3ae410089de680e8f84c68b755b42bc42c0ceb8c03dbea88a5099747091d38e", "sha256:e22bffaad04b4d16e1c091baed7f2733fc1ebb91e0c602abf1b6834d17158b1f",
"sha256:e9046e559c299b395b39ac7dbf16005308821c2f24a63cae2ab173bd6aa11616", "sha256:ec883b8e44d877bda6f94a36313a1c6063f8b1997aa091628ae2f34c7f97c8d5",
"sha256:ef6be704ae2bc8ad0ebc5cb850ee9139493b0fc4e81abcc240fb392a63ebc808", "sha256:f1baa54d50ec031d1a9beb89974108f8f2c0706f49798f4777df879df0e1adb6",
"sha256:f8dc19d92896558f9c4317ee365729ead9d7bbcf2052a9a19a3ef17abbb8ac5b" "sha256:f53a5385932cda1e2c862d89460992911a89768c65d176ff8c50cddca4d29bed"
], ],
"version": "==6.1.0" "version": "==6.2.0"
}, },
"prompt-toolkit": { "prompt-toolkit": {
"hashes": [ "hashes": [
"sha256:11adf3389a996a6d45cc277580d0d53e8a5afd281d0c9ec71b28e6f121463780", "sha256:46642344ce457641f28fc9d1c9ca939b63dadf8df128b86f1b9860e59c73a5e4",
"sha256:2519ad1d8038fd5fc8e770362237ad0364d16a7650fb5724af6997ed5515e3c1", "sha256:e7f8af9e3d70f514373bf41aa51bc33af12a6db3f71461ea47fea985defb2c31",
"sha256:977c6583ae813a37dc1c2e1b715892461fcbdaa57f6fc62f33a528c4886c8f55" "sha256:f15af68f66e664eaa559d4ac8a928111eebd5feda0c11738b5998045224829db"
], ],
"version": "==2.0.9" "version": "==2.0.10"
},
"psutil": {
"hashes": [
"sha256:028a1ec3c6197eadd11e7b46e8cc2f0720dc18ac6d7aabdb8e8c0d6c9704f000",
"sha256:503e4b20fa9d3342bcf58191bbc20a4a5ef79ca7df8972e6197cc14c5513e73d",
"sha256:863a85c1c0a5103a12c05a35e59d336e1d665747e531256e061213e2e90f63f3",
"sha256:954f782608bfef9ae9f78e660e065bd8ffcfaea780f9f2c8a133bb7cb9e826d7",
"sha256:b6e08f965a305cd84c2d07409bc16fbef4417d67b70c53b299116c5b895e3f45",
"sha256:bc96d437dfbb8865fc8828cf363450001cb04056bbdcdd6fc152c436c8a74c61",
"sha256:cf49178021075d47c61c03c0229ac0c60d5e2830f8cab19e2d88e579b18cdb76",
"sha256:d5350cb66690915d60f8b233180f1e49938756fb2d501c93c44f8fb5b970cc63",
"sha256:eba238cf1989dfff7d483c029acb0ac4fcbfc15de295d682901f0e2497e6781a"
],
"version": "==5.6.3"
}, },
"py2neo": { "py2neo": {
"hashes": [ "hashes": [
@ -591,10 +599,10 @@
}, },
"pytz": { "pytz": {
"hashes": [ "hashes": [
"sha256:26c0b32e437e54a18161324a2fca3c4b9846b74a8dccddd843113109e1116b32", "sha256:1c557d7d0e871de1f5ccd5833f60fb2550652da6be2693c1e02300743d21500d",
"sha256:c894d57500a4cd2d5c71114aaab77dbab5eabd9022308ce5ac9bb93a60a6f0c7" "sha256:b02c06db6cf09c12dd25137e563b31700d3b80fcc4ad23abb7a315f2789819be"
], ],
"version": "==2019.2" "version": "==2019.3"
}, },
"recommonmark": { "recommonmark": {
"hashes": [ "hashes": [
@ -605,36 +613,31 @@
}, },
"reportlab": { "reportlab": {
"hashes": [ "hashes": [
"sha256:065bca611829da371df97cec255239a2972119afbab57528022df8b41881a3f6", "sha256:06b7c7436fa6d4844c7637161f3297c7a96240f35622ab2d219e4fd8387c0ab2",
"sha256:329843edd93293a96b99b2e9c226066a9ed27f0f881b4933536577e1dab898cf", "sha256:0a5acf67bd9812e38ed84be8994c07a8136b0a8f4c14a1c66c9c73a9567a9a44",
"sha256:393140710488b7ffda2762a08f63671dcccdbccfed0e4c8e8ec77e5a355080a1", "sha256:1c8ca145d03e3c620866b06febb241b179197b58fb07454fbc8e9d6184cdcc93",
"sha256:3c778843f50981a1569539120f0cfa2be0ca7a80e4c61bdfc88a74c323b90b00", "sha256:2f8d785660ee316874c86abad345633ce8c652e88e03ae8a10f1fdadc72fd23d",
"sha256:44ab0741f40899936e7cc85b0a19614a483da4b476102ac58d1ac20ef6da9fc3", "sha256:4869d342352c92a812ce40555ef2a9cfbd722390d67fe61f1d6ec770e9ca41a3",
"sha256:4582272135bd2f355a616b4ac08310947d88b0d3e4f474be16175d89fa200c0d", "sha256:493e0dcd9c085d46acf4fe3f00f941e562490a74b651409039a0dee2a0d76555",
"sha256:47612270365e21581178ebbb91edabf9b3c6b4519baf2052d3f4cbe302e3ea76", "sha256:4e606e3ee9345e68cd205022d526250ad2a1164eea8f1e29d77d6ad08631b0ba",
"sha256:4f8c5e65fcfa111be309228efca92ba17f329d3dbf3bbe055094fe907ab5d4c8", "sha256:5bf91bae8995db91650fda658129c268515358b756fd16c0261a9dd641df1856",
"sha256:4ff4942cb1ca1f70a890fd35c7e1d0657d08dbdf6bdb5bc2c0dd3e30a6301cf7", "sha256:6df0730f8f715aa12333bd6d2a72eea3a989381451861186d9b5e71889454ac7",
"sha256:5b109b347ae391963ef846e41c4c65c2bc99e81f1d4eeff687635b73ee952bf5", "sha256:7195c6ea096d10c91cc470f9f0ced3ad74470d9c0fd97923b5e764597dd13671",
"sha256:5cbd56e8dea652f73f728578cb3dbc57bd100f308012fe90596085520d2cb25a", "sha256:7431c979e2b498e8e20abf458f360a451717d76c3c1bd49d1fc5697d3504f8e5",
"sha256:5dddc51b5848a2d0a6fe47e96496220a305e7d796d4a6973cc984ab1d8160ff7", "sha256:7f7f70a8d4b573d1ff65a81415b4b6ed9545630f381dff4a69307640e09d381d",
"sha256:6c81ee26753fa09062d8404f6340eefb02849608b619e3843e0d17a7cda8798f", "sha256:9945433667a46f054d1125b4ca86fe9ee31feb254728b38242e9a6008c135efe",
"sha256:706ffb184c4cdeabcaef3b9eaba86cbf7684467c32d308ed908917fc679f86c8", "sha256:b1cdbfc1fd54ac947b9f0114e00ab94e945db679f1e03357a3c00f3a85e73eea",
"sha256:794499adc5ad419e064523f13b0782ee2860180e79c8cd02379c4c957e1f0abb", "sha256:bf149847a2fd8f24b788a8abbf97a2b9a73edc5b1bd719384b786eb84bcad15e",
"sha256:8b7fcc98b0aed3e3e4f134f4d5a498bb9c068fdce6c6b2a9f103d3a339efd8d1", "sha256:ce514bfce2bf3e302f52aba9929fe3ac7d918cfea2f5d3e30bf9dac9658bf094",
"sha256:8bc0fe11be68207866902ee96eec6645d574d82fd6abd93c8bcdcd57ac1b4040", "sha256:d243d4c8cf1a7e78b734c03628b684ec5de25df1f02ccea2e10fbd217430cb72",
"sha256:92f01e16fe65e51ffa2fe0e37da697c8b8f5d892605c05394c883a866a11efc1", "sha256:d4bee20f52b8c3c477dc780780654cafcfc0eb34d8d6960c13a34a444b431f09",
"sha256:a162484b22c52ab701b74f8c35b2a14f9ecf9694f2ab149fb38f377069743e69", "sha256:e730529bd1f62034c50f70a2b05fadbf7d1402d39ff69c9dc63db066d0ef8a46",
"sha256:a30b42d6c5ffe1ce7c677328a47386f861c3bb9057bf4de5eb0f97fe17e9b3ba", "sha256:eb54ecfbf1abe6134073b7b35fd40442c4cd81bb9a5bee1a3038b8867b721bfb",
"sha256:a7a63d35c59af1d134ec43bab75070af86e59c412289198de3788765627a611c", "sha256:f18ec70f5ee6a78b3bb4361e55f3a5ef34eb253f1e72fba76f29f0d680cd446f",
"sha256:aee6aa362cbaf9abc406944064a887a69f6f5606fa54abaecf98a78459d1d954", "sha256:f6be66f69198dcd04a79faa6052f756d35643496321858f06931c7b1ed9833ab",
"sha256:ba537b091614f3839716fb7b418e157216e213a0eab3fe7db2dfbf198fb61224", "sha256:fc5c23a53fbd97b8aab4968c8548ce5cea4a54a26b4f8c1e6835df7adb8d0fe2"
"sha256:be8f70ec622b98ef830af5591ab4c0b062a67507a19ca43327da5ff350435b43",
"sha256:c380bcb032736d45bd9a90f4208547a679b7fe2327fc1187a73a2d9b58988f1d",
"sha256:cd2fdcd1e31113878d5c5c9ae17a34368a13e1c9e12d586b66b77ff806371e23",
"sha256:f59d772b504035b1468544a11269ee27648ddb2fae1efddd45ce050da2527813",
"sha256:ff1570bf8ad010c408f72822248ad2276185d473ab9a64c70ad2ec4427dda052"
], ],
"version": "==3.5.23" "version": "==3.5.28"
}, },
"requests": { "requests": {
"hashes": [ "hashes": [
@ -660,16 +663,17 @@
}, },
"snowballstemmer": { "snowballstemmer": {
"hashes": [ "hashes": [
"sha256:713e53b79cbcf97bc5245a06080a33d54a77e7cce2f789c835a143bcdb5c033e" "sha256:209f257d7533fdb3cb73bdbd24f436239ca3b2fa67d56f6ff88e86be08cc5ef0",
"sha256:df3bac3df4c2c01363f3dd2cfa78cce2840a79b9f1c2d2de9ce8d31683992f52"
], ],
"version": "==1.9.1" "version": "==2.0.0"
}, },
"soupsieve": { "soupsieve": {
"hashes": [ "hashes": [
"sha256:8662843366b8d8779dec4e2f921bebec9afd856a5ff2e82cd419acc5054a1a92", "sha256:605f89ad5fdbfefe30cdc293303665eff2d188865d4dbe4eb510bba1edfbfce3",
"sha256:a5a6166b4767725fd52ae55fee8c8b6137d9a51e9f1edea461a062a759160118" "sha256:b91d676b330a0ebd5b21719cb6e9b57c57d433671f65b9c28dd3461d9a1ed0b6"
], ],
"version": "==1.9.3" "version": "==1.9.4"
}, },
"sphinx": { "sphinx": {
"hashes": [ "hashes": [
@ -680,10 +684,10 @@
}, },
"sphinx-autodoc-typehints": { "sphinx-autodoc-typehints": {
"hashes": [ "hashes": [
"sha256:8eb1e2bc248d316a9faeca086c6133623f6d45770e342738158249356989b95c", "sha256:0d968ec3ee4f7fe7695ab6facf5cd2d74d3cea67584277458ad9b2788ebbcc3b",
"sha256:cedf37dde99096e3024ffcd498ee917c2ccf667e04e23d868d481eae2cb84910" "sha256:8edca714fd3de8e43467d7e51dd3812fe999f8874408a639f7c38a9e1a5a4eb3"
], ],
"version": "==1.7.0" "version": "==1.8.0"
}, },
"sphinxcontrib-applehelp": { "sphinxcontrib-applehelp": {
"hashes": [ "hashes": [
@ -729,10 +733,10 @@
}, },
"urllib3": { "urllib3": {
"hashes": [ "hashes": [
"sha256:b246607a25ac80bedac05c6f282e3cdaf3afb65420fd024ac94435cabe6e18d1", "sha256:3de946ffbed6e6746608990594d08faac602528ac7015ac28d33cee6a45b7398",
"sha256:dbe59173209418ae49d485b87d1681aefa36252ee85884c31346debd19463232" "sha256:9a107b99a5393caf59c7aa3c1249c16e6879447533d0887f4336dde834c7be86"
], ],
"version": "==1.25.3" "version": "==1.25.6"
}, },
"validators": { "validators": {
"hashes": [ "hashes": [

18
docs/tutorial/FullOverview.ipynb
View File

@ -59,11 +59,11 @@
"source": [ "source": [
"# Using the PyMISP objects\n", "# Using the PyMISP objects\n",
"\n", "\n",
"This page aims to give recommandations about how to efficiently use the `pymisp` library.\n", "This page aims to give recommendations about how to efficiently use the `pymisp` library.\n",
"\n", "\n",
"It is strongly recommended (read \"don't do anything else, please\") to use the library this way and never, ever modify the python dictionary you get by loading the json blob you receive from the server.\n", "It is strongly recommended (read \"don't do anything else, please\") to use the library this way and never, ever modify the python dictionary you get by loading the json blob you receive from the server.\n",
"\n", "\n",
"This library is made in a way to hide as much as the complexity as possible and we're happy to improve it is there is someting missing." "This library is made in a way to hide as much as the complexity as possible and we're happy to improve it is there is something missing."
] ]
}, },
{ {
@ -121,7 +121,7 @@
"## Set the Event date\n", "## Set the Event date\n",
"\n", "\n",
"\n", "\n",
"The date can be in many different formats. This helper makes sure it normalises it in a way that will be understood by your MISP instance." "The date can be in many different formats. This helper makes sure it normalizes it in a way that will be understood by your MISP instance."
] ]
}, },
{ {
@ -145,7 +145,7 @@
"event.set_date(d)\n", "event.set_date(d)\n",
"print(event.date)\n", "print(event.date)\n",
"\n", "\n",
"# datetime.datetime => MISP expects a day, so the hour will be droped.\n", "# datetime.datetime => MISP expects a day, so the hour will be dropped.\n",
"from datetime import datetime\n", "from datetime import datetime\n",
"d = datetime.now()\n", "d = datetime.now()\n",
"print(type(d))\n", "print(type(d))\n",
@ -159,7 +159,7 @@
"source": [ "source": [
"## Add Attribute to event\n", "## Add Attribute to event\n",
"\n", "\n",
"More usefull things: adding attributes to an event.\n", "More useful things: adding attributes to an event.\n",
"\n", "\n",
"Attributes have a bunch of parameters you can pass (if you feel like it). If you don't pass them, they'll be automatically set depending on their sane defaults.\n", "Attributes have a bunch of parameters you can pass (if you feel like it). If you don't pass them, they'll be automatically set depending on their sane defaults.\n",
"\n", "\n",
@ -263,7 +263,7 @@
"source": [ "source": [
"## Soft delete attribute\n", "## Soft delete attribute\n",
"\n", "\n",
"**Important note**: the default approach to *delete* on MISP is to do a soft delete (meaning the attribue is not displayed on the default view on MISP). The reason we do it this way is that it allows to push *delete* updates to instances we synchronize with.\n", "**Important note**: the default approach to *delete* on MISP is to do a soft delete (meaning the attribute is not displayed on the default view on MISP). The reason we do it this way is that it allows to push *delete* updates to instances we synchronize with.\n",
"\n", "\n",
"The delete method will set the default parameter of the attribute to `True`." "The delete method will set the default parameter of the attribute to `True`."
] ]
@ -533,7 +533,7 @@
"source": [ "source": [
"## Generic helper\n", "## Generic helper\n",
"\n", "\n",
"This helper is meant to be used when you alreadu have a script that does the mapping between your own code, and the MISPObject template." "This helper is meant to be used when you already have a script that does the mapping between your own code, and the MISPObject template."
] ]
}, },
{ {
@ -746,7 +746,7 @@
"source": [ "source": [
"## Edit, removes the timestamp when exporting\n", "## Edit, removes the timestamp when exporting\n",
"\n", "\n",
"If you tried to edit an event manually, and never got the updates on the instance, it is probably because the timestamps weren't updated/removed. Or you removed them all, and adding a single tag was makting every attributes as new.\n", "If you tried to edit an event manually, and never got the updates on the instance, it is probably because the timestamps weren't updated/removed. Or you removed them all, and adding a single tag was making every attributes as new.\n",
"\n", "\n",
"PyMISP got you covered." "PyMISP got you covered."
] ]
@ -803,7 +803,7 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"# Getting the API key (automatically generated on the trainig VM)" "# Getting the API key (automatically generated on the training VM)"
] ]
}, },
{ {

2
examples/feed-generator-from-redis/generator.py
View File

@ -281,7 +281,7 @@ class FeedGenerator:
# reference org # reference org
org_dict = {} org_dict = {}
org_dict['name'] = settings.org_name org_dict['name'] = settings.org_name
org_dict['uui'] = settings.org_uuid org_dict['uuid'] = settings.org_uuid
event['Orgc'] = org_dict event['Orgc'] = org_dict
# save event on disk # save event on disk

5
examples/generate_file_objects.py
View File

@ -5,7 +5,7 @@ import argparse
import json import json
try: try:
from pymisp import MISPEncode, AbstractMISP from pymisp import pymisp_json_default, AbstractMISP
from pymisp.tools import make_binary_objects from pymisp.tools import make_binary_objects
except ImportError: except ImportError:
pass pass
@ -51,7 +51,8 @@ def make_objects(path):
to_return['objects'].append(fo) to_return['objects'].append(fo)
if fo.ObjectReference: if fo.ObjectReference:
to_return['references'] += fo.ObjectReference to_return['references'] += fo.ObjectReference
return json.dumps(to_return, cls=MISPEncode) return json.dumps(to_return, default=pymisp_json_default)
if __name__ == '__main__': if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Extract indicators out of binaries and returns MISP objects.') parser = argparse.ArgumentParser(description='Extract indicators out of binaries and returns MISP objects.')

32
examples/server_sync_check_conn.py Executable file
View File

@ -0,0 +1,32 @@
#!/usr/bin/env python
import requests
import json
# Suppress those "Unverified HTTPS request is being made"
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
from keys import misp_url, misp_key, misp_verifycert
proxies = {
}
'''
Checks if the connection to a sync server works
returns json object
'''
def check_connection(connection_number):
misp_headers = {'Content-Type': 'application/json', 'Accept': 'application/json', 'Authorization': misp_key}
req = requests.get(misp_url + 'servers/testConnection/{}'.format(connection_number), verify=misp_verifycert, headers=misp_headers, proxies=proxies)
result = json.loads(req.text)
return(result)
if __name__ == "__main__":
result = check_connection(1)
print(result)

6
pymisp/__init__.py
View File

@ -1,4 +1,4 @@
__version__ = '2.4.114' __version__ = '2.4.117.1'
import logging import logging
import warnings import warnings
import sys import sys
@ -31,8 +31,8 @@ try:
warning_2020() warning_2020()
from .exceptions import PyMISPError, NewEventError, NewAttributeError, MissingDependency, NoURL, NoKey, InvalidMISPObject, UnknownMISPObjectTemplate, PyMISPInvalidFormat, MISPServerError, PyMISPNotImplementedYet, PyMISPUnexpectedResponse, PyMISPEmptyResponse # noqa from .exceptions import PyMISPError, NewEventError, NewAttributeError, MissingDependency, NoURL, NoKey, InvalidMISPObject, UnknownMISPObjectTemplate, PyMISPInvalidFormat, MISPServerError, PyMISPNotImplementedYet, PyMISPUnexpectedResponse, PyMISPEmptyResponse # noqa
from .api import PyMISP # noqa from .api import PyMISP # noqa
from .abstract import AbstractMISP, MISPEncode, MISPTag, Distribution, ThreatLevel, Analysis # noqa from .abstract import AbstractMISP, MISPEncode, pymisp_json_default, MISPTag, Distribution, ThreatLevel, Analysis # noqa
from .mispevent import MISPEvent, MISPAttribute, MISPObjectReference, MISPObjectAttribute, MISPObject, MISPUser, MISPOrganisation, MISPSighting, MISPLog, MISPShadowAttribute, MISPWarninglist, MISPTaxonomy, MISPNoticelist, MISPObjectTemplate, MISPSharingGroup, MISPRole, MISPServer, MISPFeed, MISPEventDelegation # noqa from .mispevent import MISPEvent, MISPAttribute, MISPObjectReference, MISPObjectAttribute, MISPObject, MISPUser, MISPOrganisation, MISPSighting, MISPLog, MISPShadowAttribute, MISPWarninglist, MISPTaxonomy, MISPNoticelist, MISPObjectTemplate, MISPSharingGroup, MISPRole, MISPServer, MISPFeed, MISPEventDelegation, MISPUserSetting # noqa
from .tools import AbstractMISPObjectGenerator # noqa from .tools import AbstractMISPObjectGenerator # noqa
from .tools import Neo4j # noqa from .tools import Neo4j # noqa
from .tools import stix # noqa from .tools import stix # noqa

186
pymisp/abstract.py
View File

@ -3,24 +3,41 @@
import sys import sys
import datetime import datetime
import json
from deprecated import deprecated
from json import JSONEncoder from json import JSONEncoder
from uuid import UUID
try:
from rapidjson import load
from rapidjson import loads
from rapidjson import dumps
import rapidjson
HAS_RAPIDJSON = True
except ImportError:
from json import load
from json import loads
from json import dumps
import json
HAS_RAPIDJSON = False
import logging import logging
from enum import Enum from enum import Enum
from .exceptions import PyMISPInvalidFormat from .exceptions import PyMISPInvalidFormat
# Try to import MutableMapping the python 3.3+ way
try:
from collections.abc import MutableMapping
except Exception:
pass
logger = logging.getLogger('pymisp') logger = logging.getLogger('pymisp')
if sys.version_info < (3, 0): if sys.version_info < (3, 0):
from collections import MutableMapping from collections import MutableMapping
import os
from cachetools import cached, LRUCache
resources_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data')
misp_objects_path = os.path.join(resources_path, 'misp-objects', 'objects')
with open(os.path.join(resources_path, 'describeTypes.json'), 'r') as f:
describe_types = load(f)['result']
# This is required because Python 2 is a pain. # This is required because Python 2 is a pain.
from datetime import tzinfo, timedelta from datetime import tzinfo, timedelta
@ -37,6 +54,62 @@ if sys.version_info < (3, 0):
def dst(self, dt): def dst(self, dt):
return timedelta(0) return timedelta(0)
class MISPFileCache(object):
# cache up to 150 JSON structures in class attribute
@staticmethod
@cached(cache=LRUCache(maxsize=150))
def _load_json(path):
if not os.path.exists(path):
return None
with open(path, 'r') as f:
data = load(f)
return data
elif sys.version_info < (3, 4):
from collections.abc import MutableMapping
from functools import lru_cache
import os
resources_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data')
misp_objects_path = os.path.join(resources_path, 'misp-objects', 'objects')
with open(os.path.join(resources_path, 'describeTypes.json'), 'r') as f:
describe_types = load(f)['result']
class MISPFileCache(object):
# cache up to 150 JSON structures in class attribute
@staticmethod
@lru_cache(maxsize=150)
def _load_json(path):
if not os.path.exists(path):
return None
with open(path, 'r') as f:
data = load(f)
return data
else:
from collections.abc import MutableMapping
from functools import lru_cache
from pathlib import Path
resources_path = Path(__file__).parent / 'data'
misp_objects_path = resources_path / 'misp-objects' / 'objects'
with (resources_path / 'describeTypes.json').open('r') as f:
describe_types = load(f)['result']
class MISPFileCache(object):
# cache up to 150 JSON structures in class attribute
@staticmethod
@lru_cache(maxsize=150)
def _load_json(path):
if not path.exists():
return None
with path.open('r') as f:
data = load(f)
return data
class Distribution(Enum): class Distribution(Enum):
your_organisation_only = 0 your_organisation_only = 0
@ -68,8 +141,8 @@ def _int_to_str(d):
return d return d
@deprecated(reason=" Use method default=pymisp_json_default instead of cls=MISPEncode", version='2.4.117', action='default')
class MISPEncode(JSONEncoder): class MISPEncode(JSONEncoder):
def default(self, obj): def default(self, obj):
if isinstance(obj, AbstractMISP): if isinstance(obj, AbstractMISP):
return obj.jsonable() return obj.jsonable()
@ -77,16 +150,46 @@ class MISPEncode(JSONEncoder):
return obj.isoformat() return obj.isoformat()
elif isinstance(obj, Enum): elif isinstance(obj, Enum):
return obj.value return obj.value
elif isinstance(obj, UUID):
return str(obj)
return JSONEncoder.default(self, obj) return JSONEncoder.default(self, obj)
class AbstractMISP(MutableMapping): if HAS_RAPIDJSON:
def pymisp_json_default(obj):
if isinstance(obj, AbstractMISP):
return obj.jsonable()
elif isinstance(obj, (datetime.datetime, datetime.date)):
return obj.isoformat()
elif isinstance(obj, Enum):
return obj.value
elif isinstance(obj, UUID):
return str(obj)
return rapidjson.default(obj)
else:
def pymisp_json_default(obj):
if isinstance(obj, AbstractMISP):
return obj.jsonable()
elif isinstance(obj, (datetime.datetime, datetime.date)):
return obj.isoformat()
elif isinstance(obj, Enum):
return obj.value
elif isinstance(obj, UUID):
return str(obj)
return json.default(obj)
class AbstractMISP(MutableMapping, MISPFileCache):
__resources_path = resources_path
__misp_objects_path = misp_objects_path
__describe_types = describe_types
def __init__(self, **kwargs): def __init__(self, **kwargs):
"""Abstract class for all the MISP objects""" """Abstract class for all the MISP objects"""
super(AbstractMISP, self).__init__() super(AbstractMISP, self).__init__()
self.__edited = True # As we create a new object, we assume it is edited self.__edited = True # As we create a new object, we assume it is edited
self.__not_jsonable = [] self.__not_jsonable = []
self.__self_defined_describe_types = None
if kwargs.get('force_timestamps') is not None: if kwargs.get('force_timestamps') is not None:
# Ignore the edited objects and keep the timestamps. # Ignore the edited objects and keep the timestamps.
@ -103,16 +206,28 @@ class AbstractMISP(MutableMapping):
setattr(AbstractMISP, 'tags', property(AbstractMISP.__get_tags, AbstractMISP.__set_tags)) setattr(AbstractMISP, 'tags', property(AbstractMISP.__get_tags, AbstractMISP.__set_tags))
@property @property
def properties(self): def describe_types(self):
"""All the class public properties that will be dumped in the dictionary, and the JSON export. if self.__self_defined_describe_types:
Note: all the properties starting with a `_` (private), or listed in __not_jsonable will be skipped. return self.__self_defined_describe_types
""" return self.__describe_types
to_return = []
for prop, value in vars(self).items(): @describe_types.setter
if prop.startswith('_') or prop in self.__not_jsonable: def describe_types(self, describe_types):
continue self.__self_defined_describe_types = describe_types
to_return.append(prop)
return to_return @property
def resources_path(self):
return self.__resources_path
@property
def misp_objects_path(self):
return self.__misp_objects_path
@misp_objects_path.setter
def misp_objects_path(self, misp_objects_path):
if sys.version_info >= (3, 0) and isinstance(misp_objects_path, str):
misp_objects_path = Path(misp_objects_path)
self.__misp_objects_path = misp_objects_path
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
"""Loading all the parameters as class properties, if they aren't `None`. """Loading all the parameters as class properties, if they aren't `None`.
@ -137,21 +252,21 @@ class AbstractMISP(MutableMapping):
def from_json(self, json_string): def from_json(self, json_string):
"""Load a JSON string""" """Load a JSON string"""
self.from_dict(**json.loads(json_string)) self.from_dict(**loads(json_string))
def to_dict(self): def to_dict(self):
"""Dump the lass to a dictionary. """Dump the class to a dictionary.
This method automatically removes the timestamp recursively in every object This method automatically removes the timestamp recursively in every object
that has been edited is order to let MISP update the event accordingly.""" that has been edited is order to let MISP update the event accordingly."""
is_edited = self.edited
to_return = {} to_return = {}
for attribute in self.properties: for attribute, val in self.items():
val = getattr(self, attribute, None)
if val is None: if val is None:
continue continue
elif isinstance(val, list) and len(val) == 0: elif isinstance(val, list) and len(val) == 0:
continue continue
if attribute == 'timestamp': if attribute == 'timestamp':
if not self.__force_timestamps and self.edited: if not self.__force_timestamps and is_edited:
# In order to be accepted by MISP, the timestamp of an object # In order to be accepted by MISP, the timestamp of an object
# needs to be either newer, or None. # needs to be either newer, or None.
# If the current object is marked as edited, the easiest is to # If the current object is marked as edited, the easiest is to
@ -167,13 +282,15 @@ class AbstractMISP(MutableMapping):
"""This method is used by the JSON encoder""" """This method is used by the JSON encoder"""
return self.to_dict() return self.to_dict()
def to_json(self): def to_json(self, sort_keys=False, indent=None):
"""Dump recursively any class of type MISPAbstract to a json string""" """Dump recursively any class of type MISPAbstract to a json string"""
return json.dumps(self, cls=MISPEncode, sort_keys=True, indent=2) return dumps(self, default=pymisp_json_default, sort_keys=sort_keys, indent=indent)
def __getitem__(self, key): def __getitem__(self, key):
try: try:
return getattr(self, key) if key[0] != '_' and key not in self.__not_jsonable:
return self.__dict__[key]
raise KeyError
except AttributeError: except AttributeError:
# Expected by pop and other dict-related methods # Expected by pop and other dict-related methods
raise KeyError raise KeyError
@ -185,10 +302,10 @@ class AbstractMISP(MutableMapping):
delattr(self, key) delattr(self, key)
def __iter__(self): def __iter__(self):
return iter(self.to_dict()) return iter({k: v for k, v in self.__dict__.items() if not (k[0] == '_' or k in self.__not_jsonable)})
def __len__(self): def __len__(self):
return len(self.to_dict()) return len([k for k in self.__dict__.keys() if not (k[0] == '_' or k in self.__not_jsonable)])
@property @property
def edited(self): def edited(self):
@ -196,15 +313,14 @@ class AbstractMISP(MutableMapping):
to the parent objects""" to the parent objects"""
if self.__edited: if self.__edited:
return self.__edited return self.__edited
for p in self.properties: for p, val in self.items():
if self.__edited:
break
val = getattr(self, p)
if isinstance(val, AbstractMISP) and val.edited: if isinstance(val, AbstractMISP) and val.edited:
self.__edited = True self.__edited = True
break
elif isinstance(val, list) and all(isinstance(a, AbstractMISP) for a in val): elif isinstance(val, list) and all(isinstance(a, AbstractMISP) for a in val):
if any(a.edited for a in val): if any(a.edited for a in val):
self.__edited = True self.__edited = True
break
return self.__edited return self.__edited
@edited.setter @edited.setter
@ -216,7 +332,9 @@ class AbstractMISP(MutableMapping):
raise Exception('edited can only be True or False') raise Exception('edited can only be True or False')
def __setattr__(self, name, value): def __setattr__(self, name, value):
if name in self.properties: if name[0] != '_' and not self.__edited and name in self.keys():
# The private members don't matter
# If we already have a key with that name, we're modifying it.
self.__edited = True self.__edited = True
super(AbstractMISP, self).__setattr__(name, value) super(AbstractMISP, self).__setattr__(name, value)

32
pymisp/api.py
View File

@ -19,7 +19,7 @@ from deprecated import deprecated
from . import __version__, warning_2020 from . import __version__, warning_2020
from .exceptions import PyMISPError, SearchError, NoURL, NoKey, PyMISPEmptyResponse from .exceptions import PyMISPError, SearchError, NoURL, NoKey, PyMISPEmptyResponse
from .mispevent import MISPEvent, MISPAttribute, MISPUser, MISPOrganisation, MISPSighting, MISPFeed, MISPObject, MISPSharingGroup from .mispevent import MISPEvent, MISPAttribute, MISPUser, MISPOrganisation, MISPSighting, MISPFeed, MISPObject, MISPSharingGroup
from .abstract import AbstractMISP, MISPEncode from .abstract import AbstractMISP, pymisp_json_default, describe_types
logger = logging.getLogger('pymisp') logger = logging.getLogger('pymisp')
@ -37,11 +37,6 @@ try:
except ImportError: except ImportError:
HAVE_REQUESTS = False HAVE_REQUESTS = False
if (3, 0) <= sys.version_info < (3, 6):
OLD_PY3 = True
else:
OLD_PY3 = False
try: try:
from requests_futures.sessions import FuturesSession from requests_futures.sessions import FuturesSession
ASYNC_OK = True ASYNC_OK = True
@ -140,24 +135,19 @@ class PyMISP(object): # pragma: no cover
@deprecated(reason="Use ExpandedPyMISP.describe_types_local", version='2.4.110', action='default') @deprecated(reason="Use ExpandedPyMISP.describe_types_local", version='2.4.110', action='default')
def get_local_describe_types(self): def get_local_describe_types(self):
with open(os.path.join(self.resources_path, 'describeTypes.json'), 'rb') as f: return describe_types
if OLD_PY3:
describe_types = json.loads(f.read().decode())
else:
describe_types = json.load(f)
return describe_types['result']
@deprecated(reason="Use ExpandedPyMISP.describe_types_remote", version='2.4.110', action='default') @deprecated(reason="Use ExpandedPyMISP.describe_types_remote", version='2.4.110', action='default')
def get_live_describe_types(self): def get_live_describe_types(self):
response = self._prepare_request('GET', urljoin(self.root_url, 'attributes/describeTypes.json')) response = self._prepare_request('GET', urljoin(self.root_url, 'attributes/describeTypes.json'))
describe_types = self._check_response(response) remote_describe_types = self._check_response(response)
if describe_types.get('error'): if remote_describe_types.get('error'):
for e in describe_types.get('error'): for e in remote_describe_types.get('error'):
raise PyMISPError('Failed: {}'.format(e)) raise PyMISPError('Failed: {}'.format(e))
describe_types = describe_types['result'] remote_describe_types = remote_describe_types['result']
if not describe_types.get('sane_defaults'): if not remote_describe_types.get('sane_defaults'):
raise PyMISPError('The MISP server your are trying to reach is outdated (<2.4.52). Please use PyMISP v2.4.51.1 (pip install -I PyMISP==v2.4.51.1) and/or contact your administrator.') raise PyMISPError('The MISP server your are trying to reach is outdated (<2.4.52). Please use PyMISP v2.4.51.1 (pip install -I PyMISP==v2.4.51.1) and/or contact your administrator.')
return describe_types return remote_describe_types
def _prepare_request(self, request_type, url, data=None, def _prepare_request(self, request_type, url, data=None,
background_callback=None, output_type='json'): background_callback=None, output_type='json'):
@ -172,7 +162,7 @@ class PyMISP(object): # pragma: no cover
if isinstance(data, dict): if isinstance(data, dict):
# Remove None values. # Remove None values.
data = {k: v for k, v in data.items() if v is not None} data = {k: v for k, v in data.items() if v is not None}
data = json.dumps(data, cls=MISPEncode) data = json.dumps(data, default=pymisp_json_default)
req = requests.Request(request_type, url, data=data) req = requests.Request(request_type, url, data=data)
if self.asynch and background_callback is not None: if self.asynch and background_callback is not None:
local_session = FuturesSession local_session = FuturesSession
@ -614,7 +604,7 @@ class PyMISP(object): # pragma: no cover
else: else:
data = attributes.to_json() data = attributes.to_json()
# _prepare_request(...) returns a requests.Response Object # _prepare_request(...) returns a requests.Response Object
resp = self._prepare_request('POST', url, json.dumps(data, cls=MISPEncode)) resp = self._prepare_request('POST', url, json.dumps(data, default=pymisp_json_default))
try: try:
responses.append(resp.json()) responses.append(resp.json())
except Exception: except Exception:
@ -1068,7 +1058,7 @@ class PyMISP(object): # pragma: no cover
url = urljoin(self.root_url, 'shadow_attributes/{}/{}'.format(path, id)) url = urljoin(self.root_url, 'shadow_attributes/{}/{}'.format(path, id))
if path in ['add', 'edit']: if path in ['add', 'edit']:
query = {'request': {'ShadowAttribute': attribute}} query = {'request': {'ShadowAttribute': attribute}}
response = self._prepare_request('POST', url, json.dumps(query, cls=MISPEncode)) response = self._prepare_request('POST', url, json.dumps(query, default=pymisp_json_default))
elif path == 'view': elif path == 'view':
response = self._prepare_request('GET', url) response = self._prepare_request('GET', url)
else: # accept or discard else: # accept or discard

131
pymisp/aping.py
View File

@ -18,8 +18,11 @@ import sys
from . import __version__ from . import __version__
from .exceptions import MISPServerError, PyMISPUnexpectedResponse, PyMISPNotImplementedYet, PyMISPError, NoURL, NoKey from .exceptions import MISPServerError, PyMISPUnexpectedResponse, PyMISPNotImplementedYet, PyMISPError, NoURL, NoKey
from .api import everything_broken, PyMISP from .api import everything_broken, PyMISP
from .mispevent import MISPEvent, MISPAttribute, MISPSighting, MISPLog, MISPObject, MISPUser, MISPOrganisation, MISPShadowAttribute, MISPWarninglist, MISPTaxonomy, MISPGalaxy, MISPNoticelist, MISPObjectReference, MISPObjectTemplate, MISPSharingGroup, MISPRole, MISPServer, MISPFeed, MISPEventDelegation, MISPCommunity from .mispevent import MISPEvent, MISPAttribute, MISPSighting, MISPLog, MISPObject, \
from .abstract import MISPEncode, MISPTag, AbstractMISP MISPUser, MISPOrganisation, MISPShadowAttribute, MISPWarninglist, MISPTaxonomy, \
MISPGalaxy, MISPNoticelist, MISPObjectReference, MISPObjectTemplate, MISPSharingGroup, \
MISPRole, MISPServer, MISPFeed, MISPEventDelegation, MISPCommunity, MISPUserSetting
from .abstract import pymisp_json_default, MISPTag, AbstractMISP, describe_types
SearchType = TypeVar('SearchType', str, int) SearchType = TypeVar('SearchType', str, int)
# str: string to search / list: values to search (OR) / dict: {'OR': [list], 'NOT': [list], 'AND': [list]} # str: string to search / list: values to search (OR) / dict: {'OR': [list], 'NOT': [list], 'AND': [list]}
@ -78,11 +81,14 @@ class ExpandedPyMISP(PyMISP):
if recommended_version_tup < pymisp_version_tup[:3]: if recommended_version_tup < pymisp_version_tup[:3]:
logger.info(f"The version of PyMISP recommended by the MISP instance (response['version']) is older than the one you're using now ({__version__}). If you have a problem, please upgrade the MISP instance or use an older PyMISP version.") logger.info(f"The version of PyMISP recommended by the MISP instance (response['version']) is older than the one you're using now ({__version__}). If you have a problem, please upgrade the MISP instance or use an older PyMISP version.")
elif pymisp_version_tup[:3] < recommended_version_tup: elif pymisp_version_tup[:3] < recommended_version_tup:
logger.warning(f"The version of PyMISP recommended by the MI)SP instance ({response['version']}) is newer than the one you're using now ({__version__}). Please upgrade PyMISP.") logger.warning(f"The version of PyMISP recommended by the MISP instance ({response['version']}) is newer than the one you're using now ({__version__}). Please upgrade PyMISP.")
misp_version = self.misp_instance_version misp_version = self.misp_instance_version
if 'version' in misp_version: if 'version' in misp_version:
self._misp_version = tuple(int(v) for v in misp_version['version'].split('.')) self._misp_version = tuple(int(v) for v in misp_version['version'].split('.'))
# Get the user information
self._current_user, self._current_role, self._current_user_settings = self.get_user(pythonify=True, expanded=True)
except Exception as e: except Exception as e:
raise PyMISPError(f'Unable to connect to MISP ({self.root_url}). Please make sure the API key and the URL are correct (http/https is required): {e}') raise PyMISPError(f'Unable to connect to MISP ({self.root_url}). Please make sure the API key and the URL are correct (http/https is required): {e}')
@ -106,16 +112,14 @@ class ExpandedPyMISP(PyMISP):
@property @property
def describe_types_local(self): def describe_types_local(self):
'''Returns the content of describe types from the package''' '''Returns the content of describe types from the package'''
with (self.resources_path / 'describeTypes.json').open() as f: return describe_types
describe_types = json.load(f)
return describe_types['result']
@property @property
def describe_types_remote(self): def describe_types_remote(self):
'''Returns the content of describe types from the remote instance''' '''Returns the content of describe types from the remote instance'''
response = self._prepare_request('GET', 'attributes/describeTypes.json') response = self._prepare_request('GET', 'attributes/describeTypes.json')
describe_types = self._check_response(response, expect_json=True) remote_describe_types = self._check_response(response, expect_json=True)
return describe_types['result'] return remote_describe_types['result']
@property @property
def recommended_pymisp_version(self): def recommended_pymisp_version(self):
@ -1263,8 +1267,9 @@ class ExpandedPyMISP(PyMISP):
to_return.append(u) to_return.append(u)
return to_return return to_return
def get_user(self, user: Union[MISPUser, int, str, UUID]='me', pythonify: bool=False): def get_user(self, user: Union[MISPUser, int, str, UUID]='me', pythonify: bool=False, expanded: bool=False):
'''Get a user. `me` means the owner of the API key doing the query.''' '''Get a user. `me` means the owner of the API key doing the query.
expanded also returns a MISPRole and a MISPUserSetting'''
user_id = self.__get_uuid_or_id_from_abstract_misp(user) user_id = self.__get_uuid_or_id_from_abstract_misp(user)
user = self._prepare_request('GET', f'users/view/{user_id}') user = self._prepare_request('GET', f'users/view/{user_id}')
user = self._check_response(user, expect_json=True) user = self._check_response(user, expect_json=True)
@ -1272,7 +1277,20 @@ class ExpandedPyMISP(PyMISP):
return user return user
u = MISPUser() u = MISPUser()
u.from_dict(**user) u.from_dict(**user)
if not expanded:
return u return u
else:
if self._old_misp((2, 4, 117), '2020-01-01', sys._getframe().f_code.co_name):
return u, None, None
r = MISPRole()
r.from_dict(**user['Role'])
usersettings = []
if user['UserSetting']:
for name, value in user['UserSetting'].items():
us = MISPUserSetting()
us.from_dict(**{'name': name, 'value': value})
usersettings.append(us)
return u, r, usersettings
def add_user(self, user: MISPUser, pythonify: bool=False): def add_user(self, user: MISPUser, pythonify: bool=False):
'''Add a new user''' '''Add a new user'''
@ -1290,7 +1308,10 @@ class ExpandedPyMISP(PyMISP):
user_id = self.__get_uuid_or_id_from_abstract_misp(user) user_id = self.__get_uuid_or_id_from_abstract_misp(user)
else: else:
user_id = self.__get_uuid_or_id_from_abstract_misp(user_id) user_id = self.__get_uuid_or_id_from_abstract_misp(user_id)
updated_user = self._prepare_request('POST', f'admin/users/edit/{user_id}', data=user) url = f'users/edit/{user_id}'
if self._current_role.perm_admin or self._current_role.perm_site_admin:
url = f'admin/{url}'
updated_user = self._prepare_request('POST', url, data=user)
updated_user = self._check_response(updated_user, expect_json=True) updated_user = self._check_response(updated_user, expect_json=True)
if not (self.global_pythonify or pythonify) or 'errors' in updated_user: if not (self.global_pythonify or pythonify) or 'errors' in updated_user:
return updated_user return updated_user
@ -1305,6 +1326,10 @@ class ExpandedPyMISP(PyMISP):
response = self._prepare_request('POST', f'admin/users/delete/{user_id}') response = self._prepare_request('POST', f'admin/users/delete/{user_id}')
return self._check_response(response, expect_json=True) return self._check_response(response, expect_json=True)
def change_user_password(self, new_password: str, user: Union[MISPUser, int, str, UUID]=None):
response = self._prepare_request('POST', f'users/change_pw', data={'password': new_password})
return self._check_response(response, expect_json=True)
# ## END User ### # ## END User ###
# ## BEGIN Role ### # ## BEGIN Role ###
@ -1367,7 +1392,7 @@ class ExpandedPyMISP(PyMISP):
**kwargs): **kwargs):
'''Search in the MISP instance '''Search in the MISP instance
:param returnFormat: Set the return format of the search (Currently supported: json, xml, openioc, suricata, snort - more formats are being moved to restSearch with the goal being that all searches happen through this API). Can be passed as the first parameter after restSearch or via the JSON payload. :param return_format: Set the return format of the search (Currently supported: json, xml, openioc, suricata, snort - more formats are being moved to restSearch with the goal being that all searches happen through this API). Can be passed as the first parameter after restSearch or via the JSON payload.
:param limit: Limit the number of results returned, depending on the scope (for example 10 attributes or 10 full events). :param limit: Limit the number of results returned, depending on the scope (for example 10 attributes or 10 full events).
:param page: If a limit is set, sets the page to be returned. page 3, limit 100 will return records 201->300). :param page: If a limit is set, sets the page to be returned. page 3, limit 100 will return records 201->300).
:param value: Search for the given value in the attributes' value field. :param value: Search for the given value in the attributes' value field.
@ -1413,7 +1438,7 @@ class ExpandedPyMISP(PyMISP):
''' '''
return_formats = ['openioc', 'json', 'xml', 'suricata', 'snort', 'text', 'rpz', 'csv', 'cache', 'stix', 'stix2'] return_formats = ['openioc', 'json', 'xml', 'suricata', 'snort', 'text', 'rpz', 'csv', 'cache', 'stix', 'stix2', 'yara', 'yara-json', 'attack', 'attack-sightings']
if controller not in ['events', 'attributes', 'objects', 'sightings']: if controller not in ['events', 'attributes', 'objects', 'sightings']:
raise ValueError('controller has to be in {}'.format(', '.join(['events', 'attributes', 'objects']))) raise ValueError('controller has to be in {}'.format(', '.join(['events', 'attributes', 'objects'])))
@ -1728,6 +1753,19 @@ class ExpandedPyMISP(PyMISP):
to_return.append(ml) to_return.append(ml)
return to_return return to_return
def search_feeds(self, value: Optional[SearchParameterTypes]=None, pythonify: Optional[bool]=False):
'''Search in the feeds cached on the servers'''
response = self._prepare_request('POST', '/feeds/searchCaches', data={'value': value})
normalized_response = self._check_response(response, expect_json=True)
if not (self.global_pythonify or pythonify) or 'errors' in normalized_response:
return normalized_response
to_return = []
for feed in normalized_response:
f = MISPFeed()
f.from_dict(**feed)
to_return.append(f)
return to_return
# ## END Search methods ### # ## END Search methods ###
# ## BEGIN Communities ### # ## BEGIN Communities ###
@ -1933,6 +1971,61 @@ class ExpandedPyMISP(PyMISP):
# ## END Statistics ### # ## END Statistics ###
# ## BEGIN User Settings ###
def user_settings(self, pythonify: bool=False):
"""Get all the user settings."""
user_settings = self._prepare_request('GET', 'user_settings')
user_settings = self._check_response(user_settings, expect_json=True)
if not (self.global_pythonify or pythonify) or 'errors' in user_settings:
return user_settings
to_return = []
for user_setting in user_settings:
u = MISPUserSetting()
u.from_dict(**user_setting)
to_return.append(u)
return to_return
def get_user_setting(self, user_setting: str, user: Union[MISPUser, int, str, UUID]=None, pythonify: bool=False):
'''Get an user setting'''
query = {'setting': user_setting}
if user:
query['user_id'] = self.__get_uuid_or_id_from_abstract_misp(user)
response = self._prepare_request('POST', f'user_settings/getSetting')
user_setting = self._check_response(response, expect_json=True)
if not (self.global_pythonify or pythonify) or 'errors' in user_setting:
return user_setting
u = MISPUserSetting()
u.from_dict(**user_setting)
return u
def set_user_setting(self, user_setting: str, value: Union[str, dict], user: Union[MISPUser, int, str, UUID]=None, pythonify: bool=False):
'''Get an user setting'''
query = {'setting': user_setting}
if isinstance(value, dict):
value = json.dumps(value)
query['value'] = value
if user:
query['user_id'] = self.__get_uuid_or_id_from_abstract_misp(user)
response = self._prepare_request('POST', f'user_settings/setSetting', data=query)
user_setting = self._check_response(response, expect_json=True)
if not (self.global_pythonify or pythonify) or 'errors' in user_setting:
return user_setting
u = MISPUserSetting()
u.from_dict(**user_setting)
return u
def delete_user_setting(self, user_setting: str, user: Union[MISPUser, int, str, UUID]=None):
'''Delete a user setting'''
query = {'setting': user_setting}
if user:
query['user_id'] = self.__get_uuid_or_id_from_abstract_misp(user)
response = self._prepare_request('POST', f'user_settings/delete', data=query)
return self._check_response(response, expect_json=True)
# ## END User Settings ###
# ## BEGIN Global helpers ### # ## BEGIN Global helpers ###
def change_sharing_group_on_entity(self, misp_entity: AbstractMISP, sharing_group_id, pythonify: bool=False): def change_sharing_group_on_entity(self, misp_entity: AbstractMISP, sharing_group_id, pythonify: bool=False):
@ -1952,13 +2045,13 @@ class ExpandedPyMISP(PyMISP):
raise PyMISPError('The misp_entity must be MISPEvent, MISPObject or MISPAttribute') raise PyMISPError('The misp_entity must be MISPEvent, MISPObject or MISPAttribute')
def tag(self, misp_entity: Union[AbstractMISP, str], tag: str): def tag(self, misp_entity: Union[AbstractMISP, str], tag: str, local: bool=False):
"""Tag an event or an attribute. misp_entity can be a UUID""" """Tag an event or an attribute. misp_entity can be a UUID"""
if 'uuid' in misp_entity: if 'uuid' in misp_entity:
uuid = misp_entity.uuid uuid = misp_entity.uuid
else: else:
uuid = misp_entity uuid = misp_entity
to_post = {'uuid': uuid, 'tag': tag} to_post = {'uuid': uuid, 'tag': tag, 'local': local}
response = self._prepare_request('POST', 'tags/attachTagToObject', data=to_post) response = self._prepare_request('POST', 'tags/attachTagToObject', data=to_post)
return self._check_response(response, expect_json=True) return self._check_response(response, expect_json=True)
@ -2005,6 +2098,12 @@ class ExpandedPyMISP(PyMISP):
return str(obj) return str(obj)
if isinstance(obj, (int, str)): if isinstance(obj, (int, str)):
return obj return obj
if isinstance(obj, dict) and len(obj.keys()) == 1:
# We have an object in that format: {'Event': {'id': 2, ...}}
# We need to get the content of that dictionary
obj = obj[list(obj.keys())[0]]
if self._old_misp((2, 4, 113), '2020-01-01', sys._getframe().f_code.co_name, message='MISP now accepts UUIDs to access entiries, usinf it is a lot safer across instances. Just update your MISP instance, plz.'): if self._old_misp((2, 4, 113), '2020-01-01', sys._getframe().f_code.co_name, message='MISP now accepts UUIDs to access entiries, usinf it is a lot safer across instances. Just update your MISP instance, plz.'):
if 'id' in obj: if 'id' in obj:
return obj['id'] return obj['id']
@ -2098,7 +2197,7 @@ class ExpandedPyMISP(PyMISP):
if isinstance(data, dict): # Else, we can directly json encode. if isinstance(data, dict): # Else, we can directly json encode.
# Remove None values. # Remove None values.
data = {k: v for k, v in data.items() if v is not None} data = {k: v for k, v in data.items() if v is not None}
data = json.dumps(data, cls=MISPEncode) data = json.dumps(data, default=pymisp_json_default)
if kw_params: if kw_params:
# CakePHP params in URL # CakePHP params in URL

6
pymisp/data/describeTypes.json
View File

@ -159,6 +159,7 @@
"btc", "btc",
"cc-number", "cc-number",
"comment", "comment",
"dash",
"hex", "hex",
"iban", "iban",
"other", "other",
@ -535,6 +536,10 @@
"default_category": "Other", "default_category": "Other",
"to_ids": 0 "to_ids": 0
}, },
"dash": {
"default_category": "Financial fraud",
"to_ids": 1
},
"date-of-birth": { "date-of-birth": {
"default_category": "Person", "default_category": "Person",
"to_ids": 0 "to_ids": 0
@ -1111,6 +1116,7 @@
"counter", "counter",
"country-of-residence", "country-of-residence",
"cpe", "cpe",
"dash",
"date-of-birth", "date-of-birth",
"datetime", "datetime",
"dns-soa-email", "dns-soa-email",

2
pymisp/data/misp-objects

@ -1 +1 @@
Subproject commit c381598c3d89c6f7f50a0781fb37e7785a3296b2 Subproject commit 58d6722f5e276a0ec5889e6e67316b7401960542

231
pymisp/mispevent.py
View File

@ -17,6 +17,7 @@ from deprecated import deprecated
from .abstract import AbstractMISP from .abstract import AbstractMISP
from .exceptions import UnknownMISPObjectTemplate, InvalidMISPObject, PyMISPError, NewEventError, NewAttributeError from .exceptions import UnknownMISPObjectTemplate, InvalidMISPObject, PyMISPError, NewEventError, NewAttributeError
logger = logging.getLogger('pymisp') logger = logging.getLogger('pymisp')
if sys.version_info < (3, 0): if sys.version_info < (3, 0):
@ -109,18 +110,11 @@ class MISPAttribute(AbstractMISP):
:strict: If false, fallback to sane defaults for the attribute type if the ones passed by the user are incorrect :strict: If false, fallback to sane defaults for the attribute type if the ones passed by the user are incorrect
""" """
super(MISPAttribute, self).__init__() super(MISPAttribute, self).__init__()
if not describe_types: if describe_types:
ressources_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data') self.describe_types = describe_types
with open(os.path.join(ressources_path, 'describeTypes.json'), 'rb') as f: self.__categories = self.describe_types['categories']
if OLD_PY3: self.__category_type_mapping = self.describe_types['category_type_mappings']
t = json.loads(f.read().decode()) self.__sane_default = self.describe_types['sane_defaults']
else:
t = json.load(f)
describe_types = t['result']
self.__categories = describe_types['categories']
self._types = describe_types['types']
self.__category_type_mapping = describe_types['category_type_mappings']
self.__sane_default = describe_types['sane_defaults']
self.__strict = strict self.__strict = strict
self._data = None self._data = None
self.uuid = str(uuid.uuid4()) self.uuid = str(uuid.uuid4())
@ -130,7 +124,7 @@ class MISPAttribute(AbstractMISP):
@property @property
def known_types(self): def known_types(self):
"""Returns a list of all the known MISP attributes types""" """Returns a list of all the known MISP attributes types"""
return self._types return self.describe_types['types']
@property @property
def malware_binary(self): def malware_binary(self):
@ -204,8 +198,8 @@ class MISPAttribute(AbstractMISP):
return misp_sighting return misp_sighting
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Attribute'): if 'Attribute' in kwargs:
kwargs = kwargs.get('Attribute') kwargs = kwargs['Attribute']
if kwargs.get('type') and kwargs.get('category'): if kwargs.get('type') and kwargs.get('category'):
if kwargs['type'] not in self.__category_type_mapping[kwargs['category']]: if kwargs['type'] not in self.__category_type_mapping[kwargs['category']]:
if self.__strict: if self.__strict:
@ -218,7 +212,7 @@ class MISPAttribute(AbstractMISP):
if self.type is None: if self.type is None:
raise NewAttributeError('The type of the attribute is required.') raise NewAttributeError('The type of the attribute is required.')
if self.type not in self.known_types: if self.type not in self.known_types:
raise NewAttributeError('{} is invalid, type has to be in {}'.format(self.type, (', '.join(self._types)))) raise NewAttributeError('{} is invalid, type has to be in {}'.format(self.type, (', '.join(self.known_types))))
type_defaults = self.__sane_default[self.type] type_defaults = self.__sane_default[self.type]
@ -226,6 +220,13 @@ class MISPAttribute(AbstractMISP):
if self.value is None: if self.value is None:
raise NewAttributeError('The value of the attribute is required.') raise NewAttributeError('The value of the attribute is required.')
if self.type == 'datetime' and isinstance(self.value, str): if self.type == 'datetime' and isinstance(self.value, str):
try:
if '.' in self.value:
self.value = datetime.datetime.strptime(self.value, "%Y-%m-%dT%H:%M:%S.%f")
else:
self.value = datetime.datetime.strptime(self.value, "%Y-%m-%dT%H:%M:%S")
except ValueError:
# Slower, but if the other ones fail, that's a good fallback
self.value = parse(self.value) self.value = parse(self.value)
# Default values # Default values
@ -278,14 +279,11 @@ class MISPAttribute(AbstractMISP):
raise NewAttributeError('If the distribution is set to sharing group, a sharing group ID is required (cannot be {}).'.format(self.sharing_group_id)) raise NewAttributeError('If the distribution is set to sharing group, a sharing group ID is required (cannot be {}).'.format(self.sharing_group_id))
if kwargs.get('Tag'): if kwargs.get('Tag'):
for tag in kwargs.pop('Tag'): [self.add_tag(tag) for tag in kwargs.pop('Tag')]
self.add_tag(tag)
if kwargs.get('Sighting'): if kwargs.get('Sighting'):
for sighting in kwargs.pop('Sighting'): [self.add_sighting(sighting) for sighting in kwargs.pop('Sighting')]
self.add_sighting(sighting)
if kwargs.get('ShadowAttribute'): if kwargs.get('ShadowAttribute'):
for s_attr in kwargs.pop('ShadowAttribute'): [self.add_shadow_attribute(s_attr) for s_attr in kwargs.pop('ShadowAttribute')]
self.add_shadow_attribute(s_attr)
# If the user wants to disable correlation, let them. Defaults to False. # If the user wants to disable correlation, let them. Defaults to False.
self.disable_correlation = kwargs.pop("disable_correlation", False) self.disable_correlation = kwargs.pop("disable_correlation", False)
@ -425,31 +423,18 @@ class MISPEvent(AbstractMISP):
def __init__(self, describe_types=None, strict_validation=False, **kwargs): def __init__(self, describe_types=None, strict_validation=False, **kwargs):
super(MISPEvent, self).__init__(**kwargs) super(MISPEvent, self).__init__(**kwargs)
ressources_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'data')
if strict_validation: if strict_validation:
with open(os.path.join(ressources_path, 'schema.json'), 'rb') as f: schema_file = 'schema.json'
if OLD_PY3:
self.__json_schema = json.loads(f.read().decode())
else: else:
self.__json_schema = json.load(f) schema_file = 'schema-lax.json'
if sys.version_info >= (3, 4):
self.__json_schema = self._load_json(self.resources_path / schema_file)
else: else:
with open(os.path.join(ressources_path, 'schema-lax.json'), 'rb') as f: self.__json_schema = self._load_json(os.path.join(self.resources_path, schema_file))
if OLD_PY3:
self.__json_schema = json.loads(f.read().decode())
else:
self.__json_schema = json.load(f)
if describe_types: if describe_types:
# This variable is used in add_attribute in order to avoid duplicating the structure # This variable is used in add_attribute in order to avoid duplicating the structure
self._describe_types = describe_types self.describe_types = describe_types
else:
with open(os.path.join(ressources_path, 'describeTypes.json'), 'rb') as f:
if OLD_PY3:
t = json.loads(f.read().decode())
else:
t = json.load(f)
self._describe_types = t['result']
self._types = self._describe_types['types']
self.Attribute = [] self.Attribute = []
self.Object = [] self.Object = []
self.RelatedEvent = [] self.RelatedEvent = []
@ -457,7 +442,7 @@ class MISPEvent(AbstractMISP):
@property @property
def known_types(self): def known_types(self):
return self._types return self.describe_types['types']
@property @property
def org(self): def org(self):
@ -554,8 +539,8 @@ class MISPEvent(AbstractMISP):
raise NewEventError('Invalid format for the date: {} - {}'.format(date, type(date))) raise NewEventError('Invalid format for the date: {} - {}'.format(date, type(date)))
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Event'): if 'Event' in kwargs:
kwargs = kwargs.get('Event') kwargs = kwargs['Event']
# Required value # Required value
self.info = kwargs.pop('info', None) self.info = kwargs.pop('info', None)
if self.info is None: if self.info is None:
@ -587,8 +572,7 @@ class MISPEvent(AbstractMISP):
if kwargs.get('date'): if kwargs.get('date'):
self.set_date(kwargs.pop('date')) self.set_date(kwargs.pop('date'))
if kwargs.get('Attribute'): if kwargs.get('Attribute'):
for a in kwargs.pop('Attribute'): [self.add_attribute(**a) for a in kwargs.pop('Attribute')]
self.add_attribute(**a)
# All other keys # All other keys
if kwargs.get('id'): if kwargs.get('id'):
@ -615,11 +599,9 @@ class MISPEvent(AbstractMISP):
sub_event.load(rel_event) sub_event.load(rel_event)
self.RelatedEvent.append({'Event': sub_event}) self.RelatedEvent.append({'Event': sub_event})
if kwargs.get('Tag'): if kwargs.get('Tag'):
for tag in kwargs.pop('Tag'): [self.add_tag(tag) for tag in kwargs.pop('Tag')]
self.add_tag(tag)
if kwargs.get('Object'): if kwargs.get('Object'):
for obj in kwargs.pop('Object'): [self.add_object(obj) for obj in kwargs.pop('Object')]
self.add_object(obj)
if kwargs.get('Org'): if kwargs.get('Org'):
self.Org = MISPOrganisation() self.Org = MISPOrganisation()
self.Org.from_dict(**kwargs.pop('Org')) self.Org.from_dict(**kwargs.pop('Org'))
@ -720,7 +702,7 @@ class MISPEvent(AbstractMISP):
if isinstance(value, list): if isinstance(value, list):
attr_list = [self.add_attribute(type=type, value=a, **kwargs) for a in value] attr_list = [self.add_attribute(type=type, value=a, **kwargs) for a in value]
else: else:
attribute = MISPAttribute(describe_types=self._describe_types) attribute = MISPAttribute(describe_types=self.describe_types)
attribute.from_dict(type=type, value=value, **kwargs) attribute.from_dict(type=type, value=value, **kwargs)
self.attributes.append(attribute) self.attributes.append(attribute)
self.edited = True self.edited = True
@ -879,8 +861,8 @@ class MISPObjectReference(AbstractMISP):
super(MISPObjectReference, self).__init__() super(MISPObjectReference, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('ObjectReference'): if 'ObjectReference' in kwargs:
kwargs = kwargs.get('ObjectReference') kwargs = kwargs['ObjectReference']
super(MISPObjectReference, self).from_dict(**kwargs) super(MISPObjectReference, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -895,8 +877,8 @@ class MISPObjectTemplate(AbstractMISP):
super(MISPObjectTemplate, self).__init__() super(MISPObjectTemplate, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('ObjectTemplate'): if 'ObjectTemplate' in kwargs:
kwargs = kwargs.get('ObjectTemplate') kwargs = kwargs['ObjectTemplate']
super(MISPObjectTemplate, self).from_dict(**kwargs) super(MISPObjectTemplate, self).from_dict(**kwargs)
@ -906,8 +888,8 @@ class MISPUser(AbstractMISP):
super(MISPUser, self).__init__() super(MISPUser, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('User'): if 'User' in kwargs:
kwargs = kwargs.get('User') kwargs = kwargs['User']
super(MISPUser, self).from_dict(**kwargs) super(MISPUser, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -922,8 +904,8 @@ class MISPOrganisation(AbstractMISP):
super(MISPOrganisation, self).__init__() super(MISPOrganisation, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Organisation'): if 'Organisation' in kwargs:
kwargs = kwargs.get('Organisation') kwargs = kwargs['Organisation']
super(MISPOrganisation, self).from_dict(**kwargs) super(MISPOrganisation, self).from_dict(**kwargs)
@ -933,8 +915,8 @@ class MISPFeed(AbstractMISP):
super(MISPFeed, self).__init__() super(MISPFeed, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Feed'): if 'Feed' in kwargs:
kwargs = kwargs.get('Feed') kwargs = kwargs['Feed']
super(MISPFeed, self).from_dict(**kwargs) super(MISPFeed, self).from_dict(**kwargs)
@ -944,8 +926,8 @@ class MISPWarninglist(AbstractMISP):
super(MISPWarninglist, self).__init__() super(MISPWarninglist, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Warninglist'): if 'Warninglist' in kwargs:
kwargs = kwargs.get('Warninglist') kwargs = kwargs['Warninglist']
super(MISPWarninglist, self).from_dict(**kwargs) super(MISPWarninglist, self).from_dict(**kwargs)
@ -955,8 +937,8 @@ class MISPTaxonomy(AbstractMISP):
super(MISPTaxonomy, self).__init__() super(MISPTaxonomy, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Taxonomy'): if 'Taxonomy' in kwargs:
kwargs = kwargs.get('Taxonomy') kwargs = kwargs['Taxonomy']
super(MISPTaxonomy, self).from_dict(**kwargs) super(MISPTaxonomy, self).from_dict(**kwargs)
@ -966,8 +948,8 @@ class MISPGalaxy(AbstractMISP):
super(MISPGalaxy, self).__init__() super(MISPGalaxy, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Galaxy'): if 'Galaxy' in kwargs:
kwargs = kwargs.get('Galaxy') kwargs = kwargs['Galaxy']
super(MISPGalaxy, self).from_dict(**kwargs) super(MISPGalaxy, self).from_dict(**kwargs)
@ -977,8 +959,8 @@ class MISPNoticelist(AbstractMISP):
super(MISPNoticelist, self).__init__() super(MISPNoticelist, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Noticelist'): if 'Noticelist' in kwargs:
kwargs = kwargs.get('Noticelist') kwargs = kwargs['Noticelist']
super(MISPNoticelist, self).from_dict(**kwargs) super(MISPNoticelist, self).from_dict(**kwargs)
@ -988,8 +970,8 @@ class MISPRole(AbstractMISP):
super(MISPRole, self).__init__() super(MISPRole, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Role'): if 'Role' in kwargs:
kwargs = kwargs.get('Role') kwargs = kwargs['Role']
super(MISPRole, self).from_dict(**kwargs) super(MISPRole, self).from_dict(**kwargs)
@ -999,8 +981,8 @@ class MISPServer(AbstractMISP):
super(MISPServer, self).__init__() super(MISPServer, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Server'): if 'Server' in kwargs:
kwargs = kwargs.get('Server') kwargs = kwargs['Server']
super(MISPServer, self).from_dict(**kwargs) super(MISPServer, self).from_dict(**kwargs)
@ -1010,8 +992,8 @@ class MISPSharingGroup(AbstractMISP):
super(MISPSharingGroup, self).__init__() super(MISPSharingGroup, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('SharingGroup'): if 'SharingGroup' in kwargs:
kwargs = kwargs.get('SharingGroup') kwargs = kwargs['SharingGroup']
super(MISPSharingGroup, self).from_dict(**kwargs) super(MISPSharingGroup, self).from_dict(**kwargs)
@ -1021,8 +1003,8 @@ class MISPLog(AbstractMISP):
super(MISPLog, self).__init__() super(MISPLog, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Log'): if 'Log' in kwargs:
kwargs = kwargs.get('Log') kwargs = kwargs['Log']
super(MISPLog, self).from_dict(**kwargs) super(MISPLog, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -1035,8 +1017,8 @@ class MISPEventDelegation(AbstractMISP):
super(MISPEventDelegation, self).__init__() super(MISPEventDelegation, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('EventDelegation'): if 'EventDelegation' in kwargs:
kwargs = kwargs.get('EventDelegation') kwargs = kwargs['EventDelegation']
super(MISPEventDelegation, self).from_dict(**kwargs) super(MISPEventDelegation, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -1058,8 +1040,8 @@ class MISPSighting(AbstractMISP):
:type: Type of the sighting :type: Type of the sighting
:timestamp: Timestamp associated to the sighting :timestamp: Timestamp associated to the sighting
""" """
if kwargs.get('Sighting'): if 'Sighting' in kwargs:
kwargs = kwargs.get('Sighting') kwargs = kwargs['Sighting']
super(MISPSighting, self).from_dict(**kwargs) super(MISPSighting, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -1081,11 +1063,16 @@ class MISPObjectAttribute(MISPAttribute):
def from_dict(self, object_relation, value, **kwargs): def from_dict(self, object_relation, value, **kwargs):
self.object_relation = object_relation self.object_relation = object_relation
self.value = value self.value = value
if 'Attribute' in kwargs:
kwargs = kwargs['Attribute']
# Initialize the new MISPAttribute # Initialize the new MISPAttribute
# Get the misp attribute type from the definition # Get the misp attribute type from the definition
self.type = kwargs.pop('type', None) self.type = kwargs.pop('type', None)
if self.type is None: if self.type is None:
self.type = self._definition.get('misp-attribute') self.type = self._definition.get('misp-attribute')
if 'category' not in kwargs and 'categories' in self._definition:
# Get first category in the list from the object template as default
self.category = self._definition['categories'][0]
self.disable_correlation = kwargs.pop('disable_correlation', None) self.disable_correlation = kwargs.pop('disable_correlation', None)
if self.disable_correlation is None: if self.disable_correlation is None:
# The correlation can be disabled by default in the object definition. # The correlation can be disabled by default in the object definition.
@ -1098,6 +1085,8 @@ class MISPObjectAttribute(MISPAttribute):
if not self.type: if not self.type:
raise NewAttributeError("The type of the attribute is required. Is the object template missing?") raise NewAttributeError("The type of the attribute is required. Is the object template missing?")
super(MISPObjectAttribute, self).from_dict(**dict(self, **kwargs)) super(MISPObjectAttribute, self).from_dict(**dict(self, **kwargs))
# FIXME New syntax python3 only, keep for later.
# super(MISPObjectAttribute, self).from_dict(**{**self, **kwargs})
def __repr__(self): def __repr__(self):
if hasattr(self, 'value'): if hasattr(self, 'value'):
@ -1111,8 +1100,8 @@ class MISPShadowAttribute(AbstractMISP):
super(MISPShadowAttribute, self).__init__() super(MISPShadowAttribute, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('ShadowAttribute'): if 'ShadowAttribute' in kwargs:
kwargs = kwargs.get('ShadowAttribute') kwargs = kwargs['ShadowAttribute']
super(MISPShadowAttribute, self).from_dict(**kwargs) super(MISPShadowAttribute, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
@ -1127,14 +1116,28 @@ class MISPCommunity(AbstractMISP):
super(MISPCommunity, self).__init__() super(MISPCommunity, self).__init__()
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Community'): if 'Community' in kwargs:
kwargs = kwargs.get('Community') kwargs = kwargs['Community']
super(MISPCommunity, self).from_dict(**kwargs) super(MISPCommunity, self).from_dict(**kwargs)
def __repr__(self): def __repr__(self):
return '<{self.__class__.__name__}(name={self.name}, uuid={self.uuid})'.format(self=self) return '<{self.__class__.__name__}(name={self.name}, uuid={self.uuid})'.format(self=self)
class MISPUserSetting(AbstractMISP):
def __init__(self):
super(MISPUserSetting, self).__init__()
def from_dict(self, **kwargs):
if 'UserSetting' in kwargs:
kwargs = kwargs['UserSetting']
super(MISPUserSetting, self).from_dict(**kwargs)
def __repr__(self):
return '<{self.__class__.__name__}(name={self.setting}'.format(self=self)
class MISPObject(AbstractMISP): class MISPObject(AbstractMISP):
def __init__(self, name, strict=False, standalone=False, default_attributes_parameters={}, **kwargs): def __init__(self, name, strict=False, standalone=False, default_attributes_parameters={}, **kwargs):
@ -1155,20 +1158,7 @@ class MISPObject(AbstractMISP):
self.name = name self.name = name
self._known_template = False self._known_template = False
if kwargs.get('misp_objects_path_custom'): self._set_template(kwargs.get('misp_objects_path_custom'))
# If misp_objects_path_custom is given, and an object with the given name exists, use that.
self._known_template = self._load_template_path(os.path.join(kwargs.get('misp_objects_path_custom'), self.name, 'definition.json'))
if not self._known_template:
# Check if the object is known in the default templates bundled in with PyMISP
misp_objects_path = os.path.join(os.path.abspath(os.path.dirname(sys.modules['pymisp'].__file__)), 'data', 'misp-objects', 'objects')
self._known_template = self._load_template_path(os.path.join(misp_objects_path, self.name, 'definition.json'))
if not self._known_template and self._strict:
raise UnknownMISPObjectTemplate('{} is unknown in the MISP object directory.'.format(self.name))
else:
# Then we have no meta-category, template_uuid, description and template_version
pass
self.uuid = str(uuid.uuid4()) self.uuid = str(uuid.uuid4())
self.__fast_attribute_access = defaultdict(list) # Hashtable object_relation: [attributes] self.__fast_attribute_access = defaultdict(list) # Hashtable object_relation: [attributes]
@ -1204,13 +1194,9 @@ class MISPObject(AbstractMISP):
self.update_not_jsonable('ObjectReference') self.update_not_jsonable('ObjectReference')
def _load_template_path(self, template_path): def _load_template_path(self, template_path):
if not os.path.exists(template_path): self._definition = self._load_json(template_path)
if not self._definition:
return False return False
with open(template_path, 'rb') as f:
if OLD_PY3:
self._definition = json.loads(f.read().decode())
else:
self._definition = json.load(f)
setattr(self, 'meta-category', self._definition['meta-category']) setattr(self, 'meta-category', self._definition['meta-category'])
self.template_uuid = self._definition['uuid'] self.template_uuid = self._definition['uuid']
self.description = self._definition['description'] self.description = self._definition['description']
@ -1220,11 +1206,24 @@ class MISPObject(AbstractMISP):
def force_misp_objects_path_custom(self, misp_objects_path_custom, object_name=None): def force_misp_objects_path_custom(self, misp_objects_path_custom, object_name=None):
if object_name: if object_name:
self.name = object_name self.name = object_name
template_path = os.path.join(misp_objects_path_custom, self.name, 'definition.json') self._set_template(misp_objects_path_custom)
self._known_template = self._load_template_path(template_path) def _set_template(self, misp_objects_path_custom=None):
if not self._known_template: if misp_objects_path_custom:
raise UnknownMISPObjectTemplate('{} is unknown in the MISP object directory ({}).'.format(self.name, template_path)) # If misp_objects_path_custom is given, and an object with the given name exists, use that.
self.misp_objects_path = misp_objects_path_custom
# Try to get the template
if sys.version_info >= (3, 4):
self._known_template = self._load_template_path(self.misp_objects_path / self.name / 'definition.json')
else:
self._known_template = self._load_template_path(os.path.join(self.misp_objects_path, self.name, 'definition.json'))
if not self._known_template and self._strict:
raise UnknownMISPObjectTemplate('{} is unknown in the MISP object directory.'.format(self.name))
else:
# Then we have no meta-category, template_uuid, description and template_version
pass
@property @property
def disable_validation(self): def disable_validation(self):
@ -1254,8 +1253,8 @@ class MISPObject(AbstractMISP):
raise PyMISPError('All the attributes have to be of type MISPObjectReference.') raise PyMISPError('All the attributes have to be of type MISPObjectReference.')
def from_dict(self, **kwargs): def from_dict(self, **kwargs):
if kwargs.get('Object'): if 'Object' in kwargs:
kwargs = kwargs.get('Object') kwargs = kwargs['Object']
if self._known_template: if self._known_template:
if kwargs.get('template_uuid') and kwargs['template_uuid'] != self.template_uuid: if kwargs.get('template_uuid') and kwargs['template_uuid'] != self.template_uuid:
if self._strict: if self._strict:
@ -1283,11 +1282,9 @@ class MISPObject(AbstractMISP):
else: else:
self.timestamp = datetime.datetime.fromtimestamp(int(ts), UTC()) self.timestamp = datetime.datetime.fromtimestamp(int(ts), UTC())
if kwargs.get('Attribute'): if kwargs.get('Attribute'):
for a in kwargs.pop('Attribute'): [self.add_attribute(**a) for a in kwargs.pop('Attribute')]
self.add_attribute(**a)
if kwargs.get('ObjectReference'): if kwargs.get('ObjectReference'):
for r in kwargs.pop('ObjectReference'): [self.add_reference(**r) for r in kwargs.pop('ObjectReference')]
self.add_reference(**r)
# Not supported yet - https://github.com/MISP/PyMISP/issues/168 # Not supported yet - https://github.com/MISP/PyMISP/issues/168
# if kwargs.get('Tag'): # if kwargs.get('Tag'):
@ -1337,7 +1334,7 @@ class MISPObject(AbstractMISP):
logger.warning("The value of the attribute you're trying to add is None or empty string, skipping it. Object relation: {}".format(object_relation)) logger.warning("The value of the attribute you're trying to add is None or empty string, skipping it. Object relation: {}".format(object_relation))
return None return None
if self._known_template: if self._known_template:
if self._definition['attributes'].get(object_relation): if object_relation in self._definition['attributes']:
attribute = MISPObjectAttribute(self._definition['attributes'][object_relation]) attribute = MISPObjectAttribute(self._definition['attributes'][object_relation])
else: else:
# Woopsie, this object_relation is unknown, no sane defaults for you. # Woopsie, this object_relation is unknown, no sane defaults for you.
@ -1347,6 +1344,8 @@ class MISPObject(AbstractMISP):
attribute = MISPObjectAttribute({}) attribute = MISPObjectAttribute({})
# Overwrite the parameters of self._default_attributes_parameters with the ones of value # Overwrite the parameters of self._default_attributes_parameters with the ones of value
attribute.from_dict(object_relation=object_relation, **dict(self._default_attributes_parameters, **value)) attribute.from_dict(object_relation=object_relation, **dict(self._default_attributes_parameters, **value))
# FIXME New syntax python3 only, keep for later.
# attribute.from_dict(object_relation=object_relation, **{**self._default_attributes_parameters, **value})
self.__fast_attribute_access[object_relation].append(attribute) self.__fast_attribute_access[object_relation].append(attribute)
self.Attribute.append(attribute) self.Attribute.append(attribute)
self.edited = True self.edited = True

2
pymisp/tools/create_misp_object.py
View File

@ -52,7 +52,7 @@ def make_macho_objects(lief_parsed, misp_file, standalone=True, default_attribut
def make_binary_objects(filepath=None, pseudofile=None, filename=None, standalone=True, default_attributes_parameters={}): def make_binary_objects(filepath=None, pseudofile=None, filename=None, standalone=True, default_attributes_parameters={}):
misp_file = FileObject(filepath=filepath, pseudofile=pseudofile, filename=filename, misp_file = FileObject(filepath=filepath, pseudofile=pseudofile, filename=filename,
standalone=standalone, default_attributes_parameters=default_attributes_parameters) standalone=standalone, default_attributes_parameters=default_attributes_parameters)
if HAS_LIEF and filepath or (pseudofile and filename): if HAS_LIEF and (filepath or (pseudofile and filename)):
try: try:
if filepath: if filepath:
lief_parsed = lief.parse(filepath=filepath) lief_parsed = lief.parse(filepath=filepath)

2
pymisp/tools/elfobject.py
View File

@ -25,6 +25,7 @@ except ImportError:
class ELFObject(AbstractMISPObjectGenerator): class ELFObject(AbstractMISPObjectGenerator):
def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs): def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs):
super(ELFObject, self).__init__('elf', standalone=standalone, **kwargs)
if not HAS_PYDEEP: if not HAS_PYDEEP:
logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git") logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git")
if not HAS_LIEF: if not HAS_LIEF:
@ -44,7 +45,6 @@ class ELFObject(AbstractMISPObjectGenerator):
self.__elf = parsed self.__elf = parsed
else: else:
raise InvalidMISPObject('Not a lief.ELF.Binary: {}'.format(type(parsed))) raise InvalidMISPObject('Not a lief.ELF.Binary: {}'.format(type(parsed)))
super(ELFObject, self).__init__('elf', standalone=standalone, **kwargs)
self.generate_attributes() self.generate_attributes()
def generate_attributes(self): def generate_attributes(self):

6
pymisp/tools/emailobject.py
View File

@ -13,6 +13,9 @@ logger = logging.getLogger('pymisp')
class EMailObject(AbstractMISPObjectGenerator): class EMailObject(AbstractMISPObjectGenerator):
def __init__(self, filepath=None, pseudofile=None, attach_original_email=True, standalone=True, **kwargs): def __init__(self, filepath=None, pseudofile=None, attach_original_email=True, standalone=True, **kwargs):
# PY3 way:
# super().__init__('file')
super(EMailObject, self).__init__('email', standalone=standalone, **kwargs)
if filepath: if filepath:
with open(filepath, 'rb') as f: with open(filepath, 'rb') as f:
self.__pseudofile = BytesIO(f.read()) self.__pseudofile = BytesIO(f.read())
@ -20,9 +23,6 @@ class EMailObject(AbstractMISPObjectGenerator):
self.__pseudofile = pseudofile self.__pseudofile = pseudofile
else: else:
raise InvalidMISPObject('File buffer (BytesIO) or a path is required.') raise InvalidMISPObject('File buffer (BytesIO) or a path is required.')
# PY3 way:
# super().__init__('file')
super(EMailObject, self).__init__('email', standalone=standalone, **kwargs)
self.__email = message_from_bytes(self.__pseudofile.getvalue(), policy=policy.default) self.__email = message_from_bytes(self.__pseudofile.getvalue(), policy=policy.default)
if attach_original_email: if attach_original_email:
self.add_attribute('eml', value='Full email.eml', data=self.__pseudofile) self.add_attribute('eml', value='Full email.eml', data=self.__pseudofile)

6
pymisp/tools/fileobject.py
View File

@ -29,6 +29,9 @@ except ImportError:
class FileObject(AbstractMISPObjectGenerator): class FileObject(AbstractMISPObjectGenerator):
def __init__(self, filepath=None, pseudofile=None, filename=None, standalone=True, **kwargs): def __init__(self, filepath=None, pseudofile=None, filename=None, standalone=True, **kwargs):
# PY3 way:
# super().__init__('file')
super(FileObject, self).__init__('file', standalone=standalone, **kwargs)
if not HAS_PYDEEP: if not HAS_PYDEEP:
logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git") logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git")
if not HAS_MAGIC: if not HAS_MAGIC:
@ -49,9 +52,6 @@ class FileObject(AbstractMISPObjectGenerator):
self.__pseudofile = pseudofile self.__pseudofile = pseudofile
else: else:
raise InvalidMISPObject('File buffer (BytesIO) or a path is required.') raise InvalidMISPObject('File buffer (BytesIO) or a path is required.')
# PY3 way:
# super().__init__('file')
super(FileObject, self).__init__('file', standalone=standalone, **kwargs)
self.__data = self.__pseudofile.getvalue() self.__data = self.__pseudofile.getvalue()
self.generate_attributes() self.generate_attributes()

6
pymisp/tools/machoobject.py
View File

@ -26,6 +26,9 @@ except ImportError:
class MachOObject(AbstractMISPObjectGenerator): class MachOObject(AbstractMISPObjectGenerator):
def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs): def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs):
# Python3 way
# super().__init__('elf')
super(MachOObject, self).__init__('macho', standalone=standalone, **kwargs)
if not HAS_PYDEEP: if not HAS_PYDEEP:
logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git") logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git")
if not HAS_LIEF: if not HAS_LIEF:
@ -45,9 +48,6 @@ class MachOObject(AbstractMISPObjectGenerator):
self.__macho = parsed self.__macho = parsed
else: else:
raise InvalidMISPObject('Not a lief.MachO.Binary: {}'.format(type(parsed))) raise InvalidMISPObject('Not a lief.MachO.Binary: {}'.format(type(parsed)))
# Python3 way
# super().__init__('elf')
super(MachOObject, self).__init__('macho', standalone=standalone, **kwargs)
self.generate_attributes() self.generate_attributes()
def generate_attributes(self): def generate_attributes(self):

6
pymisp/tools/peobject.py
View File

@ -26,6 +26,9 @@ except ImportError:
class PEObject(AbstractMISPObjectGenerator): class PEObject(AbstractMISPObjectGenerator):
def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs): def __init__(self, parsed=None, filepath=None, pseudofile=None, standalone=True, **kwargs):
# Python3 way
# super().__init__('pe')
super(PEObject, self).__init__('pe', standalone=standalone, **kwargs)
if not HAS_PYDEEP: if not HAS_PYDEEP:
logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git") logger.warning("Please install pydeep: pip install git+https://github.com/kbandla/pydeep.git")
if not HAS_LIEF: if not HAS_LIEF:
@ -45,9 +48,6 @@ class PEObject(AbstractMISPObjectGenerator):
self.__pe = parsed self.__pe = parsed
else: else:
raise InvalidMISPObject('Not a lief.PE.Binary: {}'.format(type(parsed))) raise InvalidMISPObject('Not a lief.PE.Binary: {}'.format(type(parsed)))
# Python3 way
# super().__init__('pe')
super(PEObject, self).__init__('pe', standalone=standalone, **kwargs)
self.generate_attributes() self.generate_attributes()
def _is_exe(self): def _is_exe(self):

6
pymisp/tools/sshauthkeyobject.py
View File

@ -12,6 +12,9 @@ logger = logging.getLogger('pymisp')
class SSHAuthorizedKeysObject(AbstractMISPObjectGenerator): class SSHAuthorizedKeysObject(AbstractMISPObjectGenerator):
def __init__(self, authorized_keys_path=None, authorized_keys_pseudofile=None, standalone=True, **kwargs): def __init__(self, authorized_keys_path=None, authorized_keys_pseudofile=None, standalone=True, **kwargs):
# PY3 way:
# super().__init__('file')
super(SSHAuthorizedKeysObject, self).__init__('ssh-authorized-keys', standalone=standalone, **kwargs)
if authorized_keys_path: if authorized_keys_path:
with open(authorized_keys_path, 'r') as f: with open(authorized_keys_path, 'r') as f:
self.__pseudofile = StringIO(f.read()) self.__pseudofile = StringIO(f.read())
@ -19,9 +22,6 @@ class SSHAuthorizedKeysObject(AbstractMISPObjectGenerator):
self.__pseudofile = authorized_keys_path self.__pseudofile = authorized_keys_path
else: else:
raise InvalidMISPObject('File buffer (StringIO) or a path is required.') raise InvalidMISPObject('File buffer (StringIO) or a path is required.')
# PY3 way:
# super().__init__('file')
super(SSHAuthorizedKeysObject, self).__init__('ssh-authorized-keys', standalone=standalone, **kwargs)
self.__data = self.__pseudofile.getvalue() self.__data = self.__pseudofile.getvalue()
self.generate_attributes() self.generate_attributes()

2
setup.py
View File

@ -41,7 +41,7 @@ setup(
], ],
install_requires=['six', 'requests', 'python-dateutil', 'jsonschema', install_requires=['six', 'requests', 'python-dateutil', 'jsonschema',
'python-dateutil', 'enum34;python_version<"3.4"', 'python-dateutil', 'enum34;python_version<"3.4"',
'functools32;python_version<"3.0"', 'deprecated'], 'functools32;python_version<"3.0"', 'deprecated', 'cachetools;python_version<"3.0"'],
extras_require={'fileobjects': ['lief>=0.8,<0.10;python_version<"3.5"', 'lief>=0.10.0.dev0;python_version>"3.5"', 'python-magic', 'pydeep'], extras_require={'fileobjects': ['lief>=0.8,<0.10;python_version<"3.5"', 'lief>=0.10.0.dev0;python_version>"3.5"', 'python-magic', 'pydeep'],
'neo': ['py2neo'], 'neo': ['py2neo'],
'openioc': ['beautifulsoup4'], 'openioc': ['beautifulsoup4'],

8
tests/mispevent_testfiles/event_obj_def_param.json
View File

@ -14,6 +14,14 @@
"to_ids": true, "to_ids": true,
"type": "filename", "type": "filename",
"value": "bar" "value": "bar"
},
{
"category": "Artifacts dropped",
"disable_correlation": false,
"object_relation": "pattern-in-file",
"to_ids": true,
"type": "pattern-in-file",
"value": "baz"
} }
], ],
"description": "File object describing a file with meta-information", "description": "File object describing a file with meta-information",

4
tests/mispevent_testfiles/existing_event.json
View File

@ -3919,7 +3919,7 @@
"date": "2017-12-14", "date": "2017-12-14",
"distribution": "3", "distribution": "3",
"id": "9616", "id": "9616",
"info": "OSINT - Attackers Deploy New ICS Attack Framework “TRITON” and Cause Operational Disruption to Critical Infrastructure", "info": "OSINT - Attackers Deploy New ICS Attack Framework \"TRITON\" and Cause Operational Disruption to Critical Infrastructure",
"org_id": "2", "org_id": "2",
"orgc_id": "2", "orgc_id": "2",
"published": false, "published": false,
@ -4019,7 +4019,7 @@
"date": "2017-10-23", "date": "2017-10-23",
"distribution": "3", "distribution": "3",
"id": "9208", "id": "9208",
"info": "Talos: “Cyber Conflict” Decoy Document Used In Real Cyber Conflict", "info": "Talos: \"Cyber Conflict\" Decoy Document Used In Real Cyber Conflict",
"org_id": "291", "org_id": "291",
"orgc_id": "291", "orgc_id": "291",
"published": true, "published": true,

4
tests/mispevent_testfiles/existing_event_edited.json
View File

@ -3922,7 +3922,7 @@
"date": "2017-12-14", "date": "2017-12-14",
"distribution": "3", "distribution": "3",
"id": "9616", "id": "9616",
"info": "OSINT - Attackers Deploy New ICS Attack Framework “TRITON” and Cause Operational Disruption to Critical Infrastructure", "info": "OSINT - Attackers Deploy New ICS Attack Framework \"TRITON\" and Cause Operational Disruption to Critical Infrastructure",
"org_id": "2", "org_id": "2",
"orgc_id": "2", "orgc_id": "2",
"published": false, "published": false,
@ -4022,7 +4022,7 @@
"date": "2017-10-23", "date": "2017-10-23",
"distribution": "3", "distribution": "3",
"id": "9208", "id": "9208",
"info": "Talos: “Cyber Conflict” Decoy Document Used In Real Cyber Conflict", "info": "Talos: \"Cyber Conflict\" Decoy Document Used In Real Cyber Conflict",
"org_id": "291", "org_id": "291",
"orgc_id": "291", "orgc_id": "291",
"published": true, "published": true,

2
tests/test.py
View File

@ -6,7 +6,7 @@ try:
except ImportError as e: except ImportError as e:
print(e) print(e)
url = 'https://localhost:8443' url = 'https://localhost:8443'
key = 'K5yV0CcxdnklzDfCKlnPniIxrMX41utQ2dG13zZ3' key = 'd6OmdDFvU3Seau3UjwvHS1y3tFQbaRNhJhDX0tjh'
import time import time

44
tests/test_mispevent.py
View File

@ -26,20 +26,20 @@ class TestMISPEvent(unittest.TestCase):
def test_simple(self): def test_simple(self):
with open('tests/mispevent_testfiles/simple.json', 'r') as f: with open('tests/mispevent_testfiles/simple.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_event(self): def test_event(self):
self.init_event() self.init_event()
self.mispevent.publish() self.mispevent.publish()
with open('tests/mispevent_testfiles/event.json', 'r') as f: with open('tests/mispevent_testfiles/event.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_loadfile(self): def test_loadfile(self):
self.mispevent.load_file('tests/mispevent_testfiles/event.json') self.mispevent.load_file('tests/mispevent_testfiles/event.json')
with open('tests/mispevent_testfiles/event.json', 'r') as f: with open('tests/mispevent_testfiles/event.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_event_tag(self): def test_event_tag(self):
self.init_event() self.init_event()
@ -50,7 +50,7 @@ class TestMISPEvent(unittest.TestCase):
self.mispevent.add_tag(new_tag) self.mispevent.add_tag(new_tag)
with open('tests/mispevent_testfiles/event_tags.json', 'r') as f: with open('tests/mispevent_testfiles/event_tags.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_attribute(self): def test_attribute(self):
self.init_event() self.init_event()
@ -62,13 +62,13 @@ class TestMISPEvent(unittest.TestCase):
self.assertEqual(attr_tags[0].name, 'osint') self.assertEqual(attr_tags[0].name, 'osint')
with open('tests/mispevent_testfiles/attribute.json', 'r') as f: with open('tests/mispevent_testfiles/attribute.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
# Fake setting an attribute ID for testing # Fake setting an attribute ID for testing
self.mispevent.attributes[0].id = 42 self.mispevent.attributes[0].id = 42
self.mispevent.delete_attribute(42) self.mispevent.delete_attribute(42)
with open('tests/mispevent_testfiles/attribute_del.json', 'r') as f: with open('tests/mispevent_testfiles/attribute_del.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_object_tag(self): def test_object_tag(self):
self.mispevent.add_object(name='file', strict=True) self.mispevent.add_object(name='file', strict=True)
@ -90,7 +90,7 @@ class TestMISPEvent(unittest.TestCase):
self.assertEqual(self.mispevent.objects[0].references[0].relationship_type, 'baz') self.assertEqual(self.mispevent.objects[0].references[0].relationship_type, 'baz')
with open('tests/mispevent_testfiles/event_obj_attr_tag.json', 'r') as f: with open('tests/mispevent_testfiles/event_obj_attr_tag.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
@unittest.skip("Not supported on MISP: https://github.com/MISP/MISP/issues/2638 - https://github.com/MISP/PyMISP/issues/168") @unittest.skip("Not supported on MISP: https://github.com/MISP/MISP/issues/2638 - https://github.com/MISP/PyMISP/issues/168")
def test_object_level_tag(self): def test_object_level_tag(self):
@ -100,7 +100,7 @@ class TestMISPEvent(unittest.TestCase):
self.mispevent.objects[0].uuid = 'a' self.mispevent.objects[0].uuid = 'a'
with open('tests/mispevent_testfiles/event_obj_tag.json', 'r') as f: with open('tests/mispevent_testfiles/event_obj_tag.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_malware(self): def test_malware(self):
with open('tests/mispevent_testfiles/simple.json', 'rb') as f: with open('tests/mispevent_testfiles/simple.json', 'rb') as f:
@ -112,7 +112,7 @@ class TestMISPEvent(unittest.TestCase):
self.assertEqual(attribute.malware_binary, pseudofile) self.assertEqual(attribute.malware_binary, pseudofile)
with open('tests/mispevent_testfiles/malware.json', 'r') as f: with open('tests/mispevent_testfiles/malware.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_existing_malware(self): def test_existing_malware(self):
self.mispevent.load_file('tests/mispevent_testfiles/malware_exist.json') self.mispevent.load_file('tests/mispevent_testfiles/malware_exist.json')
@ -127,19 +127,20 @@ class TestMISPEvent(unittest.TestCase):
sighting.from_dict(value='1', type='bar', timestamp=11111111) sighting.from_dict(value='1', type='bar', timestamp=11111111)
with open('tests/mispevent_testfiles/sighting.json', 'r') as f: with open('tests/mispevent_testfiles/sighting.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(sighting.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(sighting.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_existing_event(self): def test_existing_event(self):
self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json') self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json')
with open('tests/mispevent_testfiles/existing_event.json', 'r') as f: with open('tests/mispevent_testfiles/existing_event.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2))
self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_shadow_attributes_existing(self): def test_shadow_attributes_existing(self):
self.mispevent.load_file('tests/mispevent_testfiles/shadow.json') self.mispevent.load_file('tests/mispevent_testfiles/shadow.json')
with open('tests/mispevent_testfiles/shadow.json', 'r') as f: with open('tests/mispevent_testfiles/shadow.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
@unittest.skip("Not supported on MISP.") @unittest.skip("Not supported on MISP.")
def test_shadow_attributes(self): def test_shadow_attributes(self):
@ -152,12 +153,15 @@ class TestMISPEvent(unittest.TestCase):
del p.uuid del p.uuid
with open('tests/mispevent_testfiles/proposals.json', 'r') as f: with open('tests/mispevent_testfiles/proposals.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_default_attributes(self): def test_default_attributes(self):
self.mispevent.add_object(name='file', strict=True) self.mispevent.add_object(name='file', strict=True)
a = self.mispevent.objects[0].add_attribute('filename', value='bar', Tag=[{'name': 'blah'}]) a = self.mispevent.objects[0].add_attribute('filename', value='bar', Tag=[{'name': 'blah'}])
del a.uuid del a.uuid
a = self.mispevent.objects[0].add_attribute('pattern-in-file', value='baz')
self.assertEqual(a.category, 'Artifacts dropped')
del a.uuid
self.mispevent.add_object(name='file', strict=False, default_attributes_parameters=self.mispevent.objects[0].attributes[0]) self.mispevent.add_object(name='file', strict=False, default_attributes_parameters=self.mispevent.objects[0].attributes[0])
a = self.mispevent.objects[1].add_attribute('filename', value='baz') a = self.mispevent.objects[1].add_attribute('filename', value='baz')
del a.uuid del a.uuid
@ -165,7 +169,7 @@ class TestMISPEvent(unittest.TestCase):
self.mispevent.objects[1].uuid = 'b' self.mispevent.objects[1].uuid = 'b'
with open('tests/mispevent_testfiles/event_obj_def_param.json', 'r') as f: with open('tests/mispevent_testfiles/event_obj_def_param.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_obj_default_values(self): def test_obj_default_values(self):
self.init_event() self.init_event()
@ -181,7 +185,7 @@ class TestMISPEvent(unittest.TestCase):
self.mispevent.objects[0].uuid = 'a' self.mispevent.objects[0].uuid = 'a'
with open('tests/mispevent_testfiles/def_param.json', 'r') as f: with open('tests/mispevent_testfiles/def_param.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_event_not_edited(self): def test_event_not_edited(self):
self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json') self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json')
@ -246,7 +250,7 @@ class TestMISPEvent(unittest.TestCase):
self.assertTrue(self.mispevent.edited) self.assertTrue(self.mispevent.edited)
with open('tests/mispevent_testfiles/existing_event_edited.json', 'r') as f: with open('tests/mispevent_testfiles/existing_event_edited.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
def test_obj_by_id(self): def test_obj_by_id(self):
self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json') self.mispevent.load_file('tests/mispevent_testfiles/existing_event.json')
@ -258,7 +262,7 @@ class TestMISPEvent(unittest.TestCase):
self.mispevent.add_object(name='test_object_template', strict=True, misp_objects_path_custom='tests/mispevent_testfiles') self.mispevent.add_object(name='test_object_template', strict=True, misp_objects_path_custom='tests/mispevent_testfiles')
with self.assertRaises(InvalidMISPObject) as e: with self.assertRaises(InvalidMISPObject) as e:
# Fail on required # Fail on required
self.mispevent.to_json() self.mispevent.to_json(sort_keys=True, indent=2)
if sys.version_info >= (3, ): if sys.version_info >= (3, ):
self.assertEqual(e.exception.message, '{\'member3\'} are required.') self.assertEqual(e.exception.message, '{\'member3\'} are required.')
else: else:
@ -269,7 +273,7 @@ class TestMISPEvent(unittest.TestCase):
del a.uuid del a.uuid
with self.assertRaises(InvalidMISPObject) as e: with self.assertRaises(InvalidMISPObject) as e:
# Fail on requiredOneOf # Fail on requiredOneOf
self.mispevent.to_json() self.mispevent.to_json(sort_keys=True, indent=2)
self.assertEqual(e.exception.message, 'At least one of the following attributes is required: member1, member2') self.assertEqual(e.exception.message, 'At least one of the following attributes is required: member1, member2')
a = self.mispevent.objects[0].add_attribute('member1', value='bar') a = self.mispevent.objects[0].add_attribute('member1', value='bar')
@ -278,14 +282,14 @@ class TestMISPEvent(unittest.TestCase):
del a.uuid del a.uuid
with self.assertRaises(InvalidMISPObject) as e: with self.assertRaises(InvalidMISPObject) as e:
# member1 is not a multiple # member1 is not a multiple
self.mispevent.to_json() self.mispevent.to_json(sort_keys=True, indent=2)
self.assertEqual(e.exception.message, 'Multiple occurrences of member1 is not allowed') self.assertEqual(e.exception.message, 'Multiple occurrences of member1 is not allowed')
self.mispevent.objects[0].attributes = self.mispevent.objects[0].attributes[:2] self.mispevent.objects[0].attributes = self.mispevent.objects[0].attributes[:2]
self.mispevent.objects[0].uuid = 'a' self.mispevent.objects[0].uuid = 'a'
with open('tests/mispevent_testfiles/misp_custom_obj.json', 'r') as f: with open('tests/mispevent_testfiles/misp_custom_obj.json', 'r') as f:
ref_json = json.load(f) ref_json = json.load(f)
self.assertEqual(self.mispevent.to_json(), json.dumps(ref_json, sort_keys=True, indent=2)) self.assertEqual(self.mispevent.to_json(sort_keys=True, indent=2), json.dumps(ref_json, sort_keys=True, indent=2))
if __name__ == '__main__': if __name__ == '__main__':

62
tests/testlive_comprehensive.py
View File

@ -25,7 +25,7 @@ import logging
logging.disable(logging.CRITICAL) logging.disable(logging.CRITICAL)
try: try:
from pymisp import ExpandedPyMISP, MISPEvent, MISPOrganisation, MISPUser, Distribution, ThreatLevel, Analysis, MISPObject, MISPAttribute, MISPSighting, MISPShadowAttribute, MISPTag, MISPSharingGroup, MISPFeed, MISPServer from pymisp import ExpandedPyMISP, MISPEvent, MISPOrganisation, MISPUser, Distribution, ThreatLevel, Analysis, MISPObject, MISPAttribute, MISPSighting, MISPShadowAttribute, MISPTag, MISPSharingGroup, MISPFeed, MISPServer, MISPUserSetting
from pymisp.tools import CSVLoader, DomainIPObject, ASNObject, GenericObjectGenerator from pymisp.tools import CSVLoader, DomainIPObject, ASNObject, GenericObjectGenerator
from pymisp.exceptions import MISPServerError from pymisp.exceptions import MISPServerError
except ImportError: except ImportError:
@ -1577,7 +1577,7 @@ class TestComprehensive(unittest.TestCase):
remote_types = remote.pop('types') remote_types = remote.pop('types')
remote_categories = remote.pop('categories') remote_categories = remote.pop('categories')
remote_category_type_mappings = remote.pop('category_type_mappings') remote_category_type_mappings = remote.pop('category_type_mappings')
local = self.admin_misp_connector.describe_types_local local = dict(self.admin_misp_connector.describe_types_local)
local_types = local.pop('types') local_types = local.pop('types')
local_categories = local.pop('categories') local_categories = local.pop('categories')
local_category_type_mappings = local.pop('category_type_mappings') local_category_type_mappings = local.pop('category_type_mappings')
@ -1945,6 +1945,62 @@ class TestComprehensive(unittest.TestCase):
# Delete event # Delete event
self.admin_misp_connector.delete_event(first) self.admin_misp_connector.delete_event(first)
def test_user_settings(self):
first = self.create_simple_event()
first.distribution = 3
first.add_tag('test_publish_filter')
first.add_tag('test_publish_filter_not')
second = self.create_simple_event()
second.distribution = 3
try:
# Set
setting = self.admin_misp_connector.set_user_setting('dashboard_access', 1, pythonify=True)
setting_value = {'Tag.name': 'test_publish_filter'}
setting = self.admin_misp_connector.set_user_setting('publish_alert_filter', setting_value, pythonify=True)
self.assertTrue(isinstance(setting, MISPUserSetting))
self.assertEqual(setting.value, setting_value)
# Get
# FIXME: https://github.com/MISP/MISP/issues/5297
# setting = self.admin_misp_connector.get_user_setting('dashboard_access', pythonify=True)
# Get All
user_settings = self.admin_misp_connector.user_settings(pythonify=True)
# TODO: Make that one better
self.assertTrue(isinstance(user_settings, list))
# Test if publish_alert_filter works
first = self.admin_misp_connector.add_event(first, pythonify=True)
second = self.admin_misp_connector.add_event(second, pythonify=True)
r = self.user_misp_connector.change_user_password('Password1234')
self.assertEqual(r['message'], 'Password Changed.')
self.test_usr.autoalert = True
self.test_usr.termsaccepted = True
user = self.user_misp_connector.update_user(self.test_usr, pythonify=True)
self.assertTrue(user.autoalert)
self.admin_misp_connector.publish(first, alert=True)
self.admin_misp_connector.publish(second, alert=True)
time.sleep(10)
# FIXME https://github.com/MISP/MISP/issues/4872
# mail_logs = self.admin_misp_connector.search_logs(model='User', action='email', limit=2, pythonify=True)
mail_logs = self.admin_misp_connector.search_logs(model='User', action='email', created=datetime.now() - timedelta(seconds=30), pythonify=True)
if mail_logs:
# FIXME: On travis, the mails aren't working, so we stik that.
self.assertEqual(len(mail_logs), 3)
self.assertTrue(mail_logs[0].title.startswith(f'Email to {self.admin_misp_connector._current_user.email}'), mail_logs[0].title)
self.assertTrue(mail_logs[1].title.startswith(f'Email to {self.user_misp_connector._current_user.email}'), mail_logs[1].title)
self.assertTrue(mail_logs[2].title.startswith(f'Email to {self.user_misp_connector._current_user.email}'), mail_logs[2].title)
# Delete
# FIXME: https://github.com/MISP/MISP/issues/5297
# response = self.admin_misp_connector.delete_user_setting('publish_alert_filter')
finally:
self.test_usr.autoalert = False
self.user_misp_connector.update_user(self.test_usr)
# Delete event
self.admin_misp_connector.delete_event(first)
self.admin_misp_connector.delete_event(second)
@unittest.skipIf(sys.version_info < (3, 6), 'Not supported on python < 3.6') @unittest.skipIf(sys.version_info < (3, 6), 'Not supported on python < 3.6')
def test_communities(self): def test_communities(self):
communities = self.admin_misp_connector.communities(pythonify=True) communities = self.admin_misp_connector.communities(pythonify=True)
@ -1977,7 +2033,7 @@ class TestComprehensive(unittest.TestCase):
finally: finally:
# Delete event # Delete event
self.admin_misp_connector.delete_event(first) self.admin_misp_connector.delete_event(first)
self.admin_misp_connector.delete_event(second['Event']['id']) self.admin_misp_connector.delete_event(second)
if __name__ == '__main__': if __name__ == '__main__':

1
tests/viper-test-files Submodule

@ -0,0 +1 @@
Subproject commit 47d3c9b8d7ae69fd7a2681dd33925c055018049e

3
travis/install_travis.sh
View File

@ -11,6 +11,3 @@ else
pip install pipenv pip install pipenv
pipenv update --dev pipenv update --dev
fi fi
pushd tests
git clone https://github.com/viper-framework/viper-test-files.git
popd