Commit Graph

534 Commits (d48a013f3a06c740a03ac16cdd4e256f2f0763c7)

Author SHA1 Message Date
Richard van den Berg 9da8ed86c5 Fix errors on NIDS export when whitelist is empty 2019-07-17 12:46:43 +02:00
mokaddem e7f3d0d9df new: [timeline/*-seen] Initial import of the timeline code from the
zoidberg branch
2019-06-13 09:16:34 +02:00
mokaddem 52ae153c0e Merge branch '2.4' of github.com:MISP/MISP into galaxyMatrixImprovements 2019-06-11 15:56:10 +02:00
mokaddem 11a4bdb959 chg: [restSearch:attack] Only expose attack return format to the `event`
scope
2019-06-11 15:50:51 +02:00
Richard van den Berg 22cc03bb23 Match EDNS packets with snort rules 2019-05-17 14:34:18 +02:00
mokaddem bd1b5f6e97 chg: [export:attack] Performance improvements 2019-05-15 11:06:27 +02:00
Alexandre Dulaunoy 97ab3ddca0
chg: [yara export] fix the correct Python version is used 2019-05-13 21:33:25 +02:00
mokaddem 4fbe857f90 chg: [galaxyMatrix] Added sorting by score. Fix #4608 2019-05-13 15:07:38 +02:00
iglocska c54839d2eb Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-05-10 14:42:25 +02:00
iglocska 5ee6013dff fix: [API] Some fixes for the restsearch -> attack export 2019-05-10 14:41:50 +02:00
iglocska e899eb8b9d new: [ATT&CK] Added new export system for restsearch for ATT&CK
- Return the ATT&CK matrix data as HTML via the API
- Directly viewable via the REST client

- Greetings from the ATT&CK workshop @ Eurocontrol
2019-05-10 14:25:38 +02:00
Steve Clement 45e6d740f0 fix: [bug] getPythonVersion undefined, pull in where it is defined. 2019-05-10 07:23:14 +09:00
frpet 38a64e0ba9 rpz: action policy rename (to Local-Data)
Rename action policy "walled-garden" to "Local-Data" as per the IETF draft (and other documentation for RPZ)
2019-05-08 15:54:34 +02:00
Andras Iklody 3c6a336774
Merge pull request #4581 from pettai/RPZ-policy-action
RPZ - Add additional policy actions
2019-05-07 17:03:27 +02:00
frpet 76fcc6553a Add additional policy actions
Add the last policy actions from the RPZ draft.
* rpz-passthru allows for testing without applying changes on the returned answer.
* TCP-only forces the client over to use TCP.
2019-05-07 16:29:32 +02:00
Steve Clement fc1f15c4c0 fix: [export] Yara Export variable typo fix. Use getPythonVersion. 2019-05-07 08:37:47 +09:00
mokaddem 93673b4d4c chg: [distributionNetwork] Filter out organisations not being marked as
local. Fix #4568
2019-05-03 15:32:02 +02:00
edhoedt b9463e513c Yara export 2019-04-29 19:23:14 +02:00
iglocska 69c6562888 fix: [freetext] Also trim out no-break spaces
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
░░░░░░░░███████████████░░░░░░░░░░
░░░░░░███████████████████░░░░░░░░
░░░░░███░░░░░░░░░░░░░░████░░░░░░░
░░░░██░░░░░░░░░░░░░░░░░░░███░░░░░
░░░██░░░░░░░░░░░░░░░░░░░░░███░░░░
░░██░░███████░░░░░░██████░░██░░░░
░██░░██─────██░░░░██────██░░██░░░
░██░░█▄▄▄▄▄▄▄██░░░█▄▄▄▄▄▄██░░██░░
░██░░████─────█░░░████────█░░░██░
░██░░█────────█░░░█───────█░░░██░
██░░░██──────██░░░██─────██░░░░██
██░░░░████████░░░░░███████░░░░░██
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░███████████████░░░░░░░░░░░█
█░░░████░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██
░██░░░░░░░░░░░░░░░░░░░░░░░░░░░░█░
░░███░░░░░░░░░░░░░░░░░░░░░░░░░██░
░░░░██░░░░░░░░░░░░░░░░░░░░░░░██░░
2019-04-26 09:39:10 +02:00
chrisr3d c527077b1c Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-04-12 15:25:22 +02:00
chrisr3d 50c18eebb1
fix: [stix restSearch] Fixed output json format in case of empty results 2019-04-12 14:46:57 +02:00
chrisr3d 2b8f655415
fix: [stix restSearch[ Quick file extension clarification
- Depending on the format (.stix or .stix2)
- Impacting temporary files, it is thus for
  debugging purpose in case of error
2019-04-12 14:41:54 +02:00
iglocska 8076dbfad1 fix: [refanging] Removed invalid pattern 2019-04-09 15:51:13 +02:00
iglocska 5eecd75e5b fix: [bug] Typo causing "\" to be stripped from attributes where it shouldn't be stripped 2019-04-09 15:48:29 +02:00
iglocska a3381b8196 new: [refanging] Attributes automatically refanged in beforeValidate, fixes #4442 2019-04-09 14:53:39 +02:00
iglocska 510b781762 fix: [freetext] Stop parsing dates as phone numbers 2019-03-20 13:10:53 +01:00
iglocska 09ae8a5364 fix: [bro] typo fixed that caused an exception, fixes #4343 2019-03-20 07:02:50 +01:00
iglocska beed84a335 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2019-03-19 17:23:22 +01:00
Liviu Valsan 4656a5c1fa Fixing bug when exporting to Bro MISP attributes from events that contain a percentage sign inside the event info 2019-03-19 14:46:16 +01:00
Nikos Filippakis 9d59b10368 Publish events to Kafka
Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2019-03-18 15:53:22 +01:00
Andras Iklody 37e9ebdc39
Merge pull request #4313 from pettai/RPZexport
Add $time for Plugin.RPZ_serial
2019-03-15 17:41:34 +02:00
frpet 06b1d74c7a Add $time for serial
Add $time for generating unixtime as serial
2019-03-15 15:28:14 +01:00
mokaddem b8334521d3 chg: Updated comments 2019-03-15 11:52:37 +01:00
mokaddem 7ce6ef34de Merge branch '2.4' of github.com:MISP/MISP into extendedDistributionGraph 2019-03-15 10:59:38 +01:00
iglocska fb26e3495f chg: [cakephp version] bump 2019-03-14 08:46:10 +01:00
mokaddem d67af3c4dd fix: [distributionGraph] Transform associative array into regular array 2019-03-12 10:59:51 +01:00
mokaddem 8a5cce8cb3 chg: [distributionNetwork] Improved consistency in event index and
improved UX - WiP
2019-03-12 10:47:16 +01:00
mokaddem ead0b96e13 chg: [distributionGraphNetwork] Adjusted gravitationalConstant and mass 2019-03-01 14:55:51 +01:00
mokaddem 24d8f197ef new: [DistributionGraph] Added pie chart on sharing group. fix #4101 2019-03-01 10:56:33 +01:00
mokaddem ef045e01b3 chg: [distributionGraph] Added support of sharing group - WiP 2019-02-27 15:34:20 +01:00
mokaddem 431529c81c chg: [attackMatrix] UI: improved color scale - WiP 2019-02-11 17:54:29 +01:00
chrisr3d 4f4fe45633
fix: [stix2 export] Fixed attribute counting on restSearch 2019-02-02 12:09:52 +01:00
iglocska 4dd53eb8e3 fix: [freetext import] Handle cases where a value can be both a hash and a btc address better 2019-01-22 08:58:02 +01:00
iglocska a9a47fb46c new: [cache export] Added the includeEventUuid flag to the output 2019-01-17 15:04:01 +01:00
Daniel Roethlisberger 5b4079637a new: [attributes] Add cdhash attribute, 40+ digit hash, default Payload delivery, ids=1 (#3965) 2018-12-19 20:19:49 +01:00
chrisr3d 87190f6510
fix: [restSearch] Using the correct python version to call STIX scripts
- Using the correct python defined in virtual env,
  if available, and the default global python3
  otherwise
2018-12-07 15:31:04 +01:00
Anthony Vaccaro eab6ca62e3 fix typo in called method name 2018-11-27 10:02:25 +10:00
iglocska db5d61725a chg: [bro] Preparation for the move to restsearch
- also fixed some edge-case issues
2018-11-24 21:35:50 +01:00
iglocska 86a27e7c31 fix: [CSV] Fixed some defaults for the CSV export 2018-11-23 13:47:06 +01:00
iglocska a28909d366 new: [freetext] Added BTC recognition, fixes #3864 2018-11-13 12:06:00 +01:00
iglocska ce3c78cd7d Merge branch 'sighting_api' into 2.4 2018-10-29 20:20:17 +01:00
iglocska bbc8a8bf4d fix: [API] minor fixes to the sightings api
- fixed duplicate sighting tags in XML output
- added attribute value to the sighting
2018-10-29 20:18:29 +01:00
Hannah Ward 608ddaa969
fix: aws would error if asked to del non-existing 2018-10-26 11:01:44 +01:00
Sami Mokaddem ff5f5faf02 new: [sighting/api] xml output format + improved error feedback 2018-10-23 13:06:37 +02:00
Sami Mokaddem 01cba114f2 fix: [sightings/api] now support json output format 2018-10-23 12:17:54 +02:00
Sami Mokaddem 99e5f560a8 new: [sighting/api] trying to follow the new API architecture. JSON
export is broken but CSV is working. WIP...
2018-10-23 11:24:03 +02:00
iglocska 1187fb2a27 new: [API] Added CSV as return format for event index 2018-10-21 22:47:22 +02:00
www-data f9183dee3b Merge branch '2.4' into py-virtualenv 2018-10-15 17:09:18 +09:00
iglocska 158d0580b3 new: [API] Added a new export that simply hashes all values with a requested hash format 2018-10-09 11:21:35 +02:00
iglocska 40b1a4a271 chg: [CSV] Added timestamp in CSV output with include context on the event level 2018-10-08 21:43:23 +02:00
Steve Clement e26e4a2e92 Merge branch '2.4' into py-virtualenv 2018-10-08 07:45:04 +08:00
iglocska abc83000c0 fix: [CSV] boolean fields should be set to 1/0 instead of true/false 2018-10-06 18:59:24 +02:00
chrisr3d 69c4b58638 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-10-05 14:59:33 +02:00
chrisr3d a42b3ab756
fix: [restSearch] Avoiding useless stix python script calls on empty files 2018-10-05 14:58:39 +02:00
iglocska 77258728ee Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-10-05 14:50:33 +02:00
iglocska 1c8b17416f chg: [API] made the CSV export type less restrictive by default (to_ids / published ignored by default) 2018-10-05 14:49:12 +02:00
chrisr3d d65482c297
fix: [restSearch] Ignoring square brackets around STIX2 objects returned by the python script
Because they are already provided by the framing script
2018-10-05 10:28:48 +02:00
chrisr3d 09a138fd38 Merge branch '2.4' of github.com:MISP/MISP into chrisr3d_restSearch_tests 2018-10-04 23:09:11 +02:00
chrisr3d 445bd0c84c
add: [restSearch] STIX 1 & 2 export for restSearch
Features to be merged:
- Export of multiple MISP events
- Fetching events and writing them into files, each
  file containing at most a number of attributes
  defined by a limit
- Each file is then parsed instead of parsing each
  event individualy, which reduces the number of
  times the python scripts are called, reducing
  the execution time of the overall process
- The result is then returned as on single file
  read and displayed
2018-10-04 22:11:30 +02:00
chrisr3d 54b90cf8f5
fix: [restSearch] Changed how data is handled eeeeeeeeeee
- Criteria was number of events and is now number
  of attributes
- Writing data in a file until the limit number of
  attributes is reached, then writing in the next
  file and looping again until all data is written
- Then for each file, calling the python script to
  parse MISP events and translate them into STIX
- Writing parsed STIX data into 1 file used to
  return the result
2018-10-04 11:40:45 +02:00
iglocska e461029b9d new: [API] CSV export tool completely reworked 2018-10-03 07:59:46 +02:00
Steve Clement a699c5fcd4 Merge branch '2.4' into py-virtualenv 2018-10-02 20:55:46 +08:00
chrisr3d 5a87b64c74
Merge branch '2.4' of github.com:MISP/MISP into chrisr3d_restSearch_tests 2018-10-02 13:35:21 +02:00
iglocska 58b9a3c50b fix: [graph] Made the correlation graph aware of the new correlation loading 2018-10-02 07:35:34 +02:00
chrisr3d d6595e00c6
wip: [restSearch] Passing multiple events to the STIX parsing script
- atm calling the python script every 10 events
  fetched with fetchEvent
2018-09-30 20:32:38 +02:00
chrisr3d 4ac455d7c4 Merge branch '2.4' of github.com:MISP/MISP into chrisr3d_restSearch_tests 2018-09-29 19:20:19 +02:00
Steve Clement df3d87b5c8 Merge branch '2.4' into py-virtualenv 2018-09-28 15:00:46 +02:00
Steve Clement 05e0c412ef chg: [fix] Some fixed to the python virtualenv tweaks 2018-09-28 14:58:55 +02:00
Sami Mokaddem b66eeefffa fix: [eventGraph] prevents bug if object has no attributes 2018-09-28 12:05:46 +02:00
iglocska 46cb19df2d Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-09-28 09:24:40 +02:00
iglocska cfe86512a2 fix: [API] Fixed an invalid lookup in the openioc export 2018-09-27 23:02:59 +02:00
chrisr3d 72c0aa9987
fix: [restSearch] Prettifying stix packages with indents
- As it is in stix export function from Model/Event.php
2018-09-27 12:03:49 +02:00
chrisr3d aa69d6ca1a
fix: [cleanup] Fixed indentation in restSearch 2018-09-25 20:33:41 +02:00
chrisr3d 52115cdf85
fix: Added variable to have attribute with no ids flag from fetchEvent 2018-09-25 20:13:20 +02:00
chrisr3d 8a4911d18c
fix: [restSearch] Fixed variables & indent 2018-09-25 12:10:39 +02:00
chrisr3d 933af46dfb
wip: [restSearch] Stix1 export for restSearch 2018-09-24 14:52:33 +02:00
iglocska cefab3e01c new: [freetext import] Added detection for AS 2018-09-23 11:43:55 +02:00
iglocska 48c6150257 new: [Complex type tool] Detection of [1] style refanging 2018-09-21 15:08:33 +02:00
iglocska 2e7dfc9273 new: [API] Correctly handle objects in flat exports and exposed text export to event level search 2018-09-14 14:34:01 +02:00
iglocska 51b3ef61dd new: [API] Added the includeEventTags parameter to the /attributes/restSearch API
- appends all event level tags to each attribute
2018-09-09 16:49:59 +02:00
iglocska f995b561fb Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-09-09 00:25:28 +02:00
iglocska 926e973179 new: [API] Updated the RPZ export to follow the new API patterns 2018-09-08 23:44:38 +02:00
iglocska bc9524c712 fix: [feeds] Feed caching generates a lot of notices 2018-09-07 13:29:26 +02:00
iglocska 289b13be88 new: [API] set default behaviour to require to_ids and published set to 1 to be included in exports
- doesn't affect MISP json and xml formats
2018-09-06 00:20:03 +02:00
iglocska bcfc1f3a1a fix: [API] Fixed the broken CSV export 2018-09-05 11:36:31 +02:00
iglocska 57a6460d81 chg: [internal] JSONConverterTool's support for the deprecated showorg flag removed 2018-09-03 17:53:51 +02:00
iglocska 924dfcbec9 new: [API] XML export now exports both event and attribute level data
- relying on the old XMLConverterTool for event level conversions
2018-09-03 17:53:03 +02:00
iglocska 9c725ebff1 new: [API] OpenIOC export library correctly handles both events and attributes as their payload
- fixed annoying line breaks in the output
2018-09-03 17:50:59 +02:00
iglocska 7c3ddacd1e new: [API] NIDS exports now correctly support event and attribute level exports
- also, suricata/snort rules now include both the event and the attribute tags in the metadata
2018-09-03 17:50:08 +02:00
iglocska ba5bafd13f new: [API] JSON export library updated to support both attribute and event level conversions.
- relies on the old JSON library for event level conversions
2018-09-03 17:49:03 +02:00