Commit Graph

1227 Commits (fe3cd9d81425af5d4fd8d8bacbb5390381fbbfc8)

Author SHA1 Message Date
iglocska 23a2611202 fix: [API] Some API rearrange issues fixed in events/add 2018-08-08 11:27:49 +02:00
iglocska 5215ee1c5a chg: [api] reworked the CSV api to use the new standardised function calls 2018-08-06 10:49:45 +02:00
iglocska 76ede22308 new: [refactor] CSV api refactor
- performance gains
- first step in unifying all APIs
- moved the CSV data lookup into fetchattributes
- internal pagination is now more clever with a watchdog flag that can prevent unneeded executions by whatever calls fetchattributes
2018-08-05 19:10:52 +02:00
chrisr3d 0ed3f0617c Merge branch '2.4' of github.com:MISP/MISP into stix2 2018-08-03 13:51:33 +02:00
chrisr3d afbb83a827
fix: [import modules] Avoiding issues with userConfig when module is csvimport
- If users tick the checkbox to specify there is a
  header in the csv file to import, there should
  not be an error with empty userConfig header
2018-08-03 13:32:51 +02:00
chrisr3d d85131f538
fix: [stix2 export] Fixed syntax in stix2 function 2018-07-30 23:52:41 +02:00
chrisr3d ce6c8752c0 Merge branch '2.4' of github.com:MISP/MISP into stix2 2018-07-30 15:09:43 +02:00
Steve Clement 8402df48f3 chg: [except] Closed the brackets correctly on the throw except 2018-07-28 09:34:51 +02:00
iglocska 959628a4f8 chg: [csv] added the object_relation field to the CSV export 2018-07-27 14:06:38 +02:00
chrisr3d 5dc761ea7b
chg: [stix2 export] Using the RestResponse view call instead of having view files 2018-07-26 12:10:57 +02:00
chrisr3d 53ccf51e71
chg: [stix2 export] Multiple events export prepared in Controller & Model side
- Changes on automation side coming soon
2018-07-20 23:59:51 +02:00
iglocska a81894f14c chg: [CS] Changed to PSR-2
- to make contributions easier, adopted PSR-2
- used php-cs-fixer to rework the style
- *sniff sniff* Goodbye tab indentation
2018-07-19 11:48:22 +02:00
Steve Clement c607729e18 chg: [i18n] More __(); 2018-07-12 23:36:47 +02:00
Andras Iklody ad15ffa7ce
Merge pull request #3460 from kalyparker/fix-export-events-csv
fix: export events csv with CR (fix #3458)
2018-07-10 09:28:22 +02:00
kalyparker 34f15268d2 fix: export events csv with CR (fix #3458)
Export using automation functionnality for ids does not clean the special char like CRLF.
When there is a carriage return in the event info, the csv is broken.
2018-07-09 08:58:12 -07:00
Sami Mokaddem eb1b8bcba5 chg: [attackMatrix] support of quick tagging from the attackMatrix at
event view level
2018-07-09 09:55:17 +00:00
iglocska 8d567782d9 chg: [cleanup] Removed the deprecated GFI sandbox import
- Burn the heretic. Kill the mutant. Purge the unclean.
2018-07-06 10:57:44 +02:00
iglocska 5ba322076b chg: [refactor] Fixed an issue where too many events would cause a query too large for mysql to handle when querying /events/index via the API, fixes #3444 2018-07-05 18:52:25 +02:00
iglocska 73c18f8833 new: [API] Updated the timestmap handling in the restSearch APIs to use the new smart-system 2018-07-04 15:53:01 +02:00
iglocska c3158b50ba new: [edit strategy API] To support a smoother integration with the Hive, new API that describes what the edit strategy is for an event
- GET on /events/getEditStrategy/[id]
  - where id can be either a local ID or a UUID

- returns a JSON dictionary with the following fields:
  - strategy: edit | extend (edit if it's an own event, extend otherwise)
  - extensions: list of dictionaries with existing extensions created by the user's org (containing the id, uuid, info fields)

- The algorithms implementing this should prioritise as such:

1. Check if user can edit the event (strategy == edit) - if yes, edit
2. If no, check if extensions exist - if yes, edit one of those
3. If no, create a new extension to the original event
2018-07-02 17:29:53 +02:00
iglocska 4bff6092e5 fix: Correlation popup format 2018-06-23 23:59:34 +02:00
iglocska c721142f9b fix: [sync] pull not working due to invalid lookup against galaxies 2018-06-20 17:03:31 +02:00
iglocska 505cccdbaf fix: [error messages] made some of the error messages a bit more uniform 2018-06-20 15:12:23 +02:00
Sami Mokaddem e3988c73ad new: [attackMatrix] Also consider attack galaxy at event level in the
heatmap
fix: [attackMatrix] Typo in ATT&CK + division by 0 in gradiendTool
2018-06-18 14:51:29 +00:00
Sami Mokaddem 3a27009775 Merge remote-tracking branch 'upstream/2.4' into attack 2018-06-18 12:18:31 +00:00
Sami Mokaddem 929946f055 new: [attackMatrix] added instance UUID in rest response 2018-06-18 12:04:38 +00:00
Sami Mokaddem 8d145086f0 new: [attackMatrix] statistic about attack tags used in the instance
chg: [attackMatrix] moved functions in to model and matrix view into elements
2018-06-18 09:58:20 +00:00
Sami Mokaddem bc156ab13a new: [AttackMatrix] added Mobile/Pre-Attack Matrix support, UI
improvements and code refacto
2018-06-15 09:19:53 +00:00
iglocska 3e5e432436 fix: Fixed permission check for adding tags to an event 2018-06-12 17:01:54 +02:00
Sami Mokaddem 95e694f054 fix: [AttackMatrix] picking Att&ck tactic correctly redirect on the
matrix
2018-06-12 14:25:43 +00:00
Sami Mokaddem 6c8bdeaff6 chg: [attackMatrix] Restrict view to be ajax only 2018-06-12 13:16:23 +00:00
Sami Mokaddem 5f36725ede new: [attackMatrix] Ability to attach Mitre att&ck galaxy from the
matrix
2018-06-12 12:39:48 +00:00
iglocska 6890b734cc new: [UI/UX] Event lock initial version
- Show if another user is editing the event you're viewing (same org only)
2018-06-12 09:40:23 +02:00
Sami Mokaddem 34c69d00e2 fix: [eventView] Hide galaxy tags after search 2018-06-11 14:05:45 +00:00
Sami Mokaddem 5d8c2ccf5e new: [attackMatrix] legend scale of the heatmap with dynamic updates 2018-06-11 10:24:55 +00:00
Sami Mokaddem 4fdf7f6340 new: [attackMatrix] force kill chaine header order 2018-06-08 14:28:42 +00:00
Sami Mokaddem 775001f2cc new: [attackMatrix] addition of heatmap on tiles depending on occurence
of the tag
2018-06-08 14:16:40 +00:00
Sami Mokaddem cd0d75a4c6 new: Initial skeleton of Mitre attack matrix 2018-06-07 14:43:04 +00:00
Sami Mokaddem 00ec493414 chg: [UI Filtering] Do not set searchFor in the URL if no value.
After a discussion with iglocksa, it is better to fix it js side than
server side.
2018-05-22 14:16:27 +00:00
Sami Mokaddem 4d39d3296a fix: [UI filtering] be sure that '0' is not interpreted as empty. 2018-05-22 13:49:03 +00:00
iglocska 5780d0c5d8 fix: [UI filtering] Attribute quick filter broke all the tabbed filters, fixes #3247 2018-05-22 11:29:17 +02:00
iglocska 68b8266584 new: New flash message system, fixes #3252
- 3 types of flash messages (success, error, warning)
- uses bootstrap's own classes/structure
2018-05-16 19:32:38 +02:00
iglocska 645d996c14 new: Remove galaxy cluster information from the sync mechanism for now
- currently galaxy clusters aren't shared anyway, no point in blowing up the data size / processing time
2018-05-15 07:44:37 +02:00
iglocska 962461890c new: Added attribute level galaxy clusters 2018-05-14 23:20:09 +02:00
iglocska fefe92bde8 new: [Export] Added a secondary CSV export that includes more context to the UI download tool 2018-05-09 14:10:23 +02:00
Sami Mokaddem 680311f68f chg: [Controllers] sets the ajax variable globally
As well as removing useless set in controllers and accessing it instead
of passing through the request.
2018-05-07 14:44:59 +00:00
Sami Mokaddem a3d6fb3497 chg: [EventController] replaced if/else by ternary condition 2018-05-04 06:32:59 +00:00
Sami Mokaddem 507cd0ee85 chg: Trying not to break the MVC pattern
Server model is not passed to the constructor anymore, as well as the
Organisation model.
2018-05-04 06:27:54 +00:00
Sami Mokaddem 6d476814b0 Merge remote-tracking branch 'upstream/2.4' into distributionGraphDonut 2018-05-03 13:52:40 +00:00
Sami Mokaddem f4e4c7b335 chg: moved sharing group outside of the distribution progressbar (as it is a special case), distribution range is displayed when clicking on the pb labels and lots of minor improvements. 2018-05-02 13:37:48 +00:00
iglocska 5795b1974a new: Added warning about missing warninglists used for TLD resolution in the freetext import tool
- following the twitter feedback
2018-04-26 18:57:00 +02:00
Sami Mokaddem 7a94612161 new: Possibility to view connected communities and concerned sharing groups in distribution graph's tooltip 2018-04-25 09:48:03 +00:00
Sami Mokaddem 56b37d08fc Merge remote-tracking branch 'upstream/2.4' into distributionGraph 2018-04-25 07:08:54 +00:00
iglocska df80f702d0 Merge branch 'correlation_integration' into 2.4 2018-04-24 17:10:16 +02:00
iglocska 2af8bfec4e new: Added event enrichment functionality
- select and run a set of enrichments on all applicable attributes of the event
- exposed to the API
- exposed to the command line tool
- adheres to attribute distributions
2018-04-24 16:41:09 +02:00
Sami Mokaddem 828426b0c9 fix: support of filtering for distribution=0 (empty(0) is true ini
php). Also, only consider attr and obj_attr (ignoring object as they
only carry meta-data)
2018-04-23 14:33:32 +00:00
Sami Mokaddem 02b4f32c4f Possibility to filter valueInFieldAttribute with multiple value.
distribution graph support inherit distribution level
2018-04-23 14:14:06 +00:00
Sami Mokaddem bdcecfb1e6 Allow filtering attributes based on specific columns (previsouly not
accessible) like distribution.
Partial support of onClick for distribution graph.
2018-04-23 13:54:36 +00:00
Sami Mokaddem 72ca4260be Merge remote-tracking branch 'upstream/2.4' into distributionGraph 2018-04-23 12:52:12 +00:00
Sami Mokaddem f9414871b8 Initial version of the distribution graph 2018-04-23 12:51:15 +00:00
iglocska 110cff08d8 fix: Fixed empty event tags on the event index api 2018-04-23 11:48:39 +02:00
Sami Mokaddem 43e8529b9d Correlation graph in event view 2018-04-20 12:38:14 +00:00
Andras Iklody 607d203c04
Merge pull request #3170 from mokaddem/ref_graph
Extended event support and tag filtergin in the event graph
2018-04-20 13:40:48 +02:00
Sami Mokaddem 2ca3515f10 Feature: Possibility to filter on tags 2018-04-20 08:35:38 +00:00
iglocska 279a6459ff new: Preview the extended event ID / UUID
- Also, cleanup of the nasty event tag code
2018-04-19 13:56:50 +02:00
iglocska 3c438243f4 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-04-17 13:45:04 +02:00
iglocska 9b2e212b3d new: Added getEventInfoById API 2018-04-17 13:43:47 +02:00
Sami Mokaddem 368aa2f128 feature: Added support of extended event in event graph 2018-04-16 12:02:43 +00:00
Sami Mokaddem e14df3d066 Merge branch '2.4' of https://github.com/MISP/MISP into ref_graph 2018-04-16 11:15:06 +00:00
iglocska 1ec6412709 fix: Removed the validity check for the event UUID in the extended UUID field 2018-04-16 10:48:16 +02:00
iglocska c34067b8a7 fix: If no extension uuid is added to an event the editing via the UI would fail 2018-04-16 10:19:18 +02:00
iglocska 6e06642f6a chg: Added the command line functions to the automation page's parameters via the controller 2018-04-16 07:56:39 +02:00
iglocska 72f261a9b1 fix: Potentially fix an issue if no extended UUID is passed on edit 2018-04-11 16:44:59 +02:00
iglocska caf53e0c7f new: Extended event first iteration added
- when adding/editing an event, add another event's UUID as an extended event UUID to extend the targeted event with the current
- extender events can be viewed in the merged event view
2018-04-10 18:43:09 +02:00
Sami Mokaddem 4f96fb50a1 Moved event graph into its own view file 2018-04-09 15:41:13 +00:00
Sami Mokaddem 6ee5419297 feature: Draft of generic graphing from any key 2018-04-09 09:12:26 +00:00
iglocska 1cf495c201 fix: Fixed missing reason for failure if the freetext import had a single attribute fail during the saving process, fixes #3141 2018-04-07 23:57:23 +02:00
Sami Mokaddem f3b2741843 feature: Support of Tags in the event graph 2018-04-06 14:44:40 +00:00
Sami Mokaddem 921224ed40 Merge branch 'quick-fix-metacategory-graph' into ref_graph 2018-04-06 07:50:27 +00:00
Sami Mokaddem 3933baf9c9 Compute graph serverside 2018-04-04 13:12:16 +00:00
iglocska ed70624354 new: Added event/attribute add/edit to the restresponse describe functionality 2018-04-02 21:53:08 +02:00
Sami Mokaddem 5e83caf8fb Added retreiving of object templates in order to let the user choose the field we want to see in the event graph 2018-03-29 16:05:19 +00:00
iglocska 74937ea934 fix: Fixed the contactination issue from before 2018-03-28 11:41:27 +02:00
iglocska 9a692346a3 fix: Fixed a crappy event concatination bug for restsearch 2018-03-28 11:35:31 +02:00
StefanKelm 6ae842a0a4
Update EventsController.php 2018-03-27 15:40:34 +02:00
iglocska 6e7def472b Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-03-26 17:02:43 +02:00
iglocska d97e42296f new: Add event last modified to the event view 2018-03-26 17:02:03 +02:00
John Doe 606b18562b Pass attribute uuid to enrichment modules 2018-03-26 09:56:37 +08:00
iglocska 65385eb323 new: Added STIX 2.x import to the GUI 2018-03-23 14:37:43 +01:00
iglocska 5df34ea17d fix: Handle edge case scenarios where orphaned correlations would throw notices in the event view 2018-03-23 11:07:18 +01:00
Alexandre Dulaunoy f0d95887a4
Merge pull request #3063 from mokaddem/ref_graph
Event graph viewer editor
2018-03-23 09:08:20 +01:00
Sami Mokaddem 6a0abcce22 Renamed script again 2018-03-22 16:56:43 +00:00
Sami Mokaddem c78fca0ede Added possibility to edit references on the fly + edit objects on their dedicated webpage 2018-03-22 15:53:53 +00:00
iglocska 66c5594fb7 fix: Fixed error message if an attribute fails validation via the freetext import tool, fixes #3052 2018-03-19 23:32:18 +01:00
root a3a6a77611 Initial references graphs commit 2018-03-19 08:44:25 +00:00
iglocska 6e1528db1a fix: added uuid to organisations in the event index
- also unset empty sharing groups from the output
2018-03-08 12:07:30 +01:00
iglocska a38bccec77 new: Allow the searching of organisations by uuid on the event index (via the API) 2018-03-08 11:43:25 +01:00
iglocska 22e6fb57a8 fix: Reworked the way tags are attached to events on the index
- solves issues with the preview when an instance has an extremely high number of events
2018-02-28 17:09:55 +01:00
iglocska 714067c621 fix: Handle the no modules enabled error more gracefully 2018-02-27 10:29:41 +01:00
iglocska 3556df7658 fix: Nicer error message when trying to add a tag to an event that doesn't exist 2018-02-26 17:52:24 +01:00
iglocska 032844321c fix: Misleading failure message when failing to create Attributes partially fixes #2955 2018-02-25 23:20:37 +01:00
iglocska 10bd1f69c4 new: Allow requesting of misp standard format for the export modules
- just set the `require_standard_format` to true in the moduleinfo disctionary
2018-02-21 11:42:30 +01:00
Andras Iklody 297fe776fc
Merge pull request #2934 from cvandeplas/fix/modules-api
fix - allows upload of files using the misp-modules API
2018-02-18 10:07:21 +01:00
truckydev c247cfb77d
don't exlude attributes with non-exportable tag
exclude filter on attributes when tag is non-exportable
2018-02-15 17:17:50 +01:00
Christophe Vandeplas 637a500c1e fix - allows upload of files using the misp-modules API
See also #2719
2018-02-14 13:40:04 +01:00
iglocska 9af6130d43 new: Added STIX import directly to the UI 2018-02-09 11:30:28 +01:00
iglocska 23937eebb9 fix: Fixes to several cases of handling blocked access incorrectly / non-gracefully
- As reported by Christophe Vandeplas

- stix export: Ungraceful handling of attempted access of unauthorised event (no unauthorised data returned)
- import module: Allows creation of proposals to unauthorised events (no unauthorised data returned, proposals are for new attributes only meaning no automatic override triggered)
- saveFreetext: same as import module
2018-02-06 16:37:37 +01:00
iglocska 010557b042 new: Added returnMetaAttributes flag to the /events/freeTextImport API
- directly returns the raw parsing data instead of creating the attributes if set
- 177 days, 23 hours 40 minutes faster implementation than expected by @ilmoka - #PMD
2018-02-02 15:33:07 +01:00
iglocska a7f3bb7f76 fix: Load orgc data after attributes are loaded in search csv export
- functionality still needs further fixes, WIP
2018-01-25 07:45:38 +01:00
iglocska 00f711a86c fix: Fixes the object issues pointed out in #2543
- Shoutout to the debug hero finding them: @StefanKelm
2018-01-19 16:25:39 +01:00
iglocska 7f29a9a74b fix: Fixed a set of issues with sharing groups that lead to synced events not saving/updating 2018-01-18 23:34:04 +01:00
iglocska 3430383583 fix: Add timestamp to the CSV api 2018-01-18 15:59:13 +01:00
iglocska b18b64e833 new: Filter the event index on sharing group IDs, fixes #2845 2018-01-18 08:38:23 +01:00
iglocska 23adc990b9 new: Automatic cateory switching based on currently selected types for the freetext import/module triage screen 2018-01-17 10:15:23 +01:00
iglocska a7aa2358b2 fix: Remove the option for disabling sightings - it's an integral feature of the MISP core. Fixes #2820 2018-01-16 12:50:01 +01:00
iglocska 4e0fe770a7 fix: Removed debug 2018-01-15 11:11:58 +01:00
iglocska 637e2dda2d fix: Clarify scope for filter options in quick search 2018-01-15 10:59:44 +01:00
iglocska a2205fba31 new: Limit modules to a single organisation
- new settings in serverSettings
2018-01-13 12:22:14 +01:00
iglocska fee672dd76 fix: quickfilter should include attribute level tags too 2018-01-10 09:57:37 +01:00
iglocska 7b8da4979b fix: Pagination on event attributes didn't load the feed correlations 2018-01-09 14:00:50 +01:00
Andras Iklody d94c379f37
Merge pull request #2719 from cvandeplas/2.4
basic support for misp-modules via API
2017-12-19 20:07:26 +01:00
Alexandre Dulaunoy 710fe0ba7c
fix: STIX2 export is no more experimental and can be safely used 2017-12-19 12:18:19 +01:00
iglocska 92b441f37a fix: Fixed an issue where url parameters for restsearch didn't block attributes
- url parameters are bad
- shame
- SHAME
2017-12-13 19:22:27 +01:00
iglocska c16246598d new: Add tag restrictions for a single user 2017-12-08 16:31:00 +01:00
Christophe Vandeplas 7eed575c51 basic support for misp-modules via API
- mini cleanup of FileAccessTool that's not needed
- basic support for misp-modules via API (malware-samples not supported yet)
2017-12-07 18:52:31 +01:00
Christophe Vandeplas cda57ec92d fixes issue #2698 - malware-sample fails with import modules 2017-12-07 16:04:38 +01:00
iglocska 9e32f72f6c fix: Removed the requirement for a comment from the import modules
- if the comment field is set don't override it
2017-12-06 08:49:48 +01:00
iglocska 9ad39fedee fix: removed unused variable 2017-12-06 00:21:48 +01:00
iglocska 8323071b7e fix: Moved attribute_tags in the CSV export to the includeContext flag instead of the toggle-able attributes 2017-12-05 10:22:26 +01:00
iglocska 6d0550812d Merge branch '2.4' of github.com:MISP/MISP into 2.4 2017-12-05 09:24:30 +01:00
iglocska 3e112be322 new: Various improvements to the CSV export
- The @FloatingCode and @ilmoka care package
- Improved CSV performance for instances with large number of events
- Added "value" filter for CSV (use-case: I want all indicators for this value with context)
- Added attribute tags to the output of the CSV export
2017-12-05 09:21:31 +01:00
iglocska 8bb1fd678e Merge branch '2.4' of github.com:MISP/MISP into feature/tag_filter_rework 2017-12-05 00:09:37 +01:00
iglocska 4f6dba5f35 new: various improvements
- use the feed uuid caches to link directly to affected MISP events
- various UI improvements
- Feed preview pagination / POSTed event ID filters added
2017-12-05 00:05:11 +01:00
iglocska 05a89f5e87 Merge branch '2.4' into feature/tag_filter_rework 2017-11-30 22:28:35 +01:00
iglocska e760ba7b6a new: Add the possibility to limit fields for the CSV export via POST requests 2017-11-30 19:12:14 +01:00
iglocska 3b893d3b69 fix: Fixes to various issues with adding proposals via the freetext import tool
- no feedback on whether the resulting dataset will be stored as attributes/proposals
- unpublishing of the event when proposals get entered
- alerting the event creator of new proposals if coming from the freetext import tool
2017-11-29 07:59:09 +01:00
iglocska 678eecf224 new: Add attribute tag filters to the fetchEvents() functionality
- tag filters now filter on:
  - all events cotaining matching tags on event + attribute level (positive lookup)
  - all events not containing matching tags (negative lookup)
  - filter attributes within a matched event for blocked attributes (negative lookup)

- moved tag filtering to subquery filtering - should improve performance massively on larger instances when filtering on tags

- first round of implementations, more on the way
2017-11-19 21:21:32 +01:00
iglocska d09edd434c fix: Fixes an issue where assigning sharing groups based on existing IDs didn't work for event creation via the API
- expected full sharing groups as provided by the sync, references didn't work
2017-11-17 13:31:55 +01:00
iglocska 27e3faeba5 fix: Fixed silly lookup with injected event IDs on the export page for normal users
- broke instances with a few hundred k events
2017-11-13 16:32:28 +01:00
Andras Iklody 97b0edcbfd
chg: pass event_id to import modules, fixes #2612
As described by @Vince147
2017-11-03 08:43:16 +01:00
iglocska 08d71413cb fix: Fixed default distribution for upload_sample(), fixes #2608 2017-11-02 07:52:55 +01:00
iglocska 7f5c03f007 fix: Convert - to _ in csv headers
- to match the previous output
2017-10-28 19:10:34 +02:00
iglocska c02f91722f fix: Add the object fields by default to the CSV export 2017-10-27 17:53:37 +02:00
iglocska 6cd0a29f02 fix: Fixed the CSV field name for date 2017-10-27 16:38:39 +02:00
iglocska 35ad0f2f57 fix: Fixed an issue with the CVE export if no field parameters were passed 2017-10-27 16:14:37 +02:00
Cédric Bonhomme 96635dca78
the last useless coma 2017-10-27 11:16:42 +02:00
Cédric Bonhomme 5ac042da67
harmonizes arrays initializations 2017-10-27 11:04:57 +02:00
Cédric Bonhomme 9e93b61838
Enables the user to select the attributes to be included in the CSV export (event and object attributes). 2017-10-27 11:00:32 +02:00
Richard van den Berg b5972fb6e7 Speed up tag searches, fixes #2407 2017-10-20 14:22:22 +02:00
iglocska 7372831614 fix: Fixed a bug with the restSearch API 2017-10-13 16:56:13 +02:00