Commit Graph

534 Commits (e86b161d9319dea616a71b25e731847f4530ab48)

Author SHA1 Message Date
Steve Clement fa8a782e2a chg: [i18n] Added more __()'s 2018-07-12 21:32:50 +02:00
iglocska ce77ec7f65 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-07-12 14:07:53 +02:00
iglocska 64f65427be fix: [update] checkout the last checked in version of composer.json before attempting a pull 2018-07-12 14:06:51 +02:00
Andras Iklody f0964c11f6
Merge pull request #3479 from FloatingGhost/feature-send-logs-to-elasticsearch
Feature: send logs to elasticsearch
2018-07-12 12:09:53 +02:00
Steve Clement c3a5b95eb0 chg: [deps] There is no major difference between 2.1.0.17 and the dev version. 2018-07-12 09:24:19 +02:00
iglocska b43773fb2b fix: [settings] Make travis happy 2018-07-11 14:33:41 +02:00
iglocska 8f2377239d fix: [settings] Attempted fix to appease Travis 2018-07-11 10:47:44 +02:00
iglocska 089ce29d2f fix: [settings] Default setting for the attachments directory fixed 2018-07-11 10:21:24 +02:00
Hannah Ward a70f8e45d8
new: Add ability to log to elasticsearch 2018-07-10 17:01:57 +01:00
Hannah Ward bb31ee1f03
new: Add elasticsearch settings 2018-07-10 14:53:21 +01:00
iglocska 8259e1c22c new: [sync] Improvements to the pull mechanism
- moved the blacklist event skipping to the negotiation phase
- no longer need to pull and then discard events that have been blacklisted
- solves issues with slow syncs when a lot of deletions were involved

- also, moved the sync negotiation + event retrieval to UUID based lookups instead of ID
2018-07-06 08:31:00 +02:00
iglocska 4a08403e15 fix: [cleanup] Reverted lax baseurl validation 2018-07-06 08:14:47 +02:00
iglocska b00a54c7cc fix: [sync] pull giving some weird messages when an event is blocked by blacklists
- don't warn about failed pulls when the reason is a local blocking of the event.

- future improvements: remove the blocked events during the negotiation phase
2018-07-06 08:12:44 +02:00
Alexandre Dulaunoy e8d8d80aba
chg: [PyMISP] released as 2.4.93 2018-07-01 18:26:06 +02:00
iglocska 8559af8c6a fix: [Session handling] Make sure that the autoregenerate setting changes are actually saved 2018-06-30 19:42:57 +02:00
iglocska adebce6c69 Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-06-30 19:26:20 +02:00
iglocska 881649c8e9 new: [Session handling] Added checkAgent toggle 2018-06-30 19:26:02 +02:00
Andras Iklody fe3f2067be
fix: [update] recursively init and update submodules 2018-06-29 07:49:10 +02:00
iglocska 81c0fc2279 new: [API] Added unsafe URL parameter to authenticate users
- for legacy tools that cannot pass headers in HTTP requests for some insane reason
- Needs to be enabled by a site admin - default is that it is disabled
- MISP's diagnostic tool WILL complain if this is ever enabled
2018-06-27 14:25:40 +02:00
iglocska 73ced14c6e fix: [sync] Fix to the attribute level filters not being applied correctly on a full push
- Found during the investigation of #3378
2018-06-25 23:29:01 +02:00
iglocska 2aaf2c54c4 chg: [diagnostics] Make the STIX diagnostics a bit less cryptic 2018-06-22 09:34:56 +02:00
iglocska a930fdeaeb new: [i18n] Added tools to switch between languages via the server settings 2018-06-20 15:11:43 +02:00
iglocska 214df94bda chg: [i18n] Made the strings more i18n friendly across the application 2018-06-20 12:56:53 +02:00
chrisr3d 009dac1e43
add: [Diagnostic] Added maec python library requirements 2018-06-11 16:27:13 +02:00
Raphaël Vinot 551aa0baab
Bump recommended version of PyMISP 2018-06-07 12:08:26 +02:00
iglocska 1917ebd096 Merge branch 'deadlockfix' into 2.4 2018-06-05 18:07:06 +02:00
iglocska ed09fdedbf new: [performance] Made the deadlock fix optional
- old behaviour by default or if the setting is disabled
- new behaviour with non transactional attribute add / correlation add
2018-06-05 18:06:02 +02:00
Alexandre Dulaunoy a4535c1c31
Merge pull request #3301 from LDO-CERT/2.4
fix Typo in MISP settings
2018-05-31 15:37:15 +02:00
garanews 7fbae76191 fix Typo in MISP settings
fix Typo in MISP settings
2018-05-31 14:51:54 +02:00
iglocska 93333dceff chg: Added remaining parts of the pymisp / new stix diagnostic tool 2018-05-31 14:35:11 +02:00
garanews 01d4934668 fix Typo in MISP settings
fix Typo in MISP settings
2018-05-31 14:20:01 +02:00
iglocska 205e29bd8f fix: [CLI] Allow for empty baseurl via the CLI 2018-05-29 06:37:10 +02:00
iglocska 2ed4ecec02 new: Stricter validation of baseurl when coming via the API tool 2018-05-26 06:55:28 +02:00
Richard van den Berg f8dba9c3b7 Fix STIX diagnostics: use python3 2018-05-22 21:30:39 +02:00
iglocska bef3980a86 chg: [documentation] Better description of command line APIs / automation 2018-05-22 12:33:49 +02:00
Anthony Vaccaro ba96309305 Add config mapping for 'gpgconf' option in Crypt_GPG library.
This option not only sets the location of the gpgconf binary, but
if set to false, disables behaviour that shuts down running agents
when a Crypt_GPG object is destroyed. This behaviour would also
kill any long-running or daemonised agents that are running and
configured in the gpg.homedir directory.
2018-05-15 10:36:23 +10:00
iglocska d47cfefe17 fix: Added documentation of server setting modifications via the console
- also added left-off server setting for enabling attachments via ZMQ
2018-05-11 11:37:20 +02:00
iglocska 0ec042c71d chg: only run the automatic worker restart on upgrade if background processing is enabled 2018-05-09 10:04:53 +02:00
iglocska 39f66eb868 fix: Restart the workers due to the new cakephp version causing issues 2018-05-09 09:15:18 +02:00
iglocska 2af8bfec4e new: Added event enrichment functionality
- select and run a set of enrichments on all applicable attributes of the event
- exposed to the API
- exposed to the command line tool
- adheres to attribute distributions
2018-04-24 16:41:09 +02:00
iglocska 79f4730294 fix: Session.cookie_timeout could not be saved correctly, fixes #3182, fixes #3171 2018-04-24 10:07:28 +02:00
iglocska 3135031168 fix: Fixed the text of the cookie_timeout setting 2018-04-20 10:36:45 +02:00
iglocska 27cc7611f5 new: Added the cookie_timeout setting
- still needs some back-end changes for it to be active
2018-04-18 16:44:08 +02:00
iglocska af460f544b fix: Removed actual file path from the command line functions path 2018-04-16 10:14:05 +02:00
iglocska 3347ca2a11 Merge branch 'disable_auto_download' into 2.4 2018-04-16 08:12:10 +02:00
iglocska 0d2f907339 chg: Moved the command line functions' description to the server model 2018-04-16 07:57:06 +02:00
iglocska 570fe32764 new: Added improvements to the Cortex settings
- allow for configuring SSL options for Cortex
- previously the API key was not passed to Cortex on GET requests only on POST, breaking Cortex 2 compatibility
2018-04-13 14:22:08 +02:00
iglocska 474d2c2ab1 fix: autoregenerate causes intermittent logouts, changed the setting description and guidance in the server settings to reflect this 2018-04-11 14:11:28 +02:00
iglocska cf49732799 fix: Fixed issues with non string server settings when changing them via the console 2018-04-08 23:11:51 +02:00
iglocska a057bc64a7 fix: Don't try to run the testBaseURL server setting check if the user comes from the CLI 2018-04-01 17:36:51 +02:00
John Doe c713bb259e Made the auto download of attachments when loaded in the browser configurable 2018-03-28 13:55:55 +08:00
StefanKelm 11c530d658
Update Server.php 2018-03-27 16:18:54 +02:00
StefanKelm 616cc743c6
Update Server.php 2018-03-27 15:46:11 +02:00
StefanKelm 630d92df1d
Update Server.php 2018-03-27 15:15:04 +02:00
iglocska 4e6f57dcf3 new: Add authorization header for Cortex 2 integration 2018-03-26 17:36:58 +02:00
iglocska dc4e30f3db Merge branch '2.4' of github.com:MISP/MISP into 2.4 2018-02-15 16:38:52 +01:00
iglocska 5b575c7ee1 new: Added ssdeep threshold setting
- set the ssdeep value at which to consider two ssdeep hashes as correlating
2018-02-15 01:20:26 +01:00
iglocska d20e04ae6c fix: Fixed command execution for site admins
- a server setting allowing the override of the path variable for esoteric RHEL systems allowed site admins to inject arbitrary commands
- impact was limited by the setting being only accessible to the site administrator

- as reported by Michael Grolimund from Swiss Post (@grolinet)

- CVE-2018-6926
2018-02-12 17:58:47 +01:00
iglocska 9760f1144f new: New APIs to add/remove orgs and servers from sharing groups, fixes #2888
- added functions to manage the additions/removals of objects from sharing groups
- the following APIs are included:
  - /sharingGroups/addOrg/[sg_id]/[org_id]/[extend]
  - /sharingGroups/removeOrg/[sg_id]/[org_id]
  - /sharingGroups/addServer/[sg_id]/[server_id]/[all_orgs]
  - /sharingGroups/removeServer/[sg_id]/[server_id]

- All parameters are optional and can instead be passed as JSON objects such as:

  {
    "org_uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f",
    "sg_id": "49",
    "extend": 1
  }

- The API is extremely flexible with how to name objects, the following parameters are allowed:
  - Organisations:
    - org_id (The organisation's local instance ID)
    - org_uuid (The organisation's global UUID)
    - org_name (The organisation's identifier as known to the curent instance)
  - Server:
    - server_id (The server's local instance ID)
    - server_url (The URL of the server)
    - server_name (The local name of the server as assigned when adding the server)

The sharing groups can also be addressed by ID or UUID.
2018-02-02 08:46:14 +01:00
iglocska 6a29d06566 new: Tie tags into PubSub channel
- Reset the catastrophic @ilmoka enrage timer for another 5 days
2018-01-26 19:27:27 +01:00
Andras Iklody 173496c305
Merge pull request #1969 from devnull-/GPG_sign_option
Add a option to sign GPG emails
2018-01-24 11:11:03 +01:00
iglocska a7aa2358b2 fix: Remove the option for disabling sightings - it's an integral feature of the MISP core. Fixes #2820 2018-01-16 12:50:01 +01:00
iglocska 7772b9c43e new: Disable the viewing of a full organisation list by normal users
- Only site admins and sharing group editors can see organisation lists
  - this includes the org index and various statistics
- Keep in mind: Sharing group editors CAN see the full organisation list - otherwise they wouldn't be able to create sharing groups.
- Also, users CAN enumerate organisations that have created ANY data on the instance by looking at the given data
  - this includes events, proposals, discussion entries, etc
2018-01-13 16:55:01 +01:00
iglocska a2205fba31 new: Limit modules to a single organisation
- new settings in serverSettings
2018-01-13 12:22:14 +01:00
iglocska e552bdfc96 fix: opcache_reset() doesn't always exist on our favourite distro - only execute it if the function exists, fixes #2792 2018-01-09 08:52:43 +01:00
iglocska 9c17b11ab0 fix: Cleaner handling of failed connections during checkVersionCompatibility, fixes #2786
- log the real reason why the connection test failed in case of an exception (such as invalid certificate)
2018-01-08 17:10:26 +01:00
iglocska 99496cd7a8 new: Limit the max amount of time spent fetching the latest commit ID to 3 seconds max
- should help avoid the unresponsive diagnostic page issue
2017-12-22 14:43:29 +01:00
Andras Iklody ae84c13ee9
Merge pull request #2644 from jonas-koeritz/2.4
Added an option to customize the page title
2017-12-20 22:24:45 +01:00
Andras Iklody 6e7a9da729
Merge pull request #2684 from JanSkalny/fix_baseurl_trailing_slash
fix: remove trailing slash from MISP.baseurl
2017-12-19 20:09:22 +01:00
iglocska 42df9e36f3 fix: Fixed an issue with opcache not being used yet opcache_reset() being called, fixes #2727 2017-12-12 18:38:31 +01:00
Raphaël Vinot a03f9a8557 fix: documentation to enable cortex services 2017-12-05 11:36:43 +01:00
iglocska cff58988a7 fix: Some minor fixes to the attribute filtering 2017-12-01 16:58:23 +01:00
iglocska 8e18d22fe3 fix: Graceful handling of no response during getVersion pre-sync test 2017-11-30 10:16:27 +01:00
iglocska 6135468c41 new: Added full audit logging to ZMQ and Syslog, fixes #2635
- syslog now includes all audit log entries and it's separated into proper severity levels
- ZMQ logging and syslog logging are both optional features
2017-11-24 12:01:53 +01:00
Jan Skalny 6933050e2e fix: remove trailing slash from MISP.baseurl 2017-11-23 01:10:19 +01:00
Jan Skalny 084a3eb3ba fix: refresh rows in settings editor 2017-11-22 15:58:47 +01:00
Jonas Köritz 46653391e5 Added an option to customize the page title. 2017-11-16 09:04:03 +01:00
Trey Darley 8765e4106f typo fixen 2017-11-10 13:15:07 +00:00
iglocska 68f4833893 new: First version of the zmq reimplementation 2017-10-27 09:10:46 +02:00
iglocska cd9fe1883e fix: Some cleanup of the attribute filtering 2017-10-05 11:59:59 +02:00
Tristan METAYER 66a43f5511 Add an imput for search on all attributes in an event.
field to search can be modify in administration page.
2017-10-04 19:07:58 +02:00
iglocska e27d788090 fix: Potential fix to missing proposals during sync
- rather stupid adherence to push rules removed for proposal sync
2017-10-04 12:57:11 +02:00
Kyle Parrish c5d3ae7b1f RPZExport - Alternate NS
Added option to add an alternate nameserver to RPZ export.
2017-09-19 13:25:17 -04:00
iglocska 6b68250c17 fix: Added an upper limit for max correlations / event
- super edge-case test instance got crushed by memory usage
2017-09-18 16:54:27 +02:00
iglocska cfcaf0d410 chg: Made the current password confirmation requirement for any user profile edits optional
- default setting is having it off
- incredibly frustrating feature is now only enabled on demand
2017-08-18 09:05:20 +02:00
iglocska 78f07139bd chg: Redacted certain server settings that could be considered sensitive
- Encryption passwords as well as redis password are now redacted from the server settings
- Also includes the JSON dump of the server settings

- Thanks to cert.govt.nz for the security report.
2017-07-12 15:49:39 +02:00
Kevin Allix 2248846706 attachments_dir: Default value queried through a function to workaround PHP inability to have anything useful stored in a class property 2017-07-10 12:42:23 +02:00
Kevin Allix 1ea33e811a Add an optional setting attachments_dir, and adapt existing code to use that setting 2017-07-07 17:29:13 +02:00
Cédric Bonhomme 755bd03bb6
fix: The server settings page (servers/serverSettings) was crashing when the redis connection wasn't properly working. 2017-07-06 11:37:16 +02:00
Kevin Allix f612f0c627 allow a setting to NOT define a 'test' function 2017-07-04 12:13:10 +02:00
Kevin Allix 379849418b Add MISP.redis_password option 2017-07-03 17:04:47 +02:00
Andras Iklody 3cd94c7e7c Revert "Use posix_getpgid to check whether a pid is running" 2017-06-26 11:07:59 +02:00
Kevin Allix bee2dc3c49 Use posix_getpgid to check whether a pid is running 2017-06-25 22:34:55 +02:00
iglocska e9fdfb90d6 fix: Error handling of proposal sync
- don't log errors if no proposals are found
- don't throw an exception if no proposals are found
2017-06-20 10:17:25 +02:00
iglocska 57857c3a32 new: Performance improvements for the pub-sub modules
- Only load and open connection to redis for the pub-sub connection once.
- Massive performance boost when the ZMQ functionality is enabled
2017-06-16 08:41:12 +02:00
iglocska b952335711 fix: Fixed the mixbox version lookup 2017-06-14 11:32:12 +02:00
Alexandre Dulaunoy bef87dcb2d Merge branch '2.4' into ncsc-nl/mixbox 2017-06-09 08:47:36 +02:00
iglocska bce780090f new: Added User and Organisation addition/change data to the ZMQ feed 2017-05-29 16:18:37 +02:00
iglocska ab9f282a44 new: Added sightings to ZMQ pub sub system 2017-05-28 00:33:20 +10:00
iglocska 56c079642d new: Added attribute JSONs to pubsub system
- also made mispzmq a but more generic
2017-05-22 14:30:58 +02:00
Richard van den Berg 7b18dc19bb Also test for mixbox version 2017-05-19 15:22:48 +02:00
iglocska 314daa4551 new: Add instance uuid 2017-05-11 10:49:23 +02:00
iglocska 680364225e fix: Fixed a condition where no proposals downloaded generated a warning in the debug log 2017-05-11 09:26:50 +02:00
iglocska 96574ec335 new: First implementation of the feed analysis system 2017-05-08 14:22:27 +02:00
iglocska 9ccdc579a6 new: New module type: Cortex
- similar to Enrichment modules except for not having the options to run hover
2017-05-03 13:13:36 +02:00
iglocska e638c5350a new: Minimal flag added to the event index
- used by the sync, greatly reduces the data fetched / transfered on the initial sync negotiation
2017-04-13 15:09:29 +02:00
iglocska b2264d3b7b fix: Missing ; added 2017-04-12 12:08:08 +02:00
iglocska 1d48a6f2ed fix: Added logging to the testconnection post-test
- also, fixed the inverted error codes as noted by @ppanero
2017-04-12 11:53:54 +02:00
iglocska 5ab35f8d7c chg: org blacklisting enabled by default 2017-04-11 15:43:40 +02:00
iglocska 77ef188457 new: Password complexity defaults tightened, also passowrd requirements shown to users, fixes #2117 2017-04-11 10:04:07 +02:00
iglocska 6c487c393a chg: Default password policy now includes a 16 char+ string option as an alternative to the short 3/4, fixes #2117 2017-04-07 10:47:23 +02:00
iglocska c4c1de1ff3 fix: Spring cleaning
- removal of debug from the syncdebug
- cleanup of the fixes that resulted from it
- removal of the mangle sync from 2.4->2.3 (if you still have partners running 2 year old versions, time to notify them, stop syncing and unfriend on facebook)
2017-03-25 11:06:04 +01:00
iglocska 11bb4d4dfc fix: Some further fixes
- includes a fix to a compatibility test failure causing all instances to test as a legacy MISP
2017-03-24 13:09:52 +01:00
iglocska 77936a6bf9 new: Sync logging to debug issues 2017-03-24 12:38:38 +01:00
iglocska e1ea9cb300 fix: Added missing popup view file
- Also added a new test string in a file for the POST connection test
2017-03-23 17:21:52 +01:00
iglocska 3fbfe08f87 new: Added a POST server connection test
- hopefully it should help debug some issues
2017-03-23 11:52:07 +01:00
iglocska 6fcad27dae Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2017-03-23 10:02:24 +01:00
iglocska 84e4a62aba new: Update MISP from the diagnostics page
- right now it's pretty dumb, it simply pulls the same branch that the current user is on
- Any failure is shown but not acted upon, if the git pull fails the user will see it but it needs to be resolved via the command line
2017-03-23 10:01:29 +01:00
Andras Iklody b73be9e9bc Merge pull request #2068 from ppanero/regex_bugfix
testForPath regex fixed in Server.php
2017-03-21 14:20:19 +01:00
Pablo Panero b9633bcea5 testForPath regex fixed in Server.php 2017-03-21 14:11:38 +01:00
iglocska 46695b8ca2 fix: Several blacklist related fixes
- turned the functionality to a default on feature
- added indexes
- fixed some default values
2017-03-07 17:39:44 +01:00
iglocska 62cb2b66b6 new: Added a way to disable cached exports server wide for low disk space instnaces
- But please consider just adding some more space instead..
2017-03-02 10:49:18 +01:00
devnull- e4023fabb6 Merge branch '2.4' into GPG_sign_option 2017-02-28 15:47:21 +01:00
iglocska a2c9cfb18f fix: Fixed an issue with the baseurl diagnostic 2017-02-28 13:08:44 +01:00
iglocska 2184ca4e70 Merge branch '2.4' into 2.4.67 2017-02-24 19:55:46 +01:00
iglocska 3fca8de6bf new: Added activity charts to tag and galaxy cluster indeces
- bunch of small improvements additionally
2017-02-24 19:34:18 +01:00
iglocska 8067e5eb07 fix: Added a fix to growing arrays in the ApacheSecureAuth settings, fixes #1981 2017-02-23 15:03:32 +01:00
Pablo Panero 34f703dce9 Update Server.php
Duplicate entry of property
2017-02-23 12:34:05 +01:00
iglocska 6c15295bf9 fix: Some fixes for the new user admin features 2017-02-23 08:56:27 +01:00
Adrien RAFFIN f3d81a5921 new: Add reverse proxy support for test of baseurl 2017-02-21 15:57:27 +01:00
devnull- 4d7bb2a656 Description of the option 'sign' 2017-02-21 09:42:02 +01:00
iglocska c8a0b0d870 new: Advanced correlations
- experimental feature, correlate on CIDR
- can be turned on/off in the server settings
- For the emperor
2017-02-03 20:18:24 +01:00
iglocska 7c5ff0e11d fix: simplification of the proposal sync 2017-02-02 17:24:12 +01:00
iglocska 61464e5385 new: New setting to sanitise attributes on delete
- if enabled server wide, any delete of an attribute will not just set the deleted flag, but also sanitise the content fields
- fields sanitised: category, type, value, comment, to_ids
2017-02-02 13:38:05 +01:00
iglocska ee80ecfce7 new: small rework of the thread functionalities
- API get /threads/view/<thread_id> and /threads/viewEvent/<event_id>
- Added new setting to show post count on the event index including a notification if it has a post newer than 24 hours
2017-02-01 15:32:22 +01:00
iglocska 3509caf198 chg: version bump and changed default session engine to php 2017-01-31 10:38:19 +01:00
iglocska 0c68c74e85 fix: Fixed an invalid lookup for the site admin debug 2017-01-25 08:34:04 +01:00
iglocska 6f31d36796 fix: fixed an issue where setting site admin debug to false resulted in a critical warning 2017-01-25 08:31:10 +01:00
Iglocska ffd087ba38 fix: Added a warning if utf8 encoding isn't set up in the database config
- also, changed the default database config to enforce utf8
2017-01-19 16:40:23 +01:00
Iglocska 569b342ef5 fix: Cleaner fix, testBool doesn't need to run testForEmpty 2017-01-16 11:45:24 +01:00
Iglocska 7d3a04cb2e fix: Don't show value not set on boolean false values that are actually set in the server settings 2017-01-16 11:40:48 +01:00
Iglocska 396ab8046f fix: Fix empty space issues with server settings
- on input trim the string
- on the not empty check, first trim the string to warn users about existing issues
2017-01-02 16:32:11 +01:00
Iglocska 394440f761 chg: Description of session.timeout updated 2017-01-02 15:14:07 +01:00
iglocska 84f9fc6fcd new: Added session settings to the server settings
- also, new method for writing the MISP config file
2016-12-28 11:22:47 +01:00
iglocska da433c3549 Merge branch '2.4' of https://github.com/MISP/MISP into feature/disable_correlation 2016-12-22 21:01:58 +01:00
iglocska ffe880621e new: Disable correlation
- globally
- on an event level
- on an attribute level
2016-12-22 15:30:06 +01:00
Andras Iklody 017959ec19 fix: Pull not respecting negated tag rules fixed, fixes #1775 2016-12-20 17:43:24 +01:00
iglocska 3661be08cc Merge branch '2.4' of https://github.com/MISP/MISP into 2.4 2016-12-19 17:36:44 +01:00
Richard van den Berg 7687341ada Make threatlevel in E-mail subject optional 2016-12-16 17:17:20 +01:00
iglocska 6778d0dad5 new: Added a way to disable certain tags from the UI, fixes #1733
- also added a new setting to set the default posture when an event containing a tag is pushed (via the API/sync/etc)
  - new setting allows to automatically set new tags to hidden

- the hidden setting only hides the tags from the tag selection when tagging an event
2016-12-16 15:44:10 +01:00