Golbark
3a0bedb104
chg: [internal] Using Allowedlist instead of Whitelist
2020-09-01 16:28:20 +02:00
iglocska
f82e10d1fb
new: [API] added count returnformat for the REST api, fixes #6233
...
- simply counts the number of attributes/events found (on each respective scope)
2020-08-31 12:32:28 +02:00
rmkml
6da01364ba
add SHA3 Hash on Attribut.php
2020-08-23 18:26:56 +02:00
Andras Iklody
eccc3f83e9
Merge pull request #6202 from rmkml/2.4
...
add vhash (VirusTotal Hash) on Attribut.php
2020-08-17 17:05:46 +02:00
rmkml
a6eeb676c4
add vhash (VirusTotal Hash) on Attribut.php
2020-08-17 16:36:47 +02:00
Jakub Onderka
261b54e505
chg: [correlation] Use less memory when generating correlation
2020-08-17 10:54:01 +02:00
Jakub Onderka
cac497158e
chg: [internal] Reuse AttachmentTool instance
2020-08-14 12:30:12 +02:00
Jakub Onderka
ba74ddd689
fix: [UI] Showing image thumbnail
2020-08-13 15:34:32 +02:00
Jakub Onderka
83f5d3ada5
chg: [internal] Generate event date even if attachments doesn't exists
2020-08-13 15:34:32 +02:00
Jakub Onderka
a411b36666
chg: [internal] Move attachment handling to one place
2020-08-13 15:34:32 +02:00
Jakub Onderka
421e94bc2f
fix: [security] Check if user can access sharing group when uploading attachment
2020-08-06 11:52:53 +02:00
Jakub Onderka
8bb2d94c5e
fix: [security] ACL check when loading ajax tags
2020-08-04 12:23:41 +02:00
Jakub Onderka
5611374829
fix: [security] ACL check when editing multiple event attributes
2020-08-04 12:23:38 +02:00
iglocska
a2c0010d48
fix: [unicode] Temporarily escape 4 byte characters until we move the attribute value fields to mb4, fixes #5123
...
- fixes sync/feed issues related to 4 byte unicode characters
2020-07-30 09:17:27 +02:00
Jakub Onderka
d7f7bc725b
chg: [internal] Faster loading sighting
2020-07-28 09:50:28 +02:00
mokaddem
cda56bdbd3
fix: [attribute:editableFields] Typo in variable name
2020-07-24 16:19:02 +02:00
mokaddem
f53ba3675e
chg: [AttributeTags:handleAttributeTags] More generic way to handle
...
capture and association
2020-07-24 14:30:41 +02:00
mokaddem
6c1bbf0fde
chg: [attribute] Added tag handling when saving attributes and objects
2020-07-24 13:55:34 +02:00
mokaddem
5b4cef3e6c
Merge branch 'feature/tags-deletion' into fix-update-tags-on-attribute-edit
2020-07-24 11:41:27 +02:00
mokaddem
0086fcd1dd
chg: [attribute:editAttribute] Uage of `editableFields` instead of
...
hardcoded array
2020-07-24 10:59:42 +02:00
mokaddem
3fc896eaaa
fix: [attribute] Allow editing attributes
...
Added raw values fields in the `editableFields`
2020-07-23 12:44:07 +02:00
mokaddem
77f2a983b2
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2020-07-23 11:19:41 +02:00
mokaddem
d3f9184f21
fix: [attribute:edit] Prevent the edition of system reserved fields
2020-07-23 11:15:01 +02:00
iglocska
6750a05003
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2020-07-22 13:12:50 +02:00
iglocska
662cdbc7b7
fix: [sync] drop the republishing of events when the modification is merely a timestamp bump
...
- due to an already fixed issue still lingering, invalid event edits keep getting synchronised between instances
- these events still generate publish alerts erroneously
- this fix compares the previous state of the event to the modification, if there are no material changes (attributes, objects, object relations, event tags added/updated) then the publishing is dropped.
2020-07-22 13:10:52 +02:00
mokaddem
ad81c60986
Merge branch '2.4' of github.com:MISP/MISP into galaxy-cluster2.0
2020-07-21 08:09:25 +02:00
Jakub Onderka
35e2aa5837
chg: [internal] Faster generating correlations when enabling for event by toggle
2020-07-17 09:46:34 +02:00
Jakub Onderka
91a1913fed
new: [attribute] Add support for IDN domains
2020-07-16 17:23:49 +02:00
mokaddem
b3dbecb318
Merge branch '2.4' of github.com:MISP/MISP into galaxy-cluster2.0
2020-07-14 16:25:04 +02:00
Jakub Onderka
3afcf1e070
Merge pull request #5929 from JakubOnderka/fuzzy-purge
...
fix: [correlations] Purge ssdeep table after attribute delete
2020-07-13 22:48:10 +02:00
Andras Iklody
6dca14a6b0
Merge pull request #6067 from JakubOnderka/fix-composite-type-uniquenes
...
fix: [attributes] Possible duplicate attributes
2020-07-10 09:59:55 +02:00
Andras Iklody
bfb2bf1bfd
Merge pull request #6070 from JakubOnderka/cidr-correlation-optim
...
Cidr correlation optim
2020-07-07 12:23:35 +02:00
Andras Iklody
e6995dab67
Merge pull request #6035 from JakubOnderka/rest-search-optim
...
chg: [internal] Attribute REST search optimisations and error handling
2020-07-07 12:18:29 +02:00
mokaddem
f3a9481c61
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0
2020-07-01 16:22:55 +02:00
Jakub Onderka
4d95426ca8
chg: [correlations] Faster checking if CIDR is IPv4 or IPv6 version
2020-06-28 10:15:31 +02:00
Jakub Onderka
e96d632055
chg: [correlations] Just check if redis key exists
2020-06-28 10:14:40 +02:00
Jakub Onderka
08e1fb3b49
fix: [attributes] Possible duplicate attributes
2020-06-26 22:43:16 +02:00
iglocska
6b95047c1e
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2020-06-22 11:11:31 +02:00
iglocska
609bfbd450
fix: [security] missing ACL lookup on attribute correlations
...
- attribute correlation ACL checks are skipped when querying the attribute restsearch API revealing metadata about a correlating but unreachable attribute.
- Thanks to Jakub Onderka for his tireless work and for reporting this!
2020-06-22 11:00:56 +02:00
Jakub Onderka
3d12cb3f3c
chg: [internal] Attribute REST search optimisations and error handling
2020-06-19 23:45:22 +02:00
Jakub Onderka
7eb464a58f
fix: [UI] Typo
2020-06-18 16:34:01 +02:00
Jakub Onderka
5346190afb
new: [correlations] Enable CIDR correlations for ip-src|port and ip-dst|port types
2020-06-17 11:11:51 +02:00
Jakub Onderka
48c9c32e48
chg: [correlations] Faster IPv4 CIDR correlation
2020-06-17 09:17:57 +02:00
Jakub Onderka
8454ca38c7
chg: [correlations] Faster IPv6 correlation
2020-06-17 09:17:57 +02:00
Jakub Onderka
1d0d913972
chg: [correlations] Big speedup when correlating CIDR
2020-06-17 09:17:57 +02:00
mokaddem
a5e7e46cd4
Merge branch '2.4' of github.com:MISP/MISP into decaying-v2
2020-06-08 08:26:15 +02:00
Jakub Onderka
9665b8a091
fix: [internal] Removing attributes from empty event
2020-06-05 16:48:53 +02:00
mokaddem
bb167029eb
new: [userSettings] New setting `default_restsearch_parameters`
...
It allows users to supply restSearch parameters that will be injected
(and possibly overridden) into the restSearch filters.
2020-05-29 15:23:27 +02:00
mokaddem
591bd8f9c5
fix: [attribute:fetchAttribute] Prevent notices if tags not set while
...
computing decay
2020-05-29 15:20:07 +02:00
mokaddem
5c04b9a8c1
Merge remote-tracking branch 'origin/2.4' into galaxy-cluster2.0
2020-05-28 14:06:30 +02:00
Raphaël Vinot
27cdcf2c65
Merge pull request #5941 from MISP/git-commit-id
...
new: [type] git-commit-id
2020-05-26 15:03:52 +02:00
Andras Iklody
130e2970b5
Merge pull request #5942 from JakubOnderka/correlation-saving
...
Correlation saving
2020-05-26 14:28:51 +02:00
Jakub Onderka
f1a1f3d81f
chg: [correlation] When generating correlation, just fetch attributes that can correlate
2020-05-26 13:15:19 +02:00
Jakub Onderka
04c783afd7
chg: [correlations] Refactored correlation saving
...
* Always show other correlating value (useful for CIDR correlations)
* Make correlation saving faster (move more work to database, do not fetch not necessary fields)
* Fix some small bugs
2020-05-26 13:14:57 +02:00
Raphaël Vinot
c03670cf0d
new: [type] git-commit-id
2020-05-26 12:26:24 +02:00
Andras Iklody
ffed7fed4c
Merge pull request #5906 from JakubOnderka/fix-ipv4-cidr-validation
...
fix: [attribute] Do not allow for IPv4 CIDR masklen bigger than 32
2020-05-26 08:59:05 +02:00
Jakub Onderka
a8bc7868cf
fix: [correlations] Purge ssdeep table after attribute delete
2020-05-21 17:54:40 +02:00
chrisr3d
932e51d4d7
fix: [opendata export] Less confusing variable name for the parameter to only skip exporting the data and keep only the header
2020-05-15 14:08:34 +02:00
chrisr3d
2bf534a882
Merge branch '2.4' of https://github.com/MISP/MISP into opendata
2020-05-15 11:40:20 +02:00
Jakub Onderka
c8c599653e
fix: [attribute] Do not allow for IPv4 CIDR masklen bigger than 32
2020-05-14 18:13:39 +02:00
mokaddem
45d2eccd6a
fix: [attribute:simpleAddMalwareSample] Typo in loading `Object` class.
...
Fix #5864
- Was not spotted before because the fixed line was if fact doing
nothing as the class's key was already used
2020-05-14 15:00:43 +02:00
iglocska
a24fe930da
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2020-05-14 09:52:45 +02:00
iglocska
15d6c30649
new: [sync] (for now) undocumented force pull added
...
- can only be triggered via the CLI for now
- usage: /var/www/MISP/app/Console/cake Server pull [user_id] [server_id] [technique] [force]
- the force flag has to be passed as 'force' to avoid accidentally triggering it
- What it does:
- pulls ignoring the timetamp differences
- this means that even older states of events, attributes, objects are ingested
- useful for when wanting to reset an event / all events to align with an upstream server
- Caveats:
- attributes added on the low side are maintained
- tags added on the low side are maintained
- keep in mind this WILL override attributes that are soft deleted
2020-05-14 09:34:49 +02:00
Andras Iklody
049ee88aa7
Merge pull request #5889 from JakubOnderka/attribute_correlation
...
IPv6 CIDR correlations
2020-05-14 08:28:15 +02:00
Andras Iklody
f659ee5871
Merge pull request #5892 from JakubOnderka/fixed_event_freetext_feed_speedup
...
Fixed event freetext feed speedup
2020-05-12 08:43:46 +02:00
Jakub Onderka
47d32a46f4
chg: [internal] Faster saving attributes
2020-05-11 18:18:17 +02:00
chrisr3d
18d6e38b4f
chg: [restSearch] Option to skip fetching attributes/events when only the metadata is wanted
...
- As for the opendata export we do not need to get
the attributes or event, and are only interested
in using the metadata, a parameter to skip
fetching the actual data collection has been
added, and we avoid iterating through the entire
data collection.
2020-05-11 12:54:11 +02:00
Jakub Onderka
e56629daf1
fix: [correlations] Do not correlate CIDR with CIDR
2020-05-10 14:40:04 +02:00
Jakub Onderka
1cf9f3ce2e
fix: [attribute] modifyBeforeValidation fix for `domain|ip` type
2020-05-09 09:42:40 +02:00
Jakub Onderka
5509bb84f5
fix: [correlations] Return just unique values for CIDR list
2020-05-08 22:28:58 +02:00
Jakub Onderka
9269d92147
chg: [correlations] Faster inserting data to Redis
2020-05-08 22:24:28 +02:00
Jakub Onderka
29dba5637a
fix: [correlations] IPv6 CIDR correlations works
2020-05-08 21:40:26 +02:00
Jakub Onderka
7b26546252
fix: [correlations] Removed unnecessary Redis call
2020-05-08 21:39:50 +02:00
Jakub Onderka
c09ef4b8c2
fix: [correlations] Remove references to not exists type 'domain-ip'
2020-05-08 21:28:45 +02:00
Jakub Onderka
91139482cf
chg: [correlations] Use faster algorithm for IPv6 correlations
2020-05-08 21:16:33 +02:00
chrisr3d
b3a94a18f5
add: [restSearch] OpenData export module
2020-05-06 18:15:30 +02:00
chrisr3d
83fe3c1b4c
add: [restSearch] Added opendata to the valid formats
2020-05-06 18:15:18 +02:00
iglocska
eed82bff99
new: [API] added threat_level_id as a restSearch filter
2020-05-05 18:07:51 +02:00
iglocska
e2bbece354
Merge branch '2.4' of github.com:MISP/MISP into 5802
2020-04-30 08:30:01 +02:00
mokaddem
720aa4a7db
Merge remote-tracking branch 'origin/2.4' into feature/tags-deletion
2020-04-29 09:00:31 +02:00
iglocska
1c3dc3fe51
Merge branch '2.4' into 5272
2020-04-28 15:31:17 +02:00
mokaddem
94bf826841
Merge remote-tracking branch 'origin/2.4' into JakubOnderka-event_loading_speedup
2020-04-28 11:39:51 +02:00
Jakub Onderka
f0ada41963
chg: [internal] Speed up of loading event page
2020-04-27 16:00:28 +02:00
mokaddem
eebde57ee1
fix: [attribute:restSearch] Make sure to always pass all tags to Decaying's
...
computation function
2020-04-27 10:45:19 +02:00
mokaddem
3547a8a888
fix: [correlations] Update correlations on Attribute or Event
...
`distribution` change
2020-04-17 11:29:09 +02:00
mokaddem
dd1be03597
Merge branch '2.4' of github.com:MISP/MISP into galaxy-cluster2.0
2020-04-09 14:26:48 +02:00
iglocska
20e1c62d11
fix: [API] fixes to set_filter_uuid
2020-04-02 07:02:24 +02:00
iglocska
150b5f7338
fix: [search] Fixed the UUID / ID searches on the attribute scope, fixes #5636
2020-04-01 23:14:29 +02:00
mokaddem
04dcdebb1f
new: [galaxyCluster] Initial import of Galaxy2.0 codebase - WiP
2020-03-12 10:26:09 +01:00
mokaddem
4f3ed331f0
chg: Removed unwanted indentation
2020-03-06 10:58:50 +01:00
mokaddem
2061707932
fix: [attribute:validation] Better validation of IPv6-[dst/src] and
...
improved display.
fix #5682
2020-03-06 10:54:06 +01:00
mokaddem
5de5453dbd
Merge branch '2.4' of github.com:MISP/MISP into pr-feature/tags_deletion
2020-02-27 11:19:53 +01:00
mokaddem
8d11600e2b
chg: [tag] Support of untagging in Object's Attribute and other fixes
...
- deleted: 0 is correctly handled
- stopped usage of `editAttribute` from Attribute Controller
2020-02-27 11:17:21 +01:00
iglocska
9913d194fa
fix: [correlations] fix to an issue where attribute edits could purge correlations
...
- bug introduced by a merge gone wrong
- attribute edits that modify fields that do not affect the correlations (such as to_ids, comment, etc) would cause correlations to be purged
2020-02-26 14:28:29 +01:00
mokaddem
f94c693aa0
fix: [decaying] Attributes not having a DM associated will be defaulted
...
as `not decayed`
2020-02-21 14:47:52 +01:00
Tom King
bee49f7028
Merge remote-tracking branch 'upstream/2.4' into feature/tags_deletion
2020-02-13 16:17:03 +00:00
mokaddem
8e2da13e0e
Merge branch '2.4' into enforce-iso-datetime
2020-02-10 14:18:14 +01:00
iglocska
4806652448
fix: [API] several fixes to the Bro API
...
- always use flatten:1 to also include object attributes
- fix the generated full export to only include the header once
2020-01-31 11:45:23 +01:00
iglocska
bae2717eaf
new: [API] Enveloping improvements
...
- user controlled envelope settings to control memory estimation for attribute/event sizes
- logging of potentially too large events for the current memory envelope
- tuning of the default values
- added a divider for the event:attribute conversion to account for objects / event level contextualisation / correlations
2020-01-29 22:33:27 +01:00
mokaddem
b03d86221b
chg: [attribute:type] Datetime value is forced to be a valid ISO format
...
- It is converted into UTC in the server
- /attribute/add Form includes javascript validation part
2020-01-28 14:38:58 +01:00
Jakub Onderka
ed6bb367e3
chg: [UI] Disable Advanced extraction button if it is not installed
2020-01-26 18:56:41 +01:00
Jakub Onderka
73b9513a38
chg: [internal] Refactoring malware handling
2020-01-26 18:56:41 +01:00
Jakub Onderka
6f212dd97c
chg: [UI] Resizing images
2020-01-24 10:58:01 +01:00
mokaddem
d16369f4b3
Merge branch '2.4' of github.com:MISP/MISP into pr-view_picutre
2020-01-24 10:42:35 +01:00
iglocska
f88743cb7a
fix: [internal] slight tuning to the attribute restsearch memory envelop size
2020-01-23 09:45:27 +01:00
Alexandre Dulaunoy
42f106610b
chg: [attributes] new attribute type 'chrome-extension-id'
...
This attribute is used by Chrome to uniquely identify extension.
2020-01-21 09:40:25 +01:00
mokaddem
2a5112cc1e
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-final
2020-01-17 09:20:49 +01:00
mokaddem
96c1f41ffd
chg: [attributeTag] Stop pruning outdated attribute tag when syncing.
...
Will be re-enabled in the future
2020-01-16 15:13:10 +01:00
mokaddem
5c97c5da53
fix: [Attribute:editTag] Correctly escalate the timestamp refresh to the
...
Object
2020-01-16 11:02:56 +01:00
Tom King
0a4fbc80c5
Merge branch '2.4' into feature/tags_deletion
2020-01-14 10:03:44 +00:00
Alexandre Dulaunoy
7a82a9f8d7
new: [attribute type] kusto-query attribute type
...
Kusto query is the query language for the Kusto services in Azure used
to search large dataset. It's used in Windows Defender ATP Hunting-Queries
and also Azure Sentinel (Cloud-native SIEM).
To fix #5475
2019-12-28 15:21:19 +01:00
mokaddem
7797aeed94
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-final
...
Not a simple merge. Needed to fix forms and simplified how
form_seen_input works
2019-12-16 13:36:01 +01:00
mokaddem
092348ffcd
chg: [Attribute:seen] Moved conversion iso<->utc of fs/ls in dedicated
...
functions
2019-12-16 10:47:07 +01:00
iglocska
208e9e9cfc
fix: [API] include the local flag in tags when using restSearch
2019-12-11 10:29:45 +01:00
mokaddem
a1fdea754d
Merge branch 'zoidberg-timeline' into zoidberg-final
2019-12-06 15:40:51 +01:00
Alexandre Dulaunoy
a740d14593
chg: [types] new eppn attribute type added - EduPersonPricincipalName
...
Fix #5448
2019-12-05 19:22:38 +01:00
iglocska
bde75e9443
fix: [internal] attribute restsearch - handle empty lines returned from the module better
...
- no more empty lines hopefully in some exports
2019-12-05 09:42:49 +01:00
iglocska
5768ff9485
fix: [internal] hard delete of attributes after a soft delete fixed
2019-12-04 16:11:38 +01:00
iglocska
1c5afa49ed
new: [refactor] Massive internal refactor and cleanup of deprecated APIs
...
- new centralised restSearch function in AppController as entry point via all controllers
- new component handling restSearch related support functions, such as parameter mapping
- hollowed out all deprecated export functions on the event/attribute controller
- replaced with a new functionality that remaps them to restSearch
- all functionality should be maintained with all additional advantages introduced with restsearch
- additional cleanup (some unused functions removed)
2019-11-29 10:11:30 +01:00
Andras Iklody
ae5c228c05
Merge pull request #5273 from JakubOnderka/patch-54
...
fix: [internal] Remove unused function
2019-11-26 13:42:56 +01:00
Tom King
34c5570692
new: Allow for attribute tag deletion via Event or Attribute edit. Clean and return the attribute tags on response from editing an attribute, update code to remove legacy
2019-11-26 12:27:15 +00:00
iglocska
e4c82eb9ff
fix: [API] adding objects now has better validation errors
...
- instead of silently dropping attributes in certain cases
2019-11-15 14:11:24 +01:00
iglocska
0f40cef0f1
fix: [internal] Attribute/Event connectors for attribute_timestamp added
2019-11-11 16:09:54 +01:00
mokaddem
e396941dcf
fix: [attributes:restSearch] Fixed typo
2019-11-08 15:25:53 +01:00
mokaddem
437490872b
chg: [restSearch] Improved meta-search code
...
- Correctly returns nothing if search on metas does not return anything
- Renamed `orgc.sector` into `org.sector` while still being `orgc`
behind the hood
- Removed duplicated code
2019-11-08 11:37:43 +01:00
mokaddem
8e60c3d8d4
Merge branch '2.4' of github.com:MISP/MISP into feature-meta-search
2019-11-08 11:15:34 +01:00
iglocska
c4f1d4d15e
new: [SightingDB] Added integration with SightingDB
...
- Added configuration tool
- Added lookups from the event view
- Added includeSightingdb flag for the restSearch searches
- Added SightingDB search tool
- Added SightingDB connection test tool
2019-11-06 21:20:04 +01:00
mokaddem
a53a06d080
new: [attribute:restSearch] Support of Orgc and GalaxyElement meta searches
2019-11-06 11:12:30 +01:00
Jakub Onderka
caa62220ff
new: [internal] Attribute::isImage method
2019-10-12 09:34:49 +02:00
mokaddem
015ec7d989
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline
2019-10-08 14:12:38 +02:00
iglocska
173054ccd3
fix: [internal] Set attribute restsearch page to 1 if limit is set without setting the page number
2019-10-08 08:27:21 +02:00
Jakub Onderka
87b568526f
fix: [internal] Remove unused function
2019-10-05 23:17:19 +02:00
garanews
85c28ce36e
Fix some typo
...
Fix some typo
2019-10-04 13:02:59 +02:00
mokaddem
b44b369eec
Merge remote-tracking branch 'origin/2.4' into zoidberg-timeline
2019-10-02 14:35:00 +02:00
mokaddem
265f96dac8
chg: [object:quickAddAttribute] Improved feedback when creation fails
2019-10-02 14:30:34 +02:00
mokaddem
bb3cf85776
Merge branch '2.4' into zoidberg-timeline
2019-10-02 11:23:04 +02:00
Alexandre Dulaunoy
11e4884628
add: [attributes] new dash cryptocurrency address attribute type
2019-10-01 19:47:26 +02:00
iglocska
c53f34e33d
fix: [correlation] Skip correlation on tasks that modify an attribute in a way that wouldn't warrant a recorrelation, fixes #5204
...
- Only recorrelate attribute if:
- attribute is new
- attribute already exists and value, disable_correlation, type is updated
2019-09-29 21:07:35 +02:00
iglocska
8168cc79db
fix: [API] proposals overriding attributes wasn't always working as expected, fixes #4032
...
- until now it was bound to the to_ids setting (badly) which caused nothing but headache
- moved the new configuration to instead use the non-permissive nature of the given export formats
- non-permissive export: if the proposal block is enabled, override attributes
- permissive export types: ignore the proposals
The reasoning is simple: we use the permissive export types for types that can express additional structures such as proposals, IDS flags, publish flags etc (meaning the MISP JSON/XML formats for example)
2019-09-29 20:35:51 +02:00
iglocska
480e3b2969
Merge branch 'dev_session' into 2.4
2019-09-29 20:23:00 +02:00
iglocska
bf35987835
fix: [internal] Proposals block attributes setting broken when to_ids is an array
2019-09-29 18:33:32 +02:00
iglocska
40cf160c53
new: [API] Netfilter added as new export format
2019-09-25 20:17:25 +02:00
iglocska
946602a696
new: [User settings] Added user settings system
...
- set settings / user
- settings can be set by user themselves or their org admin / site admin
- added first setting: publish_alert_filter
- accepts boolean branched filter options
- supports deep logical trees
- OR/NOT/AND
- currently supports filtering on tags and the creator organisation
2019-09-25 11:50:54 +02:00
chrisr3d
c0aec75a09
Merge branch '2.4' of https://github.com/MISP/MISP into 2.4
2019-09-12 17:13:19 +02:00
chrisr3d
fbb25bad6f
new: [restSearch] restSearch module for ATT&CK Sightings
...
- Returning ATTA&CK Sightings in json format for
events and attributes with mitre-attack-pattern
galaxies attached
- For further details on the ATT&CK Sightings,
please visit https://attack.mitre.org/resources/sightings/
- Also thanks to @johnwunder for the clarification
on the output format
2019-09-12 17:03:35 +02:00
mokaddem
f6f1310a50
chg: [decaying:model] Third batch of fix from the PR review - WiP (not tested)
2019-09-12 11:17:33 +02:00
mokaddem
515f0572df
chg: [decaying] Added alias `score` to override on-the-fly the treshold
...
of a model
2019-09-06 14:55:12 +02:00
mokaddem
893dd617c8
chg: [attribute:restSearch] New paramter `includeFullModel` to attach
...
full model information
2019-09-06 11:32:54 +02:00
mokaddem
a5d06d1333
Merge branch '2.4' of github.com:MISP/MISP into decaying
2019-08-29 10:52:18 +02:00
Pierre-Jean Grenier
1994750db1
fix: Fix 'contain' param in app/Model/Attribute.php:fetchAttributes()
...
When we specified eg. 'contain': array('Event'), the merge done by the function was incorrect, and only kept more restrictive stuff,
while we wanted to get all the keys related to the Event.
2019-08-26 17:59:12 +02:00
mokaddem
ed98d73be7
Merge branch '2.4' of github.com:MISP/MISP into decaying
2019-08-22 15:33:09 +02:00
mokaddem
86ca816dcf
chg: [decaying] refact - Accept PUT and added comment for attribute
...
removal in restSearch
2019-08-21 15:53:13 +02:00
mokaddem
f59b338c47
fix: [decaying] Set default value and pre-checks
2019-08-21 12:06:49 +02:00
Andras Iklody
9bef0560fc
Merge pull request #5030 from zaphodef/feature/delete_attribute_messages
...
duh, fix a typo
2019-08-20 17:02:43 +02:00
Pierre-Jean Grenier
a5b9ec3907
duh, fix a typo
2019-08-20 17:00:21 +02:00
Andras Iklody
fa300836ab
Merge pull request #5029 from zaphodef/feature/delete_attribute_messages
...
fix: Fix messages when we try to delete an attribute
2019-08-20 16:55:52 +02:00
Pierre-Jean Grenier
30c270a39b
fix: Fix messages when we try to delete an attribute
2019-08-20 16:46:32 +02:00
Jakub Onderka
4ef9595023
fix: Replace not exists MethodNotFoundException with NotFoundException
2019-08-14 21:27:05 +02:00
mokaddem
05fe5e18e8
chg: [decaying] Allow for model parameteres override
2019-08-14 10:48:13 +02:00
mokaddem
6ba45b27f8
Merge remote-tracking branch 'origin/2.4' into decaying
2019-08-13 16:32:58 +02:00
mokaddem
0c8b7b5820
fix: [decaying] Do not access existing keys anymore
2019-08-13 15:54:03 +02:00
Christophe Vandeplas
099558b61c
new: reminder to run gen_misp_types_categories when model changes
2019-08-08 13:44:57 +02:00
iglocska
14685c45fb
fix: [API] Further fixes to /attributes/add
2019-08-06 16:33:16 +02:00
iglocska
edf0657a1d
fix: [API] Fixes to the new attribute add
2019-08-06 16:01:09 +02:00
iglocska
786d53d30b
new: [API] Attribute add rework - WIP
...
- handle attribute creation in a unified manner via captureAttributes
2019-08-06 15:55:16 +02:00
iglocska
d784903558
new: [internal] Default field list added for attributes
...
- let's try to standardised on things we output instead of doing it manually. It's a first step
2019-08-05 10:47:26 +02:00
iglocska
268cdf2417
chg: [types] email-subject added as a valid type for network activity
...
- used to describe outgoing e-mail subjects for exfiltration. Perhaps consider adding a new category for exfiltration altogether.
2019-08-05 09:43:29 +02:00
iglocska
c79ae263eb
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2019-08-02 13:45:50 +02:00
iglocska
7003faa00c
new: [API] new parameters added to attributes/restSearch to include additional context, fixes #4935 , fixes #4940 , affects MISP/PyMISP#415
...
- includeSightings: include sightings for all attributes returned
- includeCorrelations: include the correlations to other attributes (includes a light-weight event object with each attribute)
2019-08-02 13:41:20 +02:00
chrisr3d
1437c908bb
add: [Model] New attribute type weakness
...
- Describing links linking to the provided CWE lookup
2019-08-01 16:42:10 +02:00
iglocska
41a5597dd1
new: [API] Some more context for includeContext, fixes #4935
2019-07-30 14:56:47 +02:00
iglocska
d5594715b2
new: [API] includeContext now includes the additional event fields in the attributes/restSearch results (in JSON format)
2019-07-30 14:47:02 +02:00
iglocska
b83ac37ff4
new: [API] Allow adding tags via /attributes/add directly
2019-07-30 12:03:49 +02:00
mokaddem
791ef0d21c
chg: [decaying:rest] Renamed `decayed` into `excludeDecayed` for better
...
usability
2019-07-25 16:21:12 +02:00
mokaddem
28f080e581
new: [decaying:rest] Filtering out of decayed attributes
2019-07-25 16:13:42 +02:00
mokaddem
e4cc20b054
new: [decaying] Partial API support - WiP
2019-07-25 15:45:33 +02:00
Bechkalo Evgeny
f2e299ba19
fix: error during creating and deleting Attributes on PostgreSQL
2019-07-23 13:46:06 +03:00
mokaddem
821785273a
Merge remote-tracking branch 'origin/2.4' into decaying
2019-07-22 10:03:57 +02:00
mokaddem
fdf7161dc0
chg: [attribute:search] Added support of `contain` in
...
fetchAttributeSimple()
2019-07-17 16:13:31 +02:00
Alexandre Dulaunoy
020e67c154
new: [attribute-type] community-id added
...
Community-id is a new attribute type to describe a flow hashing algorithm allowing
the consumers of output from multiple traffic monitors to link each system's
flow records more easily.
Ref: https://github.com/corelight/community-id-spec
2019-07-13 08:38:43 +02:00
iglocska
53838ba277
chg: [error code] Attribute delete now responds with 403 if user is not allowed to delete, instead of 405
2019-07-12 16:56:13 +02:00
mokaddem
b8b6a170fe
chg: [attribute:restSearch] Search support for first_seen and last_seen
2019-07-04 15:08:05 +02:00
mokaddem
344f322a7d
chg: [attribute:restSearch] Added filtering conditions for first_seen
...
and last_seen
2019-07-04 13:51:36 +02:00
mokaddem
2ea4916051
Merge branch '2.4' of github.com:MISP/MISP into zoidberg-timeline
2019-06-24 09:06:11 +02:00
iglocska
73963d846c
fix: [data-massaging] Removed massaging for float type attributes
...
- it not only stripped anything but floats as expected, but additionally also rounded the value
2019-06-20 13:58:14 +02:00
mokaddem
0e209b610d
fix: [attribute:*-seen] Force seconds to be integers and allows editForm
...
for *-seen fields
2019-06-13 15:12:59 +02:00
mokaddem
e7f3d0d9df
new: [timeline/*-seen] Initial import of the timeline code from the
...
zoidberg branch
2019-06-13 09:16:34 +02:00
Andras Iklody
fbef06c0ca
Merge pull request #4635 from mokaddem/galaxyMatrixImprovements
...
Galaxy matrix improvements
2019-06-12 14:23:35 +02:00
mokaddem
329908eeda
chg: [attribute:delete] Simplified search options
2019-06-12 12:04:49 +02:00
mokaddem
b3849655e0
fix: [attributes] Correctly pass the user object and renamed delete function
2019-06-12 11:56:42 +02:00
mokaddem
892348d7fd
Merge branch '2.4' of github.com:MISP/MISP into mergeAttributeIntoObjects
2019-06-12 10:35:14 +02:00
mokaddem
52ae153c0e
Merge branch '2.4' of github.com:MISP/MISP into galaxyMatrixImprovements
2019-06-11 15:56:10 +02:00
mokaddem
11a4bdb959
chg: [restSearch:attack] Only expose attack return format to the `event`
...
scope
2019-06-11 15:50:51 +02:00
mokaddem
fed5556976
fix: [galaxyMatrix:export] Removed multiple bugs providing inconsistent
...
result
2019-06-11 14:13:17 +02:00
iglocska
25d8d6cf94
new: [API] added new restSearch filter - date
...
- deprecated to and from
- date works similarly to timestamp, accepted syntax options:
- time ranges in the shorthand format (7d or 24h, etc)
- timestamps
- fallback parsing for other formats (2019-01-01, "fortnight ago", etc)
- date ranges using lists [14d, 7d]
2019-06-07 09:49:52 +02:00
mokaddem
fe4740abd9
fix: [object:fromAttributes] SYNC support for older instances
...
(duplicate attributes and their contexts)
2019-06-06 15:11:34 +02:00
mokaddem
8d2c55fa69
Merge branch '2.4' of github.com:MISP/MISP into mergeAttributeIntoObjects
2019-06-05 12:02:17 +02:00
iglocska
aeb906b494
fix: [API] Weird responses from JSON objects fixed when data returned is empty
2019-05-30 13:49:04 +02:00
mokaddem
af15c4af79
fix: [sync] Correctly capture the attributes from a groupment into an
...
object during the sync
2019-05-27 15:40:54 +02:00
mokaddem
f766f6190a
fix: [attribute:editAttribute] synchronisation support when attributes
...
got merged into an object.
2019-05-27 13:50:22 +02:00
iglocska
93220608f1
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2019-05-23 11:26:11 +02:00
iglocska
9b8ef9d513
new: [API] AND for tag filters in restSeach added
2019-05-23 11:25:29 +02:00
iglocska
02a3a9a384
new: [API] Added object_relation as a filter for both the event/attribute restSearch functions
2019-05-23 07:56:23 +02:00
mokaddem
c9a3b96b7b
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2019-05-22 16:15:55 +02:00
mokaddem
94332afbf6
fix: [restSearche] Correctly interpret the `deleted` parameter on
...
`event` and `attribute` scope.
-- Pair programming with @iglocksa
2019-05-22 16:08:27 +02:00
iglocska
0fd8cee847
fix: [API] Allow more flexibility on the return content types
...
- also set RPZ as txt
2019-05-22 08:56:41 +02:00
mokaddem
e38358de69
fix: [Attribute:restSearch] Prevent failing if file empty
2019-05-15 11:09:39 +02:00
iglocska
5ee6013dff
fix: [API] Some fixes for the restsearch -> attack export
2019-05-10 14:41:50 +02:00
iglocska
e899eb8b9d
new: [ATT&CK] Added new export system for restsearch for ATT&CK
...
- Return the ATT&CK matrix data as HTML via the API
- Directly viewable via the REST client
- Greetings from the ATT&CK workshop @ Eurocontrol
2019-05-10 14:25:38 +02:00
iglocska
5c9332072f
new: [API] Added includeWarninglistHits to the attribute search API
2019-05-09 17:25:39 +02:00
edhoedt
b9463e513c
Yara export
2019-04-29 19:23:14 +02:00
Andras Iklody
8a6ddf3459
Merge pull request #4441 from mokaddem/eventGraph_thumbnail
...
EventGraph thumbnails
2019-04-11 11:55:45 +02:00
iglocska
232946c6eb
fix: [advanced extraction] Fixed invalid double encryption of the malware samples
2019-04-09 15:39:00 +02:00
iglocska
a3381b8196
new: [refanging] Attributes automatically refanged in beforeValidate, fixes #4442
2019-04-09 14:53:39 +02:00
mokaddem
dec3fd8808
chg: [viewPicture] Added comments
2019-04-09 14:00:36 +02:00
mokaddem
303b584c56
chg: [eventGraph] Added support of picture. Fix #4433
2019-04-09 13:56:31 +02:00
mokaddem
352dc9c1a5
new: [thumbnail] Thumbnail are now saved on the disk, greatly improving
...
performance when viewing an event
2019-04-09 12:57:21 +02:00
4ekin
5c51e78320
fix: TODO i18n strings in Attribute Model and updated default.pot
2019-04-02 16:59:49 +03:00
iglocska
b519230f28
fix: [API] fixed adding malware-samples unencrypted with the encrypt key set, fixes #4355
2019-03-24 22:30:41 +01:00
iglocska
beed84a335
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2019-03-19 17:23:22 +01:00
iglocska
7141f70b20
Merge branch 'kafka' into 2.4
2019-03-19 17:23:05 +01:00
Andras Iklody
7f93270710
Merge pull request #4342 from liviuvalsan/bug_fix_bro_export
...
Fixing bug when exporting to Bro MISP attributes from events that contain a percentage sign inside the event info
2019-03-19 16:49:00 +01:00
Liviu Valsan
4656a5c1fa
Fixing bug when exporting to Bro MISP attributes from events that contain a percentage sign inside the event info
2019-03-19 14:46:16 +01:00
Liviu Valsan
fe3241addb
Addressing performance issues for fetching attributes when blocking attributes via proposals
2019-03-19 14:23:08 +01:00
Nikos Filippakis
9d59b10368
Publish events to Kafka
...
Signed-off-by: Nikos Filippakis <nikolaos.filippakis@cern.ch>
2019-03-18 15:53:22 +01:00
iglocska
47d139c296
chg: [CS] indentation
2019-02-23 12:55:43 +01:00
Christophe Vandeplas
67efc70bf5
fix: [style] consistent space indentation
2019-02-10 13:08:55 +01:00
Alexandre Dulaunoy
14130b5229
chg: [datamodel] me being stupid
2019-02-01 09:07:32 +01:00
Alexandre Dulaunoy
c5d10979a9
chg: [datamodel] anonymised updated
2019-01-31 23:44:58 +01:00
Alexandre Dulaunoy
89116dd544
chg: [datamodel] second step validation for anonymised attribute type
2019-01-31 23:39:46 +01:00
Alexandre Dulaunoy
33d690e643
chg: [datamodel] anonymised is any category
2019-01-31 23:35:31 +01:00
Alexandre Dulaunoy
ef39349476
add: [datamodel] anonymise type added
...
Anonymised value - described with the anonymisation object via a relationship
Anonymisation object definition: https://www.misp-project.org/objects.html#_anonymisation
2019-01-31 23:19:17 +01:00
Alexandre Dulaunoy
e65c2a536c
chg: [type] zeek attribute added (Zeek is the new name of Bro)
...
Both attribute types, zeek and bro will coexist as exchange of NIDS
rules under the old names is common in various MISP sharing communities.
2019-01-30 22:53:38 +01:00
Andras Iklody
3843e9bdaa
Merge pull request #3995 from patriziotufarolo/2.4
...
fix: check also event.org_id when validating event ownership in order to fetch attributes. Fixes #1918
2019-01-29 17:38:57 +01:00
mokaddem
b64d5c82a7
fix: [attribute] Prevent undefined index on tag filtering.
...
As tags are popped from the attribute scope first, they will not be
available in the event scope.
2019-01-25 09:34:55 +01:00
iglocska
c38d553192
fix: [interna] deprecated text() function's tag filter fixed
2019-01-25 09:11:01 +01:00
iglocska
d64d57feea
fix: [filters] Negative tag filters ignored event tags on the attriute search
...
- as reported by @hel10wor1d
2019-01-25 08:35:50 +01:00
iglocska
9512043de9
fix: [performance] query tweak to fool old crappy versions of mysql
2019-01-24 16:20:57 +01:00
iglocska
27d048db0c
fix: [performance] Potential performance fix for older MySQL versions using the wrong index as key during fetchAttributes()
...
- observer a server prioritising the deleted flag index when filtering attributes, leading to a massive performance loss
- hacky solution to make deleted and object_id (during flattening) indeces unusable
2019-01-22 08:28:15 +01:00
iglocska
dbb1e01487
chg: [internal] timestamp resolution for time ranges should reorder the conditions
...
- always take from (smaller timestamp) to (larger timestamp), no matter the order which they were entered in
2019-01-17 10:12:47 +01:00
Alexandre Dulaunoy
eaacbaddd3
chg: [datamodels] fix hassh and hasshserver typo
2019-01-13 12:05:21 +01:00
Alexandre Dulaunoy
f388e0eff3
chg: [datamodels] new types hassh-md5 and hasshserver-md5 added
...
"HASSH" is a network fingerprinting standard which can be used
to identify specific Client and Server SSH implementations.
The fingerprints can be easily stored, searched and shared
in the form of an MD5 fingerprint.
Fix #4007
2019-01-13 11:47:30 +01:00
Patrizio Tufarolo
32962184fa
fix: check also event.org_id when validating event ownership in order to fetch attributes
...
Fixes #1918
2019-01-08 06:56:47 -05:00
iglocska
e2965a79e4
Merge branch '2.4' of github.com:MISP/MISP into 2.4
2018-12-31 08:04:29 +01:00
iglocska
65709a353b
Merge branch 'qb' into 2.4
2018-12-31 08:04:12 +01:00
eCrimeLabs
c920925d48
Update Attribute.php
...
Added ja3-fingerprint-md5 as a deticated data type. Also updating the object for ja3
2018-12-30 12:26:33 +01:00
Daniel Roethlisberger
5b4079637a
new: [attributes] Add cdhash attribute, 40+ digit hash, default Payload delivery, ids=1 ( #3965 )
2018-12-19 20:19:49 +01:00
Sami Mokaddem
ed4f752d3a
Merge remote-tracking branch 'origin/2.4' into querybuilder
2018-12-17 16:36:27 +01:00
co59
bbeef129c0
fix: [model] Network activity category: add x509-fingerprint-md5 and x509-fingerprint-sha256
2018-11-29 23:07:11 +01:00
iglocska
665b6ccbae
fix: [internal] if no attribute distribution is found in the event edits, set the default instead of defaulting to 0
2018-11-29 15:33:45 +01:00