MISP
/
misp-rfc
mirror of https://github.com/MISP/misp-rfc
2
0
Fork 0
Code Issues Releases Wiki Activity

merge

pull/31/head
Deborah Servili 2019-11-21 16:20:02 +01:00
parent 6ae41dc451 23b5fba81a
commit 025c2ee432
No known key found for this signature in database
GPG Key ID: 7E3A832850D4D7D1
14 changed files with 2038 additions and 582 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
README.md
View File

@ -12,7 +12,7 @@ All the formats can be freely reused by everyone.
* [misp-core-format](misp-core-format/raw.md.txt) ([markdown source](misp-core-format/raw.md)) which describes the core JSON format of MISP. Current Internet-Draft: [07](https://tools.ietf.org/html/draft-dulaunoy-misp-core-format)
* [misp-taxonomy-format](misp-taxonomy-format/raw.md.txt) ([markdown source](misp-taxonomy-format/raw.md)) which describes the taxonomy JSON format of MISP. Current Internet-Draft: [07](https://tools.ietf.org/html/draft-dulaunoy-misp-taxonomy-format)
* [misp-galaxy-format](misp-galaxy-format/raw.md.txt) which describes the [galaxy](https://github.com/MISP/misp-galaxy) template format used to expand the threat actor modelling of MISP. Current Internet-Draft: [06](https://datatracker.ietf.org/doc/draft-dulaunoy-misp-galaxy-format/)
* [misp-object-template-format](misp-object-template-format/raw.md.txt) which describes the [object](https://github.com/MISP/misp-objects) template format to add combinedand composite object to the MISP core format. Current Internet-Draft: [01](https://datatracker.ietf.org/doc/draft-dulaunoy-misp-object-template-format/)
* [misp-object-template-format](misp-object-template-format/raw.md.txt) which describes the [object](https://github.com/MISP/misp-objects) template format to add combinedand composite object to the MISP core format. Current Internet-Draft: [03](https://datatracker.ietf.org/doc/draft-dulaunoy-misp-object-template-format/)
## MISP Format in design phase and implemented in at least one software prototype

255
misp-core-format/raw.md
View File

@ -1,40 +1,42 @@
% Title = "MISP core format"
% abbrev = "MISP core format"
% category = "info"
% docName = "draft-dulaunoy-misp-core-format"
% ipr= "trust200902"
% area = "Security"
%
% date = 2018-08-08T00:00:00Z
%
% [[author]]
% initials="A."
% surname="Dulaunoy"
% fullname="Alexandre Dulaunoy"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "alexandre.dulaunoy@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = "16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1160"
% country = "Luxembourg"
% [[author]]
% initials="A."
% surname="Iklody"
% fullname="Andras Iklody"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "andras.iklody@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = "16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1160"
% country = "Luxembourg"
%%%
Title = "MISP core format"
abbrev = "MISP core format"
category = "info"
docName = "draft-dulaunoy-misp-core-format"
ipr= "trust200902"
area = "Security"
date = 2018-08-08T00:00:00Z
[[author]]
initials="A."
surname="Dulaunoy"
fullname="Alexandre Dulaunoy"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "alexandre.dulaunoy@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = "16, bd d'Avranches"
city = "Luxembourg"
code = "L-1160"
country = "Luxembourg"
[[author]]
initials="A."
surname="Iklody"
fullname="Andras Iklody"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "andras.iklody@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = "16, bd d'Avranches"
city = "Luxembourg"
code = "L-1160"
country = "Luxembourg"
%%%
.# Abstract
@ -64,7 +66,7 @@ document are to be interpreted as described in RFC 2119 [@!RFC2119].
## Overview
The MISP core format is in the JSON [@!RFC4627] format. In MISP, an event is composed of a single JSON object.
The MISP core format is in the JSON [@!RFC8259] format. In MISP, an event is composed of a single JSON object.
A capitalized key (like Event, Org) represent a data model and a non-capitalised key is just an attribute. This nomenclature
can support an implementation to represent the MISP format in another data structure.
@ -105,7 +107,7 @@ of the event. info **SHOULD** NOT be bigger than 256 characters and **SHOULD** N
info is represented as a JSON string. info **MUST** be present.
#### threat_level_id
#### threat\_level\_id
threat_level_id represents the threat level.
@ -154,13 +156,13 @@ timestamp represents a reference time when the event, or one of the attributes w
timestamp is represented as a JSON string. timestamp **MUST** be present.
#### publish_timestamp
#### publish\_timestamp
publish_timestamp represents a reference time when the event was published on the instance. published_timestamp is expressed in seconds (decimal) since 1st of January 1970 (Unix timestamp). At each publication of an event, publish_timestamp **MUST** be updated. The time zone **MUST** be UTC. If the published_timestamp is present and the published flag is set to false, the publish_timestamp represents the previous publication timestamp. If the event was never published, the published_timestamp **MUST** be set to 0.
publish_timestamp is represented as a JSON string. publish_timestamp **MUST** be present.
#### org_id
#### org\_id
org_id represents a human-readable identifier referencing an Org object of the organisation which generated the event. A human-readable identifier **MUST** be
represented as an unsigned integer.
@ -169,7 +171,7 @@ The org_id **MUST** be updated when the event is generated by a new instance.
org_id is represented as a JSON string. org_id **MUST** be present.
#### orgc_id
#### orgc\_id
orgc_id represents a human-readable identifier referencing an Orgc object of the organisation which created the event.
@ -177,7 +179,7 @@ The orgc_id and Org object **MUST** be preserved for any updates or transfer of
orgc_id is represented as a JSON string. orgc_id **MUST** be present.
#### attribute_count
#### attribute\_count
attribute_count represents the number of attributes in the event. attribute_count is expressed in decimal.
@ -204,7 +206,7 @@ distribution is represented by a JSON string. distribution **MUST** be present a
4
: Sharing Group
#### sharing_group_id
#### sharing\_group\_id
sharing\_group\_id represents a human-readable identifier referencing a Sharing Group object that defines the distribution of the event, if distribution level "4" is set. A human-readable identifier **MUST** be represented as an unsigned integer.
@ -279,7 +281,9 @@ A MISP document **MUST** at least includes category-type-value triplet described
"value": "Hello world",
"SharingGroup": [],
"ShadowAttribute": [],
"RelatedAttribute": []
"RelatedAttribute": [],
"first_seen": "2019-06-02T22:14:28.711954+00:00",
"last_seen": null
}
~~~~
@ -305,52 +309,52 @@ type represents the means through which an attribute tries to describe the inten
type is represented as a JSON string. type **MUST** be present and it **MUST** be a valid selection for the chosen category. The list of valid category-type combinations is as follows:
**Antivirus detection**
Antivirus detection
: link, comment, text, hex, attachment, other, anonymised
**Artifacts dropped**
Artifacts dropped
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, regkey, regkey|value, pattern-in-file, pattern-in-memory, pdb, stix2-pattern, yara, sigma, attachment, malware-sample, named pipe, mutex, windows-scheduled-task, windows-service-name, windows-service-displayname, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, other, cookie, gene, mime-type, anonymised
**Attribution**
Attribution
: threat-actor, campaign-name, campaign-id, whois-registrant-phone, whois-registrant-email, whois-registrant-name, whois-registrant-org, whois-registrar, whois-creation-date, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, other, dns-soa-email, anonymised
**External analysis**
: md5, sha1, sha256, filename, filename|md5, filename|sha1, filename|sha256, ip-src, ip-dst, ip-dst|port, ip-src|port, mac-address, mac-eui-64, hostname, domain, domain|ip, url, user-agent, regkey, regkey|value, AS, snort, bro, zeek, pattern-in-file, pattern-in-traffic, pattern-in-memory, vulnerability, attachment, malware-sample, link, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, github-repository, other, cortex, anonymised
External analysis
: md5, sha1, sha256, filename, filename|md5, filename|sha1, filename|sha256, ip-src, ip-dst, ip-dst|port, ip-src|port, mac-address, mac-eui-64, hostname, domain, domain|ip, url, user-agent, regkey, regkey|value, AS, snort, bro, zeek, pattern-in-file, pattern-in-traffic, pattern-in-memory, vulnerability, weakness, attachment, malware-sample, link, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, github-repository, other, cortex, anonymised, community-id
**Financial fraud**
: btc, xmr, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, phone-number, comment, text, other, hex, anonymised
Financial fraud
: btc, dash, xmr, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, phone-number, comment, text, other, hex, anonymised
**Internal reference**
Internal reference
: text, link, comment, other, hex, anonymised
**Network activity**
: ip-src, ip-dst, ip-dst|port, ip-src|port, port, hostname, domain, domain|ip, mac-address, mac-eui-64, email-dst, url, uri, user-agent, http-method, AS, snort, pattern-in-file, stix2-pattern, pattern-in-traffic, attachment, comment, text, x509-fingerprint-md5, x509-fingerprint-sha1, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hex, cookie, hostname|port, bro, zeek, anonymised
Network activity
: ip-src, ip-dst, ip-dst|port, ip-src|port, port, hostname, domain, domain|ip, mac-address, mac-eui-64, email-dst, url, uri, user-agent, http-method, AS, snort, pattern-in-file, stix2-pattern, pattern-in-traffic, attachment, comment, text, x509-fingerprint-md5, x509-fingerprint-sha1, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hex, cookie, hostname|port, bro, zeek, anonymised, community-id, email-subject
**Other**
Other
: comment, text, other, size-in-bytes, counter, datetime, cpe, port, float, hex, phone-number, boolean, anonymised
**Payload delivery**
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, mac-address, mac-eui-64, ip-src, ip-dst, ip-dst|port, ip-src|port, hostname, domain, email-src, email-dst, email-subject, email-attachment, email-body, url, user-agent, AS, pattern-in-file, pattern-in-traffic, stix2-pattern, yara, sigma, mime-type, attachment, malware-sample, link, malware-type, comment, text, hex, vulnerability, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hostname|port, email-dst-display-name, email-src-display-name, email-header, email-reply-to, email-x-mailer, email-mime-boundary, email-thread-index, email-message-id, mobile-application-id, whois-registrant-email, anonymised
Payload delivery
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, mac-address, mac-eui-64, ip-src, ip-dst, ip-dst|port, ip-src|port, hostname, domain, email-src, email-dst, email-subject, email-attachment, email-body, url, user-agent, AS, pattern-in-file, pattern-in-traffic, stix2-pattern, yara, sigma, mime-type, attachment, malware-sample, link, malware-type, comment, text, hex, vulnerability, weakness, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hostname|port, email-dst-display-name, email-src-display-name, email-header, email-reply-to, email-x-mailer, email-mime-boundary, email-thread-index, email-message-id, mobile-application-id, whois-registrant-email, anonymised
**Payload installation**
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, pattern-in-file, pattern-in-traffic, pattern-in-memory, stix2-pattern, yara, sigma, vulnerability, attachment, malware-sample, malware-type, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, mobile-application-id, other, mime-type, anonymised
Payload installation
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, pattern-in-file, pattern-in-traffic, pattern-in-memory, stix2-pattern, yara, sigma, vulnerability, weakness, attachment, malware-sample, malware-type, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, mobile-application-id, other, mime-type, anonymised
**Payload type**
Payload type
: comment, text, other, anonymised
**Persistence mechanism**
Persistence mechanism
: filename, regkey, regkey|value, comment, text, other, hex, anonymised
**Person**
Person
: first-name, middle-name, last-name, date-of-birth, place-of-birth, gender, passport-number, passport-country, passport-expiration, redress-number, nationality, visa-number, issue-date-of-the-visa, primary-residence, country-of-residence, special-service-request, frequent-flyer-number, travel-details, payment-details, place-port-of-original-embarkation, place-port-of-clearance, place-port-of-onward-foreign-destination, passenger-name-record-locator-number, comment, text, other, phone-number, identity-card-number, anonymised
**Social network**
Social network
: github-username, github-repository, github-organisation, jabber-id, twitter-id, email-src, email-dst, comment, text, other, whois-registrant-email, anonymised
**Support Tool**
Support Tool
: link, text, attachment, comment, other, hex, anonymised
**Targeting data**
Targeting data
: target-user, target-email, target-machine, target-org, target-location, target-external, comment, anonymised
Attributes are based on the usage within their different communities. Attributes can be extended on a regular basis and this reference document is updated accordingly.
@ -412,7 +416,7 @@ comment is a contextual comment field.
comment is represented by a JSON string. comment **MAY** be present.
#### sharing_group_id
#### sharing\_group\_id
sharing\_group\_id represents a human-readable identifier referencing a Sharing Group object that defines the distribution of the attribute, if distribution level "4" is set. A human-readable identifier **MUST** be represented as an unsigned integer.
@ -450,6 +454,18 @@ value represents the payload of an attribute. The format of the value is depende
value is represented by a JSON string. value **MUST** be present.
#### first_seen
first_seen represents a reference time when the attribute was first seen. first_seen is expressed as an ISO 8601 datetime up to the micro-second with time zone support.
first_seen is represented as a JSON string. first_seen **MAY** be present.
#### last_seen
last_seen represents a reference time when the attribute was last seen. last_seen is expressed as an ISO 8601 datetime up to the micro-second with time zone support.
last_seen is represented as a JSON string. last_seen **MAY** be present.
## ShadowAttribute
ShadowAttributes are 3rd party created attributes that either propose to add new information to an event or modify existing information. They are not meant to be actionable until the event creator accepts them - at which point they will be converted into attributes or modify an existing attribute.
@ -477,7 +493,9 @@ They are similar in structure to Attributes but additionally carry a reference t
"id": "1",
"name": "MISP",
"uuid": "568cce5a-0c80-412b-8fdf-1ffac0a83869"
}
},
"first_seen": "2019-06-02T22:14:28.711954+00:00",
"last_seen": null
}
~~~~
@ -501,52 +519,52 @@ type represents the means through which an attribute tries to describe the inten
type is represented as a JSON string. type **MUST** be present and it **MUST** be a valid selection for the chosen category. The list of valid category-type combinations is as follows:
**Antivirus detection**
Antivirus detection
: link, comment, text, hex, attachment, other, anonymised
**Artifacts dropped**
Artifacts dropped
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, regkey, regkey|value, pattern-in-file, pattern-in-memory, pdb, stix2-pattern, yara, sigma, attachment, malware-sample, named pipe, mutex, windows-scheduled-task, windows-service-name, windows-service-displayname, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, other, cookie, gene, mime-type, anonymised
**Attribution**
Attribution
: threat-actor, campaign-name, campaign-id, whois-registrant-phone, whois-registrant-email, whois-registrant-name, whois-registrant-org, whois-registrar, whois-creation-date, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, other, dns-soa-email, anonymised
**External analysis**
: md5, sha1, sha256, filename, filename|md5, filename|sha1, filename|sha256, ip-src, ip-dst, ip-dst|port, ip-src|port, mac-address, mac-eui-64, hostname, domain, domain|ip, url, user-agent, regkey, regkey|value, AS, snort, bro, zeek, pattern-in-file, pattern-in-traffic, pattern-in-memory, vulnerability, attachment, malware-sample, link, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, github-repository, other, cortex, anonymised
External analysis
: md5, sha1, sha256, filename, filename|md5, filename|sha1, filename|sha256, ip-src, ip-dst, ip-dst|port, ip-src|port, mac-address, mac-eui-64, hostname, domain, domain|ip, url, user-agent, regkey, regkey|value, AS, snort, bro, zeek, pattern-in-file, pattern-in-traffic, pattern-in-memory, vulnerability, weakness, attachment, malware-sample, link, comment, text, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, github-repository, other, cortex, anonymised, community-id
**Financial fraud**
: btc, xmr, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, phone-number, comment, text, other, hex, anonymised
Financial fraud
: btc, dash, xmr, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, phone-number, comment, text, other, hex, anonymised
**Internal reference**
Internal reference
: text, link, comment, other, hex, anonymised
**Network activity**
: ip-src, ip-dst, ip-dst|port, ip-src|port, port, hostname, domain, domain|ip, mac-address, mac-eui-64, email-dst, url, uri, user-agent, http-method, AS, snort, pattern-in-file, stix2-pattern, pattern-in-traffic, attachment, comment, text, x509-fingerprint-md5, x509-fingerprint-sha1, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hex, cookie, hostname|port, bro, zeek, anonymised
Network activity
: ip-src, ip-dst, ip-dst|port, ip-src|port, port, hostname, domain, domain|ip, mac-address, mac-eui-64, email-dst, url, uri, user-agent, http-method, AS, snort, pattern-in-file, stix2-pattern, pattern-in-traffic, attachment, comment, text, x509-fingerprint-md5, x509-fingerprint-sha1, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hex, cookie, hostname|port, bro, zeek, anonymised, community-id, email-subject
**Other**
Other
: comment, text, other, size-in-bytes, counter, datetime, cpe, port, float, hex, phone-number, boolean, anonymised
**Payload delivery**
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, mac-address, mac-eui-64, ip-src, ip-dst, ip-dst|port, ip-src|port, hostname, domain, email-src, email-dst, email-subject, email-attachment, email-body, url, user-agent, AS, pattern-in-file, pattern-in-traffic, stix2-pattern, yara, sigma, mime-type, attachment, malware-sample, link, malware-type, comment, text, hex, vulnerability, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hostname|port, email-dst-display-name, email-src-display-name, email-header, email-reply-to, email-x-mailer, email-mime-boundary, email-thread-index, email-message-id, mobile-application-id, whois-registrant-email, anonymised
Payload delivery
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, mac-address, mac-eui-64, ip-src, ip-dst, ip-dst|port, ip-src|port, hostname, domain, email-src, email-dst, email-subject, email-attachment, email-body, url, user-agent, AS, pattern-in-file, pattern-in-traffic, stix2-pattern, yara, sigma, mime-type, attachment, malware-sample, link, malware-type, comment, text, hex, vulnerability, weakness, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, ja3-fingerprint-md5, hassh-md5, hasshserver-md5, other, hostname|port, email-dst-display-name, email-src-display-name, email-header, email-reply-to, email-x-mailer, email-mime-boundary, email-thread-index, email-message-id, mobile-application-id, whois-registrant-email, anonymised
**Payload installation**
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, pattern-in-file, pattern-in-traffic, pattern-in-memory, stix2-pattern, yara, sigma, vulnerability, attachment, malware-sample, malware-type, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, mobile-application-id, other, mime-type, anonymised
Payload installation
: md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, cdhash, filename, filename|md5, filename|sha1, filename|sha224, filename|sha256, filename|sha384, filename|sha512, filename|sha512/224, filename|sha512/256, filename|authentihash, filename|ssdeep, filename|tlsh, filename|imphash, filename|impfuzzy, filename|pehash, pattern-in-file, pattern-in-traffic, pattern-in-memory, stix2-pattern, yara, sigma, vulnerability, weakness, attachment, malware-sample, malware-type, comment, text, hex, x509-fingerprint-sha1, x509-fingerprint-md5, x509-fingerprint-sha256, mobile-application-id, other, mime-type, anonymised
**Payload type**
Payload type
: comment, text, other, anonymised
**Persistence mechanism**
Persistence mechanism
: filename, regkey, regkey|value, comment, text, other, hex, anonymised
**Person**
Person
: first-name, middle-name, last-name, date-of-birth, place-of-birth, gender, passport-number, passport-country, passport-expiration, redress-number, nationality, visa-number, issue-date-of-the-visa, primary-residence, country-of-residence, special-service-request, frequent-flyer-number, travel-details, payment-details, place-port-of-original-embarkation, place-port-of-clearance, place-port-of-onward-foreign-destination, passenger-name-record-locator-number, comment, text, other, phone-number, identity-card-number, anonymised
**Social network**
Social network
: github-username, github-repository, github-organisation, jabber-id, twitter-id, email-src, email-dst, comment, text, other, whois-registrant-email, anonymised
**Support Tool**
Support Tool
: link, text, attachment, comment, other, hex, anonymised
**Targeting data**
Targeting data
: target-user, target-email, target-machine, target-org, target-location, target-external, comment, anonymised
Attributes are based on the usage within their different communities. Attributes can be extended on a regular basis and this reference document is updated accordingly.
@ -620,6 +638,18 @@ the sample **MUST** be encrypted using a password protected zip archive, with th
data is represented by a JSON string in base64 encoding. data **MUST** be set for shadow attributes of type malware-sample and attachment.
#### first_seen
first_seen represents a reference time when the attribute was first seen. first_seen as an ISO 8601 datetime up to the micro-second with time zone support.
first_seen is represented as a JSON string. first_seen **MAY** be present.
#### last_seen
last_seen represents a reference time when the attribute was last seen. last_seen as an ISO 8601 datetime up to the micro-second with time zone support.
last_seen is represented as a JSON string. last_seen **MAY** be present.
### Org
An Org object is composed of an uuid, name and id.
@ -658,9 +688,10 @@ The schema used is described by the template_uuid and template_version fields.
A MISP document containing an Object **MUST** contain a name, a meta-category, a description, a template_uuid and a template_version as described in the "Object Attributes" section.
### Sample Object object
### Sample Object
~~~~~
{#fig-sample-object}
~~~
"Object": {
"id": "588",
"name": "file",
@ -693,11 +724,15 @@ A MISP document containing an Object **MUST** contain a name, a meta-category, a
"object_id": "588",
"object_relation": "filename",
"value": "StarCraft.exe",
"ShadowAttribute": []
}
"ShadowAttribute": [],
"first_seen": null,
"last_seen": null
},
"first_seen": "2019-06-02T22:14:28.711954+00:00",
"last_seen": null
]
}
~~~~~
~~~
### Object Attributes
@ -732,19 +767,19 @@ description is a human-readable description of the given object type, as derived
description is represented as a JSON string. id **SHALL** be present.
#### template_uuid
#### template\_uuid
uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the template used to create the object. The uuid **MUST** be preserved
to preserve the object's association with the correct template used for creation. UUID version 4 is **RECOMMENDED** when assigning it to a new object.
#### template_version
#### template\_version
template_version represents a numeric incrementing version of the template used to create the object. It is used to associate the object to the
correct version of the template and together with the template_uuid forms an association to the correct template type and version.
version is represented as a JSON string. version **MUST** be present.
#### event_id
#### event\_id
event_id represents the human-readable identifier of the event that the object belongs to on a specific MISP instance. A human-readable identifier **MUST** be
represented as an unsigned integer.
@ -778,7 +813,7 @@ distribution is represented by a JSON string. distribution **MUST** be present a
4
: Sharing Group
#### sharing_group_id
#### sharing\_group\_id
sharing\_group\_id represents a human-readable identifier referencing a Sharing Group object that defines the distribution of the object, if distribution level "4" is set. A human-readable identifier **MUST** be represented as an unsigned integer.
@ -802,13 +837,25 @@ Attribute is an array of attributes that describe the object with data.
Each attribute in an object **MUST** contain the parent event's ID in the event_id field and the parent object's ID in the object_id field.
#### first\_seen
first_seen represents a reference time when the object was first seen. first_seen as an ISO 8601 datetime up to the micro-second with time zone support.
first_seen is represented as a JSON string. first_seen **MAY** be present.
#### last\_seen
last_seen represents a reference time when the object was last seen. last_seen as an ISO 8601 datetime up to the micro-second with time zone support.
last_seen is represented as a JSON string. last_seen **MAY** be present.
## Object References
Object References serve as a logical link between an Object and another referenced Object or Attribute. The relationship is categorised by an enumerated value from a fixed vocabulary.
The relationship_type is recommended to be taken from the MISP object relationship list [[@?MISP-R]] is **RECOMMENDED** to ensure a coherent naming of the tags
The relationship\_type is recommended to be taken from the MISP object relationship list [[@?MISP-R]] is **RECOMMENDED** to ensure a coherent naming of the tags
All Object References **MUST** contain an object_uuid, a referenced_uuid and a relationship type.
All Object References **MUST** contain an object\_uuid, a referenced\_uuid and a relationship type.
### Sample ObjectReference object
@ -892,14 +939,14 @@ deleted represents a setting that allows object references to be revoked. Revoke
deleted is represented by a JSON boolean. deleted **MUST** be present.
#### object_uuid
#### object\_uuid
object_uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the object that the given object reference belongs to. The object_uuid **MUST** be preserved
object\_uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the object that the given object reference belongs to. The object\_uuid **MUST** be preserved
to preserve the object reference's association with the object.
#### referenced_uuid
#### referenced\_uuid
referenced_uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the object or attribute that is being referenced by the object reference. The referenced_uuid **MUST** be preserved
referenced\_uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the object or attribute that is being referenced by the object reference. The referenced\_uuid **MUST** be preserved
to preserve the object reference's association with the object or attribute.
## Tag

109
misp-galaxy-format/raw.md
View File

@ -1,55 +1,56 @@
% Title = "MISP galaxy format"
% abbrev = "MISP galaxy format"
% category = "info"
% docName = "draft-dulaunoy-misp-galaxy-format"
% ipr= "trust200902"
% area = "Security"
%
% date = 2018-09-20T00:00:00Z
%
% [[author]]
% initials="A."
% surname="Dulaunoy"
% fullname="Alexandre Dulaunoy"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "alexandre.dulaunoy@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = "16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
% [[author]]
% initials="A."
% surname="Iklody"
% fullname="Andras Iklody"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "andras.iklody@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = " 16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
% [[author]]
% initials="D."
% surname="Servili"
% fullname="Deborah Servili"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "deborah.servili@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = " 16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
%%%
Title = "MISP galaxy format"
abbrev = "MISP galaxy format"
category = "info"
docName = "draft-dulaunoy-misp-galaxy-format"
ipr= "trust200902"
area = "Security"
date = 2019-10-04T00:00:00Z
[[author]]
initials="A."
surname="Dulaunoy"
fullname="Alexandre Dulaunoy"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "alexandre.dulaunoy@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = "16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
[[author]]
initials="A."
surname="Iklody"
fullname="Andras Iklody"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "andras.iklody@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = " 16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
[[author]]
initials="D."
surname="Servili"
fullname="Deborah Servili"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "deborah.servili@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = " 16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
%%%
.# Abstract
@ -74,11 +75,11 @@ document are to be interpreted as described in RFC 2119 [@!RFC2119].
A cluster is composed of a value (**MUST**), a description (**OPTIONAL**) and metadata (**OPTIONAL**).
Clusters are represented as a JSON [@!RFC4627] dictionary.
Clusters are represented as a JSON [@!RFC8259] dictionary.
## Overview
The MISP galaxy format uses the JSON [@!RFC4627] format. Each galaxy is represented as a JSON object with meta information including the following fields: name, uuid, description, version, type, authors, source, values, category.
The MISP galaxy format uses the JSON [@!RFC8259] format. Each galaxy is represented as a JSON object with meta information including the following fields: name, uuid, description, version, type, authors, source, values, category.
name defines the name of the galaxy. The name is represented as a string and **MUST** be present. The uuid represents the Universally Unique IDentifier (UUID) [@!RFC4122] of the object reference. The uuid **MUST** be preserved. For any updates or transfer of the same object reference. UUID version 4 is **RECOMMENDED** when assigning it to a new object reference and **MUST** be present. The description is represented as a string and **MUST** be present. The uuid is represented as a string and **MUST** be present. The version is represented as a decimal and **MUST** be present. The type is represented as a string and **MUST** be present and **MUST** match the name of the galaxy file. The source is represented as a string and **MUST** be present. Authors are represented as an array containing one or more authors and **MUST** be present. The category is represented as a string and **MUST** be present and describes the overall category of the galaxy such as tool or actor.
@ -104,7 +105,7 @@ Related contains a list of JSON key value pairs which describe the related value
## meta
Meta contains a list of custom defined JSON key value pairs. Users **SHOULD** reuse commonly used keys such as complexity, effectiveness, country, possible_issues, colour, motive, impact, refs, synonyms, status, date, encryption, extensions, ransomnotes, ransomnotes-filenames, ransomnotes-refs, suspected-victims, suspected-state-sponsor, type-of-incident, target-category, cfr-suspected-victims, cfr-suspected-state-sponsor, cfr-type-of-incident, cfr-target-category, attribution-confidence, payment-method, price, spoken-language wherever applicable.
Meta contains a list of custom defined JSON key value pairs. Users **SHOULD** reuse commonly used keys such as complexity, effectiveness, country, possible_issues, colour, motive, impact, refs, synonyms, status, date, encryption, extensions, ransomnotes, ransomnotes-filenames, ransomnotes-refs, suspected-victims, suspected-state-sponsor, type-of-incident, target-category, cfr-suspected-victims, cfr-suspected-state-sponsor, cfr-type-of-incident, cfr-target-category, attribution-confidence, payment-method, price, spoken-language wherever applicable. Additional meta field **MAY** be added without the need to be referenced or registered in advance.
refs, synonyms **SHALL** be used to give further informations. refs is represented as an array containing one or more strings and **SHALL** be present. synonyms is represented as an array containing one or more strings and **SHALL** be present.

258
misp-galaxy-format/raw.md.txt
View File

@ -5,8 +5,8 @@
Network Working Group A. Dulaunoy
Internet-Draft A. Iklody
Intended status: Informational D. Servili
Expires: March 24, 2019 CIRCL
September 20, 2018
Expires: April 6, 2020 CIRCL
October 4, 2019
MISP galaxy format
@ -38,11 +38,11 @@ Status of This Memo
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 24, 2019.
This Internet-Draft will expire on April 6, 2020.
Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the
Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
@ -53,9 +53,9 @@ Copyright Notice
Dulaunoy, et al. Expires March 24, 2019 [Page 1]
Dulaunoy, et al. Expires April 6, 2020 [Page 1]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
to this document. Code Components extracted from this document must
@ -72,14 +72,14 @@ Table of Contents
2.2. values . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3. related . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.4. meta . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. JSON Schema . . . . . . . . . . . . . . . . . . . . . . . . . 8
3. JSON Schema . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.1. MISP galaxy format - galaxy . . . . . . . . . . . . . . . 9
3.2. MISP galaxy format - clusters . . . . . . . . . . . . . . 9
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 13
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.1. Normative References . . . . . . . . . . . . . . . . . . 13
5.2. Informative References . . . . . . . . . . . . . . . . . 13
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 14
3.2. MISP galaxy format - clusters . . . . . . . . . . . . . . 10
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.1. Normative References . . . . . . . . . . . . . . . . . . 14
5.2. Informative References . . . . . . . . . . . . . . . . . 14
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 15
1. Introduction
@ -109,9 +109,9 @@ Table of Contents
Dulaunoy, et al. Expires March 24, 2019 [Page 2]
Dulaunoy, et al. Expires April 6, 2020 [Page 2]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
2. Format
@ -119,11 +119,11 @@ Internet-Draft MISP galaxy format September 2018
A cluster is composed of a value (MUST), a description (OPTIONAL) and
metadata (OPTIONAL).
Clusters are represented as a JSON [RFC4627] dictionary.
Clusters are represented as a JSON [RFC8259] dictionary.
2.1. Overview
The MISP galaxy format uses the JSON [RFC4627] format. Each galaxy
The MISP galaxy format uses the JSON [RFC8259] format. Each galaxy
is represented as a JSON object with meta information including the
following fields: name, uuid, description, version, type, authors,
source, values, category.
@ -165,9 +165,9 @@ Internet-Draft MISP galaxy format September 2018
Dulaunoy, et al. Expires March 24, 2019 [Page 3]
Dulaunoy, et al. Expires April 6, 2020 [Page 3]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
dest-uuid represents the target UUID which encompasses a relation of
@ -195,7 +195,9 @@ Internet-Draft MISP galaxy format September 2018
filenames, ransomnotes-refs, suspected-victims, suspected-state-
sponsor, type-of-incident, target-category, cfr-suspected-victims,
cfr-suspected-state-sponsor, cfr-type-of-incident, cfr-target-
category, attribution-confidence wherever applicable.
category, attribution-confidence, payment-method, price wherever
applicable. Additional meta field MAY be added without the need to
be referenced or registered in advance.
refs, synonyms SHALL be used to give further informations. refs is
represented as an array containing one or more strings and SHALL be
@ -216,16 +218,16 @@ Internet-Draft MISP galaxy format September 2018
complexity, effectiveness, impact, possible_issues MAY be used to
give further information in preventive-measure galaxy. complexity is
represented by an enumerated value from a fixed vocabulary and SHALL
Dulaunoy, et al. Expires April 6, 2020 [Page 4]
Internet-Draft MISP galaxy format October 2019
be present. effectiveness is represented by an enumerated value from
a fixed vocabulary and SHALL be present. impact is represented by an
Dulaunoy, et al. Expires March 24, 2019 [Page 4]
Internet-Draft MISP galaxy format September 2018
enumerated value from a fixed vocabulary and SHALL be present.
possible_issues is represented as a string and SHOULD be present.
@ -275,11 +277,9 @@ Internet-Draft MISP galaxy format September 2018
Dulaunoy, et al. Expires March 24, 2019 [Page 5]
Dulaunoy, et al. Expires April 6, 2020 [Page 5]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
{
@ -303,14 +303,16 @@ Internet-Draft MISP galaxy format September 2018
}
encryption, extensions, ransomnotes, ransomnotes-filenames,
ransomnotes-refs MAY be used to give further information in
ransomware galaxy. encryption is represented as a string and SHALL be
present. extensions is represented as an array containing one or more
strings and SHALL be present. ransomnotes is represented as an array
containing one or more strings ans SHALL be present. ransomnotes-
filenames is represented as an array containing one or more strings
ans SHALL be present. ransomnotes-refs is represented as an array
containing one or more strings ans SHALL be present.
ransomnotes-refs, payment-method, price MAY be used to give further
information in ransomware galaxy. encryption is represented as a
string and SHALL be present. extensions is represented as an array
containing one or more strings and SHALL be present. ransomnotes is
represented as an array containing one or more strings ans SHALL be
present. ransomnotes-filenames is represented as an array containing
one or more strings ans SHALL be present. ransomnotes-refs is
represented as an array containing one or more strings ans SHALL be
present. payment-method is represented as a string and SHALL be
present. price is represented as a string and SHALL be present.
Example use of the encryption, extensions, ransomnotes fields in the
ransomware galaxy:
@ -331,11 +333,9 @@ Internet-Draft MISP galaxy format September 2018
Dulaunoy, et al. Expires March 24, 2019 [Page 6]
Dulaunoy, et al. Expires April 6, 2020 [Page 6]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
{
@ -356,11 +356,44 @@ Internet-Draft MISP galaxy format September 2018
"value": "Ryuk ransomware"
}
Example use of the payment-method, price fields in the ransomware
galaxy:
{
"description": "This is most likely to affect English speaking users, since the note is written in English. English is understood worldwide, thus anyone can be harmed. The hacker spread the virus using email spam, fake updates, and harmful attachments. All your files are compromised including music, MS Office, Open Office, pictures, videos, shared online files etc..",
"meta": {
"date": "March 2017",
"encryption": "AES-128",
"extensions": [
".enc"
],
"payment-method": "Bitcoin",
"price": "0.1",
"ransomnotes": [
"Blocked Your computer has been blocked All your files are encrypted. To access your PC, you need to send to Bitcoin at the address below loading Step 1: Go to xxxxs : //wvw.coinbase.com/ siqnup Step 2: Create an account and follow the instructions Step 3: Go to the \"Buy Bitcoins\" section and then buy Bitcoin Step 4: Go to the \"Send\" section, enter the address above and the amount (0.1 Bitcoin) Step 5: Click on the button below to verify the payment, your files will be decrypted and the virus will disappear 'Check' If you try to bypass the lock, all files will be published on the Internet, as well as your login for all sites."
],
"refs": [
"https://id-ransomware.blogspot.co.il/2017/03/cryptomeister-ransomware.html"
]
},
"uuid": "4c76c845-c5eb-472c-93a1-4178f86c319b",
"value": "CryptoMeister Ransomware"
}
source-uuid, target-uuid SHALL be used to describe relationships.
source-uuid and target-uuid represent the Universally Unique
IDentifier (UUID) [RFC4122] of the value reference. source-uuid and
target-uuid MUST be preserved.
Dulaunoy, et al. Expires April 6, 2020 [Page 7]
Internet-Draft MISP galaxy format October 2019
Example use of the source-uuid, target-uuid fields in the mitre-
enterprise-attack-relationship galaxy:
@ -387,17 +420,36 @@ Internet-Draft MISP galaxy format September 2018
exhaustive list of possible values for cfr-target-category includes
"Private sector", "Government", "Civil society", "Military".
Dulaunoy, et al. Expires March 24, 2019 [Page 7]
Internet-Draft MISP galaxy format September 2018
Example use of the cfr-suspected-victims, cfr-suspected-state-
sponsor, cfr-type-of-incident, cfr-target-category fields in the
threat-actor galaxy:
Dulaunoy, et al. Expires April 6, 2020 [Page 8]
Internet-Draft MISP galaxy format October 2019
{
"meta": {
"country": "CN",
@ -441,17 +493,19 @@ Internet-Draft MISP galaxy format September 2018
formats. The main format is the MISP galaxy format used for the
clusters.
Dulaunoy, et al. Expires March 24, 2019 [Page 8]
Internet-Draft MISP galaxy format September 2018
3.1. MISP galaxy format - galaxy
Dulaunoy, et al. Expires April 6, 2020 [Page 9]
Internet-Draft MISP galaxy format October 2019
{
"$schema": "http://json-schema.org/schema#",
"title": "Validator for misp-galaxies - Galaxies",
@ -498,16 +552,16 @@ Internet-Draft MISP galaxy format September 2018
{
"$schema": "http://json-schema.org/schema#",
"title": "Validator for misp-galaxies - Clusters",
Dulaunoy, et al. Expires March 24, 2019 [Page 9]
Internet-Draft MISP galaxy format September 2018
"id": "https://www.github.com/MISP/misp-galaxies/schema_clusters.json",
"type": "object",
Dulaunoy, et al. Expires April 6, 2020 [Page 10]
Internet-Draft MISP galaxy format October 2019
"additionalProperties": false,
"properties": {
"description": {
@ -554,16 +608,16 @@ Internet-Draft MISP galaxy format September 2018
"type": "object"
},
"properties": {
Dulaunoy, et al. Expires March 24, 2019 [Page 10]
Internet-Draft MISP galaxy format September 2018
"dest-uuid": {
"type": "string"
Dulaunoy, et al. Expires April 6, 2020 [Page 11]
Internet-Draft MISP galaxy format October 2019
},
"type": {
"type": "string"
@ -610,16 +664,16 @@ Internet-Draft MISP galaxy format September 2018
"type": "string"
},
"refs": {
Dulaunoy, et al. Expires March 24, 2019 [Page 11]
Internet-Draft MISP galaxy format September 2018
"type": "array",
"uniqueItems": true,
Dulaunoy, et al. Expires April 6, 2020 [Page 12]
Internet-Draft MISP galaxy format October 2019
"items": {
"type": "string"
}
@ -666,16 +720,16 @@ Internet-Draft MISP galaxy format September 2018
"type": "array",
"uniqueItems": true,
"items": {
Dulaunoy, et al. Expires March 24, 2019 [Page 12]
Internet-Draft MISP galaxy format September 2018
"type": "string"
}
Dulaunoy, et al. Expires April 6, 2020 [Page 13]
Internet-Draft MISP galaxy format October 2019
}
},
"required": [
@ -710,10 +764,10 @@ Internet-Draft MISP galaxy format September 2018
DOI 10.17487/RFC4122, July 2005,
<https://www.rfc-editor.org/info/rfc4122>.
[RFC4627] Crockford, D., "The application/json Media Type for
JavaScript Object Notation (JSON)", RFC 4627,
DOI 10.17487/RFC4627, July 2006,
<https://www.rfc-editor.org/info/rfc4627>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
5.2. Informative References
@ -725,9 +779,11 @@ Internet-Draft MISP galaxy format September 2018
Dulaunoy, et al. Expires March 24, 2019 [Page 13]
Dulaunoy, et al. Expires April 6, 2020 [Page 14]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
[JSON-SCHEMA]
@ -781,9 +837,9 @@ Authors' Addresses
Dulaunoy, et al. Expires March 24, 2019 [Page 14]
Dulaunoy, et al. Expires April 6, 2020 [Page 15]
Internet-Draft MISP galaxy format September 2018
Internet-Draft MISP galaxy format October 2019
Deborah Servili
@ -837,4 +893,4 @@ Internet-Draft MISP galaxy format September 2018
Dulaunoy, et al. Expires March 24, 2019 [Page 15]
Dulaunoy, et al. Expires April 6, 2020 [Page 16]

210
misp-object-template-format/raw.md
View File

@ -1,40 +1,42 @@
% Title = "MISP object template format"
% abbrev = "MISP object template format"
% category = "info"
% docName = "draft-dulaunoy-misp-object-template-format"
% ipr= "trust200902"
% area = "Security"
%
% date = 2018-04-10T00:00:00Z
%
% [[author]]
% initials="A."
% surname="Dulaunoy"
% fullname="Alexandre Dulaunoy"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "alexandre.dulaunoy@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = "16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
% [[author]]
% initials="A."
% surname="Iklody"
% fullname="Andras Iklody"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "andras.iklody@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = " 16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
%%%
Title = "MISP object template format"
abbrev = "MISP object template format"
category = "info"
docName = "draft-dulaunoy-misp-object-template-format"
ipr= "trust200902"
area = "Security"
date = 2018-04-10T00:00:00Z
[[author]]
initials="A."
surname="Dulaunoy"
fullname="Alexandre Dulaunoy"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "alexandre.dulaunoy@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = "16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
[[author]]
initials="A."
surname="Iklody"
fullname="Andras Iklody"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "andras.iklody@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = " 16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
%%%
.# Abstract
@ -67,7 +69,7 @@ MISP object template elements consist of an object\_relation (**MUST**), a type
## Overview
The MISP object template format uses the JSON [@!RFC4627] format. Each template is represented as a JSON object with meta information including the following fields: uuid, requiredOneOf, description, version, meta-category, name.
The MISP object template format uses the JSON [@!RFC8259] format. Each template is represented as a JSON object with meta information including the following fields: uuid, requiredOneOf, description, version, meta-category, name.
### Object Template
@ -313,7 +315,137 @@ format is represented by a JSON list containing a list of formats that the relat
The MISP object template directory is publicly available [@?MISP-O] in a git repository. The repository contains an objects directory, which contains a directory per object type, containing a file named definition.json which contains the definition of the object template in the above described format.
A relationships directory is also included, containing a definition.json file which contains a list of MISP object relation definitions. There are more than 90 existing templates object documented in [@?MISP-O-DOC].
A relationships directory is also included, containing a definition.json file which contains a list of MISP object relation definitions. There are more than 125 existing templates object documented in [@?MISP-O-DOC].
## Existing and public MISP object templates
- tsk-chats - An Object Template to gather information from evidential or interesting exchange of messages identified during a digital forensic investigation.
- tsk-web-bookmark - An Object Template to add evidential bookmarks identified during a digital forensic investigation.
- tsk-web-cookie - An TSK-Autopsy Object Template to represent cookies identified during a forensic investigation.
- tsk-web-downloads - An Object Template to add web-downloads.
- tsk-web-history - An Object Template to share web history information.
- tsk-web-search-query - An Object Template to share web search query information.
- ail-leak - An information leak as defined by the AIL Analysis Information Leak framework.
- ais-info - Automated Indicator Sharing (AIS) Information Source Markings.
- android-permission - A set of android permissions - one or more permission(s) which can be linked to other objects (e.g. malware, app).
- annotation - An annotation object allowing analysts to add annotations, comments, executive summary to a MISP event, objects or attributes.
- anonymisation - Anonymisation object describing an anonymisation technique used to encode MISP attribute values. Reference: https://www.caida.org/tools/taxonomy/anonymization.xml.
- asn - Autonomous system object describing an autonomous system which can include one or more network operators management an entity (e.g. ISP) along with their routing policy, routing prefixes or alike.
- authenticode-signerinfo - Authenticode Signer Info.
- av-signature - Antivirus detection signature.
- bank-account - An object describing bank account information based on account description from goAML 4.0.
- bgp-hijack - Object encapsulating BGP Hijack description as specified, for example, by bgpstream.com.
- cap-alert - Common Alerting Protocol Version (CAP) alert object.
- cap-info - Common Alerting Protocol Version (CAP) info object.
- cap-resource - Common Alerting Protocol Version (CAP) resource object.
- coin-address - An address used in a cryptocurrency.
- cookie - An HTTP cookie (web cookie, browser cookie) is a small piece of data that a server sends to the user's web browser. The browser may store it and send it back with the next request to the same server. Typically, it's used to tell if two requests came from the same browser — keeping a user logged-in, for example. It remembers stateful information for the stateless HTTP protocol. (as defined by the Mozilla foundation.
- cortex - Cortex object describing a complete cortex analysis. Observables would be attribute with a relationship from this object.
- cortex-taxonomy - Cortex object describing an Cortex Taxonomy (or mini report).
- course-of-action - An object describing a specific measure taken to prevent or respond to an attack.
- cowrie - Cowrie honeypot object template.
- credential - Credential describes one or more credential(s) including password(s), api key(s) or decryption key(s).
- credit-card - A payment card like credit card, debit card or any similar cards which can be used for financial transactions.
- ddos - DDoS object describes a current DDoS activity from a specific or/and to a specific target. Type of DDoS can be attached to the object as a taxonomy.
- device - An object to define a device.
- diameter-attack - Attack as seen on diameter authentication against a GSM, UMTS or LTE network.
- domain-ip - A domain and IP address seen as a tuple in a specific time frame.
- elf - Object describing a Executable and Linkable Format.
- elf-section - Object describing a section of an Executable and Linkable Format.
- email - Email object describing an email with meta-information.
- exploit-poc - Exploit-poc object describing a proof of concept or exploit of a vulnerability. This object has often a relationship with a vulnerability object.
- facial-composite - An object which describes a facial composite.
- fail2ban - Fail2ban event.
- file - File object describing a file with meta-information.
- forensic-case - An object template to describe a digital forensic case.
- forensic-evidence - An object template to describe a digital forensic evidence.
- geolocation - An object to describe a geographic location.
- gtp-attack - GTP attack object as seen on a GSM, UMTS or LTE network.
- http-request - A single HTTP request header.
- ilr-impact - Institut Luxembourgeois de Regulation - Impact.
- ilr-notification-incident - Institut Luxembourgeois de Regulation - Notification d'incident.
- internal-reference - Internal reference.
- interpol-notice - An object which describes a Interpol notice.
- ip-api-address - IP Address information. Useful if you are pulling your ip information from ip-api.com.
- ip-port - An IP address (or domain or hostname) and a port seen as a tuple (or as a triple) in a specific time frame.
- irc - An IRC object to describe an IRC server and the associated channels.
- ja3 - JA3 is a new technique for creating SSL client fingerprints that are easy to produce and can be easily shared for threat intelligence. Fingerprints are composed of Client Hello packet; SSL Version, Accepted Ciphers, List of Extensions, Elliptic Curves, and Elliptic Curve Formats. https://github.com/salesforce/ja3.
- legal-entity - An object to describe a legal entity.
- lnk - LNK object describing a Windows LNK binary file (aka Windows shortcut).
- macho - Object describing a file in Mach-O format.
- macho-section - Object describing a section of a file in Mach-O format.
- mactime-timeline-analysis - Mactime template, used in forensic investigations to describe the timeline of a file activity.
- malware-config - Malware configuration recovered or extracted from a malicious binary.
- microblog - Microblog post like a Twitter tweet or a post on a Facebook wall.
- mutex - Object to describe mutual exclusion locks (mutex) as seen in memory or computer program.
- netflow - Netflow object describes an network object based on the Netflowv5/v9 minimal definition.
- network-connection - A local or remote network connection.
- network-socket - Network socket object describes a local or remote network connections based on the socket data structure.
- misc - An object which describes an organization.
- original-imported-file - Object describing the original file used to import data in MISP.
- passive-dns - Passive DNS records as expressed in draft-dulaunoy-dnsop-passive-dns-cof-01.
- paste - Paste or similar post from a website allowing to share privately or publicly posts.
- pcap-metadata - Network packet capture metadata.
- pe - Object describing a Portable Executable.
- pe-section - Object describing a section of a Portable Executable.
- person - An object which describes a person or an identity.
- phishing - Phishing template to describe a phishing website and its analysis.
- phishing-kit - Object to describe a phishing-kit.
- phone - A phone or mobile phone object which describe a phone.
- process - Object describing a system process.
- python-etvx-event-log - Event log object template to share information of the activities conducted on a system. .
- r2graphity - Indicators extracted from files using radare2 and graphml.
- regexp - An object describing a regular expression (regex or regexp). The object can be linked via a relationship to other attributes or objects to describe how it can be represented as a regular expression.
- registry-key - Registry key object describing a Windows registry key with value and last-modified timestamp.
- regripper-NTUser - Regripper Object template designed to present user specific configuration details extracted from the NTUSER.dat hive.
- regripper-sam-hive-single-user - Regripper Object template designed to present user profile details extracted from the SAM hive.
- regripper-sam-hive-user-group - Regripper Object template designed to present group profile details extracted from the SAM hive.
- regripper-software-hive-BHO - Regripper Object template designed to gather information of the browser helper objects installed on the system.
- regripper-software-hive-appInit-DLLS - Regripper Object template designed to gather information of the DLL files installed on the system.
- regripper-software-hive-application-paths - Regripper Object template designed to gather information of the application paths.
- regripper-software-hive-applications-installed - Regripper Object template designed to gather information of the applications installed on the system.
- regripper-software-hive-command-shell - Regripper Object template designed to gather information of the shell commands executed on the system.
- regripper-software-hive-windows-general-info - Regripper Object template designed to gather general windows information extracted from the software-hive.
- regripper-software-hive-software-run - Regripper Object template designed to gather information of the applications set to run on the system.
- regripper-software-hive-userprofile-winlogon - Regripper Object template designed to gather user profile information when the user logs onto the system, gathered from the software hive.
- regripper-system-hive-firewall-configuration - Regripper Object template designed to present firewall configuration information extracted from the system-hive.
- regripper-system-hive-general-configuration - Regripper Object template designed to present general system properties extracted from the system-hive.
- regripper-system-hive-network-information. - Regripper object template designed to gather network information from the system-hive.
- regripper-system-hive-services-drivers - Regripper Object template designed to gather information regarding the services/drivers from the system-hive.
- report - Metadata used to generate an executive level report.
- research-scanner - Information related to known scanning activity (e.g. from research projects).
- rogue-dns - Rogue DNS as defined by CERT.br.
- rtir - RTIR - Request Tracker for Incident Response.
- sandbox-report - Sandbox report.
- sb-signature - Sandbox detection signature.
- script - Object describing a computer program written to be run in a special run-time environment. The script or shell script can be used for malicious activities but also as support tools for threat analysts.
- shell-commands - Object describing a series of shell commands executed. This object can be linked with malicious files in order to describe a specific execution of shell commands.
- short-message-service - Short Message Service (SMS) object template describing one or more SMS message. Restriction of the initial format 3GPP 23.038 GSM character set doesn't apply.
- shortened-link - Shortened link and its redirect target.
- splunk - Splunk / Splunk ES object.
- ss7-attack - SS7 object of an attack seen on a GSM, UMTS or LTE network via SS7 logging.
- ssh-authorized-keys - An object to store ssh authorized keys file.
- stix2-pattern - An object describing a STIX pattern. The object can be linked via a relationship to other attributes or objects to describe how it can be represented as a STIX pattern.
- suricata - An object describing one or more Suricata rule(s) along with version and contextual information.
- target-system - Description about an targeted system, this could potentially be a compromissed internal system.
- threatgrid-report - ThreatGrid report.
- timecode - Timecode object to describe a start of video sequence (e.g. CCTV evidence) and the end of the video sequence.
- timesketch-timeline - A timesketch timeline object based on mandatory field in timesketch to describe a log entry.
- timesketch_message - A timesketch message entry.
- timestamp - A generic timestamp object to represent time including first time and last time seen. Relationship will then define the kind of time relationship.
- tor-hiddenservice - Tor hidden service (onion service) object.
- tor-node - Tor node (which protects your privacy on the internet by hiding the connection between users Internet address and the services used by the users) description which are part of the Tor network at a time.
- tracking-id - Analytics and tracking ID such as used in Google Analytics or other analytic platform.
- transaction - An object to describe a financial transaction.
- url - url object describes an url along with its normalized field (like extracted using faup parsing library) and its metadata.
- vehicle - Vehicle object template to describe a vehicle information and registration.
- victim - Victim object describes the target of an attack or abuse.
- virustotal-report - VirusTotal report.
- vulnerability - Vulnerability object describing a common vulnerability enumeration which can describe published, unpublished, under review or embargo vulnerability for software, equipments or hardware.
- whois - Whois records information for a domain name or an IP address.
- x509 - x509 object describing a X.509 certificate.
- yabin - yabin.py generates Yara rules from function prologs, for matching and hunting binaries. ref: https://github.com/AlienVault-OTX/yabin.
- yara - An object describing a YARA rule along with its version.
# Acknowledgements

548
misp-object-template-format/raw.md.txt
View File

@ -27,7 +27,7 @@ Status of This Memo
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/.
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
@ -43,7 +43,7 @@ Copyright Notice
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
@ -69,11 +69,12 @@ Table of Contents
2.1.3. Sample Object Template object . . . . . . . . . . . . 6
2.1.4. Object Relationships . . . . . . . . . . . . . . . . 9
3. Directory . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 10
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 10
5.1. Normative References . . . . . . . . . . . . . . . . . . 10
5.2. Informative References . . . . . . . . . . . . . . . . . 10
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11
3.1. Existing and public MISP object templates . . . . . . . . 10
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.1. Normative References . . . . . . . . . . . . . . . . . . 18
5.2. Informative References . . . . . . . . . . . . . . . . . 18
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19
1. Introduction
@ -108,7 +109,6 @@ Table of Contents
Dulaunoy & Iklody Expires October 12, 2018 [Page 2]
Internet-Draft MISP object template format April 2018
@ -123,18 +123,20 @@ Internet-Draft MISP object template format April 2018
MISP object templates themselves consist of a name (MUST), a meta-
category (MUST) and a description (SHOULD). They are identified by a
uuid (MUST) and a version (MUST). The list of requirements when it
comes to the contained MISP object template elements is defined in
the requirements field (OPTIONAL).
uuid (MUST) and a version (MUST). For any updates or transfer of the
same object reference. UUID version 4 is RECOMMENDED when assigning
it to a new object reference. The list of requirements when it comes
to the contained MISP object template elements is defined in the
requirements field (OPTIONAL).
MISP object template elements consist of an object_relation (MUST) a
type (MUST) an object_template_id (SHOULD) a ui_priority (SHOULD) a
list of categories (MAY), a list of sane_default values (MAY) or a
MISP object template elements consist of an object_relation (MUST), a
type (MUST), an object_template_id (SHOULD), a ui_priority (SHOULD),
a list of categories (MAY), a list of sane_default values (MAY) or a
values_list (MAY).
2.1. Overview
The MISP object template format uses the JSON [RFC4627] format. Each
The MISP object template format uses the JSON [RFC8259] format. Each
template is represented as a JSON object with meta information
including the following fields: uuid, requiredOneOf, description,
version, meta-category, name.
@ -157,10 +159,8 @@ Internet-Draft MISP object template format April 2018
be created based on the given template. The requiredOneOf field MAY
be present.
2.1.1.3. required
required is represented as a JSON list and contains a list of
attribute relationships of which all must be present in the object to
@ -170,6 +170,10 @@ Dulaunoy & Iklody Expires October 12, 2018 [Page 3]
Internet-Draft MISP object template format April 2018
2.1.1.3. required
required is represented as a JSON list and contains a list of
attribute relationships of which all must be present in the object to
be created based on the given template. The required field MAY be
present.
@ -195,7 +199,7 @@ Internet-Draft MISP object template format April 2018
list of options but can be created on the fly.
meta-category is represented as a JSON string. meta-category MUST be
present
present.
2.1.1.7. name
@ -212,11 +216,7 @@ Internet-Draft MISP object template format April 2018
attributes is represented as a JSON list. attributes MUST be present.
2.1.2.1. description
description is represented as a JSON string and contains the
description of the given attribute in the context of the object with
the given relationship. The description field MUST be present.
@ -226,6 +226,12 @@ Dulaunoy & Iklody Expires October 12, 2018 [Page 4]
Internet-Draft MISP object template format April 2018
2.1.2.1. description
description is represented as a JSON string and contains the
description of the given attribute in the context of the object with
the given relationship. The description field MUST be present.
2.1.2.2. ui-priority
ui-priority is represented by a numeric values in JSON string format
@ -268,12 +274,6 @@ Internet-Draft MISP object template format April 2018
The multiple field MAY be present.
2.1.2.7. sane_default
sane_default is represented by a JSON list containing one or several
recommended/sane values for an attribute. sane_default is mutually
exclusive with values_list.
@ -282,6 +282,12 @@ Dulaunoy & Iklody Expires October 12, 2018 [Page 5]
Internet-Draft MISP object template format April 2018
2.1.2.7. sane_default
sane_default is represented by a JSON list containing one or several
recommended/sane values for an attribute. sane_default is mutually
exclusive with values_list.
The sane_default field MAY be present.
2.1.2.8. values_list
@ -313,12 +319,6 @@ Internet-Draft MISP object template format April 2018
@ -522,7 +522,453 @@ Internet-Draft MISP object template format April 2018
A relationships directory is also included, containing a
definition.json file which contains a list of MISP object relation
definitions
definitions. There are more than 125 existing templates object
documented in [MISP-O-DOC].
3.1. Existing and public MISP object templates
o tsk-chats - An Object Template to gather information from
evidential or interesting exchange of messages identified during a
digital forensic investigation.
o tsk-web-bookmark - An Object Template to add evidential bookmarks
identified during a digital forensic investigation.
o tsk-web-cookie - An TSK-Autopsy Object Template to represent
cookies identified during a forensic investigation.
o tsk-web-downloads - An Object Template to add web-downloads.
o tsk-web-history - An Object Template to share web history
information.
o tsk-web-search-query - An Object Template to share web search
query information.
o ail-leak - An information leak as defined by the AIL Analysis
Information Leak framework.
o ais-info - Automated Indicator Sharing (AIS) Information Source
Markings.
o android-permission - A set of android permissions - one or more
permission(s) which can be linked to other objects (e.g. malware,
app).
Dulaunoy & Iklody Expires October 12, 2018 [Page 10]
Internet-Draft MISP object template format April 2018
o annotation - An annotation object allowing analysts to add
annotations, comments, executive summary to a MISP event, objects
or attributes.
o anonymisation - Anonymisation object describing an anonymisation
technique used to encode MISP attribute values. Reference:
<https://www.caida.org/tools/taxonomy/anonymization.xml>.
o asn - Autonomous system object describing an autonomous system
which can include one or more network operators management an
entity (e.g. ISP) along with their routing policy, routing
prefixes or alike.
o authenticode-signerinfo - Authenticode Signer Info.
o av-signature - Antivirus detection signature.
o bank-account - An object describing bank account information based
on account description from goAML 4.0.
o bgp-hijack - Object encapsulating BGP Hijack description as
specified, for example, by bgpstream.com.
o cap-alert - Common Alerting Protocol Version (CAP) alert object.
o cap-info - Common Alerting Protocol Version (CAP) info object.
o cap-resource - Common Alerting Protocol Version (CAP) resource
object.
o coin-address - An address used in a cryptocurrency.
o cookie - An HTTP cookie (web cookie, browser cookie) is a small
piece of data that a server sends to the user's web browser. The
browser may store it and send it back with the next request to the
same server. Typically, it's used to tell if two requests came
from the same browser -- keeping a user logged-in, for example.
It remembers stateful information for the stateless HTTP protocol.
(as defined by the Mozilla foundation.
o cortex - Cortex object describing a complete cortex analysis.
Observables would be attribute with a relationship from this
object.
o cortex-taxonomy - Cortex object describing an Cortex Taxonomy (or
mini report).
Dulaunoy & Iklody Expires October 12, 2018 [Page 11]
Internet-Draft MISP object template format April 2018
o course-of-action - An object describing a specific measure taken
to prevent or respond to an attack.
o cowrie - Cowrie honeypot object template.
o credential - Credential describes one or more credential(s)
including password(s), api key(s) or decryption key(s).
o credit-card - A payment card like credit card, debit card or any
similar cards which can be used for financial transactions.
o ddos - DDoS object describes a current DDoS activity from a
specific or/and to a specific target. Type of DDoS can be
attached to the object as a taxonomy.
o device - An object to define a device.
o diameter-attack - Attack as seen on diameter authentication
against a GSM, UMTS or LTE network.
o domain-ip - A domain and IP address seen as a tuple in a specific
time frame.
o elf - Object describing a Executable and Linkable Format.
o elf-section - Object describing a section of an Executable and
Linkable Format.
o email - Email object describing an email with meta-information.
o exploit-poc - Exploit-poc object describing a proof of concept or
exploit of a vulnerability. This object has often a relationship
with a vulnerability object.
o facial-composite - An object which describes a facial composite.
o fail2ban - Fail2ban event.
o file - File object describing a file with meta-information.
o forensic-case - An object template to describe a digital forensic
case.
o forensic-evidence - An object template to describe a digital
forensic evidence.
o geolocation - An object to describe a geographic location.
Dulaunoy & Iklody Expires October 12, 2018 [Page 12]
Internet-Draft MISP object template format April 2018
o gtp-attack - GTP attack object as seen on a GSM, UMTS or LTE
network.
o http-request - A single HTTP request header.
o ilr-impact - Institut Luxembourgeois de Regulation - Impact.
o ilr-notification-incident - Institut Luxembourgeois de Regulation
- Notification d'incident.
o internal-reference - Internal reference.
o interpol-notice - An object which describes a Interpol notice.
o ip-api-address - IP Address information. Useful if you are
pulling your ip information from ip-api.com.
o ip-port - An IP address (or domain or hostname) and a port seen as
a tuple (or as a triple) in a specific time frame.
o irc - An IRC object to describe an IRC server and the associated
channels.
o ja3 - JA3 is a new technique for creating SSL client fingerprints
that are easy to produce and can be easily shared for threat
intelligence. Fingerprints are composed of Client Hello packet;
SSL Version, Accepted Ciphers, List of Extensions, Elliptic
Curves, and Elliptic Curve Formats.
<https://github.com/salesforce/ja3>.
o legal-entity - An object to describe a legal entity.
o lnk - LNK object describing a Windows LNK binary file (aka Windows
shortcut).
o macho - Object describing a file in Mach-O format.
o macho-section - Object describing a section of a file in Mach-O
format.
o mactime-timeline-analysis - Mactime template, used in forensic
investigations to describe the timeline of a file activity.
o malware-config - Malware configuration recovered or extracted from
a malicious binary.
o microblog - Microblog post like a Twitter tweet or a post on a
Facebook wall.
Dulaunoy & Iklody Expires October 12, 2018 [Page 13]
Internet-Draft MISP object template format April 2018
o mutex - Object to describe mutual exclusion locks (mutex) as seen
in memory or computer program.
o netflow - Netflow object describes an network object based on the
Netflowv5/v9 minimal definition.
o network-connection - A local or remote network connection.
o network-socket - Network socket object describes a local or remote
network connections based on the socket data structure.
o misc - An object which describes an organization.
o original-imported-file - Object describing the original file used
to import data in MISP.
o passive-dns - Passive DNS records as expressed in draft-dulaunoy-
dnsop-passive-dns-cof-01.
o paste - Paste or similar post from a website allowing to share
privately or publicly posts.
o pcap-metadata - Network packet capture metadata.
o pe - Object describing a Portable Executable.
o pe-section - Object describing a section of a Portable Executable.
o person - An object which describes a person or an identity.
o phishing - Phishing template to describe a phishing website and
its analysis.
o phishing-kit - Object to describe a phishing-kit.
o phone - A phone or mobile phone object which describe a phone.
o process - Object describing a system process.
o python-etvx-event-log - Event log object template to share
information of the activities conducted on a system. .
o r2graphity - Indicators extracted from files using radare2 and
graphml.
o regexp - An object describing a regular expression (regex or
regexp). The object can be linked via a relationship to other
Dulaunoy & Iklody Expires October 12, 2018 [Page 14]
Internet-Draft MISP object template format April 2018
attributes or objects to describe how it can be represented as a
regular expression.
o registry-key - Registry key object describing a Windows registry
key with value and last-modified timestamp.
o regripper-NTUser - Regripper Object template designed to present
user specific configuration details extracted from the NTUSER.dat
hive.
o regripper-sam-hive-single-user - Regripper Object template
designed to present user profile details extracted from the SAM
hive.
o regripper-sam-hive-user-group - Regripper Object template designed
to present group profile details extracted from the SAM hive.
o regripper-software-hive-BHO - Regripper Object template designed
to gather information of the browser helper objects installed on
the system.
o regripper-software-hive-appInit-DLLS - Regripper Object template
designed to gather information of the DLL files installed on the
system.
o regripper-software-hive-application-paths - Regripper Object
template designed to gather information of the application paths.
o regripper-software-hive-applications-installed - Regripper Object
template designed to gather information of the applications
installed on the system.
o regripper-software-hive-command-shell - Regripper Object template
designed to gather information of the shell commands executed on
the system.
o regripper-software-hive-windows-general-info - Regripper Object
template designed to gather general windows information extracted
from the software-hive.
o regripper-software-hive-software-run - Regripper Object template
designed to gather information of the applications set to run on
the system.
o regripper-software-hive-userprofile-winlogon - Regripper Object
template designed to gather user profile information when the user
logs onto the system, gathered from the software hive.
Dulaunoy & Iklody Expires October 12, 2018 [Page 15]
Internet-Draft MISP object template format April 2018
o regripper-system-hive-firewall-configuration - Regripper Object
template designed to present firewall configuration information
extracted from the system-hive.
o regripper-system-hive-general-configuration - Regripper Object
template designed to present general system properties extracted
from the system-hive.
o regripper-system-hive-network-information. - Regripper object
template designed to gather network information from the system-
hive.
o regripper-system-hive-services-drivers - Regripper Object template
designed to gather information regarding the services/drivers from
the system-hive.
o report - Metadata used to generate an executive level report.
o research-scanner - Information related to known scanning activity
(e.g. from research projects).
o rogue-dns - Rogue DNS as defined by CERT.br.
o rtir - RTIR - Request Tracker for Incident Response.
o sandbox-report - Sandbox report.
o sb-signature - Sandbox detection signature.
o script - Object describing a computer program written to be run in
a special run-time environment. The script or shell script can be
used for malicious activities but also as support tools for threat
analysts.
o shell-commands - Object describing a series of shell commands
executed. This object can be linked with malicious files in order
to describe a specific execution of shell commands.
o short-message-service - Short Message Service (SMS) object
template describing one or more SMS message. Restriction of the
initial format 3GPP 23.038 GSM character set doesn't apply.
o shortened-link - Shortened link and its redirect target.
o splunk - Splunk / Splunk ES object.
o ss7-attack - SS7 object of an attack seen on a GSM, UMTS or LTE
network via SS7 logging.
Dulaunoy & Iklody Expires October 12, 2018 [Page 16]
Internet-Draft MISP object template format April 2018
o ssh-authorized-keys - An object to store ssh authorized keys file.
o stix2-pattern - An object describing a STIX pattern. The object
can be linked via a relationship to other attributes or objects to
describe how it can be represented as a STIX pattern.
o suricata - An object describing one or more Suricata rule(s) along
with version and contextual information.
o target-system - Description about an targeted system, this could
potentially be a compromissed internal system.
o threatgrid-report - ThreatGrid report.
o timecode - Timecode object to describe a start of video sequence
(e.g. CCTV evidence) and the end of the video sequence.
o timesketch-timeline - A timesketch timeline object based on
mandatory field in timesketch to describe a log entry.
o timesketch_message - A timesketch message entry.
o timestamp - A generic timestamp object to represent time including
first time and last time seen. Relationship will then define the
kind of time relationship.
o tor-hiddenservice - Tor hidden service (onion service) object.
o tor-node - Tor node (which protects your privacy on the internet
by hiding the connection between users Internet address and the
services used by the users) description which are part of the Tor
network at a time.
o tracking-id - Analytics and tracking ID such as used in Google
Analytics or other analytic platform.
o transaction - An object to describe a financial transaction.
o url - url object describes an url along with its normalized field
(like extracted using faup parsing library) and its metadata.
o vehicle - Vehicle object template to describe a vehicle
information and registration.
o victim - Victim object describes the target of an attack or abuse.
o virustotal-report - VirusTotal report.
Dulaunoy & Iklody Expires October 12, 2018 [Page 17]
Internet-Draft MISP object template format April 2018
o vulnerability - Vulnerability object describing a common
vulnerability enumeration which can describe published,
unpublished, under review or embargo vulnerability for software,
equipments or hardware.
o whois - Whois records information for a domain name or an IP
address.
o x509 - x509 object describing a X.509 certificate.
o yabin - yabin.py generates Yara rules from function prologs, for
matching and hunting binaries. ref: <https://github.com/
AlienVault-OTX/yabin>.
o yara - An object describing a YARA rule along with its version.
4. Acknowledgements
@ -535,29 +981,31 @@ Internet-Draft MISP object template format April 2018
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997, <https://www.rfc-
editor.org/info/rfc2119>.
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally
Unique IDentifier (UUID) URN Namespace", RFC 4122,
DOI 10.17487/RFC4122, July 2005, <https://www.rfc-
editor.org/info/rfc4122>.
DOI 10.17487/RFC4122, July 2005,
<https://www.rfc-editor.org/info/rfc4122>.
[RFC4627] Crockford, D., "The application/json Media Type for
JavaScript Object Notation (JSON)", RFC 4627,
DOI 10.17487/RFC4627, July 2006, <https://www.rfc-
editor.org/info/rfc4627>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
5.2. Informative References
[MISP-O] MISP, , "MISP Objects - shared and common object
templates", <https://github.com/MISP/misp-objects>.
[MISP-O] MISP, "MISP Objects - shared and common object templates",
<https://github.com/MISP/misp-objects>.
[MISP-O-DOC]
"MISP objects directory", 2018,
<https://www.misp-project.org/objects.html>.
Dulaunoy & Iklody Expires October 12, 2018 [Page 10]
Dulaunoy & Iklody Expires October 12, 2018 [Page 18]
Internet-Draft MISP object template format April 2018
@ -613,4 +1061,4 @@ Authors' Addresses
Dulaunoy & Iklody Expires October 12, 2018 [Page 11]
Dulaunoy & Iklody Expires October 12, 2018 [Page 19]

2
misp-query-format/raw.md
View File

@ -65,7 +65,7 @@ document are to be interpreted as described in RFC 2119 [@!RFC2119].
## Overview
The MISP query format is in the JSON [@!RFC4627] format.
The MISP query format is in the JSON [@!RFC8259] format.
## query format criteria

332
misp-query-format/raw.md.txt
View File

@ -68,23 +68,53 @@ Internet-Draft MISP query format October 2018
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Conventions and Terminology . . . . . . . . . . . . . . . 3
2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2. query format criteria . . . . . . . . . . . . . . . . . . 3
2.2.1. returnFormat . . . . . . . . . . . . . . . . . . . . 3
2.2.2. limit . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2.3. page . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2.2. limit . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.3. page . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.4. value . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.5. type . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.6. category . . . . . . . . . . . . . . . . . . . . . . 4
3. Security Considerations . . . . . . . . . . . . . . . . . . . 4
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 5
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 5
5.1. Normative References . . . . . . . . . . . . . . . . . . 5
5.2. Informative References . . . . . . . . . . . . . . . . . 5
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.6. category . . . . . . . . . . . . . . . . . . . . . . 5
2.2.7. org . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.8. tags . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.9. quickfilter . . . . . . . . . . . . . . . . . . . . . 5
2.2.10. from . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.11. to . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.12. last . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.13. eventid . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.14. withAttachments . . . . . . . . . . . . . . . . . . . 6
2.2.15. uuid . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.16. publish_timestamp . . . . . . . . . . . . . . . . . . 6
2.2.17. timestamp . . . . . . . . . . . . . . . . . . . . . . 7
2.2.18. published . . . . . . . . . . . . . . . . . . . . . . 7
2.2.19. enforceWarninglist . . . . . . . . . . . . . . . . . 7
2.2.20. to_ids . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.21. deleted . . . . . . . . . . . . . . . . . . . . . . . 7
2.2.22. includeEventUuid . . . . . . . . . . . . . . . . . . 7
2.2.23. event_timestamp . . . . . . . . . . . . . . . . . . . 7
2.2.24. sgReferenceOnly . . . . . . . . . . . . . . . . . . . 7
2.2.25. eventinfo . . . . . . . . . . . . . . . . . . . . . . 7
2.2.26. searchall . . . . . . . . . . . . . . . . . . . . . . 7
2.2.27. requested_attributes . . . . . . . . . . . . . . . . 7
2.2.28. includeContext . . . . . . . . . . . . . . . . . . . 7
3. Security Considerations . . . . . . . . . . . . . . . . . . . 7
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 7
5. References . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.1. Normative References . . . . . . . . . . . . . . . . . . 8
5.2. Informative References . . . . . . . . . . . . . . . . . 8
Dulaunoy & Iklody Expires April 11, 2019 [Page 2]
Internet-Draft MISP query format October 2018
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 8
1. Introduction
@ -103,17 +133,6 @@ Table of Contents
query format and how the query can be perform against a REST
interface.
Dulaunoy & Iklody Expires April 11, 2019 [Page 2]
Internet-Draft MISP query format October 2018
1.1. Conventions and Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
@ -124,7 +143,7 @@ Internet-Draft MISP query format October 2018
2.1. Overview
The MISP query format is in the JSON [RFC4627] format.
The MISP query format is in the JSON [RFC8259] format.
2.2. query format criteria
@ -134,18 +153,36 @@ Internet-Draft MISP query format October 2018
format. MISP allows multiple format (depending of the
configuration):
+----------+------------------------------------------------+
| value | Description |
+----------+------------------------------------------------+
| json | MISP JSON core format as described in [MISP-C] |
| xml | MISP XML format |
| openioc | OpenIOC format |
| suricata | Suricata NIDS format |
| snort | Snort NIDS format |
| csv | CSV format |
| rpz | Response policy zone format |
| text | Raw value list format |
+----------+------------------------------------------------+
Dulaunoy & Iklody Expires April 11, 2019 [Page 3]
Internet-Draft MISP query format October 2018
+----------+-------------------------------------------------+
| value | Description |
+----------+-------------------------------------------------+
| json | MISP JSON core format as described in [MISP-C] |
| xml | MISP XML format |
| openioc | OpenIOC format |
| suricata | Suricata NIDS format |
| snort | Snort NIDS format |
| csv | CSV format |
| rpz | Response policy zone format |
| text | Raw value list format |
| cache | MISP cache format (hashed values of attributes) |
+----------+-------------------------------------------------+
2.2.2. limit
@ -162,35 +199,38 @@ Internet-Draft MISP query format October 2018
starting with offset (limit * page) + 1 and ending with (limit *
(page+1)).
Dulaunoy & Iklody Expires April 11, 2019 [Page 3]
Internet-Draft MISP query format October 2018
2.2.4. value
value MAY be present. If set, the returned data set will be filtered
on the attribute value field. value MAY be a string or a sub-string,
the latter of which start with, ends with or is encapsulated in
on the attribute value field. value MUST be a string or a sub-string,
the latter of which starts with, ends with or is encapsulated in
wildcard (\%) characters.
2.2.5. type
type MAY be present. If set, the returned data set will be filtered
on the attribute type field. type MAY be a string or a sub-string,
the latter of which start with, ends with or is encapsulated in
on the attribute type field. type MUST be a string or a sub-string,
the latter of which starts with, ends with or is encapsulated in
wildcard (\%) characters. The list of valid attribute types is
described in the MISP core format [MISP-C] in the attribute type
section.
Dulaunoy & Iklody Expires April 11, 2019 [Page 4]
Internet-Draft MISP query format October 2018
2.2.6. category
category MAY be present. If set, the returned data set will be
filtered on the attribute category field. category MAY be a string or
a sub-string, the latter of which start with, ends with or is
filtered on the attribute category field. category MUST be a string
or a sub-string, the latter of which starts with, ends with or is
encapsulated in wildcard (\%) characters. The list of valid
categories is described in the MISP core format [MISP-C] in the
attribute type section.
@ -204,6 +244,124 @@ Internet-Draft MISP query format October 2018
"category": "Financial fraud"
}
2.2.7. org
org MAY be present. If set, the returned data set will be filtered
by the organisation identifier (local ID of the instance). org MUST
be the identifier of the organisation in a string format.
2.2.8. tags
tags MAY be present. If set, the returned data set will be filtered
by tags. tags MUST be a string or a sub-string, the latter of which
starts with, ends with or is encapsulated in wildcard (\%)
characters.
{
"returnFormat": "cache",
"limit": "100",
"tags": ["tlp:red", "%private%"]
}
2.2.9. quickfilter
2.2.10. from
from MAY be present. If set, the returned data set will be filtered
from a starting date. from MUST be a string represented in the format
year-month-date.
Dulaunoy & Iklody Expires April 11, 2019 [Page 5]
Internet-Draft MISP query format October 2018
{
"returnFormat": "json",
"limit": "100",
"tags": ["tlp:amber"],
"from": "2018-09-02",
"to": "2018-10-01"
}
2.2.11. to
to MAY be present. If set, the returned data set will be filtered
until the specified date. from MUST be a string represented in the
format year-month-date.
2.2.12. last
last MAY be present. If set, the returned data set will be filtered
in the number of days, hours or minutes defined (such as 5d, 12h or
30m). last MUST be a string represented in the format expressing
days, hours or minutes.
2.2.13. eventid
eventid MAY be present. If set, the returned data set will be
filtered to a specific event. eventid MUST be a string representing
the event id as an integer.
{
"returnFormat": "json",
"eventid": 1
}
2.2.14. withAttachments
withAttachments MAY be present. If set to True (1), the returned
data set will include the attachment(s) matching the query.
withAttachments MUST be an integer set as 1 (True) to include the
attachment(s). If not, the attachment(s) won't be included in the
results.
2.2.15. uuid
2.2.16. publish_timestamp
Dulaunoy & Iklody Expires April 11, 2019 [Page 6]
Internet-Draft MISP query format October 2018
2.2.17. timestamp
2.2.18. published
2.2.19. enforceWarninglist
2.2.20. to_ids
2.2.21. deleted
2.2.22. includeEventUuid
2.2.23. event_timestamp
2.2.24. sgReferenceOnly
2.2.25. eventinfo
2.2.26. searchall
2.2.27. requested_attributes
2.2.28. includeContext
3. Security Considerations
MISP threat intelligence instances might contain sensitive or
@ -216,16 +374,6 @@ Internet-Draft MISP query format October 2018
standard threat information that might already include malicious
intended inputs.
Dulaunoy & Iklody Expires April 11, 2019 [Page 4]
Internet-Draft MISP query format October 2018
4. Acknowledgements
The authors wish to thank all the MISP community who are supporting
@ -235,6 +383,17 @@ Internet-Draft MISP query format October 2018
5. References
Dulaunoy & Iklody Expires April 11, 2019 [Page 7]
Internet-Draft MISP query format October 2018
5.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
@ -242,10 +401,10 @@ Internet-Draft MISP query format October 2018
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC4627] Crockford, D., "The application/json Media Type for
JavaScript Object Notation (JSON)", RFC 4627,
DOI 10.17487/RFC4627, July 2006,
<https://www.rfc-editor.org/info/rfc4627>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
5.2. Informative References
@ -267,21 +426,6 @@ Authors' Addresses
Email: alexandre.dulaunoy@circl.lu
Dulaunoy & Iklody Expires April 11, 2019 [Page 5]
Internet-Draft MISP query format October 2018
Andras Iklody
Computer Incident Response Center Luxembourg
16, bd d'Avranches
@ -301,36 +445,4 @@ Internet-Draft MISP query format October 2018
Dulaunoy & Iklody Expires April 11, 2019 [Page 6]
Dulaunoy & Iklody Expires April 11, 2019 [Page 8]

78
misp-taxonomy-format/raw.md
View File

@ -1,40 +1,42 @@
% Title = "MISP taxonomy format"
% abbrev = "MISP taxonomy format"
% category = "info"
% docName = "draft-dulaunoy-misp-taxonomy-format"
% ipr= "trust200902"
% area = "Security"
%
% date = 2017-11-29T00:00:00Z
%
% [[author]]
% initials="A."
% surname="Dulaunoy"
% fullname="Alexandre Dulaunoy"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "alexandre.dulaunoy@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = "16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
% [[author]]
% initials="A."
% surname="Iklody"
% fullname="Andras Iklody"
% abbrev="CIRCL"
% organization = "Computer Incident Response Center Luxembourg"
% [author.address]
% email = "andras.iklody@circl.lu"
% phone = "+352 247 88444"
% [author.address.postal]
% street = " 16, bd d'Avranches"
% city = "Luxembourg"
% code = "L-1611"
% country = "Luxembourg"
%%%
Title = "MISP taxonomy format"
abbrev = "MISP taxonomy format"
category = "info"
docName = "draft-dulaunoy-misp-taxonomy-format"
ipr= "trust200902"
area = "Security"
date = 2017-11-29T00:00:00Z
[[author]]
initials="A."
surname="Dulaunoy"
fullname="Alexandre Dulaunoy"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "alexandre.dulaunoy@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = "16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
[[author]]
initials="A."
surname="Iklody"
fullname="Andras Iklody"
abbrev="CIRCL"
organization = "Computer Incident Response Center Luxembourg"
[author.address]
email = "andras.iklody@circl.lu"
phone = "+352 247 88444"
[author.address.postal]
street = " 16, bd d'Avranches"
city = "Luxembourg"
code = "L-1611"
country = "Luxembourg"
%%%
.# Abstract
@ -82,7 +84,7 @@ to describe machine tag (aka triple tag) vocabularies.
## Overview
The MISP taxonomy format uses the JSON [@!RFC4627] format. Each namespace is represented as a JSON object with meta information including the following fields: namespace, description, version, type.
The MISP taxonomy format uses the JSON [@!RFC8259] format. Each namespace is represented as a JSON object with meta information including the following fields: namespace, description, version, type.
namespace defines the overall namespace of the machine tag. The namespace is represented as a string and **MUST** be present. The description is represented as a string and **MUST** be present. A version is represented as a unsigned integer **MUST** be present. A type defines where a specific taxonomy is applicable and a type can be applicable at event, user or org level. The type is represented as an array containing one or more type and **SHOULD** be present. If a type is not mentioned, by default, the taxonomy is applicable at event level only. An exclusive boolean property **MAY** be present and defines at namespace level if the predicates are mutually exclusive.

220
misp-taxonomy-format/raw.md.txt
View File

@ -79,13 +79,13 @@ Table of Contents
4.1. Admiralty Scale Taxonomy . . . . . . . . . . . . . . . . 7
4.2. Open Source Intelligence - Classification . . . . . . . . 9
4.3. Available taxonomies in the public directory . . . . . . 11
5. JSON Schema . . . . . . . . . . . . . . . . . . . . . . . . . 19
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 22
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
7.1. Normative References . . . . . . . . . . . . . . . . . . 22
7.2. Informative References . . . . . . . . . . . . . . . . . 22
5. JSON Schema . . . . . . . . . . . . . . . . . . . . . . . . . 20
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 23
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 23
7.1. Normative References . . . . . . . . . . . . . . . . . . 23
7.2. Informative References . . . . . . . . . . . . . . . . . 23
7.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction
@ -145,7 +145,7 @@ Internet-Draft MISP taxonomy format November 2017
2.1. Overview
The MISP taxonomy format uses the JSON [RFC4627] format. Each
The MISP taxonomy format uses the JSON [RFC8259] format. Each
namespace is represented as a JSON object with meta information
including the following fields: namespace, description, version,
type.
@ -153,7 +153,7 @@ Internet-Draft MISP taxonomy format November 2017
namespace defines the overall namespace of the machine tag. The
namespace is represented as a string and MUST be present. The
description is represented as a string and MUST be present. A
version is represented as a decimal and MUST be present. A type
version is represented as a unsigned integer MUST be present. A type
defines where a specific taxonomy is applicable and a type can be
applicable at event, user or org level. The type is represented as
an array containing one or more type and SHOULD be present. If a
@ -683,11 +683,22 @@ Internet-Draft MISP taxonomy format November 2017
to support analysts to perform their analysis to get crowdsourced
support when using threat intelligence sharing platform like MISP.
common-taxonomy:
The Common Taxonomy for Law Enforcement and The National Network
of CSIRTs bridges the gap between the CSIRTs and international Law
Enforcement communities by adding a legislative framework to
facilitate the harmonisation of incident reporting to competent
authorities, the development of useful statistics and sharing
information within the entire cybercrime ecosystem.
copine-scale:
The COPINE Scale is a rating system created in Ireland and used in
the United Kingdom to categorise the severity of images of child
sex abuse.
cryptocurrency-threat:
Threats targetting cryptocurrency, based on CipherTrace report.
csirt_case_classification:
FIRST CSIRT Case Classification.
@ -701,7 +712,24 @@ Internet-Draft MISP taxonomy format November 2017
of cyber adversaries. <https://www.dni.gov/index.php/cyber-threat-
framework>
data-classification:
Data classification for data potentially at risk of exfiltration
based on table 2.1 of Solving Cyber Risk book.
dcso-sharing:
DCSO Sharing Taxonomy to classify certain types of MISP events
using the DCSO Event Guide
ddos:
Dulaunoy & Iklody Expires June 2, 2018 [Page 13]
Internet-Draft MISP taxonomy format November 2017
Distributed Denial of Service - or short: DDoS - taxonomy supports
the description of Denial of Service attacks and especially the
types they belong too.
@ -723,16 +751,13 @@ Internet-Draft MISP taxonomy format November 2017
ISM (Information Security Marking Metadata) V13 as described by
DNI.gov (Director of National Intelligence - US).
Dulaunoy & Iklody Expires June 2, 2018 [Page 13]
Internet-Draft MISP taxonomy format November 2017
domain-abuse:
Taxonomy to tag domain names used for cybercrime.
drugs:
A taxonomy based on the superclass and class of drugs, based on
<https://www.drugbank.ca/releases/latest>
economical-impact:
Economical impact is a taxonomy to describe the financial impact
as positive or negative gain to the tagged information.
@ -753,6 +778,14 @@ Internet-Draft MISP taxonomy format November 2017
(6.2.(a)) and JP 2-0, Joint Intelligence.
eu-marketop-and-publicadmin:
Dulaunoy & Iklody Expires June 2, 2018 [Page 14]
Internet-Draft MISP taxonomy format November 2017
Market operators and public administrations that must comply to
some notifications requirements under EU NIS directive.
@ -764,7 +797,9 @@ Internet-Draft MISP taxonomy format November 2017
designated by a EU security classification, the unauthorised
disclosure of which could cause varying degrees of prejudice to
the interests of the European Union or of one or more of the
Member States as described in CELEX 32013D0488
Member States as described in COUNCIL DECISION of 23 September
2013 on the security rules for protecting EU classified
information
europol-event:
EUROPOL type of events taxonomy.
@ -778,19 +813,11 @@ Internet-Draft MISP taxonomy format November 2017
uncertainty.
event-classification:
Dulaunoy & Iklody Expires June 2, 2018 [Page 14]
Internet-Draft MISP taxonomy format November 2017
Event Classification.
exercise:
Exercise is a taxonomy to describe if the information is part of
one or more cyber or crisis exercise
one or more cyber or crisis exercise.
false-positive:
This taxonomy aims to ballpark the expected amount of false
@ -799,7 +826,22 @@ Internet-Draft MISP taxonomy format November 2017
file-type:
List of known file types.
flesch-reading-ease:
Flesch Reading Ease is a revised system for determining the
comprehension difficulty of written material. The scoring of the
flesh score can have a maximum of 121.22 and there is no limit on
how low a score can be (negative score are valid).
fpf:
Dulaunoy & Iklody Expires June 2, 2018 [Page 15]
Internet-Draft MISP taxonomy format November 2017
The Future of Privacy Forum (FPF) visual guide to practical de-
identification [1] taxonomy is used to evaluate the degree of
identifiability of personal data and the types of pseudonymous
@ -833,15 +875,6 @@ Internet-Draft MISP taxonomy format November 2017
Christian Seifert, Ian Welch, Peter Komisarczuk, 'Taxonomy of
Honeypots', Technical Report CS-TR-06/12, VICTORIA UNIVERSITY OF
WELLINGTON, School of Mathematical and Computing Sciences, June
Dulaunoy & Iklody Expires June 2, 2018 [Page 15]
Internet-Draft MISP taxonomy format November 2017
2006, <http://www.mcs.vuw.ac.nz/comp/Publications/archive/CS-TR-
06/CS-TR-06-12.pdf>
@ -858,10 +891,20 @@ Internet-Draft MISP taxonomy format November 2017
taxonomy is inspired from NASA Incident Response and Management
Handbook.
Dulaunoy & Iklody Expires June 2, 2018 [Page 16]
Internet-Draft MISP taxonomy format November 2017
infoleak:
A taxonomy describing information leaks and especially information
classified as being potentially leaked.
information-security-data-source:
Taxonomy to classify the information security data sources
information-security-indicators:
Information security indicators have been standardized by the ETSI
Industrial Specification Group (ISG) ISI. These indicators
@ -890,14 +933,6 @@ Internet-Draft MISP taxonomy format November 2017
Malware Capabilities based on MAEC 5.0
maec-malware-obfuscation-methods:
Dulaunoy & Iklody Expires June 2, 2018 [Page 16]
Internet-Draft MISP taxonomy format November 2017
Obfuscation methods used by malware based on MAEC 5.0
malware_classification:
@ -910,6 +945,15 @@ Internet-Draft MISP taxonomy format November 2017
MONARC threat taxonomy.
ms-caro-malware:
Dulaunoy & Iklody Expires June 2, 2018 [Page 17]
Internet-Draft MISP taxonomy format November 2017
Malware Type and Platform classification based on Microsoft's
implementation of the Computer Antivirus Research Organization
(CARO) Naming Scheme and Malware Terminology.
@ -946,14 +990,6 @@ Internet-Draft MISP taxonomy format November 2017
to help provide a common lexicon when discussing incidents. This
priority assignment drives NCCIC urgency, pre-approved incident
response offerings, reporting requirements, and recommendations
Dulaunoy & Iklody Expires June 2, 2018 [Page 17]
Internet-Draft MISP taxonomy format November 2017
for leadership escalation. Generally, incident priority
distribution should follow a similar pattern to the graph below.
Based on <https://www.us-cert.gov/NCCIC-Cyber-Incident-Scoring-
@ -966,6 +1002,14 @@ Internet-Draft MISP taxonomy format November 2017
Status of events used in Request Tracker.
runtime-packer:
Dulaunoy & Iklody Expires June 2, 2018 [Page 18]
Internet-Draft MISP taxonomy format November 2017
Runtime or software packer used to combine compressed data with
the decompression code. The decompression code can add additional
obfuscations mechanisms including polymorphic-packer or other
@ -999,20 +1043,29 @@ Internet-Draft MISP taxonomy format November 2017
tor:
Taxonomy to describe Tor network infrastructure
type:
Taxonomy to describe different types of intelligence gathering
discipline which can be described the origin of intelligence.
use-case-applicability:
The Use Case Applicability categories reflect standard resolution
categories, to clearly display alerting rule configuration
problems.
veris:
Vocabulary for Event Recording and Incident Sharing (VERIS).
vocabulaire-des-probabilites-estimatives:
Vocabulaire des probabilites estimatives
Dulaunoy & Iklody Expires June 2, 2018 [Page 18]
Dulaunoy & Iklody Expires June 2, 2018 [Page 19]
Internet-Draft MISP taxonomy format November 2017
vocabulaire-des-probabilites-estimatives:
Vocabulaire des probabilites estimatives
workflow:
Workflow support language is a common language to support
intelligence analysts to perform their analysis on data and
@ -1058,17 +1111,17 @@ Internet-Draft MISP taxonomy format November 2017
}
}
},
"values": {
"type": "array",
"uniqueItems": true,
Dulaunoy & Iklody Expires June 2, 2018 [Page 19]
Dulaunoy & Iklody Expires June 2, 2018 [Page 20]
Internet-Draft MISP taxonomy format November 2017
"values": {
"type": "array",
"uniqueItems": true,
"items": {
"type": "object",
"additionalProperties": false,
@ -1114,17 +1167,17 @@ Internet-Draft MISP taxonomy format November 2017
"value"
]
}
}
}
},
Dulaunoy & Iklody Expires June 2, 2018 [Page 20]
Dulaunoy & Iklody Expires June 2, 2018 [Page 21]
Internet-Draft MISP taxonomy format November 2017
}
}
},
"type": "object",
"additionalProperties": false,
"properties": {
@ -1170,17 +1223,17 @@ Internet-Draft MISP taxonomy format November 2017
"$ref": "#/defs/values"
}
},
"required": [
"namespace",
"description",
Dulaunoy & Iklody Expires June 2, 2018 [Page 21]
Dulaunoy & Iklody Expires June 2, 2018 [Page 22]
Internet-Draft MISP taxonomy format November 2017
"required": [
"namespace",
"description",
"version",
"predicates"
]
@ -1200,10 +1253,10 @@ Internet-Draft MISP taxonomy format November 2017
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC4627] Crockford, D., "The application/json Media Type for
JavaScript Object Notation (JSON)", RFC 4627,
DOI 10.17487/RFC4627, July 2006,
<https://www.rfc-editor.org/info/rfc4627>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
7.2. Informative References
@ -1223,22 +1276,20 @@ Internet-Draft MISP taxonomy format November 2017
[MISP-T] MISP, "MISP Taxonomies - shared and common vocabularies of
tags", <https://github.com/MISP/misp-taxonomies>.
Dulaunoy & Iklody Expires June 2, 2018 [Page 22]
Internet-Draft MISP taxonomy format November 2017
7.3. URIs
[1] https://fpf.org/2016/04/25/a-visual-guide-to-practical-data-de-
identification/
Dulaunoy & Iklody Expires June 2, 2018 [Page 23]
Internet-Draft MISP taxonomy format November 2017
Authors' Addresses
Alexandre Dulaunoy
@ -1285,4 +1336,9 @@ Authors' Addresses
Dulaunoy & Iklody Expires June 2, 2018 [Page 23]
Dulaunoy & Iklody Expires June 2, 2018 [Page 24]

4
misp-warninglist-format/raw.md
View File

@ -72,11 +72,11 @@ document are to be interpreted as described in RFC 2119 [@!RFC2119].
# Format
Warninglists are represented as a JSON [@!RFC4627] dictionary.
Warninglists are represented as a JSON [@!RFC8259] dictionary.
## Overview
The MISP warninglist format uses the JSON [@!RFC4627] format. Each warninglist is represented as a JSON object with meta information including the following fields: name, description, version, type, matching_attributes, list.
The MISP warninglist format uses the JSON [@!RFC8259] format. Each warninglist is represented as a JSON object with meta information including the following fields: name, description, version, type, matching_attributes, list.
name defines the name of the warninglist. The name is represented as a string and **MUST** be present. The description is represented as a string and **MUST** be present. The version is represented as a decimal and **MUST** be present. matching_attributes is represented as an array containing one or more values and is **RECOMMENDED**. type is represented as a string from an non exaustive list and **MUST** be present.

8
sightingdb-format/Makefile Normal file
View File

@ -0,0 +1,8 @@
MMARK:=mmark -xml2 -page
docs = $(wildcard *.md)
all: $(docs)
$(MMARK) $< > $<.xml
xml2rfc --text $<.xml
xml2rfc --html $<.xml

202
sightingdb-format/raw.md Executable file
View File

@ -0,0 +1,202 @@
%%%
Title = "SightingDB query format"
abbrev = "SightingDB query format"
category = "info"
docName = "draft-tricaud-sightingdb-format"
ipr= "trust200902"
area = "Security"
date = 2019-11-03T00:00:00Z
[[author]]
initials="S."
surname="Tricaud"
fullname="Sebastien Tricaud"
abbrev="Devo Inc."
organization = "Devo Inc."
[author.address]
email = "sebastien.tricaud@devo.com"
phone = "+1 866-221-2254"
[author.address.postal]
street = "150 Cambridgepark Drive"
city = "Cambridge, MA"
code = "02140"
country = "USA"
%%%
.# Abstract
This document describes the format used by SightingDB to give automated context to a given Attribute
by counting occurrences and tracking times of observability.
SightingDB was designed to provide to MISP a Scalable and Fast way to store and retrieve Attributes.
{mainmatter}
# Introduction
Adding context to any Attribute is the key that makes it useful. While there exist numerous ways of doing it,
SightingDB does it by just counting.
Whenever somebody retrieves an Attribute, this counting is provided, allowing anyone to understand whenever something
was observed few or many times.
## Conventions and Terminology
The key words "**MUST**", "**MUST NOT**", "**REQUIRED**", "**SHALL**", "**SHALL NOT**",
"**SHOULD**", "**SHOULD NOT**", "**RECOMMENDED**", "**MAY**", and "**OPTIONAL**" in this
document are to be interpreted as described in RFC 2119 [@!RFC2119].
# Format
## Overview
The SightingDB format is in JSON [@!RFC8259] format and used to query a SightingDB compatible connector. In SightingDB, a Sighting Object is composed of a single JSON object. This object contains the following fields: value, first_seen, last_seen, count, tags, ttl and manifold.
### Attribute Storage
The fields described previously describe an Attribute and all the required characteristics. However they are stored in a Namespace. A Namespace is similar to a path in a file-system where the same file can be stored in multiple places.
### Namespace
A Namespace with multiple levels **MUST** be separated with the slash '/' character. There is no specification on how they are structured, since it depends on the use cases.
A Namespace starting with the underscore '_' character means it is private and internal to SightingDB. There are all reserved for the engine and **MUST** NOT be used.
Reserved namespaces are:
_expired/<namespace>: Which contains all the attributes that expired, preserving the origin namespace
_shadow/<namespace>: When a value is searched and does not exists, it is stored there
_stats: Statistics
_config: Configuration
_all: All the Attributes in one place, used to retrieve the 'manifold' property.
The Attribute Key MUST always be the last part of the Namespace.
#### Sample Namespaces
/Organization1/service/ipv4: Store values for ipv4 keys in /Organization1/service
/everything/domain: Store domains in /everything
### Attribute fields
#### value
The attribute value, used to store and retrieve information about an attribute. Note that value is not returned back in the JSON object, since it is queried, it is known. The Value is described in a section below, as it is very specific and can be either "as is", a hash, encoded in base64 or any other convenient mechanism.
The value implementation **MUST** offer at least: 1) Raw value 2) Base64 URL Encoded 3) SHA256 Hash
#### first_seen
Time in UTC of the first time this value was captured
#### last_seen
Time in UTC of the last time this value was captured
#### count
How many time this value was written
#### tags
Tags follow how they are defined in MISP using the MISP Taxonomy. Each Tag is separated with the ';' character.
#### ttl
Time To Live, represents the expiration in seconds since the time the Attribute was created. Once it has expired, it moves in the private Namespace _expired.
When an Attribute has this field set to 0, it means it is not set to expired. This is the default behavior.
When an Attribute has this field set to a number greater than 0, the expiration status is computed only at retrieval time.
#### manifold
When a given Attribute Value is stored in different namespaces, the manifold field keeps track of them so it returns in how many different places this attributes exists. This is a simple counter.
## SightingDB Format - One Attribute
~~~~
{
"value":"127.0.0.1",
"first_seen":1530394819,
"last_seen":1572933618,
"count":578391,
"tags":"",
"ttl":0,
"manifold": 17
}
~~~~
## Value
The value submitted can be in multiple format according to the use-case. Any implementation **MUST** offer three alternatives:
1) Raw value: where nothing is encoded and the value is stored AS IS, such as show in the example above with the One Attribute in JSON.
2) SHA256: which prevents from seeing content (see Security Considerations), has a fixed size and is convenient for most requirements
3) Base64 URL: Where the specification of Base64 is followed, except the characters conflicting with an URL argument are replaced
The value is configured as part of the Namespace. The private "_config" Namespace prefix stores this value storage mechanism.
### Configuring the value format for a Namespace
If one has the Namespace "/Organization1/BU1/ip" and want to store those IP addresses in SHA256, it will be configured like this:
The Namespace is kept but prefixed by "_config" and has a json object about value format set.
"/_config/Organization1/BU1/ip"
~~~~
{
"value_format":"SHA256"
}
~~~~
Where "value_format" is either: "SHA256", "RAW" or "BASE64URL".
## Bulk
When data must be sent and received in large amounts, it is preferable to embed in JSON all the objects at once. As such, for reading
and writing, the format is the following:
~~~~
{
"items": [
{ "/your/namespace": "127.0.0.1" },
{ "/your/other/namespace": "110812f67fa1e1f0117f6f3d70241c1a42a7b07711a93c2477cc516d9042f9db" }
]
}
~~~~
Which will either store or retrieve the wanted data.
### Response
The response when retrieving sightings also has the list of items, in order, one per line of the results:
~~~~
{
"items": [
{ "first_seen":1530337182, "last_seen":1573110615, "count":93021, "tags":"", "ttl":0, "manifold": 1 },
{ "first_seen":1562930418, "last_seen":1573110404, "count":1020492, "tags":"", "ttl":8912, "manifold": 3 }
]
}
~~~~
# Security Considerations
While this document solely focuses on the format, the reference implementation is SightingDB. The authentication, the data access is not handled by SightingDB.
It is possible a value can leak if the access is too permissive.
Even a Hashed value can be discovered, as re-hashing known values would match.
# Acknowledgements
The author wish to thank all the MISP community who are supporting the creation
of open standards in threat intelligence sharing. As well as amazing feedback gathered
during the MISP Summit 2019 in Luxembourg, in particular with Alexandre Dulaunoy and
Andras Iklody.
{backmatter}

392
sightingdb-format/raw.md.txt Normal file
View File

@ -0,0 +1,392 @@
Network Working Group S. Tricaud
Internet-Draft Devo Inc.
Intended status: Informational November 3, 2019
Expires: May 6, 2020
SightingDB query format
draft-tricaud-sightingdb-format
Abstract
This document describes the format used by SightingDB to give
automated context to a given Attribute by counting occurrences and
tracking times of observability. SightingDB was designed to provide
to MISP a Scalable and Fast way to store and retrieve Attributes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 6, 2020.
Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Tricaud Expires May 6, 2020 [Page 1]
Internet-Draft SightingDB query format November 2019
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Conventions and Terminology . . . . . . . . . . . . . . . 2
2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1.1. Attribute Storage . . . . . . . . . . . . . . . . . . 2
2.1.2. Namespace . . . . . . . . . . . . . . . . . . . . . . 3
2.1.3. Attribute fields . . . . . . . . . . . . . . . . . . 3
2.2. SightingDB Format - One Attribute . . . . . . . . . . . . 4
2.3. Value . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.3.1. Configuring the value format for a Namespace . . . . 5
2.4. Bulk . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4.1. Response . . . . . . . . . . . . . . . . . . . . . . 6
3. Security Considerations . . . . . . . . . . . . . . . . . . . 6
4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 6
5. Normative References . . . . . . . . . . . . . . . . . . . . 6
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 6
1. Introduction
Adding context to any Attribute is the key that makes it useful.
While there exist numerous ways of doing it, SightingDB does it by
just counting. Whenever somebody retrieves an Attribute, this
counting is provided, allowing anyone to understand whenever
something was observed few or many times.
1.1. Conventions and Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
2. Format
2.1. Overview
The SightingDB format is in JSON [RFC8259] format and used to query a
SightingDB compatible connector. In SightingDB, a Sighting Object is
composed of a single JSON object. This object contains the following
fields: value, first_seen, last_seen, count, tags, ttl and manifold.
2.1.1. Attribute Storage
The fields described previously describe an Attribute and all the
required characteristics. However they are stored in a Namespace. A
Namespace is similar to a path in a file-system where the same file
can be stored in multiple places.
Tricaud Expires May 6, 2020 [Page 2]
Internet-Draft SightingDB query format November 2019
2.1.2. Namespace
A Namespace with multiple levels MUST be separated with the slash '/'
character. There is no specification on how they are structured,
since it depends on the use cases.
A Namespace starting with the underscore '_' character means it is
private and internal to SightingDB. There are all reserved for the
engine and MUST NOT be used.
Reserved namespaces are:
_expired/: Which contains all the attributes that expired, preserving
the origin namespace
_shadow/: When a value is searched and does not exists, it is stored
there
_stats: Statistics
_config: Configuration
_all: All the Attributes in one place, used to retrieve the
'manifold' property.
The Attribute Key MUST always be the last part of the Namespace.
2.1.2.1. Sample Namespaces
/Organization1/service/ipv4: Store values for ipv4 keys in
/Organization1/service
/everything/domain: Store domains in /everything
2.1.3. Attribute fields
2.1.3.1. value
The attribute value, used to store and retrieve information about an
attribute. Note that value is not returned back in the JSON object,
since it is queried, it is known. The Value is described in a
section below, as it is very specific and can be either "as is", a
hash, encoded in base64 or any other convenient mechanism.
The value implementation MUST offer at least: 1) Raw value 2) Base64
URL Encoded 3) SHA256 Hash
Tricaud Expires May 6, 2020 [Page 3]
Internet-Draft SightingDB query format November 2019
2.1.3.2. first_seen
Time in UTC of the first time this value was captured
2.1.3.3. last_seen
Time in UTC of the last time this value was captured
2.1.3.4. count
How many time this value was written
2.1.3.5. tags
Tags follow how they are defined in MISP using the MISP Taxonomy.
Each Tag is separated with the ';' character.
2.1.3.6. ttl
Time To Live, represents the expiration in seconds since the time the
Attribute was created. Once it has expired, it moves in the private
Namespace _expired.
When an Attribute has this field set to 0, it means it is not set to
expired. This is the default behavior.
When an Attribute has this field set to a number greater than 0, the
expiration status is computed only at retrieval time.
2.1.3.7. manifold
When a given Attribute Value is stored in different namespaces, the
manifold field keeps track of them so it returns in how many
different places this attributes exists. This is a simple counter.
2.2. SightingDB Format - One Attribute
{
"value":"127.0.0.1",
"first_seen":1530394819,
"last_seen":1572933618,
"count":578391,
"tags":"",
"ttl":0,
"manifold": 17
}
Tricaud Expires May 6, 2020 [Page 4]
Internet-Draft SightingDB query format November 2019
2.3. Value
The value submitted can be in multiple format according to the use-
case. Any implementation MUST offer three alternatives:
1. Raw value: where nothing is encoded and the value is stored AS
IS, such as show in the example above with the One Attribute in
JSON.
2. SHA256: which prevents from seeing content (see Security
Considerations), has a fixed size and is convenient for most
requirements
3. Base64 URL: Where the specification of Base64 is followed, except
the characters conflicting with an URL argument are replaced
The value is configured as part of the Namespace. The private
"_config" Namespace prefix stores this value storage mechanism.
2.3.1. Configuring the value format for a Namespace
If one has the Namespace "/Organization1/BU1/ip" and want to store
those IP addresses in SHA256, it will be configured like this: The
Namespace is kept but prefixed by "_config" and has a json object
about value format set. "/_config/Organization1/BU1/ip"
{
"value_format":"SHA256"
}
Where "value_format" is either: "SHA256", "RAW" or "BASE64URL".
2.4. Bulk
When data must be sent and received in large amounts, it is
preferable to embed in JSON all the objects at once. As such, for
reading and writing, the format is the following:
{
"items": [
{ "/your/namespace": "127.0.0.1" },
{ "/your/other/namespace": "110812f67fa1e1f0117f6f3d70241c1a42a7b07711a93c2477cc516d9042f9db" }
]
}
Which will either store or retrieve the wanted data.
Tricaud Expires May 6, 2020 [Page 5]
Internet-Draft SightingDB query format November 2019
2.4.1. Response
The response when retrieving sightings also has the list of items, in
order, one per line of the results:
{
"items": [
{ "first_seen":1530337182, "last_seen":1573110615, "count":93021, "tags":"", "ttl":0, "manifold": 1 },
{ "first_seen":1562930418, "last_seen":1573110404, "count":1020492, "tags":"", "ttl":8912, "manifold": 3 }
]
}
3. Security Considerations
While this document solely focuses on the format, the reference
implementation is SightingDB. The authentication, the data access is
not handled by SightingDB. It is possible a value can leak if the
access is too permissive.
Even a Hashed value can be discovered, as re-hashing known values
would match.
4. Acknowledgements
The author wish to thank all the MISP community who are supporting
the creation of open standards in threat intelligence sharing. As
well as amazing feedback gathered during the MISP Summit 2019 in
Luxembourg, in particular with Alexandre Dulaunoy and Andras Iklody.
5. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
Author's Address
Tricaud Expires May 6, 2020 [Page 6]
Internet-Draft SightingDB query format November 2019
Sebastien Tricaud
Devo Inc.
150 Cambridgepark Drive
Cambridge, MA 02140
USA
Phone: +1 866-221-2254
Email: sebastien.tricaud@devo.com
Tricaud Expires May 6, 2020 [Page 7]