Merge 98cb43668d into 0db0f8c83c

README.md

@@ -35,6 +35,7 @@ For more information: [Extending MISP with Python modules](https://www.misp-proj
 * [Cytomic Orion](misp_modules/modules/expansion/cytomic_orion.py) - An expansion module to enrich attributes in MISP and share indicators of compromise with Cytomic Orion.
 * [DBL Spamhaus](misp_modules/modules/expansion/dbl_spamhaus.py) - a hover module to check Spamhaus DBL for a domain name.
 * [DNS](misp_modules/modules/expansion/dns.py) - a simple module to resolve MISP attributes like hostname and domain to expand IP addresses attributes.
+* [DNS WhoisFreaks](https://whoisfreaks.com/products/dns-records-api.html) - a simple Whoisfreaks Module that is useful for DNS Information. Our DNS checker API is a great way to gain a more in-depth understanding of an organization's online presence.
 * [docx-enrich](misp_modules/modules/expansion/docx_enrich.py) - an enrichment module to get text out of Word document into MISP (using free-text parser).
 * [DomainTools](misp_modules/modules/expansion/domaintools.py) - a hover and expansion module to get information from [DomainTools](http://www.domaintools.com/) whois.
 * [EQL](misp_modules/modules/expansion/eql.py) - an expansion module to generate event query language (EQL) from an attribute. [Event Query Language](https://eql.readthedocs.io/en/latest/)
@@ -49,6 +50,7 @@ For more information: [Extending MISP with Python modules](https://www.misp-proj
 * [html_to_markdown](misp_modules/modules/expansion/html_to_markdown.py) - Simple HTML to markdown converter
 * [HYAS Insight](misp_modules/modules/expansion/hyasinsight.py) - a hover and expansion module to get information from [HYAS Insight](https://www.hyas.com/hyas-insight).
 * [intel471](misp_modules/modules/expansion/intel471.py) - an expansion module to get info from [Intel471](https://intel471.com).
+* [Ipgeolocation](https://ipgeolocation.io/) - an expansion and hover module for IP Intelligence Stack with [IP to Geolocation](https://ipgeolocation.io/ip-location-api.html), [Timezone](https://ipgeolocation.io/astronomy-api.html) and [Astronomy API](https://ipgeolocation.io/timezone-api.html).
 * [IPASN](misp_modules/modules/expansion/ipasn.py) - a hover and expansion to get the BGP ASN of an IP address.
 * [ipinfo.io](misp_modules/modules/expansion/ipinfo.py) - an expansion module to get additional information on an IP address using the ipinfo.io API
 * [iprep](misp_modules/modules/expansion/iprep.py) - an expansion module to get IP reputation from packetmail.net.
@@ -95,7 +97,7 @@ For more information: [Extending MISP with Python modules](https://www.misp-proj
 * [VulnDB](misp_modules/modules/expansion/vulndb.py) - a module to query [VulnDB](https://www.riskbasedsecurity.com/).
 * [Vulners](misp_modules/modules/expansion/vulners.py) - an expansion module to expand information about CVEs using Vulners API.
 * [whois](misp_modules/modules/expansion/whois.py) - a module to query a local instance of [uwhois](https://github.com/rafiot/uwhoisd).
-* [whoisfreaks](misp_modules/modules/expansion/whoisfreaks.py) - An expansion module for [whoisfreaks](https://whoisfreaks.com/) that will provide an enriched analysis of the provided domain, including WHOIS and DNS information.
+* [whoisfreaks](misp_modules/modules/expansion/whoisfreaks.py) - an expansion module for [whoisfreaks](https://whoisfreaks.com/) that will provide an enriched analysis of the provided domain, including WHOIS and DNS information. Our [Whois service](https://whoisfreaks.com/products/whois-api.html), [DNS Lookup API](https://whoisfreaks.com/products/dns-records-api.html), and [SSL analysis](https://whoisfreaks.com/products/ssl-certificate-api.html), equips organizations with comprehensive threat intelligence and attack surface analysis capabilities for enhanced security. Explore our website's product section at https://whoisfreaks.com/ for a wide range of additional services catering to threat intelligence and attack surface analysis needs.
 * [wikidata](misp_modules/modules/expansion/wiki.py) - a [wikidata](https://www.wikidata.org) expansion module.
 * [xforce](misp_modules/modules/expansion/xforceexchange.py) - an IBM X-Force Exchange expansion module.
 * [xlsx-enrich](misp_modules/modules/expansion/xlsx_enrich.py) - an enrichment module to get text out of an Excel document into MISP (using free-text parser).

docs/index.md

@@ -41,6 +41,7 @@ For more information: [Extending MISP with Python modules](https://www.circl.lu/
 * [Greynoise](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/greynoise.py) - a hover to get information from greynoise.
 * [hashdd](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/hashdd.py) - a hover module to check file hashes against [hashdd.com](http://www.hashdd.com) including NSLR dataset.
 * [hibp](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/hibp.py) - a hover module to lookup against Have I Been Pwned?
+* [Ipgeolocation](https://ipgeolocation.io/) - an expansion and hover module for IP Intelligence Stack with [IP to Geolocation](https://ipgeolocation.io/ip-location-api.html), [Timezone](https://ipgeolocation.io/astronomy-api.html) and [Astronomy API](https://ipgeolocation.io/timezone-api.html).
 * [intel471](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/intel471.py) - an expansion module to get info from [Intel471](https://intel471.com).
 * [IPASN](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipasn.py) - a hover and expansion to get the BGP ASN of an IP address.
 * [iprep](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/iprep.py) - an expansion module to get IP reputation from packetmail.net.
@@ -75,6 +76,7 @@ For more information: [Extending MISP with Python modules](https://www.circl.lu/
 * [VMray](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vmray_submit.py) - a module to submit a sample to VMray.
 * [VulnDB](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vulndb.py) - a module to query [VulnDB](https://www.riskbasedsecurity.com/).
 * [Vulners](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vulners.py) - an expansion module to expand information about CVEs using Vulners API.
+* [whoisfreaks](misp_modules/modules/expansion/whoisfreaks.py) - an expansion module for [whoisfreaks](https://whoisfreaks.com/) that will provide an enriched analysis of the provided domain, including WHOIS and DNS information. Our [Whois service](https://whoisfreaks.com/products/whois-api.html), [DNS Lookup API](https://whoisfreaks.com/products/dns-records-api.html), and [SSL analysis](https://whoisfreaks.com/products/ssl-certificate-api.html), equips organizations with comprehensive threat intelligence and attack surface analysis capabilities for enhanced security. Explore our website's product section at https://whoisfreaks.com/ for a wide range of additional services catering to threat intelligence and attack surface analysis needs.
 * [whois](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py) - a module to query a local instance of [uwhois](https://github.com/rafiot/uwhoisd).
 * [wikidata](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/wiki.py) - a [wikidata](https://www.wikidata.org) expansion module.
 * [xforce](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/xforceexchange.py) - an IBM X-Force Exchange expansion module.

documentation/README.md

@@ -796,6 +796,27 @@ Module to query an IP ASN history service (https://github.com/D4-project/IPASN-H
 
 -----
 
+#### [ipgeolocation](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipgeolocation.py)
+
+<img src=logos/ipgeolocation.png height=60>
+
+An expansion module to query IpGeolocation.io to gather more information on a given IP address. We provide data such as country name, country code, city, state, local currency, time zone, ISP, ASN, Company Details, device data from User Agent String, VPN, Proxy, Tor and threat intelligence data served globally with latency based routing.
+- **features**:
+>The module takes an IP address attribute as input and queries the IpGeolocation API.  
+>The geolocation information on the IP address is always returned.
+>
+>Depending on the subscription plan, the API returns different pieces of information for details check our [page](https://ipgeolocation.io/ip-location-api.html).
+- **input**:
+>IP address
+- **output**:
+>Additional information on the IP address, like its geolocation, the autonomous system it is included in.
+- **references**:
+>https://ipgeolocation.io/
+- **requirements**:
+>An apiKey of ipGeolocation
+
+-----
+
 #### [ipinfo](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipinfo.py)
 
 <img src=logos/ipinfo.png height=60>

documentation/mkdocs/expansion.md

@@ -793,6 +793,27 @@ Module to query an IP ASN history service (https://github.com/D4-project/IPASN-H
 
 -----
 
+#### [ipgeolocation](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipgeolocation.py)
+
+<img src=../logos/ipgeolocation.png height=60>
+
+An expansion module to query IpGeolocation.io to gather more information on a given IP address. We provide data such as country name, country code, city, state, local currency, time zone, ISP, ASN, Company Details, device data from User Agent String, VPN, Proxy, Tor and threat intelligence data served globally with latency based routing.
+- **features**:
+>The module takes an IP address attribute as input and queries the IpGeolocation API.  
+>The geolocation information on the IP address is always returned.
+>
+>Depending on the subscription plan, the API returns different pieces of information for details check our [page](https://ipgeolocation.io/ip-location-api.html).
+- **input**:
+>IP address
+- **output**:
+>Additional information on the IP address, like its geolocation, the autonomous system it is included in.
+- **references**:
+>https://ipgeolocation.io/
+- **requirements**:
+>An apiKey of ipGeolocation
+
+-----
+
 #### [ipinfo](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipinfo.py)
 
 <img src=../logos/ipinfo.png height=60>

documentation/mkdocs/index.md

@@ -42,6 +42,7 @@ For more information: [Extending MISP with Python modules](https://www.circl.lu/
 * [hashdd](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/hashdd.py) - a hover module to check file hashes against [hashdd.com](http://www.hashdd.com) including NSLR dataset.
 * [hibp](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/hibp.py) - a hover module to lookup against Have I Been Pwned?
 * [intel471](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/intel471.py) - an expansion module to get info from [Intel471](https://intel471.com).
+* [Ipgeolocation](https://ipgeolocation.io/) - an expansion and hover module for IP Intelligence Stack with [IP to Geolocation](https://ipgeolocation.io/ip-location-api.html), [Timezone](https://ipgeolocation.io/astronomy-api.html) and [Astronomy API](https://ipgeolocation.io/timezone-api.html).
 * [IPASN](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/ipasn.py) - a hover and expansion to get the BGP ASN of an IP address.
 * [iprep](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/iprep.py) - an expansion module to get IP reputation from packetmail.net.
 * [Joe Sandbox submit](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/joesandbox_submit.py) - Submit files and URLs to Joe Sandbox.
@@ -76,6 +77,7 @@ For more information: [Extending MISP with Python modules](https://www.circl.lu/
 * [VulnDB](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vulndb.py) - a module to query [VulnDB](https://www.riskbasedsecurity.com/).
 * [Vulners](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/vulners.py) - an expansion module to expand information about CVEs using Vulners API.
 * [whois](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/whois.py) - a module to query a local instance of [uwhois](https://github.com/rafiot/uwhoisd).
+* [whoisfreaks](misp_modules/modules/expansion/whoisfreaks.py) - an expansion module for [whoisfreaks](https://whoisfreaks.com/) that will provide an enriched analysis of the provided domain, including WHOIS and DNS information. Our [Whois service](https://whoisfreaks.com/products/whois-api.html), [DNS Lookup API](https://whoisfreaks.com/products/dns-records-api.html), and [SSL analysis](https://whoisfreaks.com/products/ssl-certificate-api.html), equips organizations with comprehensive threat intelligence and attack surface analysis capabilities for enhanced security. Explore our website's product section at https://whoisfreaks.com/ for a wide range of additional services catering to threat intelligence and attack surface analysis needs.
 * [wikidata](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/wiki.py) - a [wikidata](https://www.wikidata.org) expansion module.
 * [xforce](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/xforceexchange.py) - an IBM X-Force Exchange expansion module.
 * [xlsx-enrich](https://github.com/MISP/misp-modules/tree/main/misp_modules/modules/expansion/xlsx-enrich.py) - an enrichment module to get text out of an Excel document into MISP (using free-text parser).

documentation/website/expansion/ipgeolocation.json

@@ -0,0 +1,13 @@
+{
+    "description": "An expansion module to query IpGeolocation.io to gather more information on a given IP address. We provide data such as country name, country code, city, state, local currency, time zone, ISP, ASN, Company Details, device data from User Agent String, VPN, Proxy, Tor and threat intelligence data served globally with latency based routing.",
+    "logo": "ipgeolocation.png",
+    "requirements": [
+        "An apiKey of ipGeolocation"
+    ],
+    "input": "IP address",
+    "output": "Additional information on the IP address, like its geolocation, the autonomous system it is included in.",
+    "references": [
+        "https://ipgeolocation.io/"
+    ],
+    "features": "The module takes an IP address attribute as input and queries the IpGeolocation API.  \nThe geolocation information on the IP address is always returned.\n\nDepending on the subscription plan, the API returns different pieces of information for details check our [page](https://ipgeolocation.io/ip-location-api.html)."
+}

misp_modules/modules/expansion/__init__.py

@@ -20,7 +20,7 @@ __all__ = ['cuckoo_submit', 'vmray_submit', 'bgpranking', 'circl_passivedns', 'c
            'trustar_enrich', 'recordedfuture', 'html_to_markdown', 'socialscan', 'passive-ssh',
            'qintel_qsentry', 'mwdb', 'hashlookup', 'mmdb_lookup', 'ipqs_fraud_and_risk_scoring',
            'clamav', 'jinja_template_rendering','hyasinsight', 'variotdbs', 'crowdsec',
-           'extract_url_components', 'ipinfo', 'whoisfreaks']
+           'extract_url_components', 'ipinfo', 'whoisfreaks', 'ipgeolocation']
 
 
 minimum_required_fields = ('type', 'uuid', 'value')

misp_modules/modules/expansion/ipgeolocation.py

@@ -0,0 +1,87 @@
+import json
+import traceback
+
+import requests
+from pymisp import MISPAttribute, MISPEvent, MISPObject
+
+mispattributes = {
+    'input': ['ip-dst', 'ip-src'],
+    'format': 'misp_standard'
+}
+moduleinfo = {
+    'version': '1', 'author': 'IpGeolocation',
+    'description': 'Querry Using IpGeolocation.io',
+    'module-type': ['expansion', 'hover']
+}
+moduleconfig = ['apiKey']
+
+_IPGEO_MAPPING ={
+    'isp':'ISP',
+    'asn':'asn',
+    'city':'city',
+    'country_name':'country',
+    'country_code2':'country-code',
+    'latitude':'latitude',
+    'longitude':'longitude',
+    'organization':'organization',
+    'continent_name':'region',
+    'continent_code':'region-code',
+    'state_prov':'state',
+    'zipcode':'zipcode',
+    'ip':'ip-src'
+}
+
+
+def handler(q=False):
+    # Input checks
+    if q is False:
+        return False
+    request = json.loads(q)
+    if not request.get('config'):
+        return {'error' : 'IpGeolocation Configuration is missing'}
+    if not request['config'].get('apiKey'):
+        return {'error' : 'IpGeolocation apiKey is missing'}
+    
+    if request['attribute']['type'] not in mispattributes['input']:
+        return {'error': 'Unsupported attribute type.'}
+            
+    attribute = request['attribute']
+    ip = request['attribute']['value']
+    apiKey = request['config']['apiKey']
+    query = requests.get(f"https://api.ipgeolocation.io/ipgeo?apiKey={apiKey}&ip={ip}")
+    if query.status_code != 200:
+        return {'error': f'Error while querying ipGeolocation.io - {query.status_code}: {query.reason}'}
+    query = query.json()
+    # Check if the IP address is not reserved for special use
+    if query.get('message'):
+        if 'bogon' in query['message']:
+            return {'error': 'The IP address(bogon IP) is reserved for special use'}
+        else:
+            return {'error': 'Error Occurred during IP data Extraction from Message'}
+    misp_event = MISPEvent()
+    input_attribute = MISPAttribute()
+    input_attribute.from_dict(**attribute)
+    misp_event.add_attribute(**input_attribute)
+
+    ipObject = MISPObject('ip-api-address')
+    # Correct
+    for field, relation in _IPGEO_MAPPING.items():
+        ipObject.add_attribute(relation, query[field])
+    ipObject.add_reference(input_attribute.uuid, 'locates')
+    misp_event.add_object(ipObject)
+    # Return the results in MISP format
+    event = json.loads(misp_event.to_json())
+    return {
+        'results': {key: event[key] for key in ('Attribute', 'Object')}
+    }
+
+def introspection():
+    return mispattributes
+
+
+def version():
+    moduleinfo['config'] = moduleconfig
+    return moduleinfo
+
+
+