url-abuse

Go to file

Raphaël Vinot 13cab6b621 Update deps, use psslv2.		2015-09-09 14:38:56 +02:00
pyurlabuse	Use POST to send email and to get content the cache	2015-03-12 16:37:59 +01:00
web	Update deps, use psslv2.	2015-09-09 14:38:56 +02:00
.gitignore	Use POST to send email and to get content the cache	2015-03-12 16:37:59 +01:00
LICENSE	Initial commit	2015-03-02 14:55:34 +01:00
README.md	EU-PI Partnership added	2015-03-05 10:38:33 +01:00
config.ini.sample	Initial commit	2015-03-02 14:55:34 +01:00
install.sh	Initial commit	2015-03-02 14:55:34 +01:00
requirements.txt	Update deps, use psslv2.	2015-09-09 14:38:56 +02:00
run_redis.sh	Initial commit	2015-03-02 14:55:34 +01:00
runapp.py	Initial commit	2015-03-02 14:55:34 +01:00
update_deps.sh	Update deps, use psslv2.	2015-09-09 14:38:56 +02:00
url_abuse_async.py	Update deps, use psslv2.	2015-09-09 14:38:56 +02:00
worker.py	Make code compatible with python 3	2015-05-03 23:28:08 +02:00

README.md

#URL Abuse

URL Abuse is a versatile free software for URL review, analysis and black-list reporting. URL Abuse is composed of a web interface where requests are submitted asynchronously and a back-end system to process the URLs into features modules.

Features

HTTP redirects analysis and follows
Google Safe-Browsing lookup
Phishtank lookup
VirusTotal lookup and submission
URL query lookup
CIRCL Passive DNS lookup
CIRCL Passive SSL lookup
Universal WHOIS lookup for abuse contact
Sphinx search interface to RT/RTIR ticketing systems. The functionality is disabled by default but can be used to display information about existing report of malicious URLs.

Please note that some of the API services will require an API key. The API keys should be located in the root of the URL Abuse directory.

Demo

CIRCL URL Abuse is online.

Install

Install the requirements

pip install -r requirements.txt

Copy and review the configuration:

cp config.ini.sample config.ini

Install Redis and update the configuration.

Start the Redis back-end

./run_redis.sh

Start the workers (at least 10)

seq 10 | parallel -u -j 10 ./worker.py

Start the web interface

python runapp.py

Contributing

We welcome pull requests for new extensions, bug fixes.

Add a new module

Look at the existings functions/modules. The changes will have to be made in the following files:

Add the function you want to execure in url_abuse_async.py
Add a route in web/__init__.py. This route will do an async call to the function defined in url_abuse_async.py. The parameter of the function is sent in an POST object
Add a statement in web/templates/url-report.html. The data option is the parameter to pass to the javascript directive
Add a directive in web/static/main.js, it will take care of passing the parameter to the backend and regularly pull for the response of the async call

Partner

URL Abuse is being developed as part of the “European Union anti-Phishing Initiative” (EU PI) project. This project is coordinated by Cert-Lexsi and co-funded by the Prevention of and Fight against Crime programme of the European Union.