CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
AIL-framework/bin/Shutdown.py

68 lines
2.1 KiB
Python
Raw Normal View History

Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
#!/usr/bin/env python2
# -*-coding:UTF-8 -*
"""
The ZMQ_Feed_Q Module
=====================
This module is consuming the Redis-list created by the ZMQ_Feed_Q Module,
And save the paste on disk to allow others modules to work on them.
..todo:: Be able to choose to delete or not the saved paste after processing.
..todo:: Store the empty paste (unprocessed) somewhere in Redis.
..note:: Module ZMQ_Something_Q and ZMQ_Something are closely bound, always put
the same Subscriber name in both of them.
Requirements
------------
*Need running Redis instances.
*Need the ZMQ_Feed_Q Module running to be able to work properly.
"""
Cleanup (remove unused imports, more pep8 compatible)
2014-08-14 14:11:07 +02:00
import redis
import ConfigParser
Use env variables everywhere
2014-08-22 14:52:02 +02:00
import os
Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
Small fixes to make the refactoring production ready * the port for the logging is 6380 * use os.environ properly * fix typos
2014-08-22 17:35:40 +02:00
configfile = os.path.join(os.environ['AIL_BIN'], './packages/config.cfg')
Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
Cleanup (remove unused imports, more pep8 compatible)
2014-08-14 14:11:07 +02:00
Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
def main():
"""Main Function"""
# CONFIG #
cfg = ConfigParser.ConfigParser()
cfg.read(configfile)
Cleanup (remove unused imports, more pep8 compatible)
2014-08-14 14:11:07 +02:00
# REDIS
r_serv = redis.StrictRedis(host=cfg.get("Redis_Queues", "host"),
port=cfg.getint("Redis_Queues", "port"),
db=cfg.getint("Redis_Queues", "db"))
Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
Do not create a ZMQ sub if it is not required.
2014-08-19 19:53:33 +02:00
# FIXME: automatic based on the queue name.
Cleanup (remove unused imports, more pep8 compatible)
2014-08-14 14:11:07 +02:00
# ### SCRIPTS ####
Initial import of AIL framework - Analysis Information Leak framework AIL is a modular framework to analyse potential information leak from unstructured data source like pastes from Past ebin or similar services. AIL framework is flexible and can be extended to support other functionalities to mine sen sitive information
2014-08-06 11:43:40 +02:00
r_serv.sadd("SHUTDOWN_FLAGS", "Feed")
r_serv.sadd("SHUTDOWN_FLAGS", "Categ")
r_serv.sadd("SHUTDOWN_FLAGS", "Lines")
r_serv.sadd("SHUTDOWN_FLAGS", "Tokenize")
r_serv.sadd("SHUTDOWN_FLAGS", "Attributes")
r_serv.sadd("SHUTDOWN_FLAGS", "Creditcards")
r_serv.sadd("SHUTDOWN_FLAGS", "Duplicate")
r_serv.sadd("SHUTDOWN_FLAGS", "Mails")
r_serv.sadd("SHUTDOWN_FLAGS", "Onion")
r_serv.sadd("SHUTDOWN_FLAGS", "Urls")
r_serv.sadd("SHUTDOWN_FLAGS", "Feed_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Categ_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Lines_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Tokenize_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Attributes_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Creditcards_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Duplicate_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Mails_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Onion_Q")
r_serv.sadd("SHUTDOWN_FLAGS", "Urls_Q")
if __name__ == "__main__":
main()