CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
187 Commits
7 Branches
44 Tags
110 MiB
Commit Graph

88 Commits (fe08d2f9cdd3e06c9630ed9e3dba5d9f48ca29d5)

Author SHA1 Message Date
MaximeStor b7d2b64a86 Merge branch 'master' of https://github.com/CIRCL/AIL-framework into module 2016-02-06 11:28:48 +01:00
Alexandre Dulaunoy 192ee7f0ec Merge pull request #49 from Alainfou/master 
Phone module added (regex needs optimization)
 2016-02-06 11:20:46 +01:00
Alain ea52fd1068 Phone regex updated 
Might still need to be fixed / optimized, in case of maths or random numbers starting with a 0. Do not capture dates, hours, coordinates anymore. Captured formats are: e.g. +331234567890 ; 09 12 34 56 78 ; +4177/123.45.69 ; +352(0)6-23-23-23...
 2016-02-05 20:58:02 +01:00
Alain fabbfd8ae9 Update module.cfg (adding Keys and Phone section) 2016-02-05 14:00:41 -05:00
Alain 43b3556588 Starting Phone number recognition 2016-02-05 13:58:21 -05:00
MaximeStor 07513a5b37 Add modules Credential and Release 2016-02-05 16:15:09 +01:00
Raphaël Vinot aef8ab0411 Listen locally for 0MQ 2016-02-04 15:32:50 +01:00
Raphaël Vinot 5ca13c42eb Launch redis and leveldb from local directory 2016-02-04 15:24:39 +01:00
Raphaël Vinot 12aca6b760 Add script to import from local directory, use local python from env 2016-02-04 15:22:51 +01:00
Raphaël Vinot 315cb48117 Add template file for writing a new module 2016-02-03 10:33:42 +01:00
Raphaël Vinot 0d6adc2063 Add initial Travis file 2016-01-19 11:43:34 +01:00
Alexandre Dulaunoy cdd0725e88 -v option added to list the path 2015-12-22 21:37:05 +00:00
Alexandre Dulaunoy e3971ac93a Onion fetching loop deactivated by default 2014-12-22 16:06:38 +00:00
Raphaël Vinot 08ceefc375 Re-add config option 2014-12-22 16:50:25 +01:00
Raphaël Vinot 50369c6706 Revert changes on the config file due to merging messup 2014-12-22 16:29:05 +01:00
Raphaël Vinot f717f9fe89 Merge branch 'master' of github.com:CIRCL/AIL-framework 2014-12-22 15:32:48 +01:00
Raphaël Vinot 9ee61db2cf Add hotfixes 2014-12-22 15:27:02 +01:00
Raphaël Vinot 8803c8447a Publish the fetched onions on a ZMQ feed. 2014-09-30 16:55:16 +02:00
Alexandre Dulaunoy 25757b0fff A simple feeder script feeding data from pystemon to AIL. 
The configuration matches the default Redis parameters used
in the pystemon configuration.

https://github.com/cvandeplas/pystemon/blob/master/pystemon.yaml#L16
 2014-09-19 14:03:05 +02:00
Raphaël Vinot 65b9a01644 Add config file for DomainClassifier, proper reporting 2014-09-17 17:22:56 +02:00
Alexandre Dulaunoy 27b134ec03 Add proper publisher for classified domains/hostnames 2014-09-10 09:27:47 +02:00
Raphaël Vinot f017680365 fix onions, cc and domain classifier modules 2014-09-08 16:51:43 +02:00
Alexandre Dulaunoy de6e21d5a7 DomainClassifier sample configuration added 2014-09-08 16:44:05 +02:00
Alexandre Dulaunoy 246621f663 First version of the DomainClassifier 2014-09-08 16:43:21 +02:00
Alexandre Dulaunoy 1397db9691 Global queue for DomainClassifier 2014-09-08 11:07:45 +02:00
Raphaël Vinot e983c839ad Categ now listen to the Global queue 2014-09-05 17:05:45 +02:00
Raphaël Vinot 46f27ada4e More cleanup 2014-09-05 10:42:01 +02:00
Raphaël Vinot fca00beed9 Add Domain Classifier module. 
Cleanup in the config files.
 2014-09-05 10:41:00 +02:00
Raphaël Vinot b7c9e489c9 Fix the exceptions 2014-09-04 11:46:07 +02:00
Raphaël Vinot 9e8611a42d stop killing the disk when creating the word curve 2014-09-02 18:20:28 +02:00
Raphaël Vinot 7542eaf739 Update starting script. 2014-09-02 15:21:36 +02:00
Raphaël Vinot 0c6b09f379 Fix the onion module, log the valid onions. 2014-09-01 16:18:06 +02:00
Raphaël Vinot f4b89669fc The onion module now fetches the URLs it finds. 2014-08-31 22:42:12 +02:00
Raphaël Vinot abfe13436b Big refactoring, make the queues more flexible 2014-08-29 19:37:56 +02:00
Raphaël Vinot 623e876f3b Cleanup. 
* Remove useless subscriber
* Fix typo in the config file
* Update Helper accordingly
 2014-08-26 17:36:57 +02:00
Alexandre Dulaunoy 3b499a2ec8 ZMQ Publisher removed 
ZMQ Publisher removed to allow concurrent use of the scripts.
In short term, we would replace all publishing part within AIL
into pub-sub Redis to avoid ZMQ limitation.
 2014-08-26 14:38:49 +02:00
Alexandre Dulaunoy f070ac2005 cymruwhois uses dotted decimal format 2014-08-25 10:05:36 +02:00
Raphaël Vinot 3886d1b834 Small fixes to make the refactoring production ready 
* the port for the logging is 6380
* use os.environ properly
* fix typos
 2014-08-22 17:35:40 +02:00
Raphaël Vinot 78125db4ea Use env variables everywhere 2014-08-22 14:52:02 +02:00
Raphaël Vinot 277d138a5d cleanup, add FIXME 2014-08-21 14:39:17 +02:00
Raphaël Vinot 63b29176c1 move Redis_Data_Merging to Paste 2014-08-21 12:22:07 +02:00
Raphaël Vinot 50cfac857e Update config 
Make all paths in the config file relative to the home directory.
 2014-08-20 16:00:56 +02:00
Raphaël Vinot a68f5b6a0e fix subscriber names, update default config 2014-08-20 15:54:21 +02:00
Raphaël Vinot 2485ba5df2 Merge remote-tracking branch 'origin/master' into testing 
Conflicts:
	bin/ZMQ_Sub_Urls.py
 2014-08-20 15:24:10 +02:00
Raphaël Vinot 99c8cc7941 completely remove ZMQ_PubSub.py 2014-08-20 15:14:57 +02:00
Alexandre Dulaunoy 1d64dc44c8 MIME type guessing - removed one duplicate call to libmagic 2014-08-20 10:22:33 +02:00
Raphaël Vinot 8d9ffbaa53 Do not create a ZMQ sub if it is not required. 2014-08-19 19:53:33 +02:00
Raphaël Vinot 45b0bf3983 Improve the cleanup. Still some to do. 2014-08-19 19:07:07 +02:00
Raphaël Vinot f1753d67c6 Cleanup the queues. 2014-08-19 16:05:37 +02:00
Alexandre Dulaunoy e8fcea6cd6 Remove undeclared variable 2014-08-18 16:17:36 +02:00