CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
623 Commits
6 Branches
40 Tags
107 MiB
Commit Graph

623 Commits (e75d6afdd9f2fe998ee06975e8a03300aab5ea49)

Author SHA1 Message Date
Mokaddem b04bac4731 Merge branch 'newModuleInformation' into production 2017-02-14 16:08:42 +01:00
Mokaddem d520c45a15 Added support of display of the last paste processed by a module 2017-02-14 16:05:59 +01:00
Mokaddem d3b4409156 Added path for debug in SQLInjection 2017-02-14 11:23:34 +01:00
Mokaddem 135a1457e0 Merge branch 'fix-linkPaste-debug' into production 2017-02-14 11:12:22 +01:00
Mokaddem d4da3a316f Added path of the paste in the log of Categ.py 2017-02-14 10:59:47 +01:00
mokaddem 60269ccecd Merge pull request #106 from mokaddem/master 
Updated readme - feeding AIL with pystemon
 2017-02-14 09:41:55 +01:00
Mokaddem 909c4f9472 Updated feeding AIL with pystemon 2017-02-14 09:40:04 +01:00
Mokaddem b9ceb4ec79 Updated feeding AIL with pystemon 2017-02-14 09:35:19 +01:00
Mokaddem 303575b2a1 Merge remote-tracking branch 'upstream/master' into production 2017-02-14 09:31:18 +01:00
Mokaddem 1318f87230 Merge remote-tracking branch 'upstream/master' 2017-02-14 09:16:02 +01:00
Alexandre Dulaunoy 2f4e5b9016 Merge branch 'master' of github.com:CIRCL/AIL-framework 2017-01-26 07:16:44 +00:00
Alexandre Dulaunoy cf903cc212 (partially fix #90 too) using a simple Alarm (SIGNAL) when exec-timeout 
Introducing a timer (in this case 60 seconds) to ensure that the
execution time of the analysis takes less than 60 seconds. This
is a simple and standard POSIX signal handler. If the timeout
is reached, the module will process the next one.

This approach fixes the specific issues we have currently
with some inputs where the sentiment analysis takes too much time. This
fix should be improved and be more generic:

 - Introducing statistics of content which timeouts.
 - Keeping a list/queue to further process those files using a different
   analysis approach. Maybe a set of "dirty" processes to handle the edge cases
   and to not impact the overall processing and analysis.
 - Make the timer configurable per module (at least for this one).
 2017-01-26 07:11:18 +00:00
Mokaddem c422db6e6d Merge remote-tracking branch 'upstream/master' into production 2017-01-17 16:38:48 +01:00
mokaddem b3a3616b0b Merge pull request #102 from ptitdoc/bugfix1 
Bug fixes in Workers and improvement of pystemon-feeder
 2017-01-17 16:35:32 +01:00
Mokaddem ea423fde0f Added requirement, better behaviors and confirmation box 2017-01-17 15:58:16 +01:00
Mokaddem 692f13d18b Improved module + Added support of click and keyboard navigation for killing and starting modules 2017-01-17 14:31:06 +01:00
Mokaddem 3b8149d840 Added action button and support of logs 2017-01-16 17:08:48 +01:00
Mokaddem 0ecfaf00f5 Added color in front of queue related to its state 2017-01-16 15:30:33 +01:00
Mokaddem af3241c008 Overwritted listbox update function to add formating 2017-01-16 14:41:02 +01:00
Mokaddem 30f303cb99 Added new ModuleInformation 2017-01-16 12:18:23 +01:00
Olivier MEDOC 16044d4d36 pystemon-feeder: add the option bind in configuration file instead of address 
This is essentially because multiple feeds can be used in the address
configuration variable.
 2017-01-13 14:58:18 +01:00
Alexandre Dulaunoy 3b101ea8f5 (partially) Fix #91 using a simple Alarm (SIGNAL) when exec-timeout 
Introducing a timer (in this case 5 seconds) to ensure that the
execution time of the tokenizer takes less than 5 seconds. This
is a simple and standard POSIX signal handler.

This approach fixes the specific issues we have currently
with some inputs where the tokenization takes too much time. This
fix should be improved and be more generic:

 - Introducing statistics of content which timeouts.
 - Keeping a list/queue to further process those files using a different
   tokenizer approach. Maybe a set of "dirty" processes to handle the edge cases
   and to not impact the overall processing and analysis.
 - Make the timer configurable per module (at least for this one).
 2017-01-12 07:32:55 +00:00
Olivier MEDOC 9f9c265cb0 SentimentAnalysis: use lexicon file path from the ail configuration file 2017-01-11 11:07:08 +01:00
ptitdoc c0fac82031 Add pystemon path in configuration sample 2017-01-10 18:33:46 +01:00
Olivier MEDOC 83db40104c import pystemon-feeder configuration from the central configuration file 2017-01-10 18:26:36 +01:00
Olivier MEDOC 8102ff009d add all_modules.txt file so that the WebGUI can do proper cleanup 2017-01-10 18:26:31 +01:00
Olivier MEDOC 3dc014dad9 fix invalid variable propagation 2017-01-10 18:26:24 +01:00
Olivier MEDOC 68ca20db8f fix to use the new pycountry API 2017-01-10 18:26:17 +01:00
Alexandre Dulaunoy 1950a2dc0e Merge pull request #99 from mokaddem/more-feed 
Usage of sha1 to filter paste content in Mixer.py
 2017-01-09 17:06:37 +01:00
Mokaddem 500608d0a1 Usage of sha1 to filter paste content in Mixer.py 2017-01-09 17:01:10 +01:00
Alexandre Dulaunoy f5adb65cb3 Merge pull request #98 from mokaddem/more-feed 
Config 2017
 2017-01-09 15:39:53 +01:00
Mokaddem f8c28796d2 Merge branch 'more-feed' of github.com:mokaddem/AIL-framework into more-feed 2017-01-09 15:37:58 +01:00
Mokaddem 409c5b8b42 Added 2017 2017-01-09 15:36:50 +01:00
Alexandre Dulaunoy a96ba40d94 Merge pull request #96 from mokaddem/more-feed 
Handling of multiple feeders
 2017-01-09 15:29:28 +01:00
Mokaddem 29241eb49f Added a line separator between the two graphs in dashboard. 2017-01-09 14:44:14 +01:00
Mokaddem 0e39f516a9 Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed. 2017-01-09 14:12:26 +01:00
Mokaddem 24c5621d29 BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None. 2016-12-26 16:33:10 +01:00
Mokaddem 3a298f0c7c Commented Mixer.py for further multiple feeds support. 2016-12-26 16:23:18 +01:00
Mokaddem 1c2169d3bc Improved interface for multiple feeds and refresh_script. 2016-12-26 16:16:44 +01:00
Mokaddem a18c046dea Fixed bug multiple refresh instances and harmonized interface. 2016-12-23 16:15:05 +01:00
Mokaddem 03dddbc359 Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs. 2016-12-23 16:01:30 +01:00
Mokaddem 97292e0899 Updated web interface to handle new mixer module. and fixed one dependency bug. 2016-12-23 15:44:46 +01:00
Mokaddem e70b9cd15c Added basic mixer with confirugable behavior. It handles muliple feeders and performs some basic stats on them. 2016-12-23 10:31:26 +01:00
Alexandre Dulaunoy ddfe3cf66a Merge pull request #95 from mokaddem/module_handling 
Module handling
 2016-12-22 22:18:31 +01:00
Alexandre Dulaunoy 3ce850e093 Merge pull request #94 from mokaddem/website-refacto 
Website refacto, support and fixes
 2016-12-22 22:17:58 +01:00
Mokaddem 9758579753 Merge branch 'module_handling' into production 2016-12-22 10:24:30 +01:00
Mokaddem 7438f16c63 Added new config 2016-12-22 10:06:35 +01:00
Mokaddem faa8f8302d Removed typo in the number of remaining pastes to display in browseImportantPaste 2016-12-22 09:28:55 +01:00
Mokaddem 9df12cdae8 Added sub-flask in a directory 2016-12-22 08:57:45 +01:00
Mokaddem 73938cf41b Prepared moduleInfo to handle module click with curses (will come back later). 2016-12-22 08:45:53 +01:00