CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
1,028 Commits
6 Branches
43 Tags
110 MiB
Commit Graph

358 Commits (b3923428e5c5c50b0952f75b011191ceb034bae6)

Author SHA1 Message Date
= b9bd651033 Removed useless import 2017-08-23 14:06:59 +02:00
= 3b3f3aa89c Flask: Added support of LevelDB database depending of the year 2017-08-23 12:20:39 +02:00
= d632335760 Automatically start redis_levelDB with the correct date 2017-08-23 10:16:56 +02:00
= bd46c83fb6 BrowseImportantPaste: Fixed a bug where onClickListener was not set when the table was only 1 page long 2017-08-21 11:55:01 +02:00
= 409eb25a64 Deleted data-flow graph and header.html from the index + Added fixed typo where 10min was in fact 3min 2017-08-21 10:51:56 +02:00
Mokaddem fa52292029 Added comments, cleanup and added credentials search progress in Flask 2017-07-20 10:50:24 +02:00
Mokaddem cbb2909156 Added comments 2017-07-20 10:24:48 +02:00
Mokaddem 05417f405c Finished basic link web interface + solved some bugs 2017-07-20 10:04:30 +02:00
Mokaddem ef741f85c0 Added similarity checks + started link webInterface 2017-07-19 11:52:06 +02:00
Mokaddem 882ab88b00 Draft module + web link 2017-07-18 16:57:15 +02:00
Mokaddem 9b8570e40b Dirty draft webpage credential_tracking 2017-07-17 17:26:19 +02:00
Mokaddem f1a1e75c32 Restaured old example webpage 2017-07-17 16:18:21 +02:00
Mokaddem 9558aafe67 Prevent the loss of the 1 message in import_dir.py 2017-07-17 16:14:27 +02:00
Mokaddem b60e164380 Added support of onion in browseWarningPaste 2017-04-26 11:47:02 +02:00
Mokaddem 5e4d0170d4 Bug fixes, jinja2 iter0 and typo 2017-04-26 11:31:08 +02:00
Mokaddem e614da4748 Deleted print + Added curve fill for Mixer graph in Index 2017-04-25 15:52:41 +02:00
Mokaddem 71b2757ed2 New script create_new_web_module permits to rapidly create a new pre-configured webpage from a template. 2017-04-25 15:22:04 +02:00
Mokaddem 6afbf77af0 Modified skeleton template name into rawSkeleton + Added possibility to hide modules in the header. 2017-04-25 12:18:08 +02:00
Mokaddem f6e8582d9d Auto organisation of the navbar -> Web page are auto-inserted. Created skeleton webpage 2017-04-19 15:14:20 +02:00
Mokaddem 32906965d8 Draft of new organisation of the web interface. Usage of Flask's blueprint 2017-04-19 11:02:03 +02:00
Mokaddem da081bae42 Merge branch 'regexInTerms' into production 2017-04-18 16:59:40 +02:00
Mokaddem c8baabd882 Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...) 2017-04-18 15:28:21 +02:00
Mokaddem 2da4c572c7 New feature in Terms: regex and set of words support (draft) 2017-03-28 17:42:44 +02:00
Mokaddem 3261eca836 Fixed all_index newline bug + added print of what's happening 2017-03-15 14:29:49 +01:00
Mokaddem 5b7d047622 Changed os.system by os.mkdir + fixed bug during rotation 2017-03-15 14:05:13 +01:00
Mokaddem 4005b72180 Added backward support + renamed index with timestamp 2017-03-15 11:51:35 +01:00
Mokaddem 381e72ee99 Added display of number of elements inside the index + changed variables names 2017-03-15 10:07:46 +01:00
Mokaddem e4757f5ceb Added possibility to choose the index in search + Updated search page with jinja2 iter0 2017-03-15 09:39:48 +01:00
Mokaddem 41132fe0bc Added support of per_paste for term-management and plot-term 2017-02-28 15:54:39 +01:00
Mokaddem 4071bf2e91 fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value 2017-02-28 15:01:48 +01:00
Mokaddem 25adae0104 Added possibility to select in top_terms the data per_paste or not (need module testing) 2017-02-15 16:29:02 +01:00
Mokaddem d4da3a316f Added path of the paste in the log of Categ.py 2017-02-14 10:59:47 +01:00
Mokaddem 29241eb49f Added a line separator between the two graphs in dashboard. 2017-01-09 14:44:14 +01:00
Mokaddem 0e39f516a9 Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed. 2017-01-09 14:12:26 +01:00
Mokaddem 24c5621d29 BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None. 2016-12-26 16:33:10 +01:00
Mokaddem 1c2169d3bc Improved interface for multiple feeds and refresh_script. 2016-12-26 16:16:44 +01:00
Mokaddem a18c046dea Fixed bug multiple refresh instances and harmonized interface. 2016-12-23 16:15:05 +01:00
Mokaddem 03dddbc359 Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs. 2016-12-23 16:01:30 +01:00
Mokaddem 97292e0899 Updated web interface to handle new mixer module. and fixed one dependency bug. 2016-12-23 15:44:46 +01:00
Mokaddem faa8f8302d Removed typo in the number of remaining pastes to display in browseImportantPaste 2016-12-22 09:28:55 +01:00
Mokaddem 9df12cdae8 Added sub-flask in a directory 2016-12-22 08:57:45 +01:00
Mokaddem d30f3ca6c7 Added missing depency datetime in dashboard 2016-12-09 13:53:57 +01:00
Mokaddem 0054be5bab Added missing json dependency in dashboard 2016-12-09 08:50:36 +01:00
Mokaddem 8daa72789e Splitted Flask_server into module related to website sections 2016-12-09 08:46:37 +01:00
Mokaddem d3ed1bbf40 Merge branch 'website-improvement' into production 2016-12-08 10:05:55 +01:00
Mokaddem 1abba4dcf9 Added support of re-plotting the plotted terms 2016-12-08 08:44:10 +01:00
Mokaddem 84dcd5586f Started plotting same data with different dates 2016-11-25 16:42:22 +01:00
Mokaddem a3255d168c ModuleInfo correctly handle CurveManageTopSets, 
Changed queue coloring in dashboard,
ModuleInformation now have a history of executed command and better handle module killing.
 2016-11-25 11:54:16 +01:00
Mokaddem 5d269ea1ee Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal. 2016-11-24 15:05:29 +01:00
Mokaddem df8d978088 Added dynamic data loading in dataTable in search.html 2016-11-24 13:31:31 +01:00
Mokaddem 80ca49f018 Prevent browser to request reconnection for the eventStream + Added dynamic data loading in the dataTable of BrowseImportantPastes 2016-11-11 16:51:39 +01:00
Mokaddem 768fb6f2df removed server overhead for paste preview data 2016-11-10 16:41:07 +01:00
Mokaddem 699c819387 fixed bug in show concerned pastes previex en (terms managements) 2016-11-10 16:33:05 +01:00
Mokaddem 94e3034e7d Fixed bun in terms management, where action_button was not bind in other dataTable pages 2016-11-10 15:53:22 +01:00
Mokaddem 7763bfb4c7 Pastes dataTable now display only the latest clicked paste even if there were discarded queries 2016-11-10 15:39:45 +01:00
Mokaddem 2dd69fbb25 Fixed modal background stacking bug + button leading to saved paste bug fixed in search 2016-11-10 14:56:25 +01:00
Mokaddem 317005c2b4 tryed to fix html display problem in tooltip + fixed problem button showPaste in modal not pointing to the paste 2016-11-10 10:30:29 +01:00
Mokaddem 34221b083f Partially solved the browser freezing problem + tootlip is also binded at a new page generation of the DataTable 2016-11-10 09:28:55 +01:00
Mokaddem acdd1367a3 Added event source for getImportantPaste 2016-10-28 16:55:56 +02:00
Mokaddem c95000866d Dynamic table in search now load all the data + fixed bugs where tooltip where not shown on other than the first page displayed and 
tooltip interpret html (not supposed to...)
 2016-10-28 14:21:08 +02:00
Mokaddem f51808d914 Added support of html formatting in search result dynamic loading table 2016-10-28 09:48:22 +02:00
Mokaddem 4be62a4e2d Added draft of dynamic update table with search result 2016-10-28 09:28:42 +02:00
Mokaddem 340624fe1e Fixed bug in modal show paste (show more button not showing) 2016-10-27 16:18:49 +02:00
Mokaddem 7e7e679ab6 Added dataTable for duplicate in show saved paste 2016-10-27 15:53:45 +02:00
Mokaddem 59a7000527 Bootstrap support in show saved paste 2016-10-27 12:38:39 +02:00
Mokaddem ac254e0e63 Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module 2016-10-27 11:50:24 +02:00
Mokaddem 1826b170ec Added support of local paste view in dashboard for Mails event only 2016-10-27 11:27:26 +02:00
Mokaddem 7c8d414948 modified provider name into no data when there is no data 2016-10-14 16:47:22 +02:00
Mokaddem 91678179fd Added msgs when queues or feed is not running/coming 2016-10-14 16:33:54 +02:00
Alexandre Dulaunoy 2781a2f0b7 Stuck not stucked 2016-09-21 08:35:20 +02:00
Jerome St-Hilaire ea13f87c2c Fixed dependency fetching for the web interface 2016-09-15 12:14:55 -07:00
Raphaël Vinot acfe9c1336 Update Flask_server.py 2016-08-26 18:06:48 +02:00
Raphaël Vinot 3290b51c91 Update update_thirdparty.sh 2016-08-26 18:05:18 +02:00
Mokaddem 79be8ab934 Fixed introduced bug relative to chart in index webpage. 2016-08-26 15:59:53 +02:00
Mokaddem 0c760d763b Added support of PID in ModuleInformation and index.html 2016-08-26 15:27:37 +02:00
Mokaddem 5c565068a6 Removed auto-reload in dashboard if window in undefined 2016-08-26 11:25:11 +02:00
Mokaddem 181c372131 Updated dependencies 2016-08-26 11:10:39 +02:00
Mokaddem 4cb3b672be Added comments 2016-08-26 10:28:35 +02:00
Mokaddem 86b655100e Deleted useless parsing function 2016-08-26 10:01:14 +02:00
Mokaddem 07d7559060 Added dataTable support in browse paste modal. 2016-08-26 08:30:09 +02:00
Mokaddem 813cc7031a Changed lots of code into zrank 2016-08-25 17:48:12 +02:00
Mokaddem 991176fdfd Suppressed metismenu import. 2016-08-25 16:40:13 +02:00
Mokaddem fb22816236 Added module number info in index webpage 2016-08-25 16:27:32 +02:00
Mokaddem 0b5a3e0cd2 Auto generate graph div + possibility to query all sentiments data. 2016-08-25 15:06:18 +02:00
Mokaddem 21f2c43572 Individualized sentiments-graph loading 2016-08-25 13:11:16 +02:00
Mokaddem f9bf00f022 Removed autokill feature if not specified 2016-08-25 11:09:31 +02:00
Mokaddem 611454ba82 Added color in web-index reprenting modules states. 2016-08-24 18:00:05 +02:00
Mokaddem e3daa84196 Added icon in header 2016-08-23 16:09:34 +02:00
Mokaddem e12244fb53 Added show/hide curve with legend in terms-top-set. 2016-08-23 15:39:13 +02:00
Mokaddem e56187eb7e Added position of each top term in the other sets + highligth ones not present. 2016-08-23 14:52:02 +02:00
Mokaddem a64d447b7b Auto hide button add_to_chart when no data plotted. 2016-08-23 12:09:41 +02:00
Mokaddem e59b7c4e47 Added tooltip in graph for terms-plot-tool 2016-08-23 12:03:23 +02:00
Mokaddem 0eba419651 Added multiple terms plot in terms-plot-tool. 2016-08-23 11:56:32 +02:00
Mokaddem 9f2d672ec8 harmonized tooltip and popover display 2016-08-23 11:20:48 +02:00
Mokaddem 8c956c22b1 Added autohide curves in terms-top-set. 2016-08-23 11:02:36 +02:00
Mokaddem 32dfd2b181 Moved legend place in term-plot-top 2016-08-23 10:16:33 +02:00
Mokaddem f9b27e4013 Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case. 2016-08-23 09:50:03 +02:00
Mokaddem 1766ced9e6 Fixed potential XSS problem with pastes display in 'search' and 'important_paste_by_module'. 2016-08-23 09:08:17 +02:00
Mokaddem 047331df3e Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module. 2016-08-23 08:52:54 +02:00
Mokaddem 6be557890c Replaced negative hour by 24h based. 2016-08-23 07:19:51 +02:00
Mokaddem 85ab0e36ff In sentiment analysis. if today top provider set is empty, try to take yesterday data and then takes from all_providers. 2016-08-23 07:11:03 +02:00
Mokaddem 8359a6668f Suppressed useless comms 2016-08-22 21:36:19 +02:00
Mokaddem 2fd2c8af39 Removed commented code 2016-08-22 20:59:56 +02:00
Mokaddem 27ae577e00 Fixed stupid line remove in index.html... 2016-08-22 16:25:52 +02:00
Mokaddem a424069c64 Fixed bug in flask with zrange (was taking the bottom of the set instead of the top leading to get only the min instead of max) 2016-08-22 16:12:20 +02:00
Mokaddem 003c72bd7c Fixed bug with redis where the inserted key was too long, causing a crash + fixed bug taking min instead of max in terms-top-set 2016-08-22 16:01:42 +02:00
Mokaddem 2466f355ab Fixed bug in manage-terms-top-set + added auto sorting in important-paste-by-module 2016-08-22 14:54:37 +02:00
Mokaddem e144e8e1c6 Fixed bug in terms-management-interface (only display one paste) + removed get_language of all pastes due to the huge overhead. 2016-08-22 13:59:12 +02:00
Mokaddem d3c0fc5f94 Added terms trending module and web interface: 
- Curve module has been rewritten
- Added new module curve_manage_top_set
- Added all web-pages
 2016-08-22 13:35:49 +02:00
Mokaddem 841e87eb16 Fixed bug in sentiment trending, if there was no data for the avg -> there was one NaN leading to no display in the gauge. 2016-08-20 18:48:24 +02:00
Mokaddem ee8f26c7c4 In sentiment trending, if there are no providers in the today\'s set, get data from random providers 2016-08-20 18:14:19 +02:00
Mokaddem 2ed60cb9bf Fixed typo 2016-08-20 16:44:36 +02:00
Mokaddem 3f1cf705fc Deleted alert() 2016-08-20 16:41:49 +02:00
Mokaddem 1c96eac2e4 Enter simulate button click on input field for terms management and plot tool 2016-08-20 16:38:51 +02:00
Mokaddem 465d18fc73 Added terms-plot tool 2016-08-20 16:23:06 +02:00
Mokaddem fb3dea6bd2 Finnished terms-plot-top-set. Also added possibility to add terms to track or blacklist. 2016-08-20 15:59:22 +02:00
Mokaddem a5117d7cad Added top set webpage 2016-08-20 11:46:22 +02:00
Mokaddem 0b6828fb16 Correctly handle and display add or remove a term in term-frequency + started sketch of terms-plot tool. 2016-08-19 16:53:46 +02:00
Mokaddem 8d41b109e7 Added query of the data for the new tracked word 2016-08-19 14:33:01 +02:00
Mokaddem b78effe779 Added draft of terms_management + Added copyright in readme 2016-08-19 13:34:02 +02:00
Mokaddem 52b7124a6b - Modified redis connection (from levelDB to redis). 
- Added term frequency in curve.
- Modified ModuleStats and WebStats to use redis special command (incrby instead of get-set and zset) and Flask to perform the correct queries.
- Added panel color in sentiment trending
 2016-08-18 15:34:19 +02:00
Mokaddem 24072ece36 Added support of pasteName in search + added header in search.html + supressed mixed indent and trailing in Flask.py 2016-08-17 14:21:31 +02:00
Mokaddem c10f28d9df Updated isolated launch scripts + Added switch displaying queue in index 2016-08-17 13:46:01 +02:00
Mokaddem 23b65eddca Auto unchecked info checkbox in index 2016-08-17 11:25:05 +02:00
Mokaddem 4d09ab0f6b Modified icon sentiment in web-header. 2016-08-17 11:13:37 +02:00
Mokaddem 44f54b7b81 Added dependencies 2016-08-17 11:04:48 +02:00
Mokaddem f0b527167a Added script flexgauge 2016-08-17 10:50:30 +02:00
Mokaddem c12e67a35a Added tooltips + slight gauge modification 2016-08-17 10:43:43 +02:00
Mokaddem 894b9efda9 Added drop of really long line in sentiment-analysis module + Added description of sentiment module. Also, fixed bug in webpage sentiement-trending concerning avg and date range. 2016-08-17 09:46:25 +02:00
Mokaddem 1084e45f1b Added draft of filter in sentiment analysis (Discard syntaxical languages) + Added nice tooltip for sparkline. Trending displays avg in function of the number of elements processed and not for the complete week + fixed bug in gauge and canvasjs (was performing avg with only 1 graph instead of all 8). 2016-08-16 16:33:02 +02:00
Mokaddem ecd834ffb6 Added autoreload in sentiment-trending. 2016-08-16 11:27:49 +02:00
Mokaddem bb677b9061 Added avg text in sentiment-trending + fixed date range bug 2016-08-16 11:23:14 +02:00
Mokaddem cdd08d7df0 Added all providers for plot_tool in moduleStats + fixed date range bug 2016-08-16 11:07:36 +02:00
Mokaddem 8fa3c2b147 Added stacked choice or not for plot-tool (sentiments) 2016-08-16 10:49:05 +02:00
Mokaddem 4baac0b994 Tool-plot for sentiment working + Splitted script and webpage 2016-08-16 10:31:18 +02:00
Mokaddem 1e7547d16c Created draft of search for plot tool 2016-08-15 16:32:02 +02:00
Mokaddem 88ed8a239d Linked plot button with server 2016-08-15 15:54:53 +02:00
Mokaddem 82f70e61f4 Added provider checkbox 2016-08-15 15:16:31 +02:00
Mokaddem f90987d284 Added date on slider 2016-08-15 14:53:46 +02:00
Mokaddem d8c84ec674 Added link to data in analysis trending + fixed tooltip bug in sparkline 2016-08-15 12:57:39 +02:00
Mokaddem 6c7f660052 Added support of week and day average. 2016-08-13 22:17:11 +02:00
Mokaddem 1a5158b081 Added sentiment analyser module (draft) 2016-08-13 15:24:57 +02:00
Mokaddem 0431ecf434 Modified webpage display and added js. 2016-08-12 10:22:24 +02:00
Mokaddem 9c3b573af2 Splited js from html + Added some ID + small display modification. 2016-08-12 07:49:24 +02:00
Mokaddem 87a86b38d1 Added page Plot-tool 2016-08-11 21:08:43 +02:00
Mokaddem c028ca80da Separated options from the plot funcion for sparkline. 2016-08-11 17:57:23 +02:00
Mokaddem 98c4cacb98 Finished first panel in sentiment-today. 2016-08-11 17:22:50 +02:00
Mokaddem 8682fcc29c Added empty webpages for sentiments modules. 2016-08-11 11:34:22 +02:00
Mokaddem 5612a77af9 Separated webpage header code from all webpage 2016-08-11 10:55:51 +02:00
Mokaddem 0ba5637e06 Added bar chart hover binder in trending charts 2016-08-10 11:16:56 +02:00