CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
1,449 Commits
7 Branches
44 Tags
110 MiB
Commit Graph

544 Commits (01ff202ae81eebe2be33dc2ba09bc921902f30cd)

Author SHA1 Message Date
Terrtia 173c0a15c6
fix: [term_management] delete tags 2018-11-08 12:01:40 +01:00
Terrtia 4e906ff841
fix: [term_management] delete emails 2018-11-08 10:41:37 +01:00
Terrtia 0adb8b4624
chg: [terms_management] delete terms tags 2018-11-07 15:37:25 +01:00
Terrtia 46614c97c8
chg: [terms_management] delete email 2018-11-07 11:42:31 +01:00
Terrtia 2c46c7f2ab
fix: [tag export] limit custom tag lenght 2018-11-07 10:17:57 +01:00
Terrtia f91aba5f6f
chg: [tag export] UI, add custom tags 2018-11-07 09:41:58 +01:00
Terrtia 0ad80a22ca
fix: [update_thirdparty] upgrade github source 2018-11-06 17:04:02 +01:00
Terrtia a3fa5a4dcf
chg: [auto push tag] delete unwanted tags 2018-11-06 16:08:58 +01:00
Terrtia 979472df15
chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
Terrtia af1ff6225e
fix: [Search] tags display 2018-11-05 10:36:58 +01:00
Terrtia 5c67297dc8
fix: [Search] Improve search performance 
note: search by paste name is temporary disabled
 2018-11-05 10:13:48 +01:00
Terrtia 5872cf9196
fix: [Scripts] Remove absolute path 2018-11-02 16:07:27 +01:00
Terrtia ea7fd8d54c
fix: [Duplicate] show duplicate with relative path 2018-10-24 15:47:03 +02:00
Terrtia 6328cc22b7
chg: [Crawler] add domains blacklist 2018-09-28 16:29:09 +02:00
Terrtia b3a6dc8487
fix: [Crawler] remove hardcoded url 2018-09-28 15:42:06 +02:00
Terrtia 3dd20af4d1
Merge branch 'master' into onion_crawler 2018-09-28 15:29:24 +02:00
Terrtia 82e6df4b94
chg: [Crawler] domains stats + logs + clean 2018-09-28 15:23:27 +02:00
Terrtia ecb2857151
chg: [Crawler] css + limit splash RAM 2018-09-27 16:47:48 +02:00
Terrtia ff0affc679
fix: [Flask] remove hardcoded url 2018-09-26 11:59:51 +02:00
Terrtia 3a72f1548d
fix: [Flask] remove hardcoded url 2018-09-25 16:46:06 +02:00
Terrtia b3923428e5
fix: [Flask] remove hardcoded url 2018-09-25 14:37:14 +02:00
Terrtia e4a5f66a10
Merge branch 'master' into onion_crawler 2018-09-24 16:35:04 +02:00
Terrtia 874824a589
fix: [Crawler] clean 2018-09-24 16:28:55 +02:00
Terrtia 50c81773e9
chg: [Crawler] add launcher and install 2018-09-24 16:23:14 +02:00
Terrtia 6edc1ddbeb
chg: [HiddenServices] get domain related to other domains 2018-09-21 10:34:06 +02:00
Terrtia 61b4c9e579
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:56:29 +02:00
Terrtia 5eadc2755f
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:38:19 +02:00
Terrtia 5732d7b134
fix: [hashDecoded] range_type_json 2018-09-19 15:36:33 +02:00
Terrtia 912f2352ab
fix: [crawled screenshot] fix img error 2018-09-18 16:58:49 +02:00
Terrtia ce63d81878
chg: [UI] pixelate crawled screenshot by default 2018-09-18 11:51:21 +02:00
Terrtia f5b648d72a
pixelate paste screenshot 2018-09-18 11:03:40 +02:00
Terrtia 094b211259
fix: [showpaste] path transversal 2018-09-17 15:42:22 +02:00
Terrtia 0c63f2f24f
chg: [Crawler] catch server response 2018-09-17 15:35:06 +02:00
Terrtia 6f0817365a
chg: [Crawler UI] display domain information 2018-09-12 09:55:49 +02:00
Terrtia ca982e13e1
chg: [Crawled Domain] show crawled pastes by domain 2018-08-27 14:34:08 +02:00
Terrtia 40772a5732
fix: merge 2018-08-27 11:30:19 +02:00
Terrtia 823c33174a
Merge branch 'master' into onion_crawler 2018-08-27 11:20:51 +02:00
Terrtia d42dd118a4
chg: [Domain crawled] add random screenshot 2018-08-27 11:02:39 +02:00
Terrtia 6c981ea5e9
fix: [HashDecoded] typo 2018-08-24 11:58:54 +02:00
Terrtia ab452841b1
fix: [HashDecoded] change template title 2018-08-24 11:51:16 +02:00
Terrtia e6b583ee7a
chg: [Hash] add Help on graph node 2018-08-24 11:40:09 +02:00
Terrtia a586548047
Merge branch 'master' into base64 2018-08-24 10:35:28 +02:00
Terrtia 7e24943537
chg: [Crawler] crawler accept all kind of domains 2018-08-24 10:13:56 +02:00
Terrtia e9580d6775
chg: [Crawler] change BDD, save i2p links 2018-08-21 15:54:53 +02:00
Terrtia ed559d9f4a
chg: [Showpaste] add screenshot + improve onion db 2018-08-16 17:24:39 +02:00
Terrtia 54cc4f3723
fix: [UI-Showpaste] show raw paste, force text/plain mimetype 2018-08-03 09:15:34 +02:00
Terrtia 911cff70da
chg: [UI] add AIL icon, fix li links 2018-08-02 10:44:47 +02:00
Terrtia deaebaa4fd
fix: [UI-Hash] pie chart json 2018-07-27 13:51:29 +02:00
Terrtia 009f87ee10
[fix: [UI-hash] fix graph node zoom 2018-07-27 11:27:41 +02:00
Terrtia b02cbee179
chg: [UI-dashboard] add logs number configuration 2018-07-26 11:35:54 +02:00
Terrtia 0a82482db4
chg: [UI-dashboard] convert time to local 2018-07-26 10:34:43 +02:00
Terrtia 5ad5939bd5
chg: display last logs 2018-07-25 16:48:44 +02:00
Terrtia 03a37af5cf
chg: [Hash] add reprocess regex order 2018-07-25 10:47:36 +02:00
Terrtia 8c97c70a3f
Merge branch 'master' into base64 2018-07-24 15:21:32 +02:00
Terrtia 4b8c26dec8
fix: [hive auto export] fix #221 2018-07-24 14:46:35 +02:00
Terrtia 3479923f96
fix: [UI-Hash] pie decoder 2018-07-23 11:43:48 +02:00
Terrtia 9c03c40482
chg: [UI-Hash] add decoder pie 2018-07-23 11:11:52 +02:00
Terrtia 1fbd4109c2
chg: [UI-Hash] add decoded type on graph 2018-07-20 15:48:56 +02:00
Terrtia bd5f83f0eb
chg: refractor base64 encoded to hash 2018-07-20 10:32:52 +02:00
Terrtia e25dd52d64
add: Binary decoder 2018-07-18 11:45:19 +02:00
Terrtia 524d5fe604
add: hash download 2018-07-18 10:09:35 +02:00
Terrtia d9c924c38d
chg: use json to submit hash 2018-07-17 14:23:49 +02:00
Terrtia 412b012ddf
add: hash line_graph (nb/day) 2018-07-16 16:45:36 +02:00
Terrtia 6f69da0c0d
fix: paste submit css 2018-07-16 10:32:41 +02:00
Terrtia fce324e370
fix: release tiemeout alarm, chg:icon graph node 2018-07-13 11:54:14 +02:00
Terrtia 87b7facba8
add: base 64 node graph 2018-07-12 17:07:17 +02:00
Alexandre Dulaunoy 9dc211dbe9
chg: [UI] Typo fixed as requested by our favourite German. 2018-07-10 10:49:02 +02:00
Terrtia fd02085495
add: show hash info 2018-07-09 17:07:59 +02:00
Terrtia c2a976e907
add: stacked barchart, resize chart 2018-07-09 11:47:04 +02:00
Terrtia 0e5a7d8d47
add date_range stacked bar 2018-07-06 16:54:43 +02:00
Terrtia 3b4e2a18aa
fix: date format https://github.com/CIRCL/AIL-framework/issues/219 2018-07-05 14:45:34 +02:00
Terrtia cd154be6f9
fix: term_management UI bug ,https://github.com/CIRCL/AIL-framework/issues/214 2018-07-05 14:12:47 +02:00
Terrtia f7d38bea16
add: base64 date and type selector 2018-07-05 13:58:40 +02:00
Terrtia b6d855b963
change import d3.js to d3.min.js 2018-06-29 15:58:24 +02:00
Terrtia adfc8cf260
add d3 and daterange js 2018-06-29 15:52:13 +02:00
Terrtia a0a639276c
fix submit paste 2018-06-29 15:29:02 +02:00
Terrtia 98d0235fc3
Merge branch 'master' into base64 2018-06-29 14:57:46 +02:00
Terrtia 5908710b34
fix: update submitted tags link 2018-06-29 11:29:52 +02:00
Terrtia cb807bf2c6
chg: change base64 icon 2018-06-29 11:28:35 +02:00
Terrtia 16c700ddb4
new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
Terrtia 225776c69c
fix: tp and fp, errors messages on paste submission 2018-06-20 10:02:26 +02:00
Terrtia 6a89fac309
fix hive and misp error 2018-06-19 16:39:49 +02:00
Terrtia af9db53171
add tag fp and tp 2018-06-19 15:09:26 +02:00
Terrtia bf72c27711
fix path-event id map 2018-06-19 13:43:26 +02:00
Terrtia cd4c099685
fix db init 2018-06-19 11:54:34 +02:00
Terrtia 103787a07d
fix 2018-06-19 11:43:16 +02:00
Terrtia bc40bb3c38
fix hive connection + add keys sample 2018-06-19 11:42:06 +02:00
Terrtia 0ee36e484f
fix hive connection + submitted dir creation 2018-06-19 11:31:30 +02:00
Terrtia 634bf6e552
add img + fix upload api 2018-06-18 17:16:22 +02:00
Terrtia 704e6f0ad1
change default submit tag, event-path map, publish events 2018-06-18 13:58:31 +02:00
Terrtia aee1b50ef1
fix duplicate export attribute + tag export whitelist 2018-06-15 17:25:43 +02:00
Terrtia 31fc33ab03
misp event creation, auto and manual + the hive alert auto creation, manual case creation 2018-06-14 16:51:06 +02:00
Terrtia 8522d50308
modify navbar + decompress file password 2018-06-11 10:21:11 +02:00
Terrtia c64588f60e
add feature, user can submit paste on the web interface 2018-06-08 16:49:20 +02:00
Terrtia 8e9c77b364
verify file upload extention 2018-06-06 10:05:25 +02:00
Terrtia 6143bc3dce
submit users input paste 2018-06-05 16:58:04 +02:00
Terrtia c41a56e4a2
clean Flask_tags + gitignore 2018-06-01 13:36:08 +02:00
Terrtia 2b310d4367
clean 2018-06-01 11:26:45 +02:00
Terrtia cd6734f704
fix install 2018-05-31 15:35:20 +02:00
Terrtia 884d655b0c
taxonomies and galaxies update via LAUNCH 2018-05-31 13:53:29 +02:00
Terrtia 5c50de45de
add taxonomies and galaxies update and install 2018-05-31 10:07:23 +02:00
Terrtia 8a87b7b082
galaxy tag info + fix 2018-05-30 16:18:58 +02:00
Terrtia dbe67e094b
galaxies synonyms 2018-05-29 17:17:02 +02:00
Terrtia f3f6b76428
galaxy description and tags 2018-05-29 08:47:46 +02:00
Terrtia f5cae0d99c
taxonomie + add tags + tags display 2018-05-23 16:58:56 +02:00
Terrtia b9eb3ed9ba
add and delete tags features 2018-05-17 15:33:06 +02:00
Terrtia 4fbbf8bbd2
display tag on important paste and search 2018-05-17 11:00:05 +02:00
Terrtia 0d164b6f4c
add tag js and css 2018-05-16 14:48:15 +02:00
Terrtia b1b7d33c96
tags 2018-05-16 14:39:01 +02:00
Terrtia f66a528bc2
fix Duplicate, save list of duplicates on disk + prevent empty hash creation 2018-05-15 23:28:47 +02:00
Terrtia 225fe76c96
fix duplicate + get raw paste 2018-05-11 16:07:30 +02:00
Terrtia 142c1dc9b0
clean 2018-05-09 13:03:46 +02:00
Terrtia 3841c159a0
change leveldb to ardb 2018-05-07 14:50:40 +02:00
Terrtia c17b9402f7
clean 2018-05-04 14:25:47 +02:00
Terrtia 96a3df42bc
decode with redis connection 2018-05-04 13:53:29 +02:00
Terrtia a900d5f08d
add python 2 to 3 bash + fix Duplicate + clean 2018-05-03 16:21:33 +02:00
Terrtia c52caebe7c
improve Duplicate + tlsh + add time out handler + debug + clean 2018-05-02 17:07:10 +02:00
Terrtia c3428df5dc add apiKeys module 2018-04-26 14:42:39 +02:00
Terrtia 8738b7cf75 fix track term and avoid duplicate mail address 2018-04-24 16:44:37 +02:00
Terrtia f93fe9aeb2 add bitcoin module 2018-04-23 14:51:53 +02:00
Terrtia 41eb250278 fix python 3 compqtibility issues 2018-04-20 10:42:19 +02:00
Terrtia 8571a86344 frontend python 3.5 upgrade 2018-04-17 16:06:32 +02:00
Philipp Schmied 4835589ff4 Fixed typo 2018-03-01 08:50:27 +01:00
Philipp Schmied 29608ab29b Fixed e-mail notificatins for sets and regexes 2018-02-28 11:31:16 +01:00
Philipp Schmied b6fcf51309 Flask_terms.py: Fixed mapping of notifcation e-mails to terms with uppercase letters 2018-02-28 09:19:27 +01:00
Philipp Schmied 5b1f0b0212 Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms. 2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy abdf41fe23
Merge pull request #164 from mokaddem/showDiff 
Show differences between duplicates
 2018-02-21 22:25:16 +01:00
Sami Mokaddem 9125930a77 fix: canvasjs changed its folder hierarchy 2017-12-14 11:47:11 +01:00
Sami Mokaddem 304213cfa4 update: Possibility to change the authorized max line length in the config for the diff of duplicates 2017-12-12 11:00:26 +01:00
Sami Mokaddem 3fe7ecf75a feature: diff in the web interface for duplicates pastes. 2017-12-12 10:48:21 +01:00
mokaddem 9cab76cf88
Merge pull request #162 from mokaddem/sentimentBugfix 
Fix: bug between front-end and back-end date representation
 2017-12-11 16:35:38 +01:00
Paul A 4a4a1d478b Quick refactor for the JavaScript 2017-10-22 18:48:54 +02:00
Paul A f6b6777095 Fixed some code for the JavaScript (coding-style) 2017-10-20 15:44:09 +02:00
Paul A 48fc8e3464 Removed duplicate function 2017-10-20 15:32:58 +02:00
Simon Tran a41c5d55c5 Migrated Dockerfile to Ubuntu 16.04 LTS 
Fixed AIL Web dependencies installation
Added Docker documentation
 2017-10-01 02:23:32 +02:00
Sami Mokaddem 2cfb8c173b Fixed from front-end date representation bug 2017-09-27 10:46:20 +02:00
= b9bd651033 Removed useless import 2017-08-23 14:06:59 +02:00
= 3b3f3aa89c Flask: Added support of LevelDB database depending of the year 2017-08-23 12:20:39 +02:00
= d632335760 Automatically start redis_levelDB with the correct date 2017-08-23 10:16:56 +02:00
= bd46c83fb6 BrowseImportantPaste: Fixed a bug where onClickListener was not set when the table was only 1 page long 2017-08-21 11:55:01 +02:00
= 409eb25a64 Deleted data-flow graph and header.html from the index + Added fixed typo where 10min was in fact 3min 2017-08-21 10:51:56 +02:00
Mokaddem fa52292029 Added comments, cleanup and added credentials search progress in Flask 2017-07-20 10:50:24 +02:00
Mokaddem cbb2909156 Added comments 2017-07-20 10:24:48 +02:00
Mokaddem 05417f405c Finished basic link web interface + solved some bugs 2017-07-20 10:04:30 +02:00
Mokaddem ef741f85c0 Added similarity checks + started link webInterface 2017-07-19 11:52:06 +02:00
Mokaddem 882ab88b00 Draft module + web link 2017-07-18 16:57:15 +02:00
Mokaddem 9b8570e40b Dirty draft webpage credential_tracking 2017-07-17 17:26:19 +02:00
Mokaddem f1a1e75c32 Restaured old example webpage 2017-07-17 16:18:21 +02:00
Mokaddem 9558aafe67 Prevent the loss of the 1 message in import_dir.py 2017-07-17 16:14:27 +02:00
Mokaddem b60e164380 Added support of onion in browseWarningPaste 2017-04-26 11:47:02 +02:00
Mokaddem 5e4d0170d4 Bug fixes, jinja2 iter0 and typo 2017-04-26 11:31:08 +02:00
Mokaddem e614da4748 Deleted print + Added curve fill for Mixer graph in Index 2017-04-25 15:52:41 +02:00
Mokaddem 71b2757ed2 New script create_new_web_module permits to rapidly create a new pre-configured webpage from a template. 2017-04-25 15:22:04 +02:00
Mokaddem 6afbf77af0 Modified skeleton template name into rawSkeleton + Added possibility to hide modules in the header. 2017-04-25 12:18:08 +02:00
Mokaddem f6e8582d9d Auto organisation of the navbar -> Web page are auto-inserted. Created skeleton webpage 2017-04-19 15:14:20 +02:00
Mokaddem 32906965d8 Draft of new organisation of the web interface. Usage of Flask's blueprint 2017-04-19 11:02:03 +02:00
Mokaddem da081bae42 Merge branch 'regexInTerms' into production 2017-04-18 16:59:40 +02:00
Mokaddem c8baabd882 Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...) 2017-04-18 15:28:21 +02:00
Mokaddem 2da4c572c7 New feature in Terms: regex and set of words support (draft) 2017-03-28 17:42:44 +02:00
Mokaddem 3261eca836 Fixed all_index newline bug + added print of what's happening 2017-03-15 14:29:49 +01:00
Mokaddem 5b7d047622 Changed os.system by os.mkdir + fixed bug during rotation 2017-03-15 14:05:13 +01:00
Mokaddem 4005b72180 Added backward support + renamed index with timestamp 2017-03-15 11:51:35 +01:00
Mokaddem 381e72ee99 Added display of number of elements inside the index + changed variables names 2017-03-15 10:07:46 +01:00
Mokaddem e4757f5ceb Added possibility to choose the index in search + Updated search page with jinja2 iter0 2017-03-15 09:39:48 +01:00
Mokaddem 41132fe0bc Added support of per_paste for term-management and plot-term 2017-02-28 15:54:39 +01:00
Mokaddem 4071bf2e91 fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value 2017-02-28 15:01:48 +01:00
Mokaddem 25adae0104 Added possibility to select in top_terms the data per_paste or not (need module testing) 2017-02-15 16:29:02 +01:00
Mokaddem d4da3a316f Added path of the paste in the log of Categ.py 2017-02-14 10:59:47 +01:00
Mokaddem 29241eb49f Added a line separator between the two graphs in dashboard. 2017-01-09 14:44:14 +01:00
Mokaddem 0e39f516a9 Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed. 2017-01-09 14:12:26 +01:00
Mokaddem 24c5621d29 BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None. 2016-12-26 16:33:10 +01:00
Mokaddem 1c2169d3bc Improved interface for multiple feeds and refresh_script. 2016-12-26 16:16:44 +01:00
Mokaddem a18c046dea Fixed bug multiple refresh instances and harmonized interface. 2016-12-23 16:15:05 +01:00
Mokaddem 03dddbc359 Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs. 2016-12-23 16:01:30 +01:00
Mokaddem 97292e0899 Updated web interface to handle new mixer module. and fixed one dependency bug. 2016-12-23 15:44:46 +01:00
Mokaddem faa8f8302d Removed typo in the number of remaining pastes to display in browseImportantPaste 2016-12-22 09:28:55 +01:00
Mokaddem 9df12cdae8 Added sub-flask in a directory 2016-12-22 08:57:45 +01:00
Mokaddem d30f3ca6c7 Added missing depency datetime in dashboard 2016-12-09 13:53:57 +01:00
Mokaddem 0054be5bab Added missing json dependency in dashboard 2016-12-09 08:50:36 +01:00
Mokaddem 8daa72789e Splitted Flask_server into module related to website sections 2016-12-09 08:46:37 +01:00
Mokaddem d3ed1bbf40 Merge branch 'website-improvement' into production 2016-12-08 10:05:55 +01:00
Mokaddem 1abba4dcf9 Added support of re-plotting the plotted terms 2016-12-08 08:44:10 +01:00
Mokaddem 84dcd5586f Started plotting same data with different dates 2016-11-25 16:42:22 +01:00
Mokaddem a3255d168c ModuleInfo correctly handle CurveManageTopSets, 
Changed queue coloring in dashboard,
ModuleInformation now have a history of executed command and better handle module killing.
 2016-11-25 11:54:16 +01:00
Mokaddem 5d269ea1ee Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal. 2016-11-24 15:05:29 +01:00
Mokaddem df8d978088 Added dynamic data loading in dataTable in search.html 2016-11-24 13:31:31 +01:00
Mokaddem 80ca49f018 Prevent browser to request reconnection for the eventStream + Added dynamic data loading in the dataTable of BrowseImportantPastes 2016-11-11 16:51:39 +01:00
Mokaddem 768fb6f2df removed server overhead for paste preview data 2016-11-10 16:41:07 +01:00
Mokaddem 699c819387 fixed bug in show concerned pastes previex en (terms managements) 2016-11-10 16:33:05 +01:00
Mokaddem 94e3034e7d Fixed bun in terms management, where action_button was not bind in other dataTable pages 2016-11-10 15:53:22 +01:00
Mokaddem 7763bfb4c7 Pastes dataTable now display only the latest clicked paste even if there were discarded queries 2016-11-10 15:39:45 +01:00
Mokaddem 2dd69fbb25 Fixed modal background stacking bug + button leading to saved paste bug fixed in search 2016-11-10 14:56:25 +01:00
Mokaddem 317005c2b4 tryed to fix html display problem in tooltip + fixed problem button showPaste in modal not pointing to the paste 2016-11-10 10:30:29 +01:00
Mokaddem 34221b083f Partially solved the browser freezing problem + tootlip is also binded at a new page generation of the DataTable 2016-11-10 09:28:55 +01:00
Mokaddem acdd1367a3 Added event source for getImportantPaste 2016-10-28 16:55:56 +02:00
Mokaddem c95000866d Dynamic table in search now load all the data + fixed bugs where tooltip where not shown on other than the first page displayed and 
tooltip interpret html (not supposed to...)
 2016-10-28 14:21:08 +02:00
Mokaddem f51808d914 Added support of html formatting in search result dynamic loading table 2016-10-28 09:48:22 +02:00
Mokaddem 4be62a4e2d Added draft of dynamic update table with search result 2016-10-28 09:28:42 +02:00
Mokaddem 340624fe1e Fixed bug in modal show paste (show more button not showing) 2016-10-27 16:18:49 +02:00
Mokaddem 7e7e679ab6 Added dataTable for duplicate in show saved paste 2016-10-27 15:53:45 +02:00