CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
2,043 Commits
7 Branches
43 Tags
110 MiB
Commit Graph

735 Commits (d97305acc1e938249aa8d56fcfd8c544c6f37d07)

Author SHA1 Message Date
Terrtia f64c385343
chg: [Crawler] handle port: crawling + history 2019-03-22 16:48:07 +01:00
Terrtia f4cdddbc7f
chg: [UI + item submit] remove browse paste from UI + fix pasteSubmit tagging 2019-03-21 11:50:53 +01:00
Terrtia 51eb951a47
fix: [UI terms] update tags endpoint 2019-03-21 11:34:21 +01:00
Terrtia c2bc72c72e
chg: [Tags alertHandler] remove alertHandler + add quick tag search sidebar + use tag last_seen to display tagged items 2019-03-21 11:11:53 +01:00
Terrtia ce6475e96e
chg: [tags] remove/add tags by daterange + improve item loading time 2019-03-20 15:57:55 +01:00
Terrtia 8b5266706e
fix: [Tags] cast nb_max_page to int 2019-03-20 13:33:12 +01:00
Terrtia a9e86231ad
chg: [Tags] add backend, daterange selector 2019-03-20 13:25:02 +01:00
Terrtia 65e6eae00d
chg: [UI preview items] b4 migration + new template + better display + decrease item loading time + add basic animation/transition 2019-03-19 13:34:41 +01:00
Terrtia e9bac098c9
fix: [UI Tags] paste preview 2019-03-15 11:16:44 +01:00
Terrtia 23d9cb2265
chg: [UI Tags search] bootstrap 4 migration 2019-03-14 17:04:55 +01:00
Terrtia 9e4813ca4b
chg: [UI Tags] start b4 migration + add date range selector 2019-03-12 13:37:15 +01:00
Terrtia d3b3942ad7
chg: [Flask UI] refractor crawler frontend + show crawled domain 2019-03-06 15:05:58 +01:00
Terrtia 6f5f880b7d
chg: [Crawler UI] add basic tooltips 2019-03-01 11:40:43 +01:00
Terrtia 93c7db86e4
chg: [Crawler UI Flask] refractor functions, templates 2019-02-28 14:56:14 +01:00
Terrtia 460f70fee6
chg: [css] add basic toggle sidebar 2019-02-27 16:58:17 +01:00
Terrtia aaf4712877
fix: [UI crawler manual] fix min input 2019-02-26 15:39:52 +01:00
Terrtia c0d72e7d2a
chg: [Crawler UI] Crawler major refractor (end) + basic UI for manual crawler 2019-02-26 14:50:48 +01:00
Terrtia e5dca268a8
chg: [Crawler] refractor 2019-02-21 09:54:43 +01:00
Terrtia 0832784f7a
chg: [thirdparty_update] get b4 datatable 2019-02-19 11:56:08 +01:00
Terrtia da78d0552d
chg: [Crawler UI Tags] add tag by day + add crawler status + UI onion blacklist 2019-02-19 11:41:45 +01:00
Terrtia 95d63daaf4
chg: [Crawler UI] fix css + bootstrap4 migration 2019-02-15 17:00:07 +01:00
Terrtia 674a9a2591
chg: [UI] font icon v5 + navbar + sidebar 2019-02-08 17:16:58 +01:00
Terrtia c2885589cf
chg: [UI] basic navbar + sidebar + refractor 2019-02-07 17:22:44 +01:00
Terrtia 516238025f
chg: [Crawler] add bootsrap4 src + refractor crawler 2019-02-05 17:16:44 +01:00
Terrtia 423c7b1455
fix: [UI] fix flot js install 2019-02-04 14:56:54 +01:00
Terrtia 655646b7c5
fix: [hiddenServices] catch empty data 2019-01-29 16:43:56 +01:00
Terrtia c1b34bd99c
fix: [Crawler] limit max crawled pages 2019-01-29 15:38:00 +01:00
Terrtia 7a80683f13
chg: [hiddenServices] crawlers status, dynamic update 2019-01-29 14:27:50 +01:00
Terrtia b9572e0085
fix: [Crawler] typo 2019-01-29 12:19:10 +01:00
Terrtia bb301a870c
fix: [Crawler] fix onion blacklist + add crawler info 2019-01-29 12:00:14 +01:00
Terrtia c4c3810037
fix: [hiddenServices] typo 2019-01-08 15:51:39 +01:00
Terrtia 46c1105483
chg: [hiddenServices] auto update last crawled domains 2019-01-08 15:48:40 +01:00
Terrtia 77211c5496
chg: [hashDecoded] add top5 types piechart (v0.1) 2018-12-24 11:42:55 +01:00
Terrtia f023f41ae5
fix: cleaning ... 2018-12-21 15:21:08 +01:00
Terrtia e93a0b637c
chg: [hashDecoded] add option to display decoded files, fix #271 2018-12-21 15:18:07 +01:00
Terrtia 306815a31d
fix: [hiddenServices UI] fix none type 2018-12-20 09:08:14 +01:00
Terrtia 6ee4b7bf93
Merge branch 'master' into update 2018-12-19 15:23:34 +01:00
Terrtia 4e08aaa80f
chg: [UI Crawled Domains] show crawled domains by date range (options: domains tags, up domains, down domains) 
todo: Shows domain history
 2018-12-17 15:58:48 +01:00
Terrtia 31a8dfe0b3
fix: [AIL] use only relative paths 
pastes duplicates are fixed on the fly
 2018-11-21 16:45:25 +01:00
Terrtia d9b776d58a
merge master 2018-11-14 15:55:42 +01:00
Sami Mokaddem 62e46b6dc1
Merge pull request #287 from CIRCL/term_management 
Term management upgrade
 2018-11-08 14:10:25 +01:00
Terrtia 173c0a15c6
fix: [term_management] delete tags 2018-11-08 12:01:40 +01:00
Terrtia 4e906ff841
fix: [term_management] delete emails 2018-11-08 10:41:37 +01:00
Terrtia 0adb8b4624
chg: [terms_management] delete terms tags 2018-11-07 15:37:25 +01:00
Terrtia 46614c97c8
chg: [terms_management] delete email 2018-11-07 11:42:31 +01:00
Terrtia 2c46c7f2ab
fix: [tag export] limit custom tag lenght 2018-11-07 10:17:57 +01:00
Terrtia f91aba5f6f
chg: [tag export] UI, add custom tags 2018-11-07 09:41:58 +01:00
Terrtia 0ad80a22ca
fix: [update_thirdparty] upgrade github source 2018-11-06 17:04:02 +01:00
Terrtia a3fa5a4dcf
chg: [auto push tag] delete unwanted tags 2018-11-06 16:08:58 +01:00
Terrtia 979472df15
chg: [Terms] tag tracked terms 2018-11-06 13:38:37 +01:00
Terrtia af1ff6225e
fix: [Search] tags display 2018-11-05 10:36:58 +01:00
Terrtia 5c67297dc8
fix: [Search] Improve search performance 
note: search by paste name is temporary disabled
 2018-11-05 10:13:48 +01:00
Terrtia 5872cf9196
fix: [Scripts] Remove absolute path 2018-11-02 16:07:27 +01:00
Terrtia ea7fd8d54c
fix: [Duplicate] show duplicate with relative path 2018-10-24 15:47:03 +02:00
Terrtia 6328cc22b7
chg: [Crawler] add domains blacklist 2018-09-28 16:29:09 +02:00
Terrtia b3a6dc8487
fix: [Crawler] remove hardcoded url 2018-09-28 15:42:06 +02:00
Terrtia 3dd20af4d1
Merge branch 'master' into onion_crawler 2018-09-28 15:29:24 +02:00
Terrtia 82e6df4b94
chg: [Crawler] domains stats + logs + clean 2018-09-28 15:23:27 +02:00
Terrtia ecb2857151
chg: [Crawler] css + limit splash RAM 2018-09-27 16:47:48 +02:00
Terrtia ff0affc679
fix: [Flask] remove hardcoded url 2018-09-26 11:59:51 +02:00
Terrtia 3a72f1548d
fix: [Flask] remove hardcoded url 2018-09-25 16:46:06 +02:00
Terrtia b3923428e5
fix: [Flask] remove hardcoded url 2018-09-25 14:37:14 +02:00
Terrtia e4a5f66a10
Merge branch 'master' into onion_crawler 2018-09-24 16:35:04 +02:00
Terrtia 874824a589
fix: [Crawler] clean 2018-09-24 16:28:55 +02:00
Terrtia 50c81773e9
chg: [Crawler] add launcher and install 2018-09-24 16:23:14 +02:00
Terrtia 6edc1ddbeb
chg: [HiddenServices] get domain related to other domains 2018-09-21 10:34:06 +02:00
Terrtia 61b4c9e579
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:56:29 +02:00
Terrtia 5eadc2755f
chg: [Flask] add prefix in config to flask routes 2018-09-20 10:38:19 +02:00
Terrtia 5732d7b134
fix: [hashDecoded] range_type_json 2018-09-19 15:36:33 +02:00
Terrtia 912f2352ab
fix: [crawled screenshot] fix img error 2018-09-18 16:58:49 +02:00
Terrtia ce63d81878
chg: [UI] pixelate crawled screenshot by default 2018-09-18 11:51:21 +02:00
Terrtia f5b648d72a
pixelate paste screenshot 2018-09-18 11:03:40 +02:00
Terrtia 094b211259
fix: [showpaste] path transversal 2018-09-17 15:42:22 +02:00
Terrtia 0c63f2f24f
chg: [Crawler] catch server response 2018-09-17 15:35:06 +02:00
Terrtia 6f0817365a
chg: [Crawler UI] display domain information 2018-09-12 09:55:49 +02:00
Terrtia ca982e13e1
chg: [Crawled Domain] show crawled pastes by domain 2018-08-27 14:34:08 +02:00
Terrtia 40772a5732
fix: merge 2018-08-27 11:30:19 +02:00
Terrtia 823c33174a
Merge branch 'master' into onion_crawler 2018-08-27 11:20:51 +02:00
Terrtia d42dd118a4
chg: [Domain crawled] add random screenshot 2018-08-27 11:02:39 +02:00
Terrtia 6c981ea5e9
fix: [HashDecoded] typo 2018-08-24 11:58:54 +02:00
Terrtia ab452841b1
fix: [HashDecoded] change template title 2018-08-24 11:51:16 +02:00
Terrtia e6b583ee7a
chg: [Hash] add Help on graph node 2018-08-24 11:40:09 +02:00
Terrtia a586548047
Merge branch 'master' into base64 2018-08-24 10:35:28 +02:00
Terrtia 7e24943537
chg: [Crawler] crawler accept all kind of domains 2018-08-24 10:13:56 +02:00
Terrtia e9580d6775
chg: [Crawler] change BDD, save i2p links 2018-08-21 15:54:53 +02:00
Terrtia ed559d9f4a
chg: [Showpaste] add screenshot + improve onion db 2018-08-16 17:24:39 +02:00
Terrtia 54cc4f3723
fix: [UI-Showpaste] show raw paste, force text/plain mimetype 2018-08-03 09:15:34 +02:00
Terrtia 911cff70da
chg: [UI] add AIL icon, fix li links 2018-08-02 10:44:47 +02:00
Terrtia deaebaa4fd
fix: [UI-Hash] pie chart json 2018-07-27 13:51:29 +02:00
Terrtia 009f87ee10
[fix: [UI-hash] fix graph node zoom 2018-07-27 11:27:41 +02:00
Terrtia b02cbee179
chg: [UI-dashboard] add logs number configuration 2018-07-26 11:35:54 +02:00
Terrtia 0a82482db4
chg: [UI-dashboard] convert time to local 2018-07-26 10:34:43 +02:00
Terrtia 5ad5939bd5
chg: display last logs 2018-07-25 16:48:44 +02:00
Terrtia 03a37af5cf
chg: [Hash] add reprocess regex order 2018-07-25 10:47:36 +02:00
Terrtia 8c97c70a3f
Merge branch 'master' into base64 2018-07-24 15:21:32 +02:00
Terrtia 4b8c26dec8
fix: [hive auto export] fix #221 2018-07-24 14:46:35 +02:00
Terrtia 3479923f96
fix: [UI-Hash] pie decoder 2018-07-23 11:43:48 +02:00
Terrtia 9c03c40482
chg: [UI-Hash] add decoder pie 2018-07-23 11:11:52 +02:00
Terrtia 1fbd4109c2
chg: [UI-Hash] add decoded type on graph 2018-07-20 15:48:56 +02:00
Terrtia bd5f83f0eb
chg: refractor base64 encoded to hash 2018-07-20 10:32:52 +02:00
Terrtia e25dd52d64
add: Binary decoder 2018-07-18 11:45:19 +02:00
Terrtia 524d5fe604
add: hash download 2018-07-18 10:09:35 +02:00
Terrtia d9c924c38d
chg: use json to submit hash 2018-07-17 14:23:49 +02:00
Terrtia 412b012ddf
add: hash line_graph (nb/day) 2018-07-16 16:45:36 +02:00
Terrtia 6f69da0c0d
fix: paste submit css 2018-07-16 10:32:41 +02:00
Terrtia fce324e370
fix: release tiemeout alarm, chg:icon graph node 2018-07-13 11:54:14 +02:00
Terrtia 87b7facba8
add: base 64 node graph 2018-07-12 17:07:17 +02:00
Alexandre Dulaunoy 9dc211dbe9
chg: [UI] Typo fixed as requested by our favourite German. 2018-07-10 10:49:02 +02:00
Terrtia fd02085495
add: show hash info 2018-07-09 17:07:59 +02:00
Terrtia c2a976e907
add: stacked barchart, resize chart 2018-07-09 11:47:04 +02:00
Terrtia 0e5a7d8d47
add date_range stacked bar 2018-07-06 16:54:43 +02:00
Terrtia 3b4e2a18aa
fix: date format https://github.com/CIRCL/AIL-framework/issues/219 2018-07-05 14:45:34 +02:00
Terrtia cd154be6f9
fix: term_management UI bug ,https://github.com/CIRCL/AIL-framework/issues/214 2018-07-05 14:12:47 +02:00
Terrtia f7d38bea16
add: base64 date and type selector 2018-07-05 13:58:40 +02:00
Terrtia b6d855b963
change import d3.js to d3.min.js 2018-06-29 15:58:24 +02:00
Terrtia adfc8cf260
add d3 and daterange js 2018-06-29 15:52:13 +02:00
Terrtia a0a639276c
fix submit paste 2018-06-29 15:29:02 +02:00
Terrtia 98d0235fc3
Merge branch 'master' into base64 2018-06-29 14:57:46 +02:00
Terrtia 5908710b34
fix: update submitted tags link 2018-06-29 11:29:52 +02:00
Terrtia cb807bf2c6
chg: change base64 icon 2018-06-29 11:28:35 +02:00
Terrtia 16c700ddb4
new: add base64 display on paste + base64 statistics 2018-06-29 10:02:29 +02:00
Terrtia 225776c69c
fix: tp and fp, errors messages on paste submission 2018-06-20 10:02:26 +02:00
Terrtia 6a89fac309
fix hive and misp error 2018-06-19 16:39:49 +02:00
Terrtia af9db53171
add tag fp and tp 2018-06-19 15:09:26 +02:00
Terrtia bf72c27711
fix path-event id map 2018-06-19 13:43:26 +02:00
Terrtia cd4c099685
fix db init 2018-06-19 11:54:34 +02:00
Terrtia 103787a07d
fix 2018-06-19 11:43:16 +02:00
Terrtia bc40bb3c38
fix hive connection + add keys sample 2018-06-19 11:42:06 +02:00
Terrtia 0ee36e484f
fix hive connection + submitted dir creation 2018-06-19 11:31:30 +02:00
Terrtia 634bf6e552
add img + fix upload api 2018-06-18 17:16:22 +02:00
Terrtia 704e6f0ad1
change default submit tag, event-path map, publish events 2018-06-18 13:58:31 +02:00
Terrtia aee1b50ef1
fix duplicate export attribute + tag export whitelist 2018-06-15 17:25:43 +02:00
Terrtia 31fc33ab03
misp event creation, auto and manual + the hive alert auto creation, manual case creation 2018-06-14 16:51:06 +02:00
Terrtia 8522d50308
modify navbar + decompress file password 2018-06-11 10:21:11 +02:00
Terrtia c64588f60e
add feature, user can submit paste on the web interface 2018-06-08 16:49:20 +02:00
Terrtia 8e9c77b364
verify file upload extention 2018-06-06 10:05:25 +02:00
Terrtia 6143bc3dce
submit users input paste 2018-06-05 16:58:04 +02:00
Terrtia c41a56e4a2
clean Flask_tags + gitignore 2018-06-01 13:36:08 +02:00
Terrtia 2b310d4367
clean 2018-06-01 11:26:45 +02:00
Terrtia cd6734f704
fix install 2018-05-31 15:35:20 +02:00
Terrtia 884d655b0c
taxonomies and galaxies update via LAUNCH 2018-05-31 13:53:29 +02:00
Terrtia 5c50de45de
add taxonomies and galaxies update and install 2018-05-31 10:07:23 +02:00
Terrtia 8a87b7b082
galaxy tag info + fix 2018-05-30 16:18:58 +02:00
Terrtia dbe67e094b
galaxies synonyms 2018-05-29 17:17:02 +02:00
Terrtia f3f6b76428
galaxy description and tags 2018-05-29 08:47:46 +02:00
Terrtia f5cae0d99c
taxonomie + add tags + tags display 2018-05-23 16:58:56 +02:00
Terrtia b9eb3ed9ba
add and delete tags features 2018-05-17 15:33:06 +02:00
Terrtia 4fbbf8bbd2
display tag on important paste and search 2018-05-17 11:00:05 +02:00
Terrtia 0d164b6f4c
add tag js and css 2018-05-16 14:48:15 +02:00
Terrtia b1b7d33c96
tags 2018-05-16 14:39:01 +02:00
Terrtia f66a528bc2
fix Duplicate, save list of duplicates on disk + prevent empty hash creation 2018-05-15 23:28:47 +02:00
Terrtia 225fe76c96
fix duplicate + get raw paste 2018-05-11 16:07:30 +02:00
Terrtia 142c1dc9b0
clean 2018-05-09 13:03:46 +02:00
Terrtia 3841c159a0
change leveldb to ardb 2018-05-07 14:50:40 +02:00
Terrtia c17b9402f7
clean 2018-05-04 14:25:47 +02:00
Terrtia 96a3df42bc
decode with redis connection 2018-05-04 13:53:29 +02:00
Terrtia a900d5f08d
add python 2 to 3 bash + fix Duplicate + clean 2018-05-03 16:21:33 +02:00
Terrtia c52caebe7c
improve Duplicate + tlsh + add time out handler + debug + clean 2018-05-02 17:07:10 +02:00
Terrtia c3428df5dc add apiKeys module 2018-04-26 14:42:39 +02:00
Terrtia 8738b7cf75 fix track term and avoid duplicate mail address 2018-04-24 16:44:37 +02:00
Terrtia f93fe9aeb2 add bitcoin module 2018-04-23 14:51:53 +02:00
Terrtia 41eb250278 fix python 3 compqtibility issues 2018-04-20 10:42:19 +02:00
Terrtia 8571a86344 frontend python 3.5 upgrade 2018-04-17 16:06:32 +02:00
Philipp Schmied 4835589ff4 Fixed typo 2018-03-01 08:50:27 +01:00
Philipp Schmied 29608ab29b Fixed e-mail notificatins for sets and regexes 2018-02-28 11:31:16 +01:00
Philipp Schmied b6fcf51309 Flask_terms.py: Fixed mapping of notifcation e-mails to terms with uppercase letters 2018-02-28 09:19:27 +01:00
Philipp Schmied 5b1f0b0212 Implemented email notifications (bin: config.cfg additions, email sending via analyzer scripts; var: Changes to add notifications via terms management); terms_management: Fixed click handlers not being added to all tracked terms. 2018-02-27 15:12:02 +01:00
Alexandre Dulaunoy abdf41fe23
Merge pull request #164 from mokaddem/showDiff 
Show differences between duplicates
 2018-02-21 22:25:16 +01:00
Sami Mokaddem 9125930a77 fix: canvasjs changed its folder hierarchy 2017-12-14 11:47:11 +01:00
Sami Mokaddem 304213cfa4 update: Possibility to change the authorized max line length in the config for the diff of duplicates 2017-12-12 11:00:26 +01:00
Sami Mokaddem 3fe7ecf75a feature: diff in the web interface for duplicates pastes. 2017-12-12 10:48:21 +01:00
mokaddem 9cab76cf88
Merge pull request #162 from mokaddem/sentimentBugfix 
Fix: bug between front-end and back-end date representation
 2017-12-11 16:35:38 +01:00
Paul A 4a4a1d478b Quick refactor for the JavaScript 2017-10-22 18:48:54 +02:00
Paul A f6b6777095 Fixed some code for the JavaScript (coding-style) 2017-10-20 15:44:09 +02:00
Paul A 48fc8e3464 Removed duplicate function 2017-10-20 15:32:58 +02:00
Simon Tran a41c5d55c5 Migrated Dockerfile to Ubuntu 16.04 LTS 
Fixed AIL Web dependencies installation
Added Docker documentation
 2017-10-01 02:23:32 +02:00
Sami Mokaddem 2cfb8c173b Fixed from front-end date representation bug 2017-09-27 10:46:20 +02:00
= b9bd651033 Removed useless import 2017-08-23 14:06:59 +02:00
= 3b3f3aa89c Flask: Added support of LevelDB database depending of the year 2017-08-23 12:20:39 +02:00
= d632335760 Automatically start redis_levelDB with the correct date 2017-08-23 10:16:56 +02:00
= bd46c83fb6 BrowseImportantPaste: Fixed a bug where onClickListener was not set when the table was only 1 page long 2017-08-21 11:55:01 +02:00
= 409eb25a64 Deleted data-flow graph and header.html from the index + Added fixed typo where 10min was in fact 3min 2017-08-21 10:51:56 +02:00
Mokaddem fa52292029 Added comments, cleanup and added credentials search progress in Flask 2017-07-20 10:50:24 +02:00
Mokaddem cbb2909156 Added comments 2017-07-20 10:24:48 +02:00
Mokaddem 05417f405c Finished basic link web interface + solved some bugs 2017-07-20 10:04:30 +02:00
Mokaddem ef741f85c0 Added similarity checks + started link webInterface 2017-07-19 11:52:06 +02:00
Mokaddem 882ab88b00 Draft module + web link 2017-07-18 16:57:15 +02:00
Mokaddem 9b8570e40b Dirty draft webpage credential_tracking 2017-07-17 17:26:19 +02:00
Mokaddem f1a1e75c32 Restaured old example webpage 2017-07-17 16:18:21 +02:00
Mokaddem 9558aafe67 Prevent the loss of the 1 message in import_dir.py 2017-07-17 16:14:27 +02:00
Mokaddem b60e164380 Added support of onion in browseWarningPaste 2017-04-26 11:47:02 +02:00
Mokaddem 5e4d0170d4 Bug fixes, jinja2 iter0 and typo 2017-04-26 11:31:08 +02:00
Mokaddem e614da4748 Deleted print + Added curve fill for Mixer graph in Index 2017-04-25 15:52:41 +02:00
Mokaddem 71b2757ed2 New script create_new_web_module permits to rapidly create a new pre-configured webpage from a template. 2017-04-25 15:22:04 +02:00
Mokaddem 6afbf77af0 Modified skeleton template name into rawSkeleton + Added possibility to hide modules in the header. 2017-04-25 12:18:08 +02:00
Mokaddem f6e8582d9d Auto organisation of the navbar -> Web page are auto-inserted. Created skeleton webpage 2017-04-19 15:14:20 +02:00
Mokaddem 32906965d8 Draft of new organisation of the web interface. Usage of Flask's blueprint 2017-04-19 11:02:03 +02:00
Mokaddem da081bae42 Merge branch 'regexInTerms' into production 2017-04-18 16:59:40 +02:00
Mokaddem c8baabd882 Updated TermsFrequency related modules + Fixed bugs (encoding, behaviors, ...) 2017-04-18 15:28:21 +02:00
Mokaddem 2da4c572c7 New feature in Terms: regex and set of words support (draft) 2017-03-28 17:42:44 +02:00
Mokaddem 3261eca836 Fixed all_index newline bug + added print of what's happening 2017-03-15 14:29:49 +01:00
Mokaddem 5b7d047622 Changed os.system by os.mkdir + fixed bug during rotation 2017-03-15 14:05:13 +01:00
Mokaddem 4005b72180 Added backward support + renamed index with timestamp 2017-03-15 11:51:35 +01:00
Mokaddem 381e72ee99 Added display of number of elements inside the index + changed variables names 2017-03-15 10:07:46 +01:00
Mokaddem e4757f5ceb Added possibility to choose the index in search + Updated search page with jinja2 iter0 2017-03-15 09:39:48 +01:00
Mokaddem 41132fe0bc Added support of per_paste for term-management and plot-term 2017-02-28 15:54:39 +01:00
Mokaddem 4071bf2e91 fixed a bug in manage_term (usage of loop.index0) + fixed bug in top_set where graph was not displaying the correct per_paste value 2017-02-28 15:01:48 +01:00
Mokaddem 25adae0104 Added possibility to select in top_terms the data per_paste or not (need module testing) 2017-02-15 16:29:02 +01:00
Mokaddem d4da3a316f Added path of the paste in the log of Categ.py 2017-02-14 10:59:47 +01:00
Mokaddem 29241eb49f Added a line separator between the two graphs in dashboard. 2017-01-09 14:44:14 +01:00
Mokaddem 0e39f516a9 Added sleep in helper for multiple listening interfaces (reduce useless work), Added a simple way to pre-process feed before sending it to the global feed. 2017-01-09 14:12:26 +01:00
Mokaddem 24c5621d29 BUG FIX in Flask_trendingmodule, Fixed member_set when get_top_relevant_data is None. 2016-12-26 16:33:10 +01:00
Mokaddem 1c2169d3bc Improved interface for multiple feeds and refresh_script. 2016-12-26 16:16:44 +01:00
Mokaddem a18c046dea Fixed bug multiple refresh instances and harmonized interface. 2016-12-23 16:15:05 +01:00
Mokaddem 03dddbc359 Reduced refresh rate of processed_pastes, synchro graphs and adjusted max on each graphs. 2016-12-23 16:01:30 +01:00
Mokaddem 97292e0899 Updated web interface to handle new mixer module. and fixed one dependency bug. 2016-12-23 15:44:46 +01:00
Mokaddem faa8f8302d Removed typo in the number of remaining pastes to display in browseImportantPaste 2016-12-22 09:28:55 +01:00
Mokaddem 9df12cdae8 Added sub-flask in a directory 2016-12-22 08:57:45 +01:00
Mokaddem d30f3ca6c7 Added missing depency datetime in dashboard 2016-12-09 13:53:57 +01:00
Mokaddem 0054be5bab Added missing json dependency in dashboard 2016-12-09 08:50:36 +01:00
Mokaddem 8daa72789e Splitted Flask_server into module related to website sections 2016-12-09 08:46:37 +01:00
Mokaddem d3ed1bbf40 Merge branch 'website-improvement' into production 2016-12-08 10:05:55 +01:00
Mokaddem 1abba4dcf9 Added support of re-plotting the plotted terms 2016-12-08 08:44:10 +01:00
Mokaddem 84dcd5586f Started plotting same data with different dates 2016-11-25 16:42:22 +01:00
Mokaddem a3255d168c ModuleInfo correctly handle CurveManageTopSets, 
Changed queue coloring in dashboard,
ModuleInformation now have a history of executed command and better handle module killing.
 2016-11-25 11:54:16 +01:00
Mokaddem 5d269ea1ee Added date in Duplicate for better distinguish duplicate. Also, added a timeout for the dataTable in duplicate inside the modal. 2016-11-24 15:05:29 +01:00
Mokaddem df8d978088 Added dynamic data loading in dataTable in search.html 2016-11-24 13:31:31 +01:00
Mokaddem 80ca49f018 Prevent browser to request reconnection for the eventStream + Added dynamic data loading in the dataTable of BrowseImportantPastes 2016-11-11 16:51:39 +01:00
Mokaddem 768fb6f2df removed server overhead for paste preview data 2016-11-10 16:41:07 +01:00
Mokaddem 699c819387 fixed bug in show concerned pastes previex en (terms managements) 2016-11-10 16:33:05 +01:00
Mokaddem 94e3034e7d Fixed bun in terms management, where action_button was not bind in other dataTable pages 2016-11-10 15:53:22 +01:00
Mokaddem 7763bfb4c7 Pastes dataTable now display only the latest clicked paste even if there were discarded queries 2016-11-10 15:39:45 +01:00
Mokaddem 2dd69fbb25 Fixed modal background stacking bug + button leading to saved paste bug fixed in search 2016-11-10 14:56:25 +01:00
Mokaddem 317005c2b4 tryed to fix html display problem in tooltip + fixed problem button showPaste in modal not pointing to the paste 2016-11-10 10:30:29 +01:00
Mokaddem 34221b083f Partially solved the browser freezing problem + tootlip is also binded at a new page generation of the DataTable 2016-11-10 09:28:55 +01:00
Mokaddem acdd1367a3 Added event source for getImportantPaste 2016-10-28 16:55:56 +02:00
Mokaddem c95000866d Dynamic table in search now load all the data + fixed bugs where tooltip where not shown on other than the first page displayed and 
tooltip interpret html (not supposed to...)
 2016-10-28 14:21:08 +02:00
Mokaddem f51808d914 Added support of html formatting in search result dynamic loading table 2016-10-28 09:48:22 +02:00
Mokaddem 4be62a4e2d Added draft of dynamic update table with search result 2016-10-28 09:28:42 +02:00
Mokaddem 340624fe1e Fixed bug in modal show paste (show more button not showing) 2016-10-27 16:18:49 +02:00
Mokaddem 7e7e679ab6 Added dataTable for duplicate in show saved paste 2016-10-27 15:53:45 +02:00
Mokaddem 59a7000527 Bootstrap support in show saved paste 2016-10-27 12:38:39 +02:00
Mokaddem ac254e0e63 Added support of browsing concerned paste in dashboard for all modules + Show in the page what was the concerned module 2016-10-27 11:50:24 +02:00
Mokaddem 1826b170ec Added support of local paste view in dashboard for Mails event only 2016-10-27 11:27:26 +02:00
Mokaddem 7c8d414948 modified provider name into no data when there is no data 2016-10-14 16:47:22 +02:00
Mokaddem 91678179fd Added msgs when queues or feed is not running/coming 2016-10-14 16:33:54 +02:00
Alexandre Dulaunoy 2781a2f0b7 Stuck not stucked 2016-09-21 08:35:20 +02:00
Jerome St-Hilaire ea13f87c2c Fixed dependency fetching for the web interface 2016-09-15 12:14:55 -07:00
Raphaël Vinot acfe9c1336 Update Flask_server.py 2016-08-26 18:06:48 +02:00
Raphaël Vinot 3290b51c91 Update update_thirdparty.sh 2016-08-26 18:05:18 +02:00
Mokaddem 79be8ab934 Fixed introduced bug relative to chart in index webpage. 2016-08-26 15:59:53 +02:00
Mokaddem 0c760d763b Added support of PID in ModuleInformation and index.html 2016-08-26 15:27:37 +02:00
Mokaddem 5c565068a6 Removed auto-reload in dashboard if window in undefined 2016-08-26 11:25:11 +02:00
Mokaddem 181c372131 Updated dependencies 2016-08-26 11:10:39 +02:00
Mokaddem 4cb3b672be Added comments 2016-08-26 10:28:35 +02:00
Mokaddem 86b655100e Deleted useless parsing function 2016-08-26 10:01:14 +02:00
Mokaddem 07d7559060 Added dataTable support in browse paste modal. 2016-08-26 08:30:09 +02:00
Mokaddem 813cc7031a Changed lots of code into zrank 2016-08-25 17:48:12 +02:00
Mokaddem 991176fdfd Suppressed metismenu import. 2016-08-25 16:40:13 +02:00
Mokaddem fb22816236 Added module number info in index webpage 2016-08-25 16:27:32 +02:00
Mokaddem 0b5a3e0cd2 Auto generate graph div + possibility to query all sentiments data. 2016-08-25 15:06:18 +02:00
Mokaddem 21f2c43572 Individualized sentiments-graph loading 2016-08-25 13:11:16 +02:00
Mokaddem f9bf00f022 Removed autokill feature if not specified 2016-08-25 11:09:31 +02:00
Mokaddem 611454ba82 Added color in web-index reprenting modules states. 2016-08-24 18:00:05 +02:00
Mokaddem e3daa84196 Added icon in header 2016-08-23 16:09:34 +02:00
Mokaddem e12244fb53 Added show/hide curve with legend in terms-top-set. 2016-08-23 15:39:13 +02:00
Mokaddem e56187eb7e Added position of each top term in the other sets + highligth ones not present. 2016-08-23 14:52:02 +02:00
Mokaddem a64d447b7b Auto hide button add_to_chart when no data plotted. 2016-08-23 12:09:41 +02:00
Mokaddem e59b7c4e47 Added tooltip in graph for terms-plot-tool 2016-08-23 12:03:23 +02:00
Mokaddem 0eba419651 Added multiple terms plot in terms-plot-tool. 2016-08-23 11:56:32 +02:00
Mokaddem 9f2d672ec8 harmonized tooltip and popover display 2016-08-23 11:20:48 +02:00
Mokaddem 8c956c22b1 Added autohide curves in terms-top-set. 2016-08-23 11:02:36 +02:00
Mokaddem 32dfd2b181 Moved legend place in term-plot-top 2016-08-23 10:16:33 +02:00
Mokaddem f9b27e4013 Fixed bug in curve preventing to track concerned_pastes. Now all tracked words are put in lower case. 2016-08-23 09:50:03 +02:00
Mokaddem 1766ced9e6 Fixed potential XSS problem with pastes display in 'search' and 'important_paste_by_module'. 2016-08-23 09:08:17 +02:00
Mokaddem 047331df3e Added save of paste_baseName in set for searches. This is done in ModuleStats to not surcharge the Global module. 2016-08-23 08:52:54 +02:00
Mokaddem 6be557890c Replaced negative hour by 24h based. 2016-08-23 07:19:51 +02:00
Mokaddem 85ab0e36ff In sentiment analysis. if today top provider set is empty, try to take yesterday data and then takes from all_providers. 2016-08-23 07:11:03 +02:00
Mokaddem 8359a6668f Suppressed useless comms 2016-08-22 21:36:19 +02:00
Mokaddem 2fd2c8af39 Removed commented code 2016-08-22 20:59:56 +02:00
Mokaddem 27ae577e00 Fixed stupid line remove in index.html... 2016-08-22 16:25:52 +02:00
Mokaddem a424069c64 Fixed bug in flask with zrange (was taking the bottom of the set instead of the top leading to get only the min instead of max) 2016-08-22 16:12:20 +02:00
Mokaddem 003c72bd7c Fixed bug with redis where the inserted key was too long, causing a crash + fixed bug taking min instead of max in terms-top-set 2016-08-22 16:01:42 +02:00
Mokaddem 2466f355ab Fixed bug in manage-terms-top-set + added auto sorting in important-paste-by-module 2016-08-22 14:54:37 +02:00
Mokaddem e144e8e1c6 Fixed bug in terms-management-interface (only display one paste) + removed get_language of all pastes due to the huge overhead. 2016-08-22 13:59:12 +02:00
Mokaddem d3c0fc5f94 Added terms trending module and web interface: 
- Curve module has been rewritten
- Added new module curve_manage_top_set
- Added all web-pages
 2016-08-22 13:35:49 +02:00
Mokaddem 841e87eb16 Fixed bug in sentiment trending, if there was no data for the avg -> there was one NaN leading to no display in the gauge. 2016-08-20 18:48:24 +02:00
Mokaddem ee8f26c7c4 In sentiment trending, if there are no providers in the today\'s set, get data from random providers 2016-08-20 18:14:19 +02:00
Mokaddem 2ed60cb9bf Fixed typo 2016-08-20 16:44:36 +02:00
Mokaddem 3f1cf705fc Deleted alert() 2016-08-20 16:41:49 +02:00
Mokaddem 1c96eac2e4 Enter simulate button click on input field for terms management and plot tool 2016-08-20 16:38:51 +02:00
Mokaddem 465d18fc73 Added terms-plot tool 2016-08-20 16:23:06 +02:00
Mokaddem fb3dea6bd2 Finnished terms-plot-top-set. Also added possibility to add terms to track or blacklist. 2016-08-20 15:59:22 +02:00
Mokaddem a5117d7cad Added top set webpage 2016-08-20 11:46:22 +02:00
Mokaddem 0b6828fb16 Correctly handle and display add or remove a term in term-frequency + started sketch of terms-plot tool. 2016-08-19 16:53:46 +02:00
Mokaddem 8d41b109e7 Added query of the data for the new tracked word 2016-08-19 14:33:01 +02:00
Mokaddem b78effe779 Added draft of terms_management + Added copyright in readme 2016-08-19 13:34:02 +02:00
Mokaddem 52b7124a6b - Modified redis connection (from levelDB to redis). 
- Added term frequency in curve.
- Modified ModuleStats and WebStats to use redis special command (incrby instead of get-set and zset) and Flask to perform the correct queries.
- Added panel color in sentiment trending
 2016-08-18 15:34:19 +02:00
Mokaddem 24072ece36 Added support of pasteName in search + added header in search.html + supressed mixed indent and trailing in Flask.py 2016-08-17 14:21:31 +02:00
Mokaddem c10f28d9df Updated isolated launch scripts + Added switch displaying queue in index 2016-08-17 13:46:01 +02:00
Mokaddem 23b65eddca Auto unchecked info checkbox in index 2016-08-17 11:25:05 +02:00
Mokaddem 4d09ab0f6b Modified icon sentiment in web-header. 2016-08-17 11:13:37 +02:00
Mokaddem 44f54b7b81 Added dependencies 2016-08-17 11:04:48 +02:00
Mokaddem f0b527167a Added script flexgauge 2016-08-17 10:50:30 +02:00
Mokaddem c12e67a35a Added tooltips + slight gauge modification 2016-08-17 10:43:43 +02:00
Mokaddem 894b9efda9 Added drop of really long line in sentiment-analysis module + Added description of sentiment module. Also, fixed bug in webpage sentiement-trending concerning avg and date range. 2016-08-17 09:46:25 +02:00
Mokaddem 1084e45f1b Added draft of filter in sentiment analysis (Discard syntaxical languages) + Added nice tooltip for sparkline. Trending displays avg in function of the number of elements processed and not for the complete week + fixed bug in gauge and canvasjs (was performing avg with only 1 graph instead of all 8). 2016-08-16 16:33:02 +02:00
Mokaddem ecd834ffb6 Added autoreload in sentiment-trending. 2016-08-16 11:27:49 +02:00
Mokaddem bb677b9061 Added avg text in sentiment-trending + fixed date range bug 2016-08-16 11:23:14 +02:00
Mokaddem cdd08d7df0 Added all providers for plot_tool in moduleStats + fixed date range bug 2016-08-16 11:07:36 +02:00
Mokaddem 8fa3c2b147 Added stacked choice or not for plot-tool (sentiments) 2016-08-16 10:49:05 +02:00
Mokaddem 4baac0b994 Tool-plot for sentiment working + Splitted script and webpage 2016-08-16 10:31:18 +02:00
Mokaddem 1e7547d16c Created draft of search for plot tool 2016-08-15 16:32:02 +02:00
Mokaddem 88ed8a239d Linked plot button with server 2016-08-15 15:54:53 +02:00
Mokaddem 82f70e61f4 Added provider checkbox 2016-08-15 15:16:31 +02:00
Mokaddem f90987d284 Added date on slider 2016-08-15 14:53:46 +02:00
Mokaddem d8c84ec674 Added link to data in analysis trending + fixed tooltip bug in sparkline 2016-08-15 12:57:39 +02:00
Mokaddem 6c7f660052 Added support of week and day average. 2016-08-13 22:17:11 +02:00
Mokaddem 1a5158b081 Added sentiment analyser module (draft) 2016-08-13 15:24:57 +02:00
Mokaddem 0431ecf434 Modified webpage display and added js. 2016-08-12 10:22:24 +02:00
Mokaddem 9c3b573af2 Splited js from html + Added some ID + small display modification. 2016-08-12 07:49:24 +02:00
Mokaddem 87a86b38d1 Added page Plot-tool 2016-08-11 21:08:43 +02:00
Mokaddem c028ca80da Separated options from the plot funcion for sparkline. 2016-08-11 17:57:23 +02:00
Mokaddem 98c4cacb98 Finished first panel in sentiment-today. 2016-08-11 17:22:50 +02:00
Mokaddem 8682fcc29c Added empty webpages for sentiments modules. 2016-08-11 11:34:22 +02:00
Mokaddem 5612a77af9 Separated webpage header code from all webpage 2016-08-11 10:55:51 +02:00
Mokaddem 0ba5637e06 Added bar chart hover binder in trending charts 2016-08-10 11:16:56 +02:00
Mokaddem 24318f6a86 Added the part 'other' in Trending chartscd 2016-08-10 10:42:06 +02:00
Mokaddem 1ca8cf69f6 Added trendingchart.js 2016-08-10 09:38:15 +02:00
Mokaddem b249676fe9 Removed console log 2016-08-09 16:54:53 +02:00
Mokaddem 507c83e06d In index webpage, reset value graph if no paste received 2016-08-09 16:48:57 +02:00
Mokaddem 943016d6ca Fixed one of the small display bug in DUplicate_ssdeep_v2 (was displaying [tlsh, tlsh] instead of simply tlsh) 2016-08-09 15:06:55 +02:00
Mokaddem 0d16a8beb7 Fixed bug where Listener was not placed if the row was not displayed (BrowseWarningPaste) 2016-08-09 12:30:40 +02:00
Mokaddem e2ba4060ee restaured loading gif when new tab is shown (BrowseWarningPaste) 2016-08-09 12:07:20 +02:00
Mokaddem a80f6ca0c2 Bug fix related with redis: 
Fixed typo key in redis for module creditcard and sqlinjection
Modified Curve redisLvlDb server
Modified Url.py so that it forwards name of protocol from saved protocolsfile
Added Cache control in Flask
Modified key-tab name into keys-tab
 2016-08-09 11:59:36 +02:00
Mokaddem 084737dd26 Fixed bug in flask not correctly displaying the number of paste/providers 2016-08-08 11:39:17 +02:00
Mokaddem 78c611fead Added warning_paste module and created related webpages. 
Fixed a Faup bug in credential (multiple instanciation) and added correc populate_set_out in concerned modules (creditcard, credential, ...).
Linked browse_warning_paste module and Flask function with redis (created new sets).
 2016-08-08 09:17:44 +02:00
Mokaddem 3523d79a05 Started browse important paste feature. Still need to link with redis. + Fixed a bug in Flask (String to list parser) 2016-08-04 16:57:34 +02:00
Mokaddem 55ef917ef1 Added few comments 2016-08-04 12:10:56 +02:00
Mokaddem d9316771cd Draft: added new duplicate hash comparison - tlsh 2016-08-04 11:55:38 +02:00
Mokaddem 50d2848a40 Tryed to fix the glob_tabvar bug 2016-08-02 16:39:57 +02:00
Mokaddem 9e82ec82b7 Fixed a bug in the hover of provider graph (happened when there was an 'other' in other graphs) 2016-08-02 16:17:25 +02:00
Mokaddem 9d37ebc860 Fixed a division by 0 in Flask + Harmonized webpage item positioning 2016-08-02 16:00:31 +02:00
Mokaddem f3cf2f853a Fixed a bug in the hover of providers graph. 
When we click on a part in providers graphs, we hover an item, then we click on a random part not belonging to providers graphs, and we re-hover one part of the providers graphs, there was an undefined variable (The random click overrided it).
 2016-08-01 11:08:53 +02:00
Mokaddem feb1717dc6 fixed size bug, which in some cases display the total size instead of the average + few cleanup + commented out log scale in moduleStats.js and added large number division in case of div by 0 2016-07-29 16:52:50 +02:00
Mokaddem a827e8ca3a Big refactoring and added lot of comments + fixed bug attaching multiple binder on barChart 2016-07-29 14:30:41 +02:00
Mokaddem e8fae17cfd Fixed small race condition bug in ModuleStat.py and Added highlight of bar belonging to the same serie in moduleStats.js 2016-07-29 10:36:44 +02:00
Mokaddem c2bf2ba630 Added tooltip in moduleStat above each bar chart + Added highlight of the selected bar. (Still not working for category 'Other') 2016-07-28 16:35:30 +02:00
Mokaddem 09fdaa3e39 Added tooltip for moduleStats bar chart 2016-07-28 12:06:54 +02:00
Mokaddem acec508f53 fixed multiple binding bug in moduleStats + Added multiple bar display for 'other pie part' + Added ModuleStats tab in all other pages 2016-07-28 09:52:07 +02:00