CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
1,310 Commits
7 Branches
44 Tags
110 MiB
Commit Graph

1310 Commits (9c6a1ea8ecc506c715028bd874b8be850b00838e)

Author SHA1 Message Date
Thirion Aurélien 9c6a1ea8ec
Merge pull request #349 from kovacsbalu/fix-paste-encoding 
Fix #314
 2019-05-16 14:30:33 +02:00
kovacsbalu f61d830678 Use default encoding error from redis. 2019-05-16 14:24:03 +02:00
Thirion Aurélien e09d49a658
Merge pull request #350 from kovacsbalu/fix-crawler-rotation 
fix: [crawler] rotation
 2019-05-16 13:31:09 +02:00
kovacsbalu 7765ab92e0 Hopp, single quote :) 2019-05-15 10:00:51 +02:00
kovacsbalu 6092f482e6 Fix crawler rotation 
Before this, crawler processed prioritized onions and after all starts prioritized regular.
 2019-05-15 09:57:18 +02:00
kovacsbalu 05079c143c Fix #314 
Replace char on redis encoding error.
Try to use local file on other error.
 2019-05-14 16:06:20 +02:00
Terrtia 0389b9c23b
chg: [crawler] manual/auto crawler: always save screenshots 2019-05-13 14:24:16 +02:00
Terrtia 254441f193
chg: [crawler] manual/auto crawler: always save screenshots 2019-05-13 13:56:43 +02:00
Terrtia 2606220c2b
fix: [faup] fix new return types (bytes to str) 2019-05-06 13:38:13 +02:00
Terrtia a4c03b4ba4
fix: [Crawler] force domains/subdomains lower case (rfc4343) 2019-05-06 11:46:20 +02:00
Terrtia cc61c99290
fix: [showpaste] fix: #346, avoid None screenshots 2019-05-06 10:00:39 +02:00
Terrtia 97fa1ddca4
fix: [python requirements] rename file 2019-05-03 17:02:56 +02:00
Terrtia 9b66fa703f
fix: [crawler] typo: domains down 2019-05-03 10:04:20 +02:00
Terrtia 1bf752fbfc
Merge branch 'master' of https://github.com/CIRCL/AIL-framework 2019-05-02 13:44:29 +02:00
Terrtia cf74adec65
chg; [doc crawler] use the torproject torrc 2019-05-02 13:43:54 +02:00
Steve Clement 71a6860323 fix: [travis] LAUNCH.sh needs -l to launch... 2019-05-01 22:03:32 +09:00
Steve Clement 9f65bc3788 chg: [travis] Travis has his own venv where it installs "stuff". Now we detect and us it in the launcher. 2019-05-01 21:31:47 +09:00
Steve Clement 57a344cfab fix: [travis] Next round of travis fixes. LAUNCH.sh is the only launch script needed. 
chg: [installer] Be way more quiet, watching unzips is only fun during development.
chg: [installer] Make the arch one +x
 2019-05-01 20:49:07 +09:00
Steve Clement 34ff6116a6 fix: [installer] Avoid doing funky **sudo pip install** moves, it breaks python on package managed python installs, if ever, use the **--user** flag. 2019-05-01 20:36:41 +09:00
Steve Clement 1b5e4f1bcb chg: [travis] Require Python 3.6 to make build faster. 2019-05-01 20:17:53 +09:00
Steve Clement e5616312d9 chg: [doc] Some stats on build status/gitter etc. 2019-05-01 18:47:16 +09:00
Steve Clement d039b8e5e0 fix: [travis] Try and require xenial (16.04) and see if it works better. 2019-05-01 18:40:58 +09:00
Steve Clement d10d4a5cc5 fix: [travis ] There are issues on the 14.04 build system of Travis. This fixes it temporarily. 2019-05-01 18:39:27 +09:00
Alexandre Dulaunoy 0f2b139625
chg: [hashDecoded] cleanup for the VT message + PEP 2019-04-30 15:50:06 +02:00
Terrtia f30befa0eb
fix: [Onion] typo 2019-04-29 11:46:28 +02:00
Terrtia a183695ddb
chg: [crawler UI] bootstrap 4 migration: show domains by daterange + bug fix hidden services tags 2019-04-26 15:14:29 +02:00
Terrtia ff14fe6279
fix: [submit paste] remove item full path, use rel path 2019-04-26 10:22:02 +02:00
Terrtia 3e6fbc8f79
fix: [paste submit] update broken endpoint + get item date 2019-04-26 10:04:07 +02:00
Terrtia 3207949c5f
fix: [update v1.5] create update config file 2019-04-25 17:17:12 +02:00
Terrtia da9c6a3b8a
fix: [update v1.5] fix update thirdparty 2019-04-25 17:06:10 +02:00
Terrtia b4e352bb74
fix: [Update] typo 2019-04-25 16:43:17 +02:00
Alexandre Dulaunoy ecd14b91d9
Merge pull request #342 from CIRCL/advanced_crawler 
Advanced_crawler + tag by daterange + bootstrap 4 migration + bugs fix
 2019-04-25 15:39:19 +02:00
Terrtia 41e6b4ec59
fix: [v1.5 background update screenshot] remove duplicate files 2019-04-25 14:39:38 +02:00
Terrtia 499c07a93e
fix: [update v1.5] remove unbuffer 2019-04-25 14:21:02 +02:00
Terrtia fc2c1422ff
fix: [Crawler] unpack_url 2019-04-25 13:54:06 +02:00
Terrtia 39d0bfa65a
chg: [overview] update db doc 2019-04-24 16:42:05 +02:00
Terrtia 50944b1d61
fix: [LAUNCH] fix typo 2019-04-24 16:37:44 +02:00
Terrtia e6dca7f8bf
chg: [update v1.5] add background update: screenshots_crawled 2019-04-24 16:19:35 +02:00
Terrtia 9868833c77
chg: [crawled screenshot] use sha256 as filepath 2019-04-24 14:09:04 +02:00
Terrtia 99b9c95638
fix: [update] fix is_fork check 2019-04-23 15:30:47 +02:00
Terrtia 35c188829f
chg: [config update] refractor, auto update, create config if file doesnt exist 2019-04-23 15:04:45 +02:00
Terrtia 4ecff21194
chg: [UI crawler] add splash dashboard 2019-04-23 14:26:26 +02:00
Terrtia 2a1cd4a009
chg: [Onion, crawler config] auto crawler: add config by url, fix onions tagging + filter subdomains 2019-04-23 11:15:34 +02:00
Terrtia 6fdf7c2123
chg: [UI crawler] status/remove auto crawler 2019-04-18 16:57:51 +02:00
Terrtia ecca5b11d0
fix: [UI dashboard] typo 2019-04-18 11:01:11 +02:00
Terrtia de285ed64c
chg: [UI DOC] add dynamic update message + add bootstrap 4 migration message + add update doc (DB) 2019-04-18 10:56:00 +02:00
Terrtia 7a5fac9a43
fix: [UI show domain] fix domain/port unpacking 2019-04-18 09:54:25 +02:00
Terrtia ddaad5db93
chg: [v1.5 background update] create update stats 2019-04-17 17:07:09 +02:00
Terrtia 4bff603d4f
chg: [settings UI] add background update progress 2019-04-17 14:39:29 +02:00
Terrtia 68077316eb
chg: [git directory] add git helper 2019-04-16 17:26:35 +02:00