Raphaël Vinot
|
9b4bbb2450
|
fix: Missing cast
|
2020-02-16 23:51:04 +01:00 |
Raphaël Vinot
|
7d2f50174d
|
chg: Bump dependencies, Improve HTTP errors handling
Fix https://github.com/CIRCL/lookyloo/issues/66
|
2020-02-16 23:38:42 +01:00 |
Raphaël Vinot
|
09b3b16910
|
fix: Properly check keys in cache
|
2020-02-11 17:26:01 +01:00 |
Raphaël Vinot
|
794d9745ab
|
fix: wrong variable in last commit
|
2020-02-11 17:09:15 +01:00 |
Raphaël Vinot
|
e7b97d23b7
|
fix: Better handling of cache issues
|
2020-02-11 17:03:25 +01:00 |
Raphaël Vinot
|
4234e44470
|
new: Add initial redirects columns
|
2020-02-03 22:25:48 +01:00 |
Raphaël Vinot
|
89edef68ab
|
new: Improve index
|
2020-02-03 18:30:41 +01:00 |
Raphaël Vinot
|
c9cbf27acc
|
chg: Fix typing
|
2020-01-24 11:25:53 +01:00 |
Raphaël Vinot
|
f1309ce5e3
|
new: Upload optional cookie file
|
2020-01-24 10:17:41 +01:00 |
Raphaël Vinot
|
f1d83d29cf
|
new: Add support for user defined cookies, bump deps
|
2020-01-23 10:52:50 +01:00 |
Raphaël Vinot
|
d34233ad5c
|
chg: Use poetry instead of pipenv
|
2020-01-21 17:39:18 +01:00 |
Raphaël Vinot
|
2d82303ef6
|
chg: Support cookies from/to splash (no UI yet)
|
2020-01-21 11:47:36 +01:00 |
Raphaël Vinot
|
0aaa814224
|
new: Initial commit for cookies tracking. No update on the view yet.
|
2020-01-10 17:28:15 +01:00 |
Raphaël Vinot
|
dec6920cb8
|
chg: Add typing
|
2020-01-06 15:32:38 +01:00 |
Raphaël Vinot
|
5cfe5306b2
|
fix: Skip UA fetch on cloudflare website until the lib is updated
|
2019-12-17 14:29:58 +01:00 |
Raphaël Vinot
|
4c86b65c56
|
fix: Avoid bypass with "http://user:pwd@host.tld"
|
2019-07-05 18:42:18 +02:00 |
Raphaël Vinot
|
a6693535de
|
fix: Allow to disable scraping private IPs.
|
2019-07-05 16:27:23 +02:00 |
Raphaël Vinot
|
8165913206
|
fix: Initialize cache only once
|
2019-06-25 18:08:52 +02:00 |
Raphaël Vinot
|
320105aca1
|
fix: Get most common user agents from willshouse.com behind cloudflare
|
2019-05-27 15:34:26 +02:00 |
Raphaël Vinot
|
66545c26a5
|
new: keep track of metadata about OS and Browser when scraping
|
2019-04-07 23:54:16 +02:00 |
Raphaël Vinot
|
12b8e4f949
|
chg: Improve async processing
|
2019-04-05 16:12:54 +02:00 |
Raphaël Vinot
|
da3d1fe392
|
fix: Avoid loading the cache multiple times
|
2019-04-05 15:07:22 +02:00 |
Raphaël Vinot
|
6b10e74470
|
fix: do not initialize the redis cache more than once.
|
2019-04-05 14:48:25 +02:00 |
Raphaël Vinot
|
0d937b02c9
|
fix: Properly remove files if the scraping failed
|
2019-04-04 00:41:15 +02:00 |
Raphaël Vinot
|
36c856ef9b
|
new: Add initial support for user-agent
|
2019-03-29 20:11:44 +01:00 |
Raphaël Vinot
|
49654b6b11
|
fix: Automatically remove failed runs.
|
2019-03-27 10:21:37 +01:00 |
Raphaël Vinot
|
fdcc566e05
|
chg: Slight cleanup
|
2019-02-18 14:29:15 +01:00 |
Raphaël Vinot
|
747ce7e52d
|
Merge branch 'anonyme' of https://github.com/Th4nat0s/lookyloo into Th4nat0s-anonyme
|
2019-02-18 13:53:04 +01:00 |
Raphaël Vinot
|
9f2b7f7b93
|
new: Add error page
|
2019-02-18 13:52:48 +01:00 |
Th4nat0s
|
8afcc30840
|
clarifie index enable/disable
|
2019-02-17 21:45:23 +00:00 |
thanat0s
|
f029b5d324
|
Add noindex button
|
2019-02-17 17:43:25 +00:00 |
Raphaël Vinot
|
189b78e20f
|
chg: pre-load the existing reports
|
2019-02-01 16:11:16 +01:00 |
Raphaël Vinot
|
1d244ef456
|
chg: Refactor code organisation
|
2019-01-30 14:30:01 +01:00 |
Raphaël Vinot
|
e5de1a9958
|
fix: Make sure sanejs is optional
|
2019-01-30 13:14:16 +01:00 |
Raphaël Vinot
|
6bc316ebcf
|
new: Initial commit for client and async scraping
|
2019-01-29 18:37:13 +01:00 |
Raphaël Vinot
|
e14a0150a0
|
chg: Assign a UUID for each scraped page, allow permalinks
|
2019-01-29 11:42:56 +01:00 |
Raphaël Vinot
|
945511d6b3
|
chg: Use PySaneJS to query SaneJS
|
2019-01-27 17:17:31 +01:00 |
Raphaël Vinot
|
bbb8c5343f
|
chg: Cleanup, use pipfile
|
2019-01-23 15:13:29 +01:00 |
Raphaël Vinot
|
2825096dc7
|
fix: Do not fail if SaneJS is not available
|
2018-07-20 11:36:54 +02:00 |
Raphaël Vinot
|
3c9b28b704
|
new: Add integration with SaneJS
|
2018-07-19 18:18:22 +02:00 |
Raphaël Vinot
|
5202630278
|
fix: Improve URL overlay
|
2018-06-29 14:03:52 +08:00 |
Raphaël Vinot
|
c8fb313d28
|
new: Download screenshot of the page.
|
2018-05-10 09:48:48 +02:00 |
Raphaël Vinot
|
6329e7747f
|
fix: Wrong URL path
|
2018-04-09 11:16:22 +02:00 |
Raphaël Vinot
|
f6345e47e1
|
new: allow to download the URLs list in a text file
|
2018-04-06 18:06:41 +02:00 |
Raphaël Vinot
|
d7cf92db21
|
new: Close button for the overlays
|
2018-04-05 22:59:45 +02:00 |
Raphaël Vinot
|
72759850ac
|
chg: Use d3v5, cleanup.
|
2018-04-05 11:17:26 +02:00 |
Raphaël Vinot
|
fb195971e8
|
chg: use pathlib everywhere, remove old tmpfiles
|
2018-03-22 18:33:42 +01:00 |
Raphaël Vinot
|
247a4a26b0
|
chg: Change cookie name, update legend, remove old pickled trees
Fix #18
|
2018-03-22 17:22:19 +01:00 |
Raphaël Vinot
|
32ba719d5c
|
new: Add Icons
|
2018-03-15 09:33:38 +01:00 |
Raphaël Vinot
|
22f1474120
|
new: Link to URL overlay
|
2018-02-23 18:45:12 +01:00 |