Raphaël Vinot
|
1882c9d71a
|
fix: Please mypy
|
2020-05-15 11:39:16 +02:00 |
Raphaël Vinot
|
d3971cc47c
|
new: Add stats
|
2020-05-13 17:31:27 +02:00 |
Raphaël Vinot
|
cd972076b5
|
new: Add url to do a full export, and get the page
|
2020-05-12 16:53:10 +02:00 |
Raphaël Vinot
|
754ae9354d
|
new: Add email notification feature
|
2020-05-11 19:01:06 +02:00 |
Raphaël Vinot
|
70c642b0f6
|
fix: Do not display the full path to the capture
|
2020-05-07 15:46:45 +02:00 |
Raphaël Vinot
|
487222b81e
|
fix: Do not fail when getting the UA breaks
|
2020-05-02 01:41:29 +02:00 |
Raphaël Vinot
|
996bd74e30
|
fix: Missing f-string
|
2020-04-24 15:57:16 +02:00 |
Raphaël Vinot
|
ae49fd2c11
|
new: Add support to pass more parameters via the API
|
2020-04-21 18:41:57 +02:00 |
Raphaël Vinot
|
b1034739ad
|
chg: Fix mypy, bump deps
|
2020-04-20 16:52:46 +02:00 |
Raphaël Vinot
|
a0c906d3dc
|
new: Initial view for 3rd party modules
|
2020-04-20 16:41:42 +02:00 |
Raphaël Vinot
|
de7f50c3b0
|
chg: Improve logging
|
2020-04-03 17:51:58 +02:00 |
Raphaël Vinot
|
1c2cdad38b
|
new: Allow admin to rebuild cache
|
2020-04-01 17:44:06 +02:00 |
Raphaël Vinot
|
e62d56e555
|
chg: Bump har2tree, cleanup
|
2020-04-01 14:33:35 +02:00 |
Raphaël Vinot
|
17c3473d95
|
new: Add config files, initial support for 3rd party modules
|
2020-03-31 14:12:57 +02:00 |
Raphaël Vinot
|
c322bc1b23
|
fix: Pass proper path to the pickled dump
|
2020-03-26 14:55:01 +01:00 |
Raphaël Vinot
|
949ad58667
|
chg: Refactoring of the redirects, rename report -> capture
|
2020-03-26 01:56:24 +01:00 |
Raphaël Vinot
|
f26a02ef86
|
new: Add link redirects on index
|
2020-03-23 12:45:57 +01:00 |
Raphaël Vinot
|
c97537fbe9
|
fix: Strip tabs/spaces around URL
|
2020-03-19 14:05:19 +01:00 |
Raphaël Vinot
|
1fd48df470
|
fix: Refang the URL at the right place
|
2020-03-19 11:05:29 +01:00 |
Raphaël Vinot
|
971877954e
|
new: Add support for last redirect
|
2020-03-18 21:14:48 +01:00 |
Fafner [_KeyZee_]
|
3a8b7e6077
|
Fixing Unknown domain if only_global_lookups is set
|
2020-03-17 16:32:39 +01:00 |
Fafner [_KeyZee_]
|
5c5577b97d
|
Fixing Unknown domain if only_global_lookups is set
|
2020-03-17 16:26:31 +01:00 |
Raphaël Vinot
|
9aa995092e
|
chg: Improve error notification
|
2020-03-17 15:27:04 +01:00 |
Raphaël Vinot
|
af98c2b075
|
chg: Better handling of error messages, display them.
|
2020-03-17 14:17:18 +01:00 |
Raphaël Vinot
|
75cb4d3246
|
new: refang urls when needed
|
2020-03-16 13:51:21 +01:00 |
Raphaël Vinot
|
9b4bbb2450
|
fix: Missing cast
|
2020-02-16 23:51:04 +01:00 |
Raphaël Vinot
|
7d2f50174d
|
chg: Bump dependencies, Improve HTTP errors handling
Fix https://github.com/CIRCL/lookyloo/issues/66
|
2020-02-16 23:38:42 +01:00 |
Raphaël Vinot
|
09b3b16910
|
fix: Properly check keys in cache
|
2020-02-11 17:26:01 +01:00 |
Raphaël Vinot
|
794d9745ab
|
fix: wrong variable in last commit
|
2020-02-11 17:09:15 +01:00 |
Raphaël Vinot
|
e7b97d23b7
|
fix: Better handling of cache issues
|
2020-02-11 17:03:25 +01:00 |
Raphaël Vinot
|
4234e44470
|
new: Add initial redirects columns
|
2020-02-03 22:25:48 +01:00 |
Raphaël Vinot
|
89edef68ab
|
new: Improve index
|
2020-02-03 18:30:41 +01:00 |
Raphaël Vinot
|
c9cbf27acc
|
chg: Fix typing
|
2020-01-24 11:25:53 +01:00 |
Raphaël Vinot
|
f1309ce5e3
|
new: Upload optional cookie file
|
2020-01-24 10:17:41 +01:00 |
Raphaël Vinot
|
f1d83d29cf
|
new: Add support for user defined cookies, bump deps
|
2020-01-23 10:52:50 +01:00 |
Raphaël Vinot
|
d34233ad5c
|
chg: Use poetry instead of pipenv
|
2020-01-21 17:39:18 +01:00 |
Raphaël Vinot
|
2d82303ef6
|
chg: Support cookies from/to splash (no UI yet)
|
2020-01-21 11:47:36 +01:00 |
Raphaël Vinot
|
0aaa814224
|
new: Initial commit for cookies tracking. No update on the view yet.
|
2020-01-10 17:28:15 +01:00 |
Raphaël Vinot
|
dec6920cb8
|
chg: Add typing
|
2020-01-06 15:32:38 +01:00 |
Raphaël Vinot
|
5cfe5306b2
|
fix: Skip UA fetch on cloudflare website until the lib is updated
|
2019-12-17 14:29:58 +01:00 |
Raphaël Vinot
|
4c86b65c56
|
fix: Avoid bypass with "http://user:pwd@host.tld"
|
2019-07-05 18:42:18 +02:00 |
Raphaël Vinot
|
a6693535de
|
fix: Allow to disable scraping private IPs.
|
2019-07-05 16:27:23 +02:00 |
Raphaël Vinot
|
8165913206
|
fix: Initialize cache only once
|
2019-06-25 18:08:52 +02:00 |
Raphaël Vinot
|
320105aca1
|
fix: Get most common user agents from willshouse.com behind cloudflare
|
2019-05-27 15:34:26 +02:00 |
Raphaël Vinot
|
66545c26a5
|
new: keep track of metadata about OS and Browser when scraping
|
2019-04-07 23:54:16 +02:00 |
Raphaël Vinot
|
12b8e4f949
|
chg: Improve async processing
|
2019-04-05 16:12:54 +02:00 |
Raphaël Vinot
|
da3d1fe392
|
fix: Avoid loading the cache multiple times
|
2019-04-05 15:07:22 +02:00 |
Raphaël Vinot
|
6b10e74470
|
fix: do not initialize the redis cache more than once.
|
2019-04-05 14:48:25 +02:00 |
Raphaël Vinot
|
0d937b02c9
|
fix: Properly remove files if the scraping failed
|
2019-04-04 00:41:15 +02:00 |
Raphaël Vinot
|
36c856ef9b
|
new: Add initial support for user-agent
|
2019-03-29 20:11:44 +01:00 |
Raphaël Vinot
|
49654b6b11
|
fix: Automatically remove failed runs.
|
2019-03-27 10:21:37 +01:00 |
Raphaël Vinot
|
fdcc566e05
|
chg: Slight cleanup
|
2019-02-18 14:29:15 +01:00 |
Raphaël Vinot
|
747ce7e52d
|
Merge branch 'anonyme' of https://github.com/Th4nat0s/lookyloo into Th4nat0s-anonyme
|
2019-02-18 13:53:04 +01:00 |
Raphaël Vinot
|
9f2b7f7b93
|
new: Add error page
|
2019-02-18 13:52:48 +01:00 |
Th4nat0s
|
8afcc30840
|
clarifie index enable/disable
|
2019-02-17 21:45:23 +00:00 |
thanat0s
|
f029b5d324
|
Add noindex button
|
2019-02-17 17:43:25 +00:00 |
Raphaël Vinot
|
189b78e20f
|
chg: pre-load the existing reports
|
2019-02-01 16:11:16 +01:00 |
Raphaël Vinot
|
1d244ef456
|
chg: Refactor code organisation
|
2019-01-30 14:30:01 +01:00 |
Raphaël Vinot
|
e5de1a9958
|
fix: Make sure sanejs is optional
|
2019-01-30 13:14:16 +01:00 |
Raphaël Vinot
|
6bc316ebcf
|
new: Initial commit for client and async scraping
|
2019-01-29 18:37:13 +01:00 |
Raphaël Vinot
|
e14a0150a0
|
chg: Assign a UUID for each scraped page, allow permalinks
|
2019-01-29 11:42:56 +01:00 |
Raphaël Vinot
|
945511d6b3
|
chg: Use PySaneJS to query SaneJS
|
2019-01-27 17:17:31 +01:00 |
Raphaël Vinot
|
bbb8c5343f
|
chg: Cleanup, use pipfile
|
2019-01-23 15:13:29 +01:00 |
Raphaël Vinot
|
2825096dc7
|
fix: Do not fail if SaneJS is not available
|
2018-07-20 11:36:54 +02:00 |
Raphaël Vinot
|
3c9b28b704
|
new: Add integration with SaneJS
|
2018-07-19 18:18:22 +02:00 |
Raphaël Vinot
|
5202630278
|
fix: Improve URL overlay
|
2018-06-29 14:03:52 +08:00 |
Raphaël Vinot
|
c8fb313d28
|
new: Download screenshot of the page.
|
2018-05-10 09:48:48 +02:00 |
Raphaël Vinot
|
6329e7747f
|
fix: Wrong URL path
|
2018-04-09 11:16:22 +02:00 |
Raphaël Vinot
|
f6345e47e1
|
new: allow to download the URLs list in a text file
|
2018-04-06 18:06:41 +02:00 |
Raphaël Vinot
|
d7cf92db21
|
new: Close button for the overlays
|
2018-04-05 22:59:45 +02:00 |
Raphaël Vinot
|
72759850ac
|
chg: Use d3v5, cleanup.
|
2018-04-05 11:17:26 +02:00 |
Raphaël Vinot
|
fb195971e8
|
chg: use pathlib everywhere, remove old tmpfiles
|
2018-03-22 18:33:42 +01:00 |
Raphaël Vinot
|
247a4a26b0
|
chg: Change cookie name, update legend, remove old pickled trees
Fix #18
|
2018-03-22 17:22:19 +01:00 |
Raphaël Vinot
|
32ba719d5c
|
new: Add Icons
|
2018-03-15 09:33:38 +01:00 |
Raphaël Vinot
|
22f1474120
|
new: Link to URL overlay
|
2018-02-23 18:45:12 +01:00 |
Raphaël Vinot
|
510c35b57f
|
chg: Store iframe info and PNG
|
2018-02-09 18:30:19 +01:00 |
Raphaël Vinot
|
92ffd1b7da
|
new: Allow to download the bodies
|
2018-02-06 01:23:00 +01:00 |
Raphaël Vinot
|
f3ee006047
|
new: Generic generator for icon list
|
2018-02-01 23:58:26 +01:00 |
Raphaël Vinot
|
6f31c2bce0
|
chg: make methods.
|
2018-02-01 01:16:20 +01:00 |
Raphaël Vinot
|
8955c9c7f7
|
chg: Slight refactoring
|
2018-01-30 15:41:35 +01:00 |
Raphaël Vinot
|
40569a3a73
|
new: Improve the URL box.
Remove all tabs.
|
2018-01-22 10:49:45 +01:00 |
Raphaël Vinot
|
34c1be7695
|
Get ready for building URL tree.
|
2018-01-16 18:06:43 +01:00 |
Raphaël Vinot
|
ab37ced391
|
chg: Dragable box for URLs of a domain
|
2018-01-16 17:48:11 +01:00 |
Raphaël Vinot
|
19b0cb25f8
|
up: UI improvments
|
2017-10-10 11:16:16 +02:00 |
Raphaël Vinot
|
629370fc74
|
Cleanup
|
2017-10-05 17:38:35 +02:00 |
Raphaël Vinot
|
8b353e3e76
|
Properly use click event
|
2017-10-04 17:09:00 +02:00 |
Raphaël Vinot
|
5102546789
|
Cleanup, set favicon
|
2017-10-04 16:39:03 +02:00 |
Raphaël Vinot
|
0ac38a0f2c
|
Add tooltip for URLs
|
2017-10-04 15:13:42 +02:00 |
Raphaël Vinot
|
85b8fd2c0a
|
Cleanup and improve tree view
|
2017-09-29 17:30:01 +02:00 |
Raphaël Vinot
|
2b9213eb1f
|
Only import tree.js in the tree view page
|
2017-09-29 14:48:37 +02:00 |
Raphaël Vinot
|
8d9122b36c
|
Make the svg size generic.
|
2017-09-29 14:43:40 +02:00 |
Raphaël Vinot
|
b9c3d0b276
|
Add favicon
|
2017-09-29 14:43:08 +02:00 |
Raphaël Vinot
|
9cb772eb79
|
Add infoboxes on the tree view
|
2017-09-25 14:11:01 +01:00 |
Raphaël Vinot
|
6900e08dc4
|
Make the node length dynamic
|
2017-09-25 11:13:45 +01:00 |
Raphaël Vinot
|
17e6b6cce5
|
Add background pattern
|
2017-09-25 10:23:32 +01:00 |
Raphaël Vinot
|
d22b6a9312
|
Initial commit
|
2017-09-23 23:54:38 +01:00 |
Raphaël Vinot
|
2755855c43
|
Reverse order index page
|
2017-09-13 09:38:25 +02:00 |
Raphaël Vinot
|
9e207290eb
|
Fix template
|
2017-09-07 16:22:16 +02:00 |
Raphaël Vinot
|
7f6902d910
|
Fix last commit
|
2017-09-07 13:44:32 +02:00 |
Raphaël Vinot
|
2010ccfd56
|
Fix typo
|
2017-09-07 13:40:42 +02:00 |
Raphaël Vinot
|
b059c7b1f3
|
Make it possible to render tree containing ImgFace
See https://github.com/etetoolkit/ete/issues/302
|
2017-09-03 01:14:21 +02:00 |
Raphaël Vinot
|
7cfd51bbbe
|
Improve UI
|
2017-08-27 22:31:44 +02:00 |
Raphaël Vinot
|
500fa35670
|
Add default depth
|
2017-08-27 22:29:44 +02:00 |
Raphaël Vinot
|
ca5d07b8b8
|
Add config to run as service behind nginx
|
2017-08-12 20:12:14 +02:00 |
Raphaël Vinot
|
aa659234bb
|
Cleanup HAR_DIR if a scrap failed
|
2017-08-12 19:19:24 +02:00 |
Raphaël Vinot
|
61e6ddb064
|
Create HAR_DIR if needed
|
2017-08-12 17:45:53 +02:00 |
Raphaël Vinot
|
f7f1abc9e4
|
Add initial web interface
|
2017-07-23 19:58:08 +02:00 |