Commit Graph

50 Commits (lacus)

Author SHA1 Message Date
Raphaël Vinot 9beb65cdcd chg: Implement support for LacusCore *or* PyLacus 2022-09-23 12:32:37 +02:00
Raphaël Vinot 3c8fcb3700 chg: Bump lacuscore 2022-09-20 14:49:58 +02:00
Raphaël Vinot 1157428d88 chg: Properly handle missing HAR 2022-09-19 16:16:58 +02:00
Raphaël Vinot be3bec1252 chg: Add missing bits 2022-09-16 12:20:34 +02:00
Raphaël Vinot 68525331c5 chg: move to lacus, WiP 2022-09-14 16:47:38 +02:00
Raphaël Vinot 812c63b0f2 fix: error in UAs, typing 2022-09-05 18:58:45 +02:00
Raphaël Vinot ebbe6e3ce9 new: Pick mobile devices on capture page 2022-08-22 17:34:00 +02:00
Raphaël Vinot 35789f549b fix: Exception on invalid capture 2022-08-20 23:33:32 +02:00
Raphaël Vinot d63ea473f5 new: Autoselect browser engine based on the UA 2022-08-19 14:26:22 +02:00
Raphaël Vinot 998ef12b06 new: Add support for playwright devices and browser name (API only) 2022-08-18 11:19:32 +02:00
Raphaël Vinot be2e1ddc33 fix: properly handle listing configuration, clear None from queries before pasing to redis 2022-08-10 18:53:14 +02:00
Raphaël Vinot 49f335405e fix: Avoid exceptions on invalid requests 2022-08-05 11:28:44 +02:00
Raphaël Vinot 4280a4e11f fix: Support for document on public instances. 2022-08-04 21:28:47 +02:00
Raphaël Vinot 94bae7c5e3 chg: Avoid exception on broken captures 2022-08-04 21:11:58 +02:00
Raphaël Vinot 4f72d64735 new: Upload a file instead of submitting a URL. 2022-08-04 16:58:07 +02:00
Raphaël Vinot 3170038db7 new: dropdown to pass DoNotTrack HTTP header
Improvments on the capture page.
2022-08-03 12:07:45 +02:00
Raphaël Vinot c9381873c7 chg: cleanup the file download feature 2022-07-19 17:54:45 +02:00
Arhamyss 47ecc7a4fa new: Download file 2022-07-19 10:07:36 -04:00
Raphaël Vinot 6ba019ec83 chg: Improve somewhat the useragents available for capturing
Fix #416
2022-06-09 18:58:17 +02:00
Raphaël Vinot 1817a3e13b chg: sunday cleanup 2022-05-23 00:15:52 +02:00
Raphaël Vinot d222ae04aa new: Keep capture even if we have a network error 2022-05-03 12:23:16 +02:00
Raphaël Vinot 463d1d2d1a new: autosubmit to FOX, bump deps 2022-05-02 13:04:55 +02:00
Raphaël Vinot 1679ccf90f chg: Improve capture, ignore ssl issues. 2022-04-26 13:49:24 +02:00
Raphaël Vinot 77fbf47e73 fix: capture cleanup 2022-04-26 10:25:11 +02:00
Raphaël Vinot 147bc65992 fix: Mypy, docker 2022-04-26 00:59:57 +02:00
Raphaël Vinot 41c7e87458 fix: docker, improve error catching 2022-04-26 00:33:50 +02:00
Raphaël Vinot 5af278f84d fix: issue in playwrightcapture module 2022-04-25 15:20:05 +02:00
Raphaël Vinot 4ad898a375 chg: Use packaged playwright capture module 2022-04-25 13:34:01 +02:00
Raphaël Vinot c93a6c307d chg: properly set cookies 2022-04-24 20:17:54 +03:00
Raphaël Vinot 680eb1b309 fix: better handling if capture fails. 2022-04-21 15:48:28 +03:00
Raphaël Vinot 8d159ffba0 new: Switch away from splash to use playwright 2022-04-21 14:55:07 +03:00
Raphaël Vinot cf46dde1ed chg: Add basic pre-hook config 2022-03-31 11:30:53 +02:00
Raphaël Vinot 58b50f2b24 new: Pass optional arbitrary HTTP headers to capture 2021-11-23 12:59:56 -08:00
Raphaël Vinot bfb1e6b181 fix: Use default_public for all capture, including if submitted via the API 2021-11-02 14:58:31 -07:00
Raphaël Vinot 1f998b457f chg: use template 2021-10-18 13:06:43 +02:00
Raphaël Vinot 48fc807e7d new: Add monitoring for pickle cache status 2021-09-24 12:02:28 +02:00
Raphaël Vinot d1f673f3a7 chg: Cleanup passing listing key to and from bool in redis 2021-09-10 14:20:58 +02:00
Raphaël Vinot 9c7929569e fix: The captures are visible on the index by default. 2021-09-08 20:43:56 +02:00
Raphaël Vinot 48b632aa1e fix: Incorrect matching for listing key in capture (always false) 2021-09-08 10:53:31 +02:00
Raphaël Vinot 902c8f81b6 chg: Improve error message if the capture fails
Fix #257
2021-09-07 18:16:01 +02:00
Raphaël Vinot dfbe40a52e chg: reorder imports 2021-09-07 16:00:07 +02:00
Raphaël Vinot c09adec333 chg: Improve logging. 2021-09-01 14:08:25 +02:00
Raphaël Vinot 797de9ddb3 fix: remove datefmt from logging.basicConfig, it was a bad idea. 2021-09-01 10:40:59 +02:00
Raphaël Vinot 8a51383d7a chg: Move the process managment methods to the proper class 2021-08-27 17:28:26 +02:00
Raphaël Vinot d41b7735dd chg: Improve storage, support both modes. 2021-08-26 15:49:19 +02:00
Raphaël Vinot 407e78ae7f chg: More cleanup, support clean shutdown of multiple async captures 2021-08-25 16:40:51 +02:00
Raphaël Vinot bf700e7a7b chg: Major refactoring, move capture code to external script. 2021-08-25 13:36:48 +02:00
Raphaël Vinot f865ec912a fix: Move set/unset running to abstract
Avoid issues when a script fails unexpectedly.
2021-04-09 14:33:42 +02:00
Raphaël Vinot 6059cb5219 chg: Remove useless code 2021-03-12 16:49:04 +01:00
Raphaël Vinot ea052c7c12 fix: Rename scrape -> capture in async 2020-11-05 14:14:33 +01:00