Commit Graph

89 Commits (main)

Author SHA1 Message Date
Raphaël Vinot 658fdaeaea chg: a user config can overwrite all the settings for a capture 2024-05-07 18:46:57 +02:00
Raphaël Vinot f4f234a331 chg: refactorize contacts filtering 2024-04-09 16:49:35 +02:00
Raphaël Vinot 5368151ce0 chg: Bump deps 2024-02-05 16:33:46 +01:00
Raphaël Vinot c67f01c775 chg: Improve strict typing 2024-01-26 15:03:36 +01:00
Raphaël Vinot bd6a0f2d22 chg: cleanup with annotations 2024-01-13 01:45:45 +01:00
Raphaël Vinot ee1ad48b25 chg: Use new annotations 2024-01-12 17:15:41 +01:00
Raphaël Vinot 0f86f6cd2b fix: works better with a counter. 2024-01-08 19:03:49 +01:00
Raphaël Vinot 539121c4d1 fix: yes, it was dumb 2024-01-08 18:58:59 +01:00
Raphaël Vinot 13f7394ba2 fix: never checking archives if the current year is not in the archives 2024-01-08 18:45:01 +01:00
Raphaël Vinot 7791eff842 new: Store directories by day, refactor indexing 2023-11-16 16:54:21 +01:00
Raphaël Vinot db639d9dde chg: Improve logging for errors in cookies, bump har2tree 2023-07-27 12:43:52 +02:00
Raphaël Vinot fd9325bb0d chg: Improve logging, add lock on indexer. 2023-07-26 12:37:12 +02:00
Raphaël Vinot ef3432cbed fix: Few more improvments on lockfile and broken captures. 2023-07-25 20:16:48 +02:00
Raphaël Vinot 484aec5ddd fix: Properly handle lock file. 2023-07-25 19:29:53 +02:00
Raphaël Vinot 0c7b3d9106 fix: indexer getting stuck when we had more than one at a time 2023-07-25 17:08:00 +02:00
Raphaël Vinot 582b5956e9 new: Store capture settings, use TypedDict whenever possible. 2023-05-15 16:08:19 +02:00
Raphaël Vinot 25433b5707 chg: always use LookylooException 2023-03-12 19:25:16 +01:00
Raphaël Vinot 36d39f6076 new: Add PID in lock file, allows to check if the locking process is still there 2023-02-26 17:20:17 +01:00
Raphaël Vinot f335b4a58d fix: Use publicsuffixlist instead of publicsuffix2 2023-01-20 15:23:11 +01:00
Raphaël Vinot 01b9f8d1be fix: remove print, replace with logging 2022-12-27 01:55:18 +01:00
Raphaël Vinot d38b612c37 chg: Bump lacuscore 2022-09-23 13:13:09 +02:00
Raphaël Vinot 812c63b0f2 fix: error in UAs, typing 2022-09-05 18:58:45 +02:00
Raphaël Vinot 742a0b1799 fix: Exception when we try to update the UAs from within a PW context 2022-09-01 18:20:29 +02:00
Raphaël Vinot c6464936fc chg: Bump to poetry v1.2, remove dep on setuptools 2022-08-31 16:33:13 +02:00
Raphaël Vinot f232eba662 chg: Improve UA rendering 2022-08-23 17:44:48 +02:00
Raphaël Vinot ebbe6e3ce9 new: Pick mobile devices on capture page 2022-08-22 17:34:00 +02:00
Raphaël Vinot 4f72d64735 new: Upload a file instead of submitting a URL. 2022-08-04 16:58:07 +02:00
Raphaël Vinot c0bde711d2 new: Very basic RiskIQ support for PDNS. 2022-07-15 18:53:49 +02:00
Raphaël Vinot 96d8f1084d fix: exception on subsequent captures when the cookie jar is empty 2022-07-11 17:05:53 +02:00
Raphaël Vinot ce9ecfefdb chg: remove reference to max depth (not used since playwright), cleanup 2022-06-11 14:49:58 +02:00
Raphaël Vinot 6ba019ec83 chg: Improve somewhat the useragents available for capturing
Fix #416
2022-06-09 18:58:17 +02:00
Raphaël Vinot 1817a3e13b chg: sunday cleanup 2022-05-23 00:15:52 +02:00
Raphaël Vinot cc17b414ff chg: Remove refs to splash 2022-04-25 14:43:02 +02:00
Raphaël Vinot ae9cb3e81c chg: Bump deps 2022-03-29 21:13:02 +02:00
Raphaël Vinot 43a29f5781 fix: [Modules] Missing change in helpers, fix urlscan cache storage 2022-02-03 12:49:16 +01:00
Raphaël Vinot 5e5206ae00 fix: Catch exception if cookies are invalid 2021-11-26 14:04:07 -05:00
Raphaël Vinot 1f998b457f chg: use template 2021-10-18 13:06:43 +02:00
Raphaël Vinot 32ee474be2 chg: Improve tree creation and cache 2021-09-22 17:09:04 +02:00
Raphaël Vinot d6982392ef chg: cleanup get_user_agents
Related: #262
2021-09-13 22:32:02 +02:00
Raphaël Vinot dfbe40a52e chg: reorder imports 2021-09-07 16:00:07 +02:00
Raphaël Vinot c09adec333 chg: Improve logging. 2021-09-01 14:08:25 +02:00
Raphaël Vinot 117500b777 chg: Make archiver an index generator 2021-08-30 12:48:13 +02:00
Raphaël Vinot 8a51383d7a chg: Move the process managment methods to the proper class 2021-08-27 17:28:26 +02:00
Raphaël Vinot 8918b11a7f chg: The Tech Blog UAs can only be fetched manually, moving it to tools 2021-08-27 15:22:39 +02:00
Raphaël Vinot d41b7735dd chg: Improve storage, support both modes. 2021-08-26 15:49:19 +02:00
Raphaël Vinot 407e78ae7f chg: More cleanup, support clean shutdown of multiple async captures 2021-08-25 16:40:51 +02:00
Raphaël Vinot bf700e7a7b chg: Major refactoring, move capture code to external script. 2021-08-25 13:36:48 +02:00
Raphaël Vinot 81390d5ea0 chg: cleanup in the mail lookyloo class 2021-08-24 18:32:54 +02:00
Raphaël Vinot 8433cbcc1b chg: Cleanup archiver, initialize index captures in start 2021-08-24 17:10:14 +02:00
Raphaël Vinot 7933670941 new: Integration with urlscan.io 2021-08-10 17:38:47 +02:00