Commit Graph

275 Commits (11fcc9dd8b218cde9c1de310eaffbc2330bc9468)

Author SHA1 Message Date
Raphaël Vinot b20ddb4788 chg: quick and dirty category indexing 2024-07-23 00:21:26 +02:00
Raphaël Vinot d5fb385add chg: Improve errors handling, especially for broken CaptureSettings 2024-07-22 13:14:21 +02:00
Raphaël Vinot 804bab6d2f new: Use Pydantic for CaptureSettings 2024-07-19 16:12:24 +02:00
Raphaël Vinot 40ad6d0031
fix: Make sure listing setting is properly handled. 2024-07-19 01:14:32 +02:00
Raphaël Vinot 604daefa43 chg: Load capture settigns from a single entry point, force types 2024-07-15 17:40:07 +02:00
Raphaël Vinot 74e0f851b2 fix: python <3.10 compat 2024-06-24 16:16:20 +02:00
Raphaël Vinot 994f70adcf fix: Only rewrite the index files if there are changes. 2024-06-24 15:24:55 +02:00
Raphaël Vinot 37e92dbaf1 fix: Avoid crash if capture timestamp for UUID is incorrect
Related to crash and probably #922
2024-06-23 15:10:19 +02:00
Raphaël Vinot 59503d6378 fix: Typo in website starter 2024-06-13 14:24:53 +02:00
Raphaël Vinot 8ad2292d68 fix: Properly use --max-requests-jitter 2024-06-13 12:43:47 +02:00
Raphaël Vinot bb23a5ffb4 chg: Make the archive capture importer generic 2024-06-11 20:07:38 +02:00
Raphaël Vinot 702671255b chg: force restart webserver more often 2024-06-11 11:02:25 +02:00
Raphaël Vinot 0f6d5c1649 fix: Speedup quick cache init 2024-06-11 11:02:25 +02:00
Raphaël Vinot 0a973be5dd fix: Avoid stashing the web interface when lacus becomes unavailable 2024-06-11 11:02:25 +02:00
Raphaël Vinot 658fdaeaea chg: a user config can overwrite all the settings for a capture 2024-05-07 18:46:57 +02:00
Adrian Maraj 427c6ed465 chg: fixes 2024-04-30 16:50:48 +02:00
Adrian Maraj ae0c9e3449 new: simplified version to automatically report url 2024-04-30 15:32:14 +02:00
Raphaël Vinot 9eb14548e3 fix: run cleanup properly when auto_report == 0 2024-04-29 17:26:46 +02:00
Raphaël Vinot 9f9c709c30 fix: Remove empty dirs from indexes 2024-04-24 14:54:51 +03:00
Raphaël Vinot 466a3c5614 new: Basic support for CERT PL phishing truncated hash HTML structure
Fix #905
2024-04-11 17:47:52 +02:00
Raphaël Vinot 0f4ef013c9 new: Index and views for identifiers 2024-03-14 00:56:28 +01:00
Raphaël Vinot 926c0da23e chg: Disable index cache for backgroupd processes 2024-03-12 12:02:10 +01:00
Raphaël Vinot e40796c92e chg: merge index and reindex methods. 2024-03-11 00:14:07 +01:00
Raphaël Vinot d2df33aa5c fix: use a more direct way to index 2024-03-09 15:33:10 +01:00
Raphaël Vinot 24cc00fe96 chg: clear cache when it is not needed 2024-03-08 15:50:47 +01:00
Raphaël Vinot 07a6fe8f49 chg: Bump deps 2024-03-08 10:36:04 +01:00
Raphaël Vinot caab916ff1 fix: Missing file 2024-03-05 21:03:36 +01:00
Raphaël Vinot e45b7c4346 new: Indexer for *all* the captures 2024-03-05 20:51:21 +01:00
Raphaël Vinot 9e302a9b14 new: Add shodan hash on favicon views 2024-02-26 19:09:48 +01:00
Raphaël Vinot decf887b63 new: Shodan MM3H indexing 2024-02-26 17:07:23 +01:00
Raphaël Vinot 7e25747d82 fix: properly check zscore (can be 0) 2024-02-24 21:18:57 +01:00
Raphaël Vinot 4153138644 new: Add favicons in indexer 2024-02-19 16:15:52 +01:00
Raphaël Vinot e02b7392a6 chg: Only search for finished captures in the top 50, properly march CaptureStatusCore.DONE 2024-02-15 02:36:16 +01:00
Raphaël Vinot c67f01c775 chg: Improve strict typing 2024-01-26 15:03:36 +01:00
Raphaël Vinot fcfe9751f3 fix: potential race condition when checking if a capture is ongoing or not 2024-01-25 14:21:57 +01:00
Raphaël Vinot 86dfb20122 chg; Bump PyLacus 2024-01-16 00:27:43 +01:00
Raphaël Vinot 60c8d7e78d fix: Do not set priority to None 2024-01-14 02:18:21 +01:00
Raphaël Vinot bd6a0f2d22 chg: cleanup with annotations 2024-01-13 01:45:45 +01:00
Raphaël Vinot ee1ad48b25 chg: Use new annotations 2024-01-12 17:15:41 +01:00
Raphaël Vinot d06da1aa52 fix: Avoid exception if a file is missing on s3 2024-01-08 21:02:54 +01:00
Raphaël Vinot f7c45b5039 fix: Add proper path in set to check 2024-01-08 16:50:48 +01:00
Raphaël Vinot 79f5b728d0 fix: do not attempt to remove a directory too early 2024-01-08 16:37:48 +01:00
Raphaël Vinot d60a4e56db chg: Update indexes only when needed 2024-01-08 16:27:12 +01:00
Raphaël Vinot 89dbef8683 chg: Avoid to discard the index lock too soon 2023-11-21 16:50:15 +01:00
Raphaël Vinot 9031141b61 chg: Remove empty dirs when everything has been archived 2023-11-21 11:50:09 +01:00
Raphaël Vinot 6d61645d97 chg: remove index when all the captures are archived 2023-11-20 23:48:56 +01:00
Raphaël Vinot efe2124753 fix: Quit BG indexer when shutdown is requested. Improve exceptions handling in archiver 2023-11-20 11:45:41 +01:00
Raphaël Vinot 11a3b6b2f9 chg: Improve indexes cleanup 2023-11-18 03:20:49 +01:00
Raphaël Vinot ff27808320 fix: Path in index may be the full path (old format) 2023-11-18 02:47:43 +01:00
Raphaël Vinot cd11df7ac4 fix: Update index files to remove archived (or simply gone) captures 2023-11-18 02:39:21 +01:00