CIRCL
/
lookyloo
mirror of https://github.com/CIRCL/lookyloo
2
0
Fork 0
Code Issues Releases Wiki Activity
2,432 Commits
10 Branches
33 Tags
22 MiB
Commit Graph

65 Commits (9bc29ee9645a3e3a60942b3e052119cb1cf33642)

Author SHA1 Message Date
Raphaël Vinot 7a37751447 chg: Allow to stop indexing before all the captures are indexed 2024-09-27 12:26:34 +02:00
Raphaël Vinot 1085932ad2 new: Force indexing of a specific capture from the UI 
This should also make the indexing a lot faster.
 2024-08-29 13:32:45 +02:00
Raphaël Vinot b20ddb4788 chg: quick and dirty category indexing 2024-07-23 00:21:26 +02:00
Raphaël Vinot 466a3c5614 new: Basic support for CERT PL phishing truncated hash HTML structure 
Fix #905
 2024-04-11 17:47:52 +02:00
Raphaël Vinot 0f4ef013c9 new: Index and views for identifiers 2024-03-14 00:56:28 +01:00
Raphaël Vinot 926c0da23e chg: Disable index cache for backgroupd processes 2024-03-12 12:02:10 +01:00
Raphaël Vinot d2df33aa5c fix: use a more direct way to index 2024-03-09 15:33:10 +01:00
Raphaël Vinot 24cc00fe96 chg: clear cache when it is not needed 2024-03-08 15:50:47 +01:00
Raphaël Vinot 07a6fe8f49 chg: Bump deps 2024-03-08 10:36:04 +01:00
Raphaël Vinot e45b7c4346 new: Indexer for *all* the captures 2024-03-05 20:51:21 +01:00
Raphaël Vinot 9e302a9b14 new: Add shodan hash on favicon views 2024-02-26 19:09:48 +01:00
Raphaël Vinot decf887b63 new: Shodan MM3H indexing 2024-02-26 17:07:23 +01:00
Raphaël Vinot 4153138644 new: Add favicons in indexer 2024-02-19 16:15:52 +01:00
Raphaël Vinot bd6a0f2d22 chg: cleanup with annotations 2024-01-13 01:45:45 +01:00
Raphaël Vinot ee1ad48b25 chg: Use new annotations 2024-01-12 17:15:41 +01:00
Raphaël Vinot 89dbef8683 chg: Avoid to discard the index lock too soon 2023-11-21 16:50:15 +01:00
Raphaël Vinot efe2124753 fix: Quit BG indexer when shutdown is requested. Improve exceptions handling in archiver 2023-11-20 11:45:41 +01:00
Raphaël Vinot ce76218657 fix: build backlog pickles in reverse order 2023-11-16 23:58:07 +01:00
Raphaël Vinot 7791eff842 new: Store directories by day, refactor indexing 2023-11-16 16:54:21 +01:00
Raphaël Vinot 2920f796fe fix: Speedup generating pickles in BG 2023-10-09 10:26:37 +02:00
Raphaël Vinot 532b68dd07 fix: Avoid exception when attempting to move a capture 2023-09-18 00:33:59 +02:00
Raphaël Vinot c203aa91b9 chg: Avoid directory listing as much as possible in archiver, allow shutdown 2023-08-04 14:02:45 +02:00
Raphaël Vinot 5fca6b13ea chg: Show stacktrace when we cannot build the pickle 2023-08-04 13:15:39 +02:00
Raphaël Vinot 959b7ca96d fix: use glob with path instead of rglob (faster)) 2023-08-04 13:15:03 +02:00
Raphaël Vinot 4be8186cc6 chg: Improve readability of the background indexer 2023-07-30 16:59:41 +02:00
Raphaël Vinot ea2ded9beb fix: properly handle missing title in cache 2023-07-27 15:21:06 +02:00
Raphaël Vinot ebfc2f00a5 fix: Exception when a formerly broken capture is re-processed and works 2023-07-27 14:56:39 +02:00
Raphaël Vinot 855485984f fix: handle gracefully empty lists in hset, and duplicates UUIDs 2023-07-26 22:16:00 +02:00
Raphaël Vinot fd9325bb0d chg: Improve logging, add lock on indexer. 2023-07-26 12:37:12 +02:00
Raphaël Vinot f60457a484 fix: Put the max captures counter at the right place... 2023-07-26 11:45:22 +02:00
Raphaël Vinot fc5850e147 chg: Avoid building old pickles forever 2023-07-26 11:38:40 +02:00
Raphaël Vinot a18f8f9675 chg: do not discard capture without HAR files 
They are often just captures with an error file.
 2023-07-25 20:29:30 +02:00
Raphaël Vinot ef3432cbed fix: Few more improvments on lockfile and broken captures. 2023-07-25 20:16:48 +02:00
Raphaël Vinot 484aec5ddd fix: Properly handle lock file. 2023-07-25 19:29:53 +02:00
Raphaël Vinot 345a2f3f45 fix: Import method from the right file 2023-07-25 17:16:59 +02:00
Raphaël Vinot 3c50474ce4 fix: check if a tree.pickle.gz exists in the background indexer 2023-07-25 17:13:28 +02:00
Raphaël Vinot 0c7b3d9106 fix: indexer getting stuck when we had more than one at a time 2023-07-25 17:08:00 +02:00
Raphaël Vinot 177474e874 new: Basic support for HHHash 2023-07-21 15:48:20 +02:00
Raphaël Vinot 9995371916 chg: Normalize logging on the config file settings 2023-04-05 16:23:46 +02:00
Raphaël Vinot 36d39f6076 new: Add PID in lock file, allows to check if the locking process is still there 2023-02-26 17:20:17 +01:00
Raphaël Vinot 00370291ac new: Logging config in file 2022-11-23 15:54:22 +01:00
Raphaël Vinot a27683f090 fix: Match compressed HAR as valid for rebuild 2022-09-28 11:23:44 +02:00
Raphaël Vinot edd8d786d3 chg: Do not try to build a tree if there are no HAR files 2022-09-26 15:59:04 +02:00
Raphaël Vinot 31261e84c2 fix: Better handling of half broken captures without HAR files 2022-09-26 14:58:30 +02:00
Raphaël Vinot da33a7f5b3 chg: Avoid stacktrace when trying to generate broken capture 2022-09-23 14:46:19 +02:00
Raphaël Vinot e89e9a20cb fix: Force BG processor to index all the recent captures 2022-08-12 01:08:28 +02:00
Raphaël Vinot ef1094a331 chg: Bump deps, fix cookie issue 
Fix  #404
 2022-04-29 00:44:03 +02:00
Raphaël Vinot 83fc0bd8f4 fix: shutil.move wants str (not Path) for python<3.9 2022-04-10 12:43:56 +02:00
Kimmo Linnavuo a80b6a31e4 Use shutil.move instead of path rename when moving discarded captures 2022-04-08 15:28:06 +03:00
Raphaël Vinot cf46dde1ed chg: Add basic pre-hook config 2022-03-31 11:30:53 +02:00