CIRCL
/
lookyloo
mirror of https://github.com/CIRCL/lookyloo
2
0
Fork 0
Code Issues Releases Wiki Activity
2,279 Commits
8 Branches
29 Tags
68 MiB
Commit Graph

706 Commits (main)

Author SHA1 Message Date
Raphaël Vinot 9e302a9b14 new: Add shodan hash on favicon views 2024-02-26 19:09:48 +01:00
Raphaël Vinot decf887b63 new: Shodan MM3H indexing 2024-02-26 17:07:23 +01:00
Raphaël Vinot c6f1057708 fix: Add mimetype for favicon in tree 2024-02-22 16:14:26 +01:00
Raphaël Vinot 391166aec7 new: Many improvments when correlating ressources 2024-02-21 14:36:19 +01:00
Raphaël Vinot dcbec2efef new: use multiple modals instead of changing page in favicons and ressources views 2024-02-20 17:01:16 +01:00
Raphaël Vinot 40ca3ed9bc new: easy way to find other captures with same favicon 2024-02-20 01:13:20 +01:00
Raphaël Vinot 95a660cbda Add landing page to favicon page 2024-02-19 22:30:43 +01:00
Raphaël Vinot 4153138644 new: Add favicons in indexer 2024-02-19 16:15:52 +01:00
Raphaël Vinot 066b93b314 chg: Bump deps, maintenance, typing. 2024-02-06 10:20:30 +01:00
Raphaël Vinot 5368151ce0 chg: Bump deps 2024-02-05 16:33:46 +01:00
Raphaël Vinot c67f01c775 chg: Improve strict typing 2024-01-26 15:03:36 +01:00
Raphaël Vinot 86dfb20122 chg; Bump PyLacus 2024-01-16 00:27:43 +01:00
Raphaël Vinot 7f832f8e15 chg: Bump PySecurityTXT 2024-01-16 00:02:52 +01:00
Raphaël Vinot bd6a0f2d22 chg: cleanup with annotations 2024-01-13 01:45:45 +01:00
Raphaël Vinot a26e80b093 fix: missing future 2024-01-12 17:24:02 +01:00
Raphaël Vinot ee1ad48b25 chg: Use new annotations 2024-01-12 17:15:41 +01:00
Raphaël Vinot 004a1f6d46 fix: Exception when a URL is not in phishtank anymore 2024-01-09 00:01:22 +01:00
Raphaël Vinot 0f86f6cd2b fix: works better with a counter. 2024-01-08 19:03:49 +01:00
Raphaël Vinot 539121c4d1 fix: yes, it was dumb 2024-01-08 18:58:59 +01:00
Raphaël Vinot 13f7394ba2 fix: never checking archives if the current year is not in the archives 2024-01-08 18:45:01 +01:00
Raphaël Vinot 5c9dd1eda0 fix: Avoid exception when IPs are empty lists 2024-01-08 16:35:59 +01:00
Raphaël Vinot 67b0dc0b7e fix: [MISP] Avoid exception when a hostname starts with "file" 2024-01-08 13:29:06 +01:00
Raphaël Vinot 6e1c41da8e fix: Remove yearly redirects 2024-01-08 13:08:20 +01:00
Raphaël Vinot 2caa72e444 fix: use the right variable... 2024-01-08 12:38:54 +01:00
Raphaël Vinot 18fbbb4e7b chg: Properly use archived indexes for stats 2024-01-08 12:32:52 +01:00
Raphaël Vinot 858bff544a chg: Remove stats for archived captures 2024-01-05 16:42:47 +01:00
Raphaël Vinot cb26519dd4 chg: remove broken json files 2023-12-26 19:47:16 +01:00
Raphaël Vinot a98c1d39ee fix: Exception when dumping WhistleBlowerDict to json 2023-12-26 19:44:10 +01:00
Raphaël Vinot 4d68515d20 chg: Make mypy happy 2023-12-13 14:01:14 +01:00
Raphaël Vinot db26633ebb fix: Avoid exception when there are no historical data. 2023-12-13 12:40:33 +01:00
Raphaël Vinot 4668298943 new: CIRCL Passive DNS module 2023-12-12 16:19:01 +01:00
Raphaël Vinot 050ece57c8 chg: More cleanup in recent captures lookup 2023-11-22 15:17:11 +01:00
Raphaël Vinot 9173c072bc fix: Properly compare dates 2023-11-22 14:02:13 +01:00
Raphaël Vinot 175e7294ea fix: Always skip archived captures from sorted capture cache 2023-11-22 13:53:14 +01:00
Raphaël Vinot 3af6068dce fix: allow to run the scripts from anywhere 2023-11-20 23:52:01 +01:00
Raphaël Vinot 7791eff842 new: Store directories by day, refactor indexing 2023-11-16 16:54:21 +01:00
Raphaël Vinot 1b987c38b7 chg: avoid issue if ipasnhistory is down, improve IDNA rendering 2023-11-10 21:19:22 +01:00
Raphaël Vinot 8152e3260c fix: Avoid exception if ipasnhistory is loading 2023-11-09 11:10:42 +01:00
Raphaël Vinot 264d94cd2d fix: Pass IPs to whois instead of v4/v6... 2023-11-06 21:17:34 +01:00
Raphaël Vinot 7b142277ed fix: Properly handle unlimited requests in body hash lookup 2023-11-06 16:58:32 +01:00
Raphaël Vinot c8606ff167 fix: Do not update cached UUIDs from hostnode popup 2023-11-06 16:45:37 +01:00
Raphaël Vinot 77d5cab1e8 chg: Speedup hostnode popup 2023-11-06 16:16:09 +01:00
Raphaël Vinot 21965758f2 fix: Avoid encoding issue with titles 2023-10-26 00:05:50 +02:00
Raphaël Vinot a6fff6687d fix: Avoid exception when there is no URL to send to VT 2023-10-21 16:59:23 +02:00
Raphaël Vinot a2ba5c551d fix: allow auto_report to be "True" without any setting. 2023-10-20 15:48:28 +02:00
Raphaël Vinot 4143d5f249 chg: avoid trying to load a tree that is currenly being built 2023-10-13 20:08:58 +02:00
Raphaël Vinot 0daff9ef77 chg: settings tweaks, logging 2023-10-11 15:02:11 +02:00
Raphaël Vinot b4599492f3 fix: Avoid exception killing website if non-responsive 3rd party module. 2023-10-11 14:57:53 +02:00
Raphaël Vinot 2920f796fe fix: Speedup generating pickles in BG 2023-10-09 10:26:37 +02:00
Raphaël Vinot 1220f5926d fix: reduce calls to stat on archived dirs, improve logging 2023-09-29 15:00:40 +02:00
Raphaël Vinot 68204c2686 new: Support for rendered nodes with HTML and downloaded file 2023-09-26 18:08:04 +02:00
Raphaël Vinot 5d5b93266d new: optionally ignore IPs when comparing nodes 
Related https://github.com/Lookyloo/monitoring/issues/23
 2023-09-12 14:03:02 +02:00
Raphaël Vinot 0365c9d6c9 fix: Strip all non-URL characters from capture. 2023-09-11 16:54:50 +02:00
Raphaël Vinot 00bf8fb551 fix: Improve getting MISP instances 2023-08-29 17:30:45 +02:00
Raphaël Vinot 6034d10834 new: (WiP) Add support for multiple MISPs 2023-08-29 14:03:01 +02:00
Raphaël Vinot 72ba77377d chg: Catch empty file hash and return empry file immediately 2023-08-24 15:12:38 +02:00
Raphaël Vinot ff15f3a60a Avoid exceptions when trying to get a datablob 2023-08-24 14:55:32 +02:00
Raphaël Vinot 8b316ad185 fix: Properly search hash in embedded resources 2023-08-24 14:37:03 +02:00
Raphaël Vinot 2a843ac828 chg: Delist less captures, avoid edge case causing an exception for MISP export 2023-08-23 17:22:46 +02:00
Raphaël Vinot 447229ced3 chg: Compress HARs by default, update codebase accordingly 2023-08-11 13:16:59 +02:00
Raphaël Vinot 1f517b1562 chg: Cosmetic changes to support upcomming ete4 2023-08-10 17:31:19 +02:00
Raphaël Vinot 206e5957b5 new: Support for favicons fetching and display 
Related https://github.com/Lookyloo/PlaywrightCapture/issues/45
 2023-08-09 16:50:33 +02:00
Raphaël Vinot e256a7fe6b chg: Proper use of shutil.move, speedup initialization of CaptureCache 2023-08-08 12:41:21 +02:00
Raphaël Vinot 54674f6c5b avoid exception if pickle needs to be rebuild 2023-08-04 13:16:05 +02:00
Raphaël Vinot 35218900ed fix: MISP export when the capture is a file. 2023-07-30 17:48:16 +02:00
Raphaël Vinot 182e7af188 chg: Attempt to keep index up-to-date with cached captures 2023-07-28 14:40:33 +02:00
Raphaël Vinot c0f601e5db chg: remove incomplete_redirects from cache, was always 0/False 2023-07-28 14:05:28 +02:00
Raphaël Vinot ea2ded9beb fix: properly handle missing title in cache 2023-07-27 15:21:06 +02:00
Raphaël Vinot ebfc2f00a5 fix: Exception when a formerly broken capture is re-processed and works 2023-07-27 14:56:39 +02:00
Raphaël Vinot db639d9dde chg: Improve logging for errors in cookies, bump har2tree 2023-07-27 12:43:52 +02:00
Raphaël Vinot 855485984f fix: handle gracefully empty lists in hset, and duplicates UUIDs 2023-07-26 22:16:00 +02:00
Raphaël Vinot fd9325bb0d chg: Improve logging, add lock on indexer. 2023-07-26 12:37:12 +02:00
Raphaël Vinot ebd2b29993 fix: export compress HAR 2023-07-26 11:28:32 +02:00
Raphaël Vinot ef3432cbed fix: Few more improvments on lockfile and broken captures. 2023-07-25 20:16:48 +02:00
Raphaël Vinot 484aec5ddd fix: Properly handle lock file. 2023-07-25 19:29:53 +02:00
Raphaël Vinot 0c7b3d9106 fix: indexer getting stuck when we had more than one at a time 2023-07-25 17:08:00 +02:00
Raphaël Vinot 18b5a51c86 fix: Exception when none of the captures are cached. 2023-07-24 22:41:11 +02:00
Raphaël Vinot 0604ccfdee new: Improve the HHH details page 2023-07-24 14:53:00 +02:00
Raphaël Vinot 8cd6f5490a fix: Properly cache URL, fix mypy issue 
Related https://github.com/Lookyloo/PyLookyloo/issues/32
 2023-07-24 13:15:13 +02:00
Raphaël Vinot 36f9d3dc60 chg: Improve error reporting 
Related https://github.com/Lookyloo/PyLookyloo/issues/32
 2023-07-24 12:48:57 +02:00
Raphaël Vinot 177474e874 new: Basic support for HHHash 2023-07-21 15:48:20 +02:00
Raphaël Vinot b3000d02dd fix: 🤷 2023-07-13 16:59:52 +02:00
Raphaël Vinot f3095c7ccf chg: Add logging when comparing is impossible due to a missing pickle 2023-07-10 12:27:43 +02:00
Raphaël Vinot 3193763c8a fix: logic issue in last commit 2023-07-06 14:59:24 +02:00
Raphaël Vinot 79c9b95faf fix: Avoid issue if a cature is somewhat broken and we want to send a notification 2023-07-06 14:50:38 +02:00
Raphaël Vinot 2c5c30006c fix: index page stuck building captures for no good reason 2023-07-04 11:22:36 +02:00
Raphaël Vinot 6bb660ed82 fix: incorrect pickle rebuild, causing the index to get stuck 2023-07-03 17:04:55 +02:00
Raphaël Vinot 95a17f5770 fix: Handling of captures with errors 
Related https://github.com/Lookyloo/monitoring/issues/15
 2023-06-21 15:41:14 +02:00
Raphaël Vinot 475b424878 chg: retry a few times if remote lacus isn't available immediately 2023-06-16 12:47:50 +02:00
Raphaël Vinot 324c85cac8 new: Support for STARTTLS in the notification 2023-06-14 16:19:46 +02:00
Raphaël Vinot 0505edff0e new: Global proxy setting. 
Overwrites any proxy given by the user. Still uses the tor proxy when
needed.

Related: https://github.com/Lookyloo/lookyloo/issues/318
 2023-06-13 12:07:26 +02:00
Raphaël Vinot 97a668aa91 chg: do not export pickle 2023-06-12 16:01:30 +02:00
Raphaël Vinot 4ff9b7651d new: Support for TZ, Geo, locale, color scheme 2023-06-07 15:05:40 +02:00
Raphaël Vinot 474ab91dcd chg: Use URL from capture settings file 2023-06-05 15:21:22 +02:00
Raphaël Vinot b3b7d4f5c6 new: Add lookyloo permaurls in comparator response 2023-05-17 12:14:24 +02:00
Raphaël Vinot f8acca5e51 fix: properly ignore resources if only hostname matches 2023-05-17 11:14:53 +02:00
Raphaël Vinot 386aa6e91b fix: make the sets into tuples so endswith works 2023-05-17 09:53:20 +02:00
Raphaël Vinot 95f05a3da1 new: Display proxy if passed to the CaptureSettings 
Fix #695
 2023-05-15 16:33:49 +02:00
Raphaël Vinot 582b5956e9 new: Store capture settings, use TypedDict whenever possible. 2023-05-15 16:08:19 +02:00
Raphaël Vinot a504facf5e new: Flag to quickly say if two captures are different or not 2023-05-11 15:07:37 +02:00