Commit Graph

1982 Commits (refactor_indexing)

Author SHA1 Message Date
Raphaël Vinot f67a24f5c2 new: Store directories by day, refactor indexing (WiP) 2023-11-16 00:38:36 +01:00
Raphaël Vinot 1b987c38b7 chg: avoid issue if ipasnhistory is down, improve IDNA rendering 2023-11-10 21:19:22 +01:00
Raphaël Vinot 110815778f chg: Bump deps 2023-11-10 21:18:34 +01:00
Raphaël Vinot 8152e3260c fix: Avoid exception if ipasnhistory is loading 2023-11-09 11:10:42 +01:00
Raphaël Vinot 176e5048aa chg: Avoid default values set to "string" in API 2023-11-08 13:49:48 +01:00
Raphaël Vinot 5b42cb7f4a new: Display IDNA decoded on tree. 2023-11-07 21:12:35 +01:00
Raphaël Vinot f416851f16 new: show IDNA decoded hostname if possible 2023-11-07 20:49:21 +01:00
Raphaël Vinot 175f4b51e5 fix: Avoid exception when downloading a non-existent file 2023-11-07 16:47:34 +01:00
Raphaël Vinot 7536bbb7f2 fix: Properly initialize Lookyloo once/process 2023-11-07 16:10:47 +01:00
Raphaël Vinot f56a6084a0 fix: partial revert last commit, needs unique instance 2023-11-07 14:19:20 +01:00
Raphaël Vinot a4c643cc16 fix: Do not initialize the Lookyloo class multiple times 2023-11-07 13:49:32 +01:00
Raphaël Vinot 264d94cd2d fix: Pass IPs to whois instead of v4/v6... 2023-11-06 21:17:34 +01:00
Raphaël Vinot 7b142277ed fix: Properly handle unlimited requests in body hash lookup 2023-11-06 16:58:32 +01:00
Raphaël Vinot c8606ff167 fix: Do not update cached UUIDs from hostnode popup 2023-11-06 16:45:37 +01:00
Raphaël Vinot 22f0fbd21c chg: Bump deps 2023-11-06 16:17:21 +01:00
Raphaël Vinot 77d5cab1e8 chg: Speedup hostnode popup 2023-11-06 16:16:09 +01:00
Raphaël Vinot b06ae9de65 chg: Bump deps 2023-11-01 09:59:43 +01:00
Raphaël Vinot 02cfcd9694 chg: Bump deps 2023-10-26 00:09:25 +02:00
Raphaël Vinot 21965758f2 fix: Avoid encoding issue with titles 2023-10-26 00:05:50 +02:00
Raphaël Vinot d6e664d7ca fix: Initialize the config file with indent 2023-10-24 12:36:55 +02:00
Raphaël Vinot 1c5c178d20 fix: s3fs support was broken. 2023-10-23 15:59:14 +02:00
Raphaël Vinot 6079dfdd37 chg: Bump deps, block flask-restx 1.2 as it requires an old jsonschema for no clear reason 2023-10-23 13:58:38 +02:00
Raphaël Vinot fcaeda8f7f new: Use S3FS in archiving script instead, remove python 3.12 uspport
Also remove standalone script for updating archived indexes.
2023-10-23 13:57:44 +02:00
Raphaël Vinot bf9ff87dac new: prepare s3fs support for archives 2023-10-22 01:18:34 +02:00
Raphaël Vinot acfbd77189 chg: Bump deps, supports urllib3 2023-10-21 21:22:20 +02:00
Raphaël Vinot a6fff6687d fix: Avoid exception when there is no URL to send to VT 2023-10-21 16:59:23 +02:00
Raphaël Vinot db9ca0ea2b fix: Properly match 0/1 as string 2023-10-20 15:55:50 +02:00
Raphaël Vinot a2ba5c551d fix: allow auto_report to be "True" without any setting. 2023-10-20 15:48:28 +02:00
Raphaël Vinot a34e3a76cd chg: Bump deps 2023-10-18 14:13:28 +02:00
Raphaël Vinot 1d6026da58 chg: disable python 3.12, aiohttp doesn't supports it. 2023-10-17 23:17:13 +02:00
Raphaël Vinot 49f7558d23 chg: Bump deps, enable python 3.12 2023-10-17 23:06:41 +02:00
Raphaël Vinot 5b01147672 chg: bump deps, version 2023-10-17 09:23:38 +02:00
Raphaël Vinot 4143d5f249 chg: avoid trying to load a tree that is currenly being built 2023-10-13 20:08:58 +02:00
Raphaël Vinot 74bc8fcdd5 chg: [wip] use standalone script to update indexes on s3fs 2023-10-13 15:58:23 +02:00
Raphaël Vinot 524132e100 fix: revert support for python 3.12
Not supported by playwright: https://github.com/microsoft/playwright-python/issues/2096
2023-10-12 16:32:43 +02:00
Raphaël Vinot f599f2e031 chg: Add support for python 3.12 2023-10-12 16:15:43 +02:00
Raphaël Vinot 0daff9ef77 chg: settings tweaks, logging 2023-10-11 15:02:11 +02:00
Raphaël Vinot b4599492f3 fix: Avoid exception killing website if non-responsive 3rd party module. 2023-10-11 14:57:53 +02:00
Raphaël Vinot 5db3401079 chg: Bump deps 2023-10-11 14:55:45 +02:00
Raphaël Vinot 5ca7c5cb1d fix: Typo in last commit 2023-10-10 21:40:49 +02:00
Raphaël Vinot 3e4eb572a0 chg: auto-restart webservers after 1000 requests 2023-10-10 21:32:26 +02:00
Raphaël Vinot fe74b543bb chg: Bump deps 2023-10-09 10:27:47 +02:00
Raphaël Vinot 2920f796fe fix: Speedup generating pickles in BG 2023-10-09 10:26:37 +02:00
Raphaël Vinot 6e1e4d831c fix: Make mypy happy 2023-10-04 11:21:36 +02:00
Raphaël Vinot 466320a0e0 chg: Bump deps 2023-10-04 11:20:52 +02:00
Raphaël Vinot 04392d6922 new: Simplistic script to generate the index file in a s3fs bucket in pure python 2023-10-04 11:20:02 +02:00
Raphaël Vinot f2c9647a9e new: Don't attempt to initialize indexes if they're on a s3fs mount 2023-10-04 11:06:02 +02:00
Raphaël Vinot e3b85508f1 fix: Attempt to check if a directory is empty faster. 2023-10-02 16:16:22 +02:00
Raphaël Vinot f250cba632 chg: yet another attempt to improve checking archived captures 2023-10-02 15:50:46 +02:00
Raphaël Vinot 1220f5926d fix: reduce calls to stat on archived dirs, improve logging 2023-09-29 15:00:40 +02:00