Raphaël Vinot
|
681e136ef4
|
chg: More cleanup to use the new caching system as it should be.
|
2021-09-27 11:36:27 +02:00 |
Raphaël Vinot
|
d05b1edf48
|
new: Add logging in Indexing
|
2021-09-24 16:27:46 +02:00 |
Raphaël Vinot
|
6e9e3990c4
|
fix: Indexes not updated on tree rebuild, better handling of tree cache
|
2021-09-24 16:16:41 +02:00 |
Raphaël Vinot
|
48fc807e7d
|
new: Add monitoring for pickle cache status
|
2021-09-24 12:02:28 +02:00 |
Raphaël Vinot
|
1d3bb55343
|
chg: Rebuild trees when captures are moved.
|
2021-09-23 16:40:39 +02:00 |
Raphaël Vinot
|
87c6925c7b
|
new: IPs lookup against phishtank
|
2021-09-23 13:58:40 +02:00 |
Raphaël Vinot
|
e6e61089b6
|
chg: Speedup cache initialization
|
2021-09-23 10:29:02 +02:00 |
Raphaël Vinot
|
aa7173d27d
|
fix: a broken/missing pickle is handled elsewhere
|
2021-09-23 09:17:34 +02:00 |
Raphaël Vinot
|
19c8c54b0b
|
fix: force-rebuild pickle if needed.
|
2021-09-22 22:23:20 +02:00 |
Raphaël Vinot
|
32ee474be2
|
chg: Improve tree creation and cache
|
2021-09-22 17:09:04 +02:00 |
Raphaël Vinot
|
01fff00cad
|
chg: Slight cleanup
|
2021-09-21 18:04:22 +02:00 |
Raphaël Vinot
|
bfcb4feb9a
|
chg: Remove import
|
2021-09-17 15:10:21 +02:00 |
Raphaël Vinot
|
275b4aee93
|
fix: Avoid None in urlscan.io result.
|
2021-09-17 12:58:05 +02:00 |
Raphaël Vinot
|
e7d9c1cb1a
|
fix: return error message if urlscan isn't able to run the capture.
|
2021-09-17 09:51:52 +02:00 |
Raphaël Vinot
|
cefb3fddbf
|
new: Phishtank lookup.
|
2021-09-16 16:33:44 +02:00 |
Raphaël Vinot
|
9c552f1032
|
chg: Move modules into files
|
2021-09-16 11:22:02 +02:00 |
Raphaël Vinot
|
1d40dd44ae
|
new: Add urlscan permanent URL to MISP export
|
2021-09-15 15:42:02 +02:00 |
Raphaël Vinot
|
d6982392ef
|
chg: cleanup get_user_agents
Related: #262
|
2021-09-13 22:32:02 +02:00 |
Raphaël Vinot
|
22d8672131
|
chg: Catch timeouts when pushing to MISP
|
2021-09-10 15:25:37 +02:00 |
Raphaël Vinot
|
d1f673f3a7
|
chg: Cleanup passing listing key to and from bool in redis
|
2021-09-10 14:20:58 +02:00 |
Raphaël Vinot
|
5abc264159
|
fix: Better initialization of cache, avoid missing the error message.
|
2021-09-08 16:34:54 +02:00 |
Raphaël Vinot
|
902c8f81b6
|
chg: Improve error message if the capture fails
Fix #257
|
2021-09-07 18:16:01 +02:00 |
Raphaël Vinot
|
048e44ce91
|
fix: Avoid exception in _set_capture_cache in case of critical error
|
2021-09-07 16:00:07 +02:00 |
Raphaël Vinot
|
dfbe40a52e
|
chg: reorder imports
|
2021-09-07 16:00:07 +02:00 |
Raphaël Vinot
|
7d76d96ff7
|
fix: Avoid 500 error on the website if unable to dump pickle.
|
2021-09-06 16:30:43 +02:00 |
Raphaël Vinot
|
074bb675f6
|
fix: Handling of status code 0, export capture UUID
|
2021-09-06 16:12:41 +02:00 |
Raphaël Vinot
|
c09adec333
|
chg: Improve logging.
|
2021-09-01 14:08:25 +02:00 |
Raphaël Vinot
|
72d85c62e3
|
fix: Use redis ConnectionError to avoid exception on shutdown
|
2021-09-01 12:44:57 +02:00 |
Raphaël Vinot
|
6db708bb93
|
chg: Avoid exception if the backend is already down
|
2021-08-31 15:43:55 +02:00 |
Raphaël Vinot
|
9042b73e69
|
chg: Remove dead code, proper use of exceptions.
|
2021-08-31 13:48:19 +02:00 |
Raphaël Vinot
|
117500b777
|
chg: Make archiver an index generator
|
2021-08-30 12:48:13 +02:00 |
Raphaël Vinot
|
1bff8f1529
|
fix: cast capture dir to string for redis
|
2021-08-28 20:36:50 +02:00 |
Raphaël Vinot
|
8a51383d7a
|
chg: Move the process managment methods to the proper class
|
2021-08-27 17:28:26 +02:00 |
Raphaël Vinot
|
85e43fc677
|
chg: Make the website start a normal start script
|
2021-08-27 16:45:16 +02:00 |
Raphaël Vinot
|
8918b11a7f
|
chg: The Tech Blog UAs can only be fetched manually, moving it to tools
|
2021-08-27 15:22:39 +02:00 |
Raphaël Vinot
|
d41b7735dd
|
chg: Improve storage, support both modes.
|
2021-08-26 15:49:19 +02:00 |
Raphaël Vinot
|
407e78ae7f
|
chg: More cleanup, support clean shutdown of multiple async captures
|
2021-08-25 16:40:51 +02:00 |
Raphaël Vinot
|
bc5e41a1ca
|
fix: Properly re-set the cache if the tree was not built
|
2021-08-25 15:00:26 +02:00 |
Raphaël Vinot
|
bf700e7a7b
|
chg: Major refactoring, move capture code to external script.
|
2021-08-25 13:36:48 +02:00 |
Raphaël Vinot
|
81390d5ea0
|
chg: cleanup in the mail lookyloo class
|
2021-08-24 18:32:54 +02:00 |
Raphaël Vinot
|
8433cbcc1b
|
chg: Cleanup archiver, initialize index captures in start
|
2021-08-24 17:10:14 +02:00 |
Raphaël Vinot
|
53ceb9c329
|
chg: Cleanup when dir is moved, digit months on 2 values
|
2021-08-23 14:53:19 +02:00 |
Raphaël Vinot
|
b8109c6462
|
fix: Make mypy happy
|
2021-08-23 14:20:26 +02:00 |
Raphaël Vinot
|
d359bc7521
|
chg: Better use of cache, sanity checks
|
2021-08-23 12:17:44 +02:00 |
Raphaël Vinot
|
58b837cb6c
|
new: Archiver, refactoring.
|
2021-08-20 17:46:22 +02:00 |
Raphaël Vinot
|
6be9b69d95
|
chg: Use connection pool whenever possible
|
2021-08-18 18:01:04 +02:00 |
Raphaël Vinot
|
7495e5b4f1
|
chg: Use connection pool in indexing
|
2021-08-18 17:31:17 +02:00 |
Raphaël Vinot
|
046387a9f4
|
chg: Make mypy happy
|
2021-08-18 16:37:50 +02:00 |
Raphaël Vinot
|
3a96e27f8c
|
chg: Try using redis connection pool
|
2021-08-18 16:07:39 +02:00 |
Raphaël Vinot
|
09a69622c0
|
fix: urlscan dislikes URLs without scheme.
|
2021-08-16 11:01:00 +02:00 |