Raphaël Vinot
|
e492428bf5
|
fix: Proper call to local cache
|
2021-04-03 02:24:38 +02:00 |
Raphaël Vinot
|
82db729e96
|
fix: cache captures index in main script
|
2021-04-03 02:06:32 +02:00 |
Raphaël Vinot
|
b6ea1c24db
|
new: Query capture status
|
2021-03-31 19:25:57 +02:00 |
Raphaël Vinot
|
a8711a18f1
|
fix: Make mypy happy
|
2021-03-30 16:43:57 +02:00 |
Raphaël Vinot
|
9f88edd74d
|
new: Check if an event with the same lookyloo capture is already on MISP
Fix #181
|
2021-03-30 16:32:04 +02:00 |
Raphaël Vinot
|
acb1bf2814
|
fix: MISP export when rendered node is empty
Fix #182
|
2021-03-30 14:06:34 +02:00 |
Raphaël Vinot
|
88bd450b59
|
chg: Improve tree page (menu and stats)
|
2021-03-25 18:32:48 +01:00 |
Raphaël Vinot
|
fa6b4701c0
|
chg: update the cache at the right place.
|
2021-03-20 21:54:46 +01:00 |
Raphaël Vinot
|
b1fbf9ff74
|
fix: Properly type capture_dir in cache
|
2021-03-20 01:18:28 +01:00 |
Raphaël Vinot
|
2a55461286
|
new: Capture an URL on the rendered page, keep the session (WiP)
|
2021-03-19 17:52:20 +01:00 |
Raphaël Vinot
|
cd7b050cb0
|
chg: rename and cleanup methods
|
2021-03-18 18:47:54 +01:00 |
Raphaël Vinot
|
25c1a6dbce
|
chg: Use sorted cache whenever possible
|
2021-03-18 15:39:55 +01:00 |
Raphaël Vinot
|
49ae4490b2
|
new: (hidden) interface to search for hostnames and URLs
|
2021-03-18 00:40:14 +01:00 |
Raphaël Vinot
|
99402efa42
|
fix: allow to dump very big tree
|
2021-03-16 14:09:30 +01:00 |
Raphaël Vinot
|
1f9f5f1a9a
|
chg: Improve url and hostnames search
|
2021-03-16 13:35:59 +01:00 |
Raphaël Vinot
|
8c656062b9
|
fix: Catch exception when the tree has too many redirects.
|
2021-03-12 18:20:17 +01:00 |
Raphaël Vinot
|
46aea0fe3a
|
chg: Better error handling
|
2021-03-12 16:52:12 +01:00 |
Raphaël Vinot
|
e90fc151bd
|
fix: Avoid indexing URLs multiple times
|
2021-03-12 13:18:48 +01:00 |
Raphaël Vinot
|
4533bbdfa2
|
chg: Make MyPy happy
|
2021-03-08 14:03:03 +01:00 |
Raphaël Vinot
|
e35ceab3dd
|
chg: Better logging and error handling
|
2021-03-08 13:57:55 +01:00 |
Raphaël Vinot
|
e467f1d1eb
|
chg: Catch exceptions when the capture fails bad.
|
2021-03-08 11:44:29 +01:00 |
Raphaël Vinot
|
984a930aa8
|
chg: Avoid 500 on missing UUID
|
2021-03-07 23:57:46 +01:00 |
Raphaël Vinot
|
b09aec5b84
|
fix: un-break json dump
|
2021-02-12 12:06:26 +01:00 |
Raphaël Vinot
|
d711fd3644
|
fix: Make get legacy user config quiet
|
2021-02-05 15:24:11 +01:00 |
Raphaël Vinot
|
39dd2021dd
|
chg: Complete rework of the login system, add UI for MISP Push
|
2021-02-04 19:51:43 +01:00 |
Raphaël Vinot
|
610ffef43a
|
new: Add VT objects to MISP export
|
2021-02-02 15:23:38 +01:00 |
Raphaël Vinot
|
a68400dac7
|
chg: Add comments in MISP export
|
2021-02-02 15:23:38 +01:00 |
Raphaël Vinot
|
fca3290ef0
|
fix: Make sure to skip last year's final week in the stats
|
2021-02-02 15:23:38 +01:00 |
Raphaël Vinot
|
a7484e6cc4
|
chg: Improve MISP export
* IPs of redirects
* default tags
* auto publish
|
2021-02-01 18:09:32 +01:00 |
Raphaël Vinot
|
54ba67bb87
|
chg: Make lookyloo link org only in MISP export for private instances
|
2021-02-01 14:58:06 +01:00 |
Raphaël Vinot
|
f017ef5bef
|
chg: Add support to timeout in the MISP module
|
2021-02-01 13:01:44 +01:00 |
Raphaël Vinot
|
60ad0dc7ce
|
fix: add warning if MISP module is enabled, but instance unreachable.
|
2021-01-29 16:33:56 +01:00 |
Raphaël Vinot
|
c358c08640
|
fix: Make mypy happy.
|
2021-01-28 19:28:54 +01:00 |
Raphaël Vinot
|
acfb0d1c26
|
new: Add push to MISP feature
|
2021-01-28 18:37:47 +01:00 |
Raphaël Vinot
|
d8d2c59f8b
|
chg: Make mypy happy.
|
2021-01-25 13:14:33 +01:00 |
Raphaël Vinot
|
f9d8081efa
|
fix: hidden view
|
2021-01-18 13:26:02 +01:00 |
Raphaël Vinot
|
8bd874e2b0
|
new: Add variable width thumbnail method
|
2021-01-18 12:30:07 +01:00 |
Raphaël Vinot
|
72463cc898
|
chg: Slight cleanups and improvments
|
2021-01-14 17:28:59 +01:00 |
Raphaël Vinot
|
6149df06eb
|
chg: Make the cache entries a dataclass
Fix #99
|
2021-01-14 17:12:23 +01:00 |
Raphaël Vinot
|
61baab1fbe
|
fix: make mypy happy
|
2021-01-13 15:16:36 +01:00 |
Raphaël Vinot
|
4686b560dc
|
chg: Better handling of insanely long webpages.
|
2021-01-13 14:33:20 +01:00 |
Raphaël Vinot
|
0d68844c90
|
chg: Refactoring and documenting
|
2021-01-12 17:22:51 +01:00 |
Raphaël Vinot
|
33326c2463
|
new: Extract URLs from rendered pages
|
2021-01-08 13:03:23 +01:00 |
Raphaël Vinot
|
2e0671fd96
|
fix: Properly handle new year in week stats
|
2021-01-04 12:34:00 +01:00 |
Raphaël Vinot
|
7241ae2ea1
|
fix: Do not fail on MISP export if there are no redirects
|
2020-12-24 15:43:56 +01:00 |
Raphaël Vinot
|
9b18370fe1
|
fix: Properly generate meta keys related to the User Agents
|
2020-12-22 19:16:46 +01:00 |
Raphaël Vinot
|
fcae1d42d0
|
new: Show images in tooltip in hostnode popup
|
2020-12-22 18:23:39 +01:00 |
Raphaël Vinot
|
62cb5a9eb7
|
fix: Failure on new install
|
2020-12-11 10:26:42 +01:00 |
Raphaël Vinot
|
ed6a53d211
|
chg: small visual improvments
|
2020-12-09 21:08:34 +01:00 |
Raphaël Vinot
|
3c6eca3567
|
new: Add screenshot thumbnail on tree, move links to the end of the node
|
2020-12-09 19:11:19 +01:00 |
Raphaël Vinot
|
b73a3b7533
|
chg: Add attachements to MISP export
|
2020-12-08 14:59:34 +01:00 |
Raphaël Vinot
|
fbbfa3082d
|
chg: Fix typing
|
2020-12-07 20:54:33 +01:00 |
Raphaël Vinot
|
8eab287721
|
new: Initial implementation of MISP export
|
2020-12-07 20:50:46 +01:00 |
Raphaël Vinot
|
9b3d338d3c
|
chg: Rename analisys->submission, add unique urls on stats graph
|
2020-12-07 13:55:03 +01:00 |
Raphaël Vinot
|
7bba854e0a
|
fix: Properly call logger
|
2020-12-04 01:33:58 +01:00 |
Raphaël Vinot
|
5897f8024d
|
chg: Do not fail when sanejs is too slow to answer
|
2020-12-04 00:31:53 +01:00 |
Raphaël Vinot
|
d353463548
|
chg: Simplify modules activation
|
2020-12-03 12:33:35 +01:00 |
Raphaël Vinot
|
820c7c5a4d
|
fix: remove broken pickles if needed
|
2020-12-01 17:00:44 +01:00 |
Raphaël Vinot
|
1a7a74cccd
|
chg: Cleanup stats page
|
2020-11-27 16:27:29 +01:00 |
Fafner [_KeyZee_]
|
0e3d007bf5
|
fixing typo (again)
|
2020-11-25 12:22:45 +01:00 |
Fafner [_KeyZee_]
|
7bf13db3f2
|
fixing ttypo
|
2020-11-25 12:05:08 +01:00 |
Raphaël Vinot
|
d964549e56
|
fix: Typing, dump to json
|
2020-11-24 21:01:30 +01:00 |
Fafner [_KeyZee_]
|
d77dfb1cac
|
updating output for stats
|
2020-11-24 18:55:13 +01:00 |
Raphaël Vinot
|
c54d29601d
|
chg: Fix typing
|
2020-11-24 18:19:53 +01:00 |
Fafner [_KeyZee_]
|
bedbb8eb45
|
adding get_stats()
|
2020-11-24 16:46:01 +01:00 |
Fafner [_KeyZee_]
|
2ae7d87420
|
Adding uniq_domains
|
2020-11-24 16:44:43 +01:00 |
Raphaël Vinot
|
9c382890f5
|
fix: Improve lokking on error in capture
|
2020-11-10 20:27:01 +01:00 |
Raphaël Vinot
|
354f269218
|
new: Integrate categorization in indexing
|
2020-11-09 16:02:54 +01:00 |
Raphaël Vinot
|
9243f84295
|
fix: No exception if body_hash is not indexed
|
2020-11-06 15:59:13 +01:00 |
Raphaël Vinot
|
99c939fd7f
|
chg: Rename scrape -> capture everywhere
Related to #118
|
2020-11-03 16:34:52 +01:00 |
Raphaël Vinot
|
69d34b803c
|
fix: Duplicates on index.
|
2020-11-03 12:21:14 +01:00 |
Raphaël Vinot
|
8b1e3585ea
|
chg: Improve initial caching.
|
2020-10-29 23:25:20 +01:00 |
Raphaël Vinot
|
69f65c9a87
|
chg: Update cache on tree visualization
|
2020-10-29 17:55:43 +01:00 |
Raphaël Vinot
|
5f4a97e11b
|
fix: Typing meta
|
2020-10-29 14:19:17 +01:00 |
Raphaël Vinot
|
06a4c2cc87
|
fix: Speedup sorted cache
|
2020-10-29 14:06:38 +01:00 |
Raphaël Vinot
|
da85aa0fb7
|
fix: Pre-sort entries on index
|
2020-10-29 13:29:13 +01:00 |
Raphaël Vinot
|
2802cfd46c
|
new: Add captures categorization
|
2020-10-28 18:49:15 +01:00 |
Raphaël Vinot
|
451553e60d
|
fix: Do not overwrite the urlnodes
|
2020-10-27 01:52:28 +01:00 |
Raphaël Vinot
|
068c92d5a4
|
new: API to query hostnames
|
2020-10-27 01:42:00 +01:00 |
Raphaël Vinot
|
39f88e9121
|
new: API to query URLs
|
2020-10-27 00:02:18 +01:00 |
Raphaël Vinot
|
bdc0488e38
|
new: Hash lookup method
|
2020-10-23 20:51:15 +02:00 |
Raphaël Vinot
|
c6c4da981c
|
chg: Improve start/stop
|
2020-10-22 16:41:00 +02:00 |
Raphaël Vinot
|
f062895801
|
fix: Add max size to lru_cache
|
2020-10-13 13:03:57 +02:00 |
Raphaël Vinot
|
4f52804361
|
chg: Make new version of mypy happy
|
2020-10-12 12:15:07 +02:00 |
Raphaël Vinot
|
90a9ff9bb5
|
chg: Refactoring, add get_hashes
|
2020-10-09 18:05:25 +02:00 |
Raphaël Vinot
|
be2cd18d2b
|
chg: Cleanup mypy config
|
2020-10-01 11:48:00 +02:00 |
Raphaël Vinot
|
9ce06b8821
|
chg: Add resolved IP on hostnode page
|
2020-09-29 14:24:18 +02:00 |
Raphaël Vinot
|
ef6eb1c253
|
new: Initial support for CNAME lookups
|
2020-09-25 17:23:10 +02:00 |
Raphaël Vinot
|
3ce4f85118
|
fix: Avoid 500 when sanejs is unreachable
|
2020-09-25 12:02:40 +02:00 |
Raphaël Vinot
|
9663c5de87
|
fix: Properly load the config files when needed.
|
2020-09-21 16:54:07 +02:00 |
Raphaël Vinot
|
a7ecaee8c4
|
chg: If apikey is None in the modules, break immediately.
|
2020-09-21 16:47:49 +02:00 |
Raphaël Vinot
|
7a34095d9c
|
new: Config option for Flask IP and Port, reorganize config loading
|
2020-09-21 16:41:30 +02:00 |
Raphaël Vinot
|
b06f02e101
|
new: Expose rendered content on hostnode page
|
2020-09-09 15:04:21 +02:00 |
Raphaël Vinot
|
2b237a7416
|
new: Use rendered HTML
|
2020-09-08 19:18:55 +02:00 |
Raphaël Vinot
|
457829a23f
|
chg: cleanup and improve views
|
2020-09-04 18:40:51 +02:00 |
Raphaël Vinot
|
9f4c77d5d2
|
chg: Cleanups, allow to add context from ressources page
|
2020-09-03 16:32:53 +02:00 |
Raphaël Vinot
|
407a9a5511
|
new: Download ressource from ressources page
|
2020-09-03 14:39:38 +02:00 |
Raphaël Vinot
|
6d22085ceb
|
fix: do not crash on empty malicious file
|
2020-09-01 18:39:49 +02:00 |
Raphaël Vinot
|
ecbbceb781
|
fix: Typing
|
2020-09-01 17:54:54 +02:00 |
Raphaël Vinot
|
b16a5768ea
|
chg: Refactoring and normalizing the known hashes lookups
|
2020-09-01 17:33:36 +02:00 |
Raphaël Vinot
|
aee62136ab
|
fix: cache unknown hashes in sanejs
|
2020-09-01 00:51:44 +02:00 |
Raphaël Vinot
|
4a9a50e289
|
chg: Add icon for hostnodes with empty content.
|
2020-08-31 15:21:41 +02:00 |
Raphaël Vinot
|
a5c210a5de
|
new: Add checkmark for legitimate nodes.
|
2020-08-28 19:11:19 +02:00 |
Raphaël Vinot
|
6866b49c87
|
chg: Add support for legitimate content context
|
2020-08-28 18:24:14 +02:00 |
Raphaël Vinot
|
97b5b2d77f
|
chg: Working prototype for user-defined context
|
2020-08-28 18:03:52 +02:00 |
Raphaël Vinot
|
611b2a8ff5
|
fix: Crash if there are no ressources
|
2020-08-27 16:15:07 +02:00 |
Raphaël Vinot
|
9eb08beb3c
|
chg: Add load/dump of known legit captures
|
2020-08-27 15:48:56 +02:00 |
Raphaël Vinot
|
60b767d08f
|
chg: Normalize ressources details display, refactoring
|
2020-08-27 12:57:10 +02:00 |
Raphaël Vinot
|
11598fc3ba
|
chg: Major refactoring of the contextualization of the body contents
|
2020-08-26 16:52:59 +02:00 |
Raphaël Vinot
|
1b0cdde84e
|
chg: Normalize known content lookup
|
2020-08-25 18:00:16 +02:00 |
Raphaël Vinot
|
4d6237aae5
|
fix: Make mypy happy
|
2020-08-25 15:52:54 +02:00 |
Raphaël Vinot
|
c75b9e65fe
|
chg: Remove known content from SaneJS module
|
2020-08-25 15:51:08 +02:00 |
Raphaël Vinot
|
bdb726ca9e
|
new: Add malicious listing on hostnode
|
2020-08-24 15:31:53 +02:00 |
Raphaël Vinot
|
c5aabcf4a3
|
new: Initial commit to mark body responses as legitimate/malicious
|
2020-08-20 19:39:03 +02:00 |
Raphaël Vinot
|
fa57435433
|
chg: Sanity check in email send
|
2020-08-20 15:05:27 +02:00 |
Raphaël Vinot
|
339d0dd7e0
|
fix: Make mypy happy
|
2020-08-10 20:11:26 +02:00 |
Raphaël Vinot
|
83d8f49be9
|
fix: Missing change for last commit
|
2020-08-10 20:05:47 +02:00 |
Raphaël Vinot
|
2873773ee7
|
new: trigger to hide a capture from the front page (admin only)
|
2020-08-10 12:35:16 +02:00 |
Raphaël Vinot
|
f17acbce60
|
fix: Referer cannot be None
|
2020-07-20 23:04:11 +02:00 |
Raphaël Vinot
|
1c5f4f5710
|
fix: Do not index private captures on public instance
|
2020-07-20 13:39:08 +02:00 |
Raphaël Vinot
|
c7db7829a7
|
chg: More macros
|
2020-07-15 18:04:34 +02:00 |
Raphaël Vinot
|
0468317a20
|
chg: Improve rendering, bump deps.
|
2020-07-15 01:35:55 +02:00 |
Raphaël Vinot
|
89725b8ed0
|
new: Download embedded resource
|
2020-07-11 02:10:56 +02:00 |
Raphaël Vinot
|
e5e4e4972e
|
new: Add visualisation for embedded resources.
|
2020-07-10 18:57:16 +02:00 |
Raphaël Vinot
|
db0d5b3f2d
|
chg: Remove unused parameter.
|
2020-07-08 15:52:26 +02:00 |
Raphaël Vinot
|
0c5501016c
|
fix: Rebuild caches when tree doesn't exists
|
2020-07-08 15:52:26 +02:00 |
Raphaël Vinot
|
23419a31b9
|
fix: cleanup
|
2020-07-08 15:52:26 +02:00 |
Raphaël Vinot
|
4851436999
|
chg: More improvements in body hash display
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
bfa39223c8
|
chg: Improve hash lookup rendering
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
34a5dff055
|
chg: Refactoring, use capture UUID insted of path as reference for public methods
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
95a08f5655
|
fix: Make mypy happy
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
a4bc2b2818
|
new: Improve hostnode popup, center tree on node
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
29c78d3485
|
chg: Cleanup and improve index rendering
|
2020-07-08 15:51:45 +02:00 |
Raphaël Vinot
|
7772706262
|
new: Display body hashes matches on hostnode page
|
2020-07-08 15:51:01 +02:00 |
Raphaël Vinot
|
6e13622d88
|
fix: Typing
|
2020-07-08 15:51:01 +02:00 |
Raphaël Vinot
|
6bb26c4358
|
new: Initial support for body hashes index
|
2020-07-08 15:51:01 +02:00 |
Raphaël Vinot
|
e6aded6c12
|
chg: Add domains aggregation by cookies names
|
2020-07-08 15:51:01 +02:00 |
Raphaël Vinot
|
67b41ca8fb
|
chg: Improve intergration of cookies indexing
|
2020-07-08 15:51:01 +02:00 |
Raphaël Vinot
|
e6c9f6dada
|
new: Add capture meta on the related cookies page
|
2020-07-08 15:42:13 +02:00 |
Raphaël Vinot
|
5ae7f0f7e4
|
new: Initial version of cookies indexing
|
2020-07-08 15:42:13 +02:00 |
Raphaël Vinot
|
d18f5f4f88
|
fix: Docker, capture form, error message.
|
2020-07-08 02:25:15 +02:00 |
Raphaël Vinot
|
33fb74ae83
|
new: Add referer to initial URL
|
2020-07-08 00:37:29 +02:00 |
Raphaël Vinot
|
4cfe91392a
|
chg: Docker, default to env variable for splash URL
|
2020-07-07 13:51:09 +02:00 |
Raphaël Vinot
|
316338a213
|
chg: Better error handling
|
2020-07-06 15:33:00 +02:00 |
Raphaël Vinot
|
605dc00989
|
fix: Properly handle json dumps
|
2020-07-06 14:16:17 +02:00 |
Raphaël Vinot
|
1d77a2001e
|
chg: Bump deps, improve error handling
|
2020-07-03 18:25:16 +02:00 |
Raphaël Vinot
|
41acc544ef
|
chg: depth from UI is a string...
|
2020-06-29 19:23:16 +02:00 |
Raphaël Vinot
|
ce6deabdad
|
fix: convert max_depth from config to int
|
2020-06-29 19:21:38 +02:00 |
Raphaël Vinot
|
afe95c28f6
|
new: Add max depth for scraping.
|
2020-06-29 18:00:56 +02:00 |
Raphaël Vinot
|
05de56022f
|
chg: Use capture UUID as a reference everywhere
|
2020-06-29 12:01:31 +02:00 |