1675 Commits (master)

Author	SHA1	Message	Date
terrtia	6f2668eff1	chg: [tools] reprocess objects by type	2024-03-13 14:04:23 +01:00
terrtia	13372f8c85	fix: [dashboard] fix objects links	2024-03-13 11:58:40 +01:00
terrtia	eb6adc4b98	chg: [modules] improve error output	2024-03-13 10:11:03 +01:00
terrtia	27b2679ba6	fix: [tracker] fix yara content error on empty message	2024-03-13 10:07:11 +01:00
terrtia	dc0545dfd0	chg: [RetroHunt] retro hunt on messages	2024-03-12 13:54:56 +01:00
terrtia	9031376b50	fix: [FeederImporter] add all objects in queue	2024-03-12 13:43:19 +01:00
terrtia	800098540b	chg: [correlation] description: get message content + don't show etag, cookie_name and hhash by default	2024-03-11 15:33:30 +01:00
terrtia	197ff0222d	chg: [lang] improve language detection + UI: manual translation and detection	2024-03-08 15:26:06 +01:00
terrtia	40a32fec75	fix: [module] fix SQLInjectionDetection object ID	2024-03-08 13:54:14 +01:00
terrtia	87dc619171	chg: [chats] add heatmap nb messages/hour for all messages	2024-03-05 10:56:36 +01:00
terrtia	ad039e4720	chg: [api] get chat/subchannel/thread messages	2024-02-29 16:18:22 +01:00
terrtia	e1e9609ad9	chg: [api] get object + get investigation	2024-02-29 14:56:45 +01:00
terrtia	d5e830c591	chg: [domains] add crawler status stats by domain type pie chart	2024-02-28 14:19:47 +01:00
terrtia	0d55725e28	chg: [crawler] add monthly crawled domains stats	2024-02-27 14:56:48 +01:00
terrtia	afe13185d9	chg: [test] add basic API test	2024-02-27 11:03:11 +01:00
terrtia	775b7fa868	fix: [tests] fix tests + global new file content	2024-02-27 10:15:40 +01:00
terrtia	ad63651838	chg: [api] refactor blueprint	2024-02-26 15:35:48 +01:00
terrtia	f980ab5145	fix: [favicon] fix misp object export	2024-02-21 14:56:11 +01:00
terrtia	81c4dde7b0	fix: [favicon] crawler favicon	2024-02-21 14:34:20 +01:00
terrtia	c219febd71	chg: [favicon] add favicons objects + correlation	2024-02-21 14:18:09 +01:00
terrtia	6fa8f6e0bc	fix: [vanity] length > 5 clusters	2024-02-19 15:54:15 +01:00
terrtia	784579baef	fix: [vanity] debug	2024-02-19 15:52:26 +01:00
terrtia	cb4345c871	fix: [vanity] debug	2024-02-19 15:50:20 +01:00
terrtia	4c5a0b9906	fix: [vanity] length+1 min vanity = 2	2024-02-19 15:38:51 +01:00
terrtia	273e264659	fix: [vanity] vanity builder	2024-02-19 15:31:55 +01:00
terrtia	88fbe36f70	fix: [vanity] update id domains is up + min vanity = 4	2024-02-19 15:23:16 +01:00
terrtia	f07a4b422b	chg: [vanity] add vanity domain explorer + fix blurred screenshot + fix languages search filter	2024-02-19 15:14:37 +01:00
terrtia	495ceea73d	fix: [api] get titles domain	2024-02-16 11:56:48 +01:00
terrtia	782677e8ff	chg: [domain] sort onion vanity name by prefix	2024-02-16 10:31:45 +01:00
terrtia	93d1b6fc1f	fix: [tracker] fix message trackers	2024-02-15 16:22:00 +01:00
terrtia	1e8f8e9e9c	chg: [tracker] list of last matchs, show description	2024-02-15 12:02:01 +01:00
terrtia	1a3ffe70c1	fix: [retro_hunt] api delete paused retro_hunt	2024-02-15 11:45:55 +01:00
terrtia	48ce4a76a4	fix: [retro_hunt] delete paused retro_hunt	2024-02-15 11:42:10 +01:00
terrtia	7d42315419	chg: [retro_hunt] show match + fix None subtype	2024-02-15 11:31:17 +01:00
terrtia	c260455d14	chg: [titles] add endpoints to get all titles	2024-02-15 10:13:06 +01:00
terrtia	4cf3d628db	chg: [messages] manual translation by language + add new kvrocks lang namespace	2024-02-14 16:43:29 +01:00
terrtia	50bfd92105	chg: [chat] add endpoints to download chat, subchannel and thread, + fix message translated by default	2024-02-13 16:13:18 +01:00
terrtia	304afd00aa	chg: [exif] add debug	2024-02-07 10:32:18 +01:00
terrtia	38a918e485	fix: [flask] fix escape import	2024-02-06 11:56:39 +01:00
terrtia	4168d07118	fix: [chats] fix chats image importer	2024-02-06 11:13:45 +01:00
terrtia	4c1d058e6d	fix: [language] catch libretranslate exception	2024-02-05 14:19:42 +01:00
terrtia	aa56e71631	fix: [language] crawled items, force gcld3 detection	2024-02-05 14:10:19 +01:00
terrtia	99fedf9855	fix: [LAUNCH] update screen status	2024-02-05 11:32:49 +01:00
terrtia	c1529b217d	fix: [LAUNCH] fix killall	2024-02-05 11:09:41 +01:00
terrtia	ff59dcf81d	fix: [LAUNCH] fix ENV error message	2024-02-05 09:57:53 +01:00
terrtia	b6eb6c9016	fix: [crawler] fix capture None domain name	2024-02-02 14:48:19 +01:00
terrtia	a7fd838329	fix: [tags] fix invalid tags	2024-02-02 14:41:57 +01:00
terrtia	1a2d1e41f5	fix: [tags] fix typo	2024-02-02 11:53:25 +01:00
terrtia	7295f7b32d	chg: [LAUNCH] change restart flags	2024-02-02 11:42:10 +01:00
terrtia	e6d63e3b8f	Merge remote-tracking branch 'refs/remotes/origin/dev' into dev	2024-02-02 11:33:09 +01:00
terrtia	45bdc2707b	chg: [tags] add Tag class	2024-02-02 11:15:08 +01:00
Steve Clement	1ac2fd026d	Merge branch 'ail-project:dev' into dev	2024-02-01 09:59:39 +01:00
terrtia	e4f21f05cc	fix: [D4] fix module cache	2024-01-30 14:31:09 +01:00
terrtia	5fab2326e6	fix: [misp export] fix empty event on module start	2024-01-30 11:45:43 +01:00
terrtia	194ae960fc	fix: [crawlers] fix capture return error code	2024-01-30 11:35:43 +01:00
terrtia	d1608e89e1	fix: [crawlers] fix errored capture queue	2024-01-30 11:29:42 +01:00
terrtia	fbd7e2236a	fix: [crawlers] fix errored capture start time	2024-01-30 11:24:12 +01:00
terrtia	2db8587d03	chg: [Hosts] improve perf + regex timeout + cache DNS results	2024-01-30 10:28:50 +01:00
terrtia	a10119fb6a	chg: [kvrocks] j -4 install + update to latest version	2024-01-29 16:41:59 +01:00
terrtia	896b411eaf	chg: [translation] translate chats info, users info and subchannels names	2024-01-29 14:36:53 +01:00
terrtia	6363a4f1cf	fix: [chat view] fix created_at + filter	2024-01-29 10:52:18 +01:00
terrtia	61bccecdab	chg: [chats] show NB messages by chat	2024-01-26 16:06:42 +01:00
terrtia	699453f079	chg: [relationships] add relationship engine + WIP relationships between forwarded messages/chats	2024-01-26 15:42:46 +01:00
terrtia	9c6619aed3	fix: [crawler] log UNKNOWN timeout	2024-01-26 15:31:32 +01:00
Steve Clement	9cfd230661	chg: [LAUNCH] make sure reload works	2024-01-18 11:53:13 +01:00
Steve Clement	1a02b6d768	chg: [LAUNCH] Addded a "restart" option, killAll + launchAuto	2024-01-18 11:42:37 +01:00
terrtia	edf0c4c454	chg: [message] UI translate message object	2024-01-16 14:38:29 +01:00
terrtia	f586baa0c5	fix: [Language] fix language source	2024-01-16 12:04:39 +01:00
terrtia	1c46bb4296	chg: [Language] replace pycld3 by gcld3 + clean text before language detection	2024-01-15 14:17:15 +01:00
terrtia	d6d67f6a4c	chg: [hosts] filter onion	2024-01-09 14:31:55 +01:00
terrtia	bdaa4c51c9	fix: [hosts] fix number of hosts extracted	2024-01-09 12:15:40 +01:00
terrtia	5094b2dcbb	fix: [DomClassifier] improve perf	2024-01-09 11:38:54 +01:00
terrtia	7263a9777c	fix: [DomClassifier] improve perf	2024-01-09 11:26:05 +01:00
terrtia	5c25ec0fea	fix: [DomClassifier] improve perf	2024-01-09 11:24:54 +01:00
terrtia	f851cc9f42	fix: [queue] save last timout in cache	2024-01-09 11:19:01 +01:00
terrtia	bd2ca4b319	fix: [crawler] fix api create_task	2024-01-09 09:47:49 +01:00
terrtia	a14c0484af	fix: [tags] debug	2024-01-08 15:34:32 +01:00
terrtia	be4feb7799	fix: [tags] debug	2024-01-08 15:28:31 +01:00
terrtia	259f29c10c	fix: [tags] debug	2024-01-08 15:27:08 +01:00
terrtia	bfc018f929	fix: [tags] debug	2024-01-08 15:21:37 +01:00
terrtia	f95d32d6dc	fix: [tags] debug	2024-01-08 15:02:39 +01:00
terrtia	4b21cc2939	fix: [tags] debug	2024-01-08 15:01:40 +01:00
terrtia	07c51e111f	fix: [tags] debug	2024-01-08 14:59:58 +01:00
terrtia	efb8b2d0d3	fix: [tags] debug	2024-01-08 14:58:18 +01:00
terrtia	0abc3fee0e	fix: [tags] debug	2024-01-08 14:56:20 +01:00
terrtia	8bf67cf3b6	fix: [tags] remove invalid tags	2024-01-08 14:51:40 +01:00
terrtia	d7c8262653	fix: [keys module] fix tags	2024-01-08 14:24:51 +01:00
terrtia	9716351196	Merge branch 'master' into dev	2024-01-08 14:05:34 +01:00
terrtia	0af5ea9d48	chg: [queues] timeout obj after 2 days	2024-01-08 13:50:56 +01:00
terrtia	c05f4d7833	chg: [chats] get user message ids by chat	2024-01-02 17:15:45 +01:00
Markus Lassfolk	81f9a860d4	Fix IndexError in get_last_tag_from_remote function ... This commit adds a check to ensure that the output from the subprocess command in the get_last_tag_from_remote function has a sufficient number of lines before attempting to access specific indices. This change prevents the IndexError that occurred when the git command's output was shorter than expected.	2024-01-01 21:54:29 +01:00
terrtia	70bb6757f8	chg: [correlation] UI chats filters + correation user-account/username	2023-12-13 11:51:53 +01:00
terrtia	9221e532c4	fix: [crawlers] fix task start	2023-12-12 11:32:33 +01:00
terrtia	7e9ea48c81	fix: [crawler] debug	2023-12-12 11:20:21 +01:00
terrtia	cdfc9f64e5	fix: [crawler] debug	2023-12-12 11:13:58 +01:00
terrtia	847d004c13	fix: [crawler] debug	2023-12-12 11:05:23 +01:00
terrtia	c20c41c50f	fix: [libinjection] memory leak, disable module	2023-12-12 10:30:40 +01:00
terrtia	4529a76d13	fix: [zmq importer] fix object source name	2023-12-12 10:14:59 +01:00
terrtia	235539ea42	fix: [crawler] fix capture start time	2023-12-11 09:30:09 +01:00
terrtia	5fc9b1403f	chg: [chats] add pagination	2023-12-11 00:46:15 +01:00
terrtia	943a873124	chg: [tags] searech messages tags by daterange	2023-12-09 16:50:43 +01:00
terrtia	73185f19fd	chg: [categ] messages, bypass categ module + fix correlation	2023-12-08 15:40:05 +01:00
terrtia	5b808ed416	fix: [translate] fix exception	2023-12-08 14:38:55 +01:00
terrtia	1c52c187ad	fix: [api] fix add crawler capture return	2023-12-08 10:37:58 +01:00
terrtia	cea96863ba	fix: [language] libretranslate unreachable	2023-12-07 14:59:40 +01:00
terrtia	3e591d95bc	fix: [retro_hunt] fix daterange	2023-12-07 14:40:51 +01:00
terrtia	a382b572c6	chg: [crawler] push onion discovery capture_uuid to another AIL	2023-12-07 11:28:35 +01:00
terrtia	38ce17bc8a	chg: [chats] add chats participants + user-accounts basic template	2023-12-06 16:26:26 +01:00
terrtia	bef4e69a68	chg: [chats] translate messages on demand	2023-12-04 15:47:58 +01:00
terrtia	941838ab76	chg: [chats] add discord threads, Forum channel	2023-12-04 10:26:02 +01:00
terrtia	93ef541862	chg: [chats] add messages threads	2023-11-29 16:28:25 +01:00
terrtia	f766cbebda	chg: [chats] add message file-name object + str emoticon reactions	2023-11-27 16:25:09 +01:00
terrtia	fa4f1269cd	chg: [chat users] add users icons	2023-11-24 16:35:45 +01:00
terrtia	2b8e9b43f3	chg: [chats] factorise heatmap + chat icon	2023-11-24 15:05:19 +01:00
terrtia	9fbd3f4bb6	chg: [chats] add message and blur blocks	2023-11-16 14:50:03 +01:00
terrtia	2ea5f82760	chg: [queues] add new image queue + add exif module	2023-11-15 15:45:25 +01:00
terrtia	4142ad9884	chg: [chats] add image object + show message image	2023-11-15 14:12:50 +01:00
terrtia	7bf0fe8992	chg: [chats] add heatmap nb week messages by hour	2023-11-13 14:10:24 +01:00
terrtia	54c57ea35b	chg: [chats] message object template	2023-11-08 15:46:05 +01:00
terrtia	e0f70c5072	fix: [investigations] delete obj	2023-11-08 13:35:13 +01:00
terrtia	ce989adbd3	fix: [chats] fix username meta	2023-11-08 13:07:00 +01:00
terrtia	207a6524d7	fix: [languages] fix language module	2023-11-08 11:59:06 +01:00
terrtia	6c77ca5136	fix: [chats] fix chat username	2023-11-08 11:25:30 +01:00
terrtia	5068287c76	Merge branch 'master' into dev	2023-11-08 10:42:02 +01:00
terrtia	acef57bb36	fix: [tags] fix galaxies synonyms	2023-11-08 10:41:32 +01:00
terrtia	e7f060c23d	chg: [messages] refactor get_messages_meta + add basic message template	2023-11-08 10:31:51 +01:00
terrtia	4cc9608a3f	chg: [chats explorer] show chats/subchannels creation date	2023-11-07 15:24:40 +01:00
terrtia	3fb1d0ad74	fix: [chats] fix messages replies	2023-11-07 11:24:24 +01:00
terrtia	b1d5399607	chg: [chats] add UI shortcut + networks list + show chats/subchannels info	2023-11-06 16:38:31 +01:00
terrtia	789210bcba	chg: [chats] improve UI + fix importer	2023-11-06 14:08:23 +01:00
terrtia	9125119764	chg: [chats] add chats explorer v0	2023-11-02 16:28:33 +01:00
terrtia	c5cef5fd00	chg: [core] merge master + fix object subtype correlation stats	2023-10-12 13:53:00 +02:00
terrtia	6978764b02	fix: [module] fix module obj type: language + mail	2023-10-11 14:53:12 +02:00
terrtia	623ba455ff	fix: [queues] fix ended duplicate + sync queue	2023-10-11 14:31:13 +02:00
terrtia	676b0f84ef	chg: [module + queues] track + rename object global ID by module	2023-10-11 12:06:01 +02:00
terrtia	eae57fb813	chg: [importers obj_type] importers queues: add feeder source + object global ID	2023-10-05 16:24:28 +02:00
terrtia	daf9f6fb5d	fix: [chats] message css + reply ID	2023-10-04 14:40:13 +02:00
terrtia	fee3332edb	fix: [tracker] delete yara rule, fix filter by object type	2023-09-29 15:43:37 +02:00
terrtia	c19b1f34e3	chg: [ail queues] merge	2023-09-08 10:52:55 +02:00
terrtia	bb3dad2873	chg: [objs processed] xxhash messages	2023-09-07 10:38:03 +02:00
Jean-Louis Huynen	68c17c3fbc	chg: [crawlers] submit cookies to the crawler task API	2023-08-31 16:13:20 +02:00
Jean-Louis Huynen	ed0423118e	chg: [crawlers] submit a single cookie to the crawler task API	2023-08-31 15:42:44 +02:00
Terrtia	7c73f0944a	fix: [items source] filter invalid item sources	2023-08-29 14:03:26 +02:00
Terrtia	099253f854	fix: [json importer] fix empty source name	2023-08-29 13:50:16 +02:00
Terrtia	24969610cc	fix: [items source] fix empty sources list	2023-08-29 11:59:39 +02:00
Terrtia	b32f110285	chg: [chat + user-account] correlations + usernames timeline	2023-08-28 16:29:38 +02:00
Terrtia	546d6538fd	chg: [mail exporter] add obj content extract for each regex match	2023-08-24 14:37:50 +02:00
Terrtia	c01b806ae3	chg: [mail exporter] add obj content extract for each yara rule match	2023-08-24 11:11:57 +02:00
Terrtia	843b2d3134	fix: correlations	2023-08-23 16:13:20 +02:00
Terrtia	4e3784922c	fix: typo	2023-08-23 11:47:39 +02:00
Terrtia	2145eb7b8a	fix: [title] fix None title	2023-08-23 11:46:37 +02:00
Terrtia	f44c5509da	chg: [titles] add yara tracker on title + tags domains if unsafe title tags	2023-08-23 11:16:22 +02:00
Terrtia	045aab6f34	fix: [module pasties] fix module name	2023-08-21 15:52:33 +02:00
Terrtia	0cb7431e10	chg: [modules] crawl pasties domains	2023-08-21 15:49:32 +02:00
Terrtia	3c1813ba02	chg: [core] add telegram importer + Chat object + message Object + add timeline engine	2023-08-18 11:05:21 +02:00
Terrtia	f05c7b6a93	fix: [module extractor] fix invalid yara offset	2023-08-08 11:27:57 +02:00
Terrtia	4dc5527c1a	fix: [module extractor] fix invalid yara offset	2023-08-08 11:26:16 +02:00
Terrtia	529a24c191	chg: [module extrator] add debug	2023-08-08 10:40:44 +02:00
Terrtia	bd7aa979bd	chg: [module extrator] add debug	2023-08-08 10:36:58 +02:00
Terrtia	2691000d0c	chg: [telegram fedeer] use meta of the new feeder	2023-08-02 15:49:12 +02:00
Terrtia	14a76a91d9	fix: [tags ui] fix galaxy, get number of tags enabled + add toolip helper	2023-08-01 11:07:06 +02:00
Terrtia	1aa0bd8a0e	fix: [settings] fix edit user	2023-07-31 16:25:28 +02:00
Terrtia	64252ddf75	fix: [Tracker] edit tracker filters	2023-07-31 16:00:31 +02:00
Terrtia	4eb5b4c4b9	fix: [MailExporter] improve logs + SMTP over SSL: add an option to validate self signed CA ... This exporter use STARTTLS by defaults. If the SMTP server does not support the STARTTLS extension, the client will switch to SSL instead.	2023-07-28 11:10:21 +02:00
Terrtia	68dffcd26b	chg: [api crawler] fix response + add cookiejar, proxy and frequency parameters	2023-07-25 15:57:11 +02:00
Terrtia	fe2769308b	fix: [crawler] fix down domain + domain redirection history	2023-07-18 14:30:00 +02:00
Terrtia	77e8cc6c02	Merge branch 'master' of github.com:ail-project/ail-framework	2023-07-17 15:47:32 +02:00
Terrtia	a9485928db	chg: [HHHash] add HHHash object and correlation https://www.foo.be/2023/07/HTTP-Headers-Hashing_HHHash	2023-07-17 15:47:17 +02:00
fukusuket	22dff19004	fix: [domain search] returns an empty string when searched with invalid characters	2023-07-14 18:28:34 +09:00
Terrtia	780b4f9f9c	fix: [background update] fix logger	2023-07-12 14:37:59 +02:00
Terrtia	73bfe614df	chg: [updater] refactor background updater + add v5.2 update	2023-07-12 11:36:47 +02:00
Terrtia	28c647d370	chg: [crawler har] compress HAR	2023-07-10 15:56:34 +02:00
Terrtia	c719990125	fix: [crawler] add timeout to Unknown captures	2023-07-10 11:23:44 +02:00
Thirion Aurélien	8f0e7f1434	Merge pull request #174 from fukusuket/fix-500-erro-when-invalid-lacus-url ... fix: [crawler] add exception handing for `ping_lacus`	2023-07-10 10:05:20 +02:00
fukusuket	fcd6a98fb0	fix: [crawler] added existence check for screenshot set	2023-07-09 12:50:13 +09:00
fukusuket	e35924ec22	fix: [crawler] add exception handing for ping_lacus	2023-07-08 12:11:25 +09:00
Terrtia	482fc21b5e	chg: [correlation] correlation graph, add an option to hide an object/node by pressing H + reset correlation graph	2023-07-07 16:29:32 +02:00
Terrtia	450ebdd789	chg: [etag] add new etag object	2023-07-06 11:26:32 +02:00
Terrtia	35f678245f	fix: [decoded] fix download file	2023-06-30 16:22:30 +02:00
Terrtia	0fa0984ec4	fix: [updater] fix db checker	2023-06-30 16:19:38 +02:00
Terrtia	fa8b4a67d9	fix: [correlation tags] fix tag all objects	2023-06-29 15:02:57 +02:00
Terrtia	279ec00990	fix: [correlation card decoded meta] mimetype + size	2023-06-29 11:40:57 +02:00
Terrtia	fce63d52a0	fix: [updater] remove old ARDB env	2023-06-29 09:07:00 +02:00
Terrtia	d6a24c0357	fix: [hunter + misp export] fix misp event json export + retro hunt date search and description	2023-06-29 08:48:50 +02:00
Terrtia	b4d536f174	chg: [investigation] add object comment	2023-06-26 16:28:31 +02:00
Terrtia	a0686eefcf	fix: [pystemon importer] fix base64 encoding	2023-06-26 11:24:04 +02:00
Terrtia	f1f33d6c1b	chg: [pystemon importer] fix gzipped pastes	2023-06-22 16:34:14 +02:00
Terrtia	47e1343187	fix: [crawler] same capture uuid if a domain is already crawled	2023-06-22 16:09:18 +02:00
Terrtia	b459498db2	chg: [queues] track object + check if object processed	2023-06-22 15:38:04 +02:00
Terrtia	4567c9d400	chg: [correlation graph] show message if max_nodes reached + fix cookie-name sparkline	2023-06-20 11:23:58 +02:00
Terrtia	501d10bbbd	chg: [crawler] auto tag crawled domains	2023-06-20 08:11:44 +02:00
Terrtia	c02b9f2a81	fix: [IPAdress module] empty queue if no IP ranges provided	2023-06-19 15:17:41 +02:00
Terrtia	b64c70e0cd	fix: [retro hunt] fix object tag queue + decoded content	2023-06-19 10:53:15 +02:00
Terrtia	6c21a8f8cf	fix: [daterange object] fix objects by date	2023-06-19 08:55:56 +02:00
Terrtia	84bc2c0669	chg: [correlation] add an option to remove max number of nodes if max_node == 0	2023-06-18 16:09:57 +02:00
Terrtia	e9539e640b	fix: [crawler] fix incomplete response	2023-06-18 15:09:09 +02:00
Terrtia	f8fd037bd2	chg: [object cookie-name] add new cookie-name object + correlation	2023-06-16 15:39:13 +02:00
Terrtia	ee76d20277	fix: [user] fix get user token #163	2023-06-14 14:53:07 +02:00
Terrtia	b38819fdaa	fix: [user] fix user delete #163	2023-06-14 14:40:01 +02:00
Terrtia	d4152462f5	chg: [titles] add title IDs and contents search	2023-06-13 15:29:11 +02:00