Commit Graph

82 Commits (818839d506b596ff03523ab93a2925955b8e5bfe)

Author SHA1 Message Date
cyber288 e121ff3925
Changed matching algorithm to string 2020-09-23 15:47:49 -04:00
cyber288 365fb4c00e
Changed matching algorithm to string 2020-09-23 15:46:41 -04:00
Andras Iklody 7c1de70ce0
Changed matching algorithm to string
Example for a dangerous entry: dropbox.com with the hostname algorithm and url as a valid attribute type means that https://dropbox.com/malicious/files.exe would get excluded from the automation systems when using the warninglist.

I've changed the algorithm to full string matches.
2020-09-16 00:27:03 +02:00
Kevin Holvoet a9c163ef20 Refactor last scripts, logging, central directory for downloads
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
Kevin Holvoet 610292e90f Refactor more generators 2020-07-21 13:42:50 +02:00
Kevin Holvoet 623ccd6c44 remove extra .txt extension from downloaded filed 2020-07-21 09:13:15 +02:00
Kevin Holvoet 9dc6ea9ca8 add check if downloaded file has changed on server before downloading 2020-07-21 09:11:51 +02:00
Kevin Holvoet d3e87dc7ae Refactor code to make it simpler/more uniform 2020-07-21 01:21:28 +02:00
Kevin Holvoet d32eb23a58 Chg generator-publicdns: work with new CSV format
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
2020-07-21 00:38:50 +02:00
Kevin Holvoet cb52a472e6 solved LGTM alerts 2020-07-17 12:42:34 +02:00
Kevin Holvoet 9ba47c1463 Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13 2020-07-17 12:31:29 +02:00
Kevin Holvoet 049475ab22 add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues 2020-07-17 10:06:06 +02:00
Kevin Holvoet c00b21de5f Fix Microsoft Azure generator: format changed from XML to JSON + download link changed. 2020-07-17 09:25:01 +02:00
Kevin Holvoet e0b3968635 Merge tranco scripts,:generate_tranco.py generates both full and 10k list 2020-07-17 09:23:39 +02:00
Kevin Holvoet 1d59d7f6f5 Automatically copy output to list.json file in correct folder 2020-07-16 10:02:59 +02:00
Kevin Holvoet 1f15bba220 Refactored mozilla certificate generator: solve relative path issue, remove unused code, refactor structure of code 2020-07-15 16:28:34 +02:00
Kevin Holvoet c924d72db5 renamed cisco top1m to top1k to reflect reality 2020-07-15 02:06:02 +02:00
Kevin Holvoet 2bbbb52e1c chg: chmod +x for new scripts in tools folder 2020-07-14 16:28:07 +02:00
Alexandre Dulaunoy 7a28900663
Merge branch 'main' of github.com:MISP/misp-warninglists into main 2020-07-08 14:54:14 +02:00
Jakub Onderka f493aa451d Update TLDs list 2020-07-06 16:20:48 +02:00
GlennHD e0cd7a65a0
Added tranco10k 2020-06-11 10:07:04 -05:00
Jakub Onderka 4766c457c0 Validate values in CI 2020-05-22 19:28:04 +02:00
Raphaël Vinot 300d823638 chg: Add script to make lists unique, and sort the keys.
Update covid lists.
2020-04-03 13:37:17 +02:00
iglocska c14634a031
new: added covid generators / lists 2020-04-01 12:00:55 +02:00
GlennHD 109470bf0d
Create generate_majestic-million.py
Pulls top 10K of the most referred to hosts from Majestic Million.
2020-02-03 01:01:22 -06:00
Trey Darley 17952df91e add script to generate warning list from Tranco (https://tranco-list.eu/) 2020-01-10 15:27:07 +01:00
Jean-Louis Huynen 622bd3510e
add: [wikimedia] adds a warning list for wikimedia infrastructure. 2019-09-12 11:09:16 +02:00
elhoim d22c6902b3 Added list and tool to generate list for cloudflare IP ranges. 2019-07-24 17:35:39 +02:00
Alexandre Dulaunoy 4d8ce3986f
Merge pull request #106 from SteveClement/tools
fix: [alexa] The generator wants to decode things ;)
2019-04-24 15:38:25 +02:00
Steve Clement 14a675a70a fix: [moz500] Fix the confusion about Moz.com and Mozilla.com 2019-04-24 14:42:41 +09:00
Steve Clement 88ad201afd fix: [alex] The generator wants to decode things ;) 2019-04-24 10:51:11 +09:00
Steve Clement 1be60b805a fix: [tools] Made python scripts executable. 2019-04-24 10:38:42 +09:00
Steve Clement 190312cf0f chg: [moz500] Added Pages too. Updated list 2019-04-24 10:36:22 +09:00
Steve Clement f41f976ce6 chg: [moz500] Added info how to regenerate, added provisional urls/files
to topPages.
2019-04-24 10:23:40 +09:00
Steve Clement 9e0b2ebc75 new: [list] Added Mozilla Top 500 domains 2019-04-24 09:45:56 +09:00
Steve Clement 8f1fe94b49 new: [tool] Generate The Moz top 500 Domain list from https://moz.com/top500 2019-04-23 18:00:07 +09:00
Vincent Brillault af7e5a188c
CAs: Fix final new line in json 2019-03-29 16:57:54 +01:00
Vincent Brillault d4e749c3e6
CAs: Fix json indentation (2 spaces, not 4) 2019-03-29 16:55:34 +01:00
Vincent Brillault da0dabc01b
Mozilla CA/intermediate: also match x509-fingerprint-* 2019-03-29 16:39:22 +01:00
Vincent Brillault a1c0e83bd4
Add warning lists based on Mozilla's trusted CA and Intermediates 2019-03-29 16:23:22 +01:00
Raphaël Vinot 04e8f468d9 fix: Wrong file name in the scripts 2019-03-25 11:43:27 +01:00
Alexandre Dulaunoy 48a5e32d50
new: [disposal-email] added 2019-03-24 12:02:30 +01:00
Alexandre Dulaunoy 5ed5403157
new: [VPN] lists of common VPN IPv4 and IPv6 addresses added
Source of the IPv4/IPv6 is https://github.com/ejrv/VPNs
2019-03-24 11:17:59 +01:00
Alexandre Dulaunoy a568c14332
chg: [o365 tools] fix title of the IP address warning list 2019-03-10 09:28:31 +01:00
Alexandre Dulaunoy 6e97b8bf8a
chg: [o365] separate Microsoft Office 365 lists (hostname and IP addresses) 2019-03-10 09:27:12 +01:00
Alexandre Dulaunoy 0cc3ad9d82
Merge branch 'master' of https://github.com/kx499/misp-warninglists into kx499-master 2019-03-10 09:13:03 +01:00
Faber 91d77374ff updated MS O365 script to handle json and updated list.json 2019-03-09 19:43:13 -05:00
Alexandre Dulaunoy 342a6d2ad5
chg: [tools] alexa script fixed 2019-03-09 17:25:27 +01:00
Alexandre Dulaunoy 58db3ef4b0
chg: [tools] fix cisco script 2019-03-09 17:23:42 +01:00
Alexandre Dulaunoy d24febd23c
Merge pull request #93 from ater49/master
Adding university domains warninglist
2019-03-01 22:43:14 +01:00