MISP
/
misp-warninglists
mirror of https://github.com/MISP/misp-warninglists
2
0
Fork 0
Code Issues Releases Wiki Activity
753 Commits
3 Branches
47 Tags
764 MiB
Commit Graph

129 Commits (d255fdecaf559eac4a624f73d77c44d4bdae1c1e)

Author SHA1 Message Date
Kevin Holvoet a9c163ef20 Refactor last scripts, logging, central directory for downloads 
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
 2020-07-27 10:44:30 +02:00
Kevin Holvoet 610292e90f Refactor more generators 2020-07-21 13:42:50 +02:00
Kevin Holvoet 623ccd6c44 remove extra .txt extension from downloaded filed 2020-07-21 09:13:15 +02:00
Kevin Holvoet 9dc6ea9ca8 add check if downloaded file has changed on server before downloading 2020-07-21 09:11:51 +02:00
Kevin Holvoet d3e87dc7ae Refactor code to make it simpler/more uniform 2020-07-21 01:21:28 +02:00
Kevin Holvoet d32eb23a58 Chg generator-publicdns: work with new CSV format 
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
 2020-07-21 00:38:50 +02:00
Kevin Holvoet cb52a472e6 solved LGTM alerts 2020-07-17 12:42:34 +02:00
Kevin Holvoet 9ba47c1463 Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13 2020-07-17 12:31:29 +02:00
Kevin Holvoet 049475ab22 add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues 2020-07-17 10:06:06 +02:00
Kevin Holvoet c00b21de5f Fix Microsoft Azure generator: format changed from XML to JSON + download link changed. 2020-07-17 09:25:01 +02:00
Kevin Holvoet e0b3968635 Merge tranco scripts,:generate_tranco.py generates both full and 10k list 2020-07-17 09:23:39 +02:00
Kevin Holvoet 1d59d7f6f5 Automatically copy output to list.json file in correct folder 2020-07-16 10:02:59 +02:00
Kevin Holvoet 1f15bba220 Refactored mozilla certificate generator: solve relative path issue, remove unused code, refactor structure of code 2020-07-15 16:28:34 +02:00
Kevin Holvoet c924d72db5 renamed cisco top1m to top1k to reflect reality 2020-07-15 02:06:02 +02:00
Kevin Holvoet 2bbbb52e1c chg: chmod +x for new scripts in tools folder 2020-07-14 16:28:07 +02:00
Alexandre Dulaunoy 7a28900663
Merge branch 'main' of github.com:MISP/misp-warninglists into main 2020-07-08 14:54:14 +02:00
Jakub Onderka f493aa451d Update TLDs list 2020-07-06 16:20:48 +02:00
GlennHD e0cd7a65a0
Added tranco10k 2020-06-11 10:07:04 -05:00
Jakub Onderka 4766c457c0 Validate values in CI 2020-05-22 19:28:04 +02:00
Raphaël Vinot 300d823638 chg: Add script to make lists unique, and sort the keys. 
Update covid lists.
 2020-04-03 13:37:17 +02:00
iglocska c14634a031
new: added covid generators / lists 2020-04-01 12:00:55 +02:00
GlennHD 109470bf0d
Create generate_majestic-million.py 
Pulls top 10K of the most referred to hosts from Majestic Million.
 2020-02-03 01:01:22 -06:00
Trey Darley 17952df91e add script to generate warning list from Tranco (https://tranco-list.eu/) 2020-01-10 15:27:07 +01:00
Jean-Louis Huynen 622bd3510e
add: [wikimedia] adds a warning list for wikimedia infrastructure. 2019-09-12 11:09:16 +02:00
elhoim d22c6902b3 Added list and tool to generate list for cloudflare IP ranges. 2019-07-24 17:35:39 +02:00
Alexandre Dulaunoy 4d8ce3986f
Merge pull request #106 from SteveClement/tools 
fix: [alexa] The generator wants to decode things ;)
 2019-04-24 15:38:25 +02:00
Steve Clement 14a675a70a fix: [moz500] Fix the confusion about Moz.com and Mozilla.com 2019-04-24 14:42:41 +09:00
Steve Clement 88ad201afd fix: [alex] The generator wants to decode things ;) 2019-04-24 10:51:11 +09:00
Steve Clement 1be60b805a fix: [tools] Made python scripts executable. 2019-04-24 10:38:42 +09:00
Steve Clement 190312cf0f chg: [moz500] Added Pages too. Updated list 2019-04-24 10:36:22 +09:00
Steve Clement f41f976ce6 chg: [moz500] Added info how to regenerate, added provisional urls/files 
to topPages.
 2019-04-24 10:23:40 +09:00
Steve Clement 9e0b2ebc75 new: [list] Added Mozilla Top 500 domains 2019-04-24 09:45:56 +09:00
Steve Clement 8f1fe94b49 new: [tool] Generate The Moz top 500 Domain list from https://moz.com/top500 2019-04-23 18:00:07 +09:00
Vincent Brillault af7e5a188c
CAs: Fix final new line in json 2019-03-29 16:57:54 +01:00
Vincent Brillault d4e749c3e6
CAs: Fix json indentation (2 spaces, not 4) 2019-03-29 16:55:34 +01:00
Vincent Brillault da0dabc01b
Mozilla CA/intermediate: also match x509-fingerprint-* 2019-03-29 16:39:22 +01:00
Vincent Brillault a1c0e83bd4
Add warning lists based on Mozilla's trusted CA and Intermediates 2019-03-29 16:23:22 +01:00
Raphaël Vinot 04e8f468d9 fix: Wrong file name in the scripts 2019-03-25 11:43:27 +01:00
Alexandre Dulaunoy 48a5e32d50
new: [disposal-email] added 2019-03-24 12:02:30 +01:00
Alexandre Dulaunoy 5ed5403157
new: [VPN] lists of common VPN IPv4 and IPv6 addresses added 
Source of the IPv4/IPv6 is https://github.com/ejrv/VPNs
 2019-03-24 11:17:59 +01:00
Alexandre Dulaunoy a568c14332
chg: [o365 tools] fix title of the IP address warning list 2019-03-10 09:28:31 +01:00
Alexandre Dulaunoy 6e97b8bf8a
chg: [o365] separate Microsoft Office 365 lists (hostname and IP addresses) 2019-03-10 09:27:12 +01:00
Alexandre Dulaunoy 0cc3ad9d82
Merge branch 'master' of https://github.com/kx499/misp-warninglists into kx499-master 2019-03-10 09:13:03 +01:00
Faber 91d77374ff updated MS O365 script to handle json and updated list.json 2019-03-09 19:43:13 -05:00
Alexandre Dulaunoy 342a6d2ad5
chg: [tools] alexa script fixed 2019-03-09 17:25:27 +01:00
Alexandre Dulaunoy 58db3ef4b0
chg: [tools] fix cisco script 2019-03-09 17:23:42 +01:00
Alexandre Dulaunoy d24febd23c
Merge pull request #93 from ater49/master 
Adding university domains warninglist
 2019-03-01 22:43:14 +01:00
ater49 120d6955c4 Correcting updater 2019-03-01 22:29:19 +01:00
ater49 a8c19f102f Correcting updater 2019-03-01 22:28:58 +01:00
ater49 536dae5db2 Adding updater for crl warninglist 2019-03-01 19:02:59 +01:00
ater49 c7cbb9a5d7 Adding update tool for university domains list 2019-03-01 18:48:22 +01:00
Robert Nixon 5fbec29345
Fixed TLD truncation issue 
Fixed TLD truncation issue
 2018-08-20 09:13:38 -04:00
Robert Nixon 9021404d6f
Added generate_alexa.py back and added type param 2018-08-10 09:17:49 -04:00
Robert Nixon d9457fc40a
Not sure why Travis is failing 2018-08-10 09:11:17 -04:00
Robert Nixon c468d00355
Added type for Travis 2018-08-10 08:57:27 -04:00
Robert Nixon 7246355806
Removing gen Alexa 2018-08-10 08:48:03 -04:00
Robert Nixon ed9700b150
New script to generate Cisco Umbrella Top 1000 2018-08-10 08:47:40 -04:00
Alexandre Dulaunoy 6b815d76f9
fix: various fixes + add number of elements in each lists 2018-03-20 22:34:19 +01:00
Alexandre Dulaunoy 9760088941
Merge branch 'master' of https://github.com/gizolka/misp-warninglists into gizolka-master 2018-03-20 21:29:34 +01:00
Joanna 820aef2a74 Created a converter of MISP warning lists to asciidoctor format 2018-03-20 11:35:49 +01:00
eCrimeLabs d1d5b6f109
Bugfix (l.append) 2018-02-22 10:57:31 +01:00
eCrimeLabs 3fb335eebd
Bugfix type 2018-02-22 07:53:31 +01:00
eCrimeLabs f42b95706e
Fixed typo 2018-02-21 23:30:02 +01:00
eCrimeLabs ab4bca460e
"type": "cidr", 2018-02-21 23:29:30 +01:00
eCrimeLabs 4716207fec
Update generate-amazon-aws.py 2018-02-21 22:05:02 +00:00
eCrimeLabs f0c996b8de
Generate json file of Amazon AWS IP's 2018-02-21 22:50:13 +01:00
Alexandre Dulaunoy ac2beeea02
add: Microsoft Azure Datacenter IP Ranges added including tool to 
generate the JSON.

Fix #43
 2017-12-29 15:39:52 +01:00
Alexandre Dulaunoy 23d4b68de3
add: list of Microsoft office365/azure in China + extraction tool added 
fix #42
 2017-12-29 15:30:57 +01:00
rmarsollier 02b4bf8a52 adding wikipedia scrapper for google domains 2017-06-19 09:09:33 +02:00
Raphaël Vinot cdef6f192e Update lists, add schema. 2017-02-12 21:01:36 +01:00
Nicolas Bareil f3b9178b55 typo in the name 2017-01-27 15:42:54 +01:00
Edvard Rejthar 7ad930e5fb Checks for open resolvers in the list of IPs. 
Is able to fetch the MISP warning list a say if there are some resolvers.
 2016-12-07 18:30:52 +01:00
Alexandre Dulaunoy 338241e499 Add version and name to the office365 warning list 2016-12-06 12:39:45 +01:00
Alexandre Dulaunoy 42816083a3 Office 365 URLs and IP address ranges added 2016-12-06 12:16:34 +01:00
Xavier Claude bf8b467a3e Add a tool to generate public dns resolver list 
The tool generate two lists, one for IPv4 (list4.json) and one for IPv6
(list6.json) to allow the user to only enable one of the two.

The list is downloaded from http://public-dns.info/ and a sample of the
list was tested with:

for dns in $( awk -F ","  '{ print $1 }' < nameservers.csv ) ; do dig +noedns @$dns google.com | grep NOERROR 1>/dev/null  || echo $dns ; done

~95% of the tested servers responded. So the list is not all crap.
 2016-08-03 19:59:55 +02:00
Xavier Claude 16a4265688 Actualy put alexa 1000 top domains in the output list 2016-08-03 15:54:13 +02:00
Xavier Claude 4075a58165 Write the alexa top1M zip file after download 2016-08-03 15:52:28 +02:00
Xavier Claude b856e9897c Fix alexa top1M url 2016-08-03 15:43:35 +02:00
Alexandre Dulaunoy 34ce028295 Alexa top 1000 MISP warning list added including generation tool 2016-05-28 21:31:40 +02:00