Commit Graph

453 Commits (a9c163ef20391b16e271eb84e8ba46d91a460907)

Author SHA1 Message Date
Kevin Holvoet a9c163ef20 Refactor last scripts, logging, central directory for downloads
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
Kevin Holvoet 610292e90f Refactor more generators 2020-07-21 13:42:50 +02:00
Kevin Holvoet 623ccd6c44 remove extra .txt extension from downloaded filed 2020-07-21 09:13:15 +02:00
Kevin Holvoet 9dc6ea9ca8 add check if downloaded file has changed on server before downloading 2020-07-21 09:11:51 +02:00
Kevin Holvoet d3e87dc7ae Refactor code to make it simpler/more uniform 2020-07-21 01:21:28 +02:00
Kevin Holvoet d32eb23a58 Chg generator-publicdns: work with new CSV format
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
2020-07-21 00:38:50 +02:00
Kevin Holvoet cb52a472e6 solved LGTM alerts 2020-07-17 12:42:34 +02:00
Kevin Holvoet 9ba47c1463 Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13 2020-07-17 12:31:29 +02:00
Kevin Holvoet feb3ceefda Merge remote-tracking branch 'upstream/main' into main 2020-07-17 10:24:24 +02:00
Kevin Holvoet 6837ba016c Updated lists after updating scripts 2020-07-17 10:10:04 +02:00
Kevin Holvoet 049475ab22 add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues 2020-07-17 10:06:06 +02:00
Kevin Holvoet c00b21de5f Fix Microsoft Azure generator: format changed from XML to JSON + download link changed. 2020-07-17 09:25:01 +02:00
Kevin Holvoet e0b3968635 Merge tranco scripts,:generate_tranco.py generates both full and 10k list 2020-07-17 09:23:39 +02:00
Kevin Holvoet 1d59d7f6f5 Automatically copy output to list.json file in correct folder 2020-07-16 10:02:59 +02:00
Kevin Holvoet 1f15bba220 Refactored mozilla certificate generator: solve relative path issue, remove unused code, refactor structure of code 2020-07-15 16:28:34 +02:00
Kevin Holvoet c924d72db5 renamed cisco top1m to top1k to reflect reality 2020-07-15 02:06:02 +02:00
Alexandre Dulaunoy 79dc2cba16
Merge pull request #153 from Wiscy-Security/main
Change tool/scripts permission + update tranco lists
2020-07-14 21:51:40 +02:00
Kevin Holvoet cde6059d73 Update Tranco & Tranco10k list 2020-07-14 16:38:12 +02:00
Kevin Holvoet 2bbbb52e1c chg: chmod +x for new scripts in tools folder 2020-07-14 16:28:07 +02:00
Alexandre Dulaunoy db0bf3305f
chg: [whats-my-ip] fix 152 2020-07-09 08:34:43 +02:00
Alexandre Dulaunoy 7a28900663
Merge branch 'main' of github.com:MISP/misp-warninglists into main 2020-07-08 14:54:14 +02:00
Alexandre Dulaunoy 1acdef286a
Merge pull request #151 from JakubOnderka/tlds-update
Update TLDs list
2020-07-08 07:23:18 +02:00
Jakub Onderka f493aa451d Update TLDs list 2020-07-06 16:20:48 +02:00
Alexandre Dulaunoy 8cd3e148cb
Merge pull request #150 from houey/patch-3
adding forms.gle which is for google forms.
2020-06-24 07:57:36 +02:00
Houston c6a361c7a1
adding forms.gle which is for google forms.
adding forms.gle to the list. This is a short link for Google Forms managed by Google Firebase
2020-06-23 17:06:04 -05:00
Alexandre Dulaunoy 6611f8b977
Merge pull request #149 from houey/patch-2
added gvt1.com to Google domains warning list.
2020-06-20 11:52:06 +02:00
Houston fbab4c59dc
added gvt1.com to Google domains warning list. 2020-06-19 11:03:32 -05:00
Alexandre Dulaunoy 82fdf528e9
Merge pull request #148 from GlennHD/master
Fixed typo in list.json of Tranco10k
2020-06-12 08:51:28 +02:00
GlennHD dfa999922e
Fixed typo
Fixed typo in list.json
2020-06-11 11:30:36 -05:00
Alexandre Dulaunoy 067bd58464
chg: [jq] all 2020-06-11 18:19:26 +02:00
Alexandre Dulaunoy 6ea9f8a687
Merge branch 'GlennHD-master' 2020-06-11 18:16:35 +02:00
Alexandre Dulaunoy 32d98c4588
chg: [tranco10k] jq all the things 2020-06-11 18:16:03 +02:00
GlennHD 8dc39ef910
Update README.md 2020-06-11 10:09:33 -05:00
GlennHD 6b0adad5e3
Added Tranco10k list 2020-06-11 10:09:17 -05:00
GlennHD a64b67ed85
Create tranco10k list.json 2020-06-11 10:08:36 -05:00
GlennHD e0cd7a65a0
Added tranco10k 2020-06-11 10:07:04 -05:00
Alexandre Dulaunoy 5a6c70003b
Merge pull request #146 from GlennHD/patch-3
Added Majestic Million to Readme
2020-06-10 12:46:38 +02:00
GlennHD e40852136f
Added Majestic Million to Readme
Added Majestic Million to Readme
2020-06-09 14:48:11 -05:00
Andras Iklody 1e97ba0da3
Merge pull request #145 from JakubOnderka/validate-values
Validate values in CI
2020-05-24 14:05:09 +02:00
Jakub Onderka 4766c457c0 Validate values in CI 2020-05-22 19:28:04 +02:00
Alexandre Dulaunoy 78590f3526
chg: [amazon-aws] updated to the latest version 2020-05-19 16:28:49 +02:00
Alexandre Dulaunoy 470b86dba4
chg: [microsoft-office365] updated to the latest version 2020-05-19 16:26:42 +02:00
Alexandre Dulaunoy 691b9a31c3
Merge pull request #143 from bartblaze/patch-9
Update list.json
2020-05-19 16:25:12 +02:00
Bart d22e04f776
Update list.json
Make hostname only, same for another one already in the list.
2020-04-30 20:48:23 +02:00
Bart cd9b83878e
Update list.json 2020-04-29 22:07:46 +02:00
Raphaël Vinot 2c116cbd1f fix: Sort entries 2020-04-23 11:56:05 +02:00
Andras Iklody 58150d8b22
chg: [covid] added covidmemory.lu 2020-04-13 11:59:05 +02:00
iglocska 28687d90d5
jq the covid lists 2020-04-07 06:14:20 +02:00
iglocska 64e86acbf5
Merge branch 'master' of github.com:MISP/misp-warninglists 2020-04-07 06:13:06 +02:00
iglocska 10ddaf06d2
chg: covid lists bumped 2020-04-07 06:10:44 +02:00