Commit Graph

489 Commits (d2755d409b5a883656aef4144f2e446b902361f6)

Author SHA1 Message Date
Alexandre Dulaunoy d2755d409b
Merge pull request #164 from cyber288/main
Change hostname type to string type for multiple lists
2020-09-24 10:34:00 +02:00
Alexandre Dulaunoy 346fa68e47
Merge pull request #163 from rhaist/patch-1
Create requirements.txt
2020-09-24 10:33:11 +02:00
cyber288 7e0e72bb5c
Changed matching algorithm to string 2020-09-23 15:58:03 -04:00
cyber288 5e03dc549a
Changed matching algorithm to string 2020-09-23 15:57:34 -04:00
cyber288 625a484306
Update version number 2020-09-23 15:56:05 -04:00
cyber288 85310d7a11
Update version number 2020-09-23 15:55:34 -04:00
cyber288 58481c2213
Update version number 2020-09-23 15:55:01 -04:00
cyber288 818839d506
Fix date 2020-09-23 15:54:31 -04:00
cyber288 709bdd94f0
Changed matching algorithm to string 2020-09-23 15:53:55 -04:00
cyber288 47c341fe1c
Changed matching algorithm to string 2020-09-23 15:52:53 -04:00
cyber288 29bd7bf7a0
Changed matching algorithm to string 2020-09-23 15:52:00 -04:00
cyber288 2d08c4ce35
Changed matching algorithm to string 2020-09-23 15:51:18 -04:00
cyber288 5e85c66bae
Changed matching algorithm to string 2020-09-23 15:50:36 -04:00
cyber288 e121ff3925
Changed matching algorithm to string 2020-09-23 15:47:49 -04:00
cyber288 365fb4c00e
Changed matching algorithm to string 2020-09-23 15:46:41 -04:00
Robert Haist 6494e55411
Create requirements.txt
Those are the additional Python3 requirements I needed to generate all the lists.
2020-09-23 09:33:02 +02:00
Alexandre Dulaunoy bcbb62d734
chg: [automatic] updated 2020-09-16 09:04:39 +02:00
Andras Iklody e0736d0b7a
fix: changed parsing algorithm to string, see #7c1de70 2020-09-16 00:27:55 +02:00
Andras Iklody 7c1de70ce0
Changed matching algorithm to string
Example for a dangerous entry: dropbox.com with the hostname algorithm and url as a valid attribute type means that https://dropbox.com/malicious/files.exe would get excluded from the automation systems when using the warninglist.

I've changed the algorithm to full string matches.
2020-09-16 00:27:03 +02:00
Alexandre Dulaunoy fa7cbb52f7
chg: [automatic] updated 2020-09-11 14:33:42 +02:00
Alexandre Dulaunoy ae3f68338a
Merge pull request #162 from Wiscy-Security/main
Refactor last scripts, central logging, central directory for downloads, automation script
2020-09-11 14:26:00 +02:00
Kevin Holvoet d7f01edbac Merge upstream, update lists, fix conflicts.
Merge remote-tracking branch 'upstream/main' into main
2020-09-11 13:48:51 +02:00
Alexandre Dulaunoy 17e6f1b761
Merge pull request #161 from bartblaze/patch-1
Add new domains
2020-09-11 06:48:47 +02:00
Bart 40ebba72d3
Add new domains 2020-09-10 21:27:06 +02:00
Alexandre Dulaunoy f19e7274e3
chg: [tranco] updated 2020-08-11 10:35:50 +02:00
Alexandre Dulaunoy e2bb53492a
chg: [public-dns] updated 2020-08-11 10:33:32 +02:00
Alexandre Dulaunoy fa325114ab
chg: [microsoft-azure] updated 2020-08-11 10:31:56 +02:00
Alexandre Dulaunoy 963381c839
chg: [tld] updated to the latest version 2020-08-11 10:29:32 +02:00
Alexandre Dulaunoy a2355251ac
chg: [aws] updated 2020-08-10 14:46:50 +02:00
Alexandre Dulaunoy fea9465eb5
chg: [office 365] updated 2020-08-10 14:45:58 +02:00
Alexandre Dulaunoy 746e0ba5d0
chg: [office 365] updated 2020-08-10 14:45:05 +02:00
Alexandre Dulaunoy fa350ba446
Merge pull request #157 from sustefil/fix-issue-156
Fix generator.py:download_to_file
2020-08-05 10:38:56 +02:00
Filip Suster 9557844065 Fix generator.py:download_to_file
When some script which is using generator.py module (e.g. generate-publicdns.py) is run for the
first time, the file is missing and unhandled exception is thrown
2020-08-03 17:24:39 +02:00
Kevin Holvoet aa81303fd0 Merge remote-tracking branch 'upstream/main' into main 2020-07-27 10:56:55 +02:00
Kevin Holvoet a9c163ef20 Refactor last scripts, logging, central directory for downloads
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
Alexandre Dulaunoy 4636dc5640
chg: [mozilla-intermediate-CA] updated to the latest version 2020-07-27 10:17:17 +02:00
Alexandre Dulaunoy 23418c5483
Merge pull request #154 from Wiscy-Security/main
Refactoring of code + updates of warninglists
2020-07-27 10:07:44 +02:00
Kevin Holvoet 610292e90f Refactor more generators 2020-07-21 13:42:50 +02:00
Kevin Holvoet 623ccd6c44 remove extra .txt extension from downloaded filed 2020-07-21 09:13:15 +02:00
Kevin Holvoet 9dc6ea9ca8 add check if downloaded file has changed on server before downloading 2020-07-21 09:11:51 +02:00
Kevin Holvoet d3e87dc7ae Refactor code to make it simpler/more uniform 2020-07-21 01:21:28 +02:00
Kevin Holvoet d32eb23a58 Chg generator-publicdns: work with new CSV format
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
2020-07-21 00:38:50 +02:00
Kevin Holvoet cb52a472e6 solved LGTM alerts 2020-07-17 12:42:34 +02:00
Kevin Holvoet 9ba47c1463 Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13 2020-07-17 12:31:29 +02:00
Kevin Holvoet feb3ceefda Merge remote-tracking branch 'upstream/main' into main 2020-07-17 10:24:24 +02:00
Kevin Holvoet 6837ba016c Updated lists after updating scripts 2020-07-17 10:10:04 +02:00
Kevin Holvoet 049475ab22 add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues 2020-07-17 10:06:06 +02:00
Kevin Holvoet c00b21de5f Fix Microsoft Azure generator: format changed from XML to JSON + download link changed. 2020-07-17 09:25:01 +02:00
Kevin Holvoet e0b3968635 Merge tranco scripts,:generate_tranco.py generates both full and 10k list 2020-07-17 09:23:39 +02:00
Kevin Holvoet 1d59d7f6f5 Automatically copy output to list.json file in correct folder 2020-07-16 10:02:59 +02:00