Commit Graph

116 Commits (a6da32235f04b4c26fe9d1478bf385410232c872)

Author SHA1 Message Date
Alexandre Dulaunoy e72f112dfb
chg: [generator] logging updated 2022-02-01 16:52:09 +01:00
Alexandre Dulaunoy 697aec2821
fix: [stackpath] now a txt file is included on the stackpath 2022-01-04 15:45:01 +01:00
Alexandre Dulaunoy 40b8d3834a
chg: [[publicdns] update the golden list 2021-09-23 10:50:09 +02:00
Jakub Onderka 5687ebf976 new: Include dnscrypt-resolvers IP addresses to public-dns lists 2021-09-15 11:44:51 +02:00
Alexandre Dulaunoy d68a78085c
chg: [tools] add golden IPv4 DNS server in the generate-publicdns.py script
One of the source is regularly dropping 9.9.9.9 as a source of
publicdns. This additional list can be used to ensure these known golden
DNS public resolver are there.
2021-07-13 09:21:21 +02:00
Jakub Onderka 00d5e40ac1 new: List of known SMTP receiving IP addresses 2021-06-14 23:26:44 +02:00
Jakub Onderka e7401c9cbe new: List of known SMTP sending IP ranges 2021-06-14 23:09:39 +02:00
Jakub Onderka 6c9eff2dcc new: Generator for Akamai 2021-06-13 15:48:55 +02:00
Jakub Onderka 98f7390e88 chg: [cidr] Consolidate CIDR networks 2021-06-13 14:54:48 +02:00
Alexandre Dulaunoy 0c8d517366
Merge pull request #188 from JakubOnderka/office365-cn
chg: [microsoft-office365-cn] Update to latest version
2021-06-13 07:36:08 +02:00
Jakub Onderka 7dfdb1dbaf chg: [microsoft-office365-cn] Update to latest version 2021-06-12 18:38:05 +02:00
Jakub Onderka d66a51e537 new: [crl] Genreate domains and IPs directly from Mozilla intermediate list 2021-06-12 11:28:13 +02:00
Jakub Onderka 8f2691535b fix: [mozilla-IntermediateCA] Typo 2021-06-10 17:57:35 +02:00
Jakub Onderka c9320d8ca9 new: [google-gmail-sending-ips] Add generator and update to latest version 2021-06-10 16:26:26 +02:00
Jakub Onderka 14ee657daf new: [google-gcp] Add generator and update to latest version 2021-06-10 16:07:29 +02:00
Jakub Onderka a6c3455944 new: [azure] List for Azure China, Germany and US Gov cloud 2021-06-10 11:15:32 +02:00
Jakub Onderka 104bf6883f new: [second-level-tlds] Add generator and update to latest version 2021-06-10 10:36:57 +02:00
Jakub Onderka aee9919ac4 fix: [moz] Moz is not Mozilla 2021-06-10 10:02:20 +02:00
Jakub Onderka 8a2c3dc760 fix: [publicdns] IP addresses should be cidr 2021-06-10 09:59:04 +02:00
Kevin Holvoet 08a0a32813 generate-stackpath.py: Added scraper on website to get new link for ipblocks.txt file 2021-06-02 14:51:58 +02:00
Alexandre Dulaunoy 329092675a
new: [tools] simple python script to generate the list of warning-lists
in Markdown
2021-05-01 10:59:07 +02:00
Alexandre Dulaunoy 16313352e2
chg: [tools] fix the link for the documentation generator 2021-05-01 10:32:52 +02:00
Alexandre Dulaunoy 9f426c8099
chg: [stackpath] host IPv6 addresses are without subnet 2021-04-21 16:42:55 +02:00
Kevin Holvoet 68666c298e Gave execute permissions to generate_phone_numbers.py 2021-04-21 12:10:30 +02:00
Kevin Holvoet 37bc01ae21 Created new Stackpath CDN IP list 2021-04-21 12:09:59 +02:00
Raphaël Vinot b2ed7f766b fix: python 3.9 compat, take 2 2020-11-24 18:49:43 +01:00
Raphaël Vinot 46fd402d43 fix: python 3.9 compat 2020-11-24 18:47:20 +01:00
chrisr3d 7154bfc01f
add: A few more phone numbers regexes 2020-10-31 22:06:20 +01:00
chrisr3d 6d49d3386f
add: Added regexes for the american fictitious numbers in the list 2020-10-30 23:29:18 +01:00
chrisr3d 8a629209f2
chg: Changed name to be displayed as warning and description 2020-10-27 11:19:05 +01:00
chrisr3d a50c06c9eb
chg: Turned the regexes for audiovisual works into a single one 2020-10-27 10:38:45 +01:00
chrisr3d 543406dff4
add: New Warninglist for phone numbers that should never be attributed
- First examples filling the list of regexes: the
  phone numbers used for audiovisual works, or
  the communications companies internal numbers.
  Those phone numbers are reserved and should
  never be given to any user
- We'll add as well the numbers reserved for the
  american audiovisual works soon
2020-10-27 04:03:44 +01:00
hugeek 14fc1e9cf0 Adding replacement for wildcard and dash inputs 2020-09-30 15:43:57 -04:00
cyber288 7e0e72bb5c
Changed matching algorithm to string 2020-09-23 15:58:03 -04:00
cyber288 e121ff3925
Changed matching algorithm to string 2020-09-23 15:47:49 -04:00
cyber288 365fb4c00e
Changed matching algorithm to string 2020-09-23 15:46:41 -04:00
Andras Iklody 7c1de70ce0
Changed matching algorithm to string
Example for a dangerous entry: dropbox.com with the hostname algorithm and url as a valid attribute type means that https://dropbox.com/malicious/files.exe would get excluded from the automation systems when using the warninglist.

I've changed the algorithm to full string matches.
2020-09-16 00:27:03 +02:00
Kevin Holvoet a9c163ef20 Refactor last scripts, logging, central directory for downloads
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
Kevin Holvoet 610292e90f Refactor more generators 2020-07-21 13:42:50 +02:00
Kevin Holvoet 623ccd6c44 remove extra .txt extension from downloaded filed 2020-07-21 09:13:15 +02:00
Kevin Holvoet 9dc6ea9ca8 add check if downloaded file has changed on server before downloading 2020-07-21 09:11:51 +02:00
Kevin Holvoet d3e87dc7ae Refactor code to make it simpler/more uniform 2020-07-21 01:21:28 +02:00
Kevin Holvoet d32eb23a58 Chg generator-publicdns: work with new CSV format
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
2020-07-21 00:38:50 +02:00
Kevin Holvoet cb52a472e6 solved LGTM alerts 2020-07-17 12:42:34 +02:00
Kevin Holvoet 9ba47c1463 Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13 2020-07-17 12:31:29 +02:00
Kevin Holvoet 049475ab22 add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues 2020-07-17 10:06:06 +02:00
Kevin Holvoet c00b21de5f Fix Microsoft Azure generator: format changed from XML to JSON + download link changed. 2020-07-17 09:25:01 +02:00
Kevin Holvoet e0b3968635 Merge tranco scripts,:generate_tranco.py generates both full and 10k list 2020-07-17 09:23:39 +02:00
Kevin Holvoet 1d59d7f6f5 Automatically copy output to list.json file in correct folder 2020-07-16 10:02:59 +02:00
Kevin Holvoet 1f15bba220 Refactored mozilla certificate generator: solve relative path issue, remove unused code, refactor structure of code 2020-07-15 16:28:34 +02:00