Commit Graph

132 Commits (aa939f482ffc29a53631529d67b76b49c48282e4)

Author SHA1 Message Date
csandera d91c41438b generator-vpn.py did not saved the generated list because of incorrect destination folder name 2023-01-10 19:41:59 +01:00
Alexandre Dulaunoy fe923ca1d7
new: [generate-chrome-crux-1m] New generator added for the Cached Chrome Top Million Websites
"Recent research showed that the top million most popular websites published by Google Chrome via their UX Report (CrUX) is significantly more accurate than other top lists like the Alexa Top Million and Tranco Top Million.

This repository caches a CSV version of the Chrome top sites, queried
from the CrUX data in Google BigQuery. You can browse all of the cached
lists here. The most up-to-date top million global websites can be
downloaded directly at:
https://raw.githubusercontent.com/zakird/crux-top-lists/main/data/global/current.csv.gz."

Ref: https://github.com/zakird/crux-top-lists
2023-01-02 11:18:35 +01:00
Alexandre Dulaunoy 811869148f
chg: [generator] add an option to handle gzip file
- It doesn't change the default behavior assuming it's a non-gzip file
- If the `gzip_enable` is set to `True` then the destination file is uncompressed
2023-01-02 11:16:49 +01:00
Jakub Onderka d4845c59fe
fix: [microsoft-azure-appid] Correct type 2022-12-01 13:38:11 +00:00
Alexandre Dulaunoy 917e009e27
fix: [tools] typo fixed 2022-12-01 11:32:27 +01:00
temperatestage a9386a4b42
Create generate-microsoft-azure-appid.py 2022-11-18 23:03:50 +11:00
David Cruciani 3adfc0e309 fix: [generator] remove header 2022-10-13 08:19:33 +02:00
David Cruciani 49922d0635 chg: [majestic_million] numbers parameter 2022-10-10 15:51:11 +02:00
Alexandre Dulaunoy 14d19fa58f
Merge pull request #226 from jberkers42/jberkers42-tenable
Add generator script for Tenable
2022-10-05 15:52:48 +02:00
John Berkers 3069e9ab00 Add generator script for Tenable 2022-10-05 18:03:03 +11:00
Alexandre Dulaunoy 70733ad25b
chg: [dns] add 1.1.1.1 as golden 2022-09-09 16:22:54 +02:00
Alexandre Dulaunoy 3e8138338e
chg: [tools] vpn generation updated following the removal of
github.com/ejrv/VPNs
2022-09-01 10:49:26 +02:00
Alexandre Dulaunoy cd1baa3de5
fix: [public-dns] Update the public-dns generator to remove non valid hostnames
Fix reported issue #214
2022-08-16 08:26:14 +02:00
John Berkers 8ea33dd725 Update StackPatch ipblocks URL 2022-08-16 11:42:09 +10:00
Alexandre Dulaunoy cbb5c8f6a7
chg: [tools] fix generate vpn uri 2022-03-31 13:47:25 +02:00
Loïc Jaquemet f974c15776
VPN list is outdated
github.com/ejrv/VPNs/ has not been refreshed in 24 months.
https://github.com/X4BNet/lists_vpn/ seems to have replicated that effort for ipv4. No new source found for ipv6
Pros:
- Code for generation is in repository

Cons:
- Unsure about quality. Lots of ASN as source of data.
- No IPV6 data

Comparable open source sources:
- Firehol https://github.com/firehol/blocklist-ipsets / datacenter.ipset . Source data (https://github.com/client9/ipcat) is outdated from 2019
- https://github.com/jhassine/server-ip-addresses limited provider list
- https://incolumitas.com/pages/Datacenter-IP-API/ API, smaller list

Others sources seem commercial (Udger, securitytrails)
2022-03-30 09:41:37 -06:00
Alexandre Dulaunoy e72f112dfb
chg: [generator] logging updated 2022-02-01 16:52:09 +01:00
Alexandre Dulaunoy 697aec2821
fix: [stackpath] now a txt file is included on the stackpath 2022-01-04 15:45:01 +01:00
Alexandre Dulaunoy 40b8d3834a
chg: [[publicdns] update the golden list 2021-09-23 10:50:09 +02:00
Jakub Onderka 5687ebf976 new: Include dnscrypt-resolvers IP addresses to public-dns lists 2021-09-15 11:44:51 +02:00
Alexandre Dulaunoy d68a78085c
chg: [tools] add golden IPv4 DNS server in the generate-publicdns.py script
One of the source is regularly dropping 9.9.9.9 as a source of
publicdns. This additional list can be used to ensure these known golden
DNS public resolver are there.
2021-07-13 09:21:21 +02:00
Jakub Onderka 00d5e40ac1 new: List of known SMTP receiving IP addresses 2021-06-14 23:26:44 +02:00
Jakub Onderka e7401c9cbe new: List of known SMTP sending IP ranges 2021-06-14 23:09:39 +02:00
Jakub Onderka 6c9eff2dcc new: Generator for Akamai 2021-06-13 15:48:55 +02:00
Jakub Onderka 98f7390e88 chg: [cidr] Consolidate CIDR networks 2021-06-13 14:54:48 +02:00
Alexandre Dulaunoy 0c8d517366
Merge pull request #188 from JakubOnderka/office365-cn
chg: [microsoft-office365-cn] Update to latest version
2021-06-13 07:36:08 +02:00
Jakub Onderka 7dfdb1dbaf chg: [microsoft-office365-cn] Update to latest version 2021-06-12 18:38:05 +02:00
Jakub Onderka d66a51e537 new: [crl] Genreate domains and IPs directly from Mozilla intermediate list 2021-06-12 11:28:13 +02:00
Jakub Onderka 8f2691535b fix: [mozilla-IntermediateCA] Typo 2021-06-10 17:57:35 +02:00
Jakub Onderka c9320d8ca9 new: [google-gmail-sending-ips] Add generator and update to latest version 2021-06-10 16:26:26 +02:00
Jakub Onderka 14ee657daf new: [google-gcp] Add generator and update to latest version 2021-06-10 16:07:29 +02:00
Jakub Onderka a6c3455944 new: [azure] List for Azure China, Germany and US Gov cloud 2021-06-10 11:15:32 +02:00
Jakub Onderka 104bf6883f new: [second-level-tlds] Add generator and update to latest version 2021-06-10 10:36:57 +02:00
Jakub Onderka aee9919ac4 fix: [moz] Moz is not Mozilla 2021-06-10 10:02:20 +02:00
Jakub Onderka 8a2c3dc760 fix: [publicdns] IP addresses should be cidr 2021-06-10 09:59:04 +02:00
Kevin Holvoet 08a0a32813 generate-stackpath.py: Added scraper on website to get new link for ipblocks.txt file 2021-06-02 14:51:58 +02:00
Alexandre Dulaunoy 329092675a
new: [tools] simple python script to generate the list of warning-lists
in Markdown
2021-05-01 10:59:07 +02:00
Alexandre Dulaunoy 16313352e2
chg: [tools] fix the link for the documentation generator 2021-05-01 10:32:52 +02:00
Alexandre Dulaunoy 9f426c8099
chg: [stackpath] host IPv6 addresses are without subnet 2021-04-21 16:42:55 +02:00
Kevin Holvoet 68666c298e Gave execute permissions to generate_phone_numbers.py 2021-04-21 12:10:30 +02:00
Kevin Holvoet 37bc01ae21 Created new Stackpath CDN IP list 2021-04-21 12:09:59 +02:00
Raphaël Vinot b2ed7f766b fix: python 3.9 compat, take 2 2020-11-24 18:49:43 +01:00
Raphaël Vinot 46fd402d43 fix: python 3.9 compat 2020-11-24 18:47:20 +01:00
chrisr3d 7154bfc01f
add: A few more phone numbers regexes 2020-10-31 22:06:20 +01:00
chrisr3d 6d49d3386f
add: Added regexes for the american fictitious numbers in the list 2020-10-30 23:29:18 +01:00
chrisr3d 8a629209f2
chg: Changed name to be displayed as warning and description 2020-10-27 11:19:05 +01:00
chrisr3d a50c06c9eb
chg: Turned the regexes for audiovisual works into a single one 2020-10-27 10:38:45 +01:00
chrisr3d 543406dff4
add: New Warninglist for phone numbers that should never be attributed
- First examples filling the list of regexes: the
  phone numbers used for audiovisual works, or
  the communications companies internal numbers.
  Those phone numbers are reserved and should
  never be given to any user
- We'll add as well the numbers reserved for the
  american audiovisual works soon
2020-10-27 04:03:44 +01:00
hugeek 14fc1e9cf0 Adding replacement for wildcard and dash inputs 2020-09-30 15:43:57 -04:00
cyber288 7e0e72bb5c
Changed matching algorithm to string 2020-09-23 15:58:03 -04:00