MISP
/
misp-warninglists
mirror of https://github.com/MISP/misp-warninglists
2
0
Fork 0
Code Issues Releases Wiki Activity
misp-warninglists/generate_all.sh

44 lines
1.3 KiB
Bash
Raw Normal View History

Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
#!/bin/bash
set -e
set -x
pushd tools
chg: [tools] removed Alexa from generate_all
2023-05-21 15:01:12 +02:00
# python3 generate_alexa.py # not updated since February 1, 2023 and offline after July 31, 2023
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
python3 generate-amazon-aws.py
python3 generate-cisco.py
python3 generate-cloudflare.py
python3 generate-covid.py
new: [crl] Genreate domains and IPs directly from Mozilla intermediate list
2021-06-11 20:55:48 +02:00
python3 generate-crl-ip-domains.py
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
python3 generate-disposal.py
# TODO: Google page on Wikipedia does not exist anymore
# Suggestion came to use a passivetotal whois search for org:Google LLC
#python3 generate-google.py > lists/google/list.json
chg: [warning-list] updated
2023-10-23 10:00:56 +02:00
#python3 generate_majestic-million.py -n 10000
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
python3 generate-microsoft-azure.py
python3 generate_mozilla_certificates.py
python3 generate_moz-top500.py
python3 generate-office365.py
add: New Warninglist for phone numbers that should never be attributed - First examples filling the list of regexes: the phone numbers used for audiovisual works, or the communications companies internal numbers. Those phone numbers are reserved and should never be given to any user - We'll add as well the numbers reserved for the american audiovisual works soon
2020-10-27 04:03:44 +01:00
python3 generate_phone_numbers.py
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
python3 generate-publicdns.py
Add stackpath to generate_all.sh script
2021-04-21 13:48:03 +02:00
python3 generate-stackpath.py
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
python3 generate-tlds.py
python3 generate_tranco.py
python3 generate-university-domain-list.py
python3 generate-vpn.py
python3 generate-wikimedia.py
fix: [script] typo in one of the script name
2021-06-10 19:05:31 +02:00
python3 generate-second-level-tlds.py
new: [google-gcp] Add generator and update to latest version
2021-06-10 16:06:37 +02:00
python3 generate-google-gcp.py
Added google bot updater + list
2023-04-05 14:33:08 +02:00
python3 generate-google-bot.py
new: [google-gmail-sending-ips] Add generator and update to latest version
2021-06-10 16:25:13 +02:00
python3 generate-google-gmail-sending-ips.py
new: List of known SMTP receiving IP addresses
2021-06-14 23:26:44 +02:00
python3 generate-smtp.py
Updates to reflect addition of Tenable Warninglists - Update generate_all.sh - Update README.md
2022-10-06 01:40:06 +02:00
python3 generate-tenable.py
new: [microsoft-azure-appid] generation added
2022-11-18 16:36:28 +01:00
python3 generate-microsoft-azure-appid.py
chg: [generate_all] add generate-chrome-crux-1m.py
2023-01-02 11:25:37 +01:00
python3 generate-chrome-crux-1m.py
Added DigitalSide.IT warninglist (false positive detection)
2023-06-22 15:34:20 +02:00
python3 generate-digitalside.py
new: [openai chatgpt] OpenAI source bot added
2023-08-08 11:45:53 +02:00
python3 generate-gptbot.py
new: [cisco umbrella block pages] hostname, IPv4 and IPv6 addresses some minor clean-up and final update
2023-08-25 16:11:41 +02:00
python3 generate-cisco-umbrella-blockpage.py
Refactor last scripts, logging, central directory for downloads * Refactored generate_moz-top50.py * Download all file to new /tmp file to centralize all downloads * Add central logging to generators.log file * Create Bash script that generates all warninglists * Add /tmp folder and extra files to .gitignore * Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
popd
./jq_all_the_things.sh