104bf6883f
new: [second-level-tlds] Add generator and update to latest version
2021-06-10 10:36:57 +02:00
aee9919ac4
fix: [moz] Moz is not Mozilla
2021-06-10 10:02:20 +02:00
8a2c3dc760
fix: [publicdns] IP addresses should be cidr
2021-06-10 09:59:04 +02:00
08a0a32813
generate-stackpath.py: Added scraper on website to get new link for ipblocks.txt file
2021-06-02 14:51:58 +02:00
Alexandre Dulaunoy
329092675a
new: [tools] simple python script to generate the list of warning-lists
...
in Markdown
2021-05-01 10:59:07 +02:00
Alexandre Dulaunoy
16313352e2
chg: [tools] fix the link for the documentation generator
2021-05-01 10:32:52 +02:00
Alexandre Dulaunoy
9f426c8099
chg: [stackpath] host IPv6 addresses are without subnet
2021-04-21 16:42:55 +02:00
68666c298e
Gave execute permissions to generate_phone_numbers.py
2021-04-21 12:10:30 +02:00
37bc01ae21
Created new Stackpath CDN IP list
2021-04-21 12:09:59 +02:00
b2ed7f766b
fix: python 3.9 compat, take 2
2020-11-24 18:49:43 +01:00
46fd402d43
fix: python 3.9 compat
2020-11-24 18:47:20 +01:00
7154bfc01f
add: A few more phone numbers regexes
2020-10-31 22:06:20 +01:00
6d49d3386f
add: Added regexes for the american fictitious numbers in the list
2020-10-30 23:29:18 +01:00
8a629209f2
chg: Changed name to be displayed as warning and description
2020-10-27 11:19:05 +01:00
a50c06c9eb
chg: Turned the regexes for audiovisual works into a single one
2020-10-27 10:38:45 +01:00
543406dff4
add: New Warninglist for phone numbers that should never be attributed
...
- First examples filling the list of regexes: the
phone numbers used for audiovisual works, or
the communications companies internal numbers.
Those phone numbers are reserved and should
never be given to any user
- We'll add as well the numbers reserved for the
american audiovisual works soon
2020-10-27 04:03:44 +01:00
14fc1e9cf0
Adding replacement for wildcard and dash inputs
2020-09-30 15:43:57 -04:00
7e0e72bb5c
Changed matching algorithm to string
2020-09-23 15:58:03 -04:00
e121ff3925
Changed matching algorithm to string
2020-09-23 15:47:49 -04:00
365fb4c00e
Changed matching algorithm to string
2020-09-23 15:46:41 -04:00
7c1de70ce0
Changed matching algorithm to string
...
Example for a dangerous entry: dropbox.com with the hostname algorithm and url as a valid attribute type means that https://dropbox.com/malicious/files.exe would get excluded from the automation systems when using the warninglist.
I've changed the algorithm to full string matches.
2020-09-16 00:27:03 +02:00
a9c163ef20
Refactor last scripts, logging, central directory for downloads
...
* Refactored generate_moz-top50.py
* Download all file to new /tmp file to centralize all downloads
* Add central logging to generators.log file
* Create Bash script that generates all warninglists
* Add /tmp folder and extra files to .gitignore
* Start adding exception handling in download_to_file and write_to_file
2020-07-27 10:44:30 +02:00
610292e90f
Refactor more generators
2020-07-21 13:42:50 +02:00
623ccd6c44
remove extra .txt extension from downloaded filed
2020-07-21 09:13:15 +02:00
9dc6ea9ca8
add check if downloaded file has changed on server before downloading
2020-07-21 09:11:51 +02:00
d3e87dc7ae
Refactor code to make it simpler/more uniform
2020-07-21 01:21:28 +02:00
d32eb23a58
Chg generator-publicdns: work with new CSV format
...
1. The CSV format has changed with the update on 2020-07-14.
2. The script also generates IPv4, IPv6, and the hostname lists at once.
3. Downloaded file added to .gitignore
2020-07-21 00:38:50 +02:00
cb52a472e6
solved LGTM alerts
2020-07-17 12:42:34 +02:00
9ba47c1463
Added multiple lists from Cisco Umbrella list. Solves issue #24 and #13
2020-07-17 12:31:29 +02:00
049475ab22
add .gitignore for downloaded files, refactor code for generators: use central module, remove useless code, fix minor issues
2020-07-17 10:06:06 +02:00
c00b21de5f
Fix Microsoft Azure generator: format changed from XML to JSON + download link changed.
2020-07-17 09:25:01 +02:00
e0b3968635
Merge tranco scripts,:generate_tranco.py generates both full and 10k list
2020-07-17 09:23:39 +02:00
1d59d7f6f5
Automatically copy output to list.json file in correct folder
2020-07-16 10:02:59 +02:00
1f15bba220
Refactored mozilla certificate generator: solve relative path issue, remove unused code, refactor structure of code
2020-07-15 16:28:34 +02:00
c924d72db5
renamed cisco top1m to top1k to reflect reality
2020-07-15 02:06:02 +02:00
2bbbb52e1c
chg: chmod +x for new scripts in tools folder
2020-07-14 16:28:07 +02:00
Alexandre Dulaunoy
7a28900663
Merge branch 'main' of github.com:MISP/misp-warninglists into main
2020-07-08 14:54:14 +02:00
f493aa451d
Update TLDs list
2020-07-06 16:20:48 +02:00
e0cd7a65a0
Added tranco10k
2020-06-11 10:07:04 -05:00
4766c457c0
Validate values in CI
2020-05-22 19:28:04 +02:00
300d823638
chg: Add script to make lists unique, and sort the keys.
...
Update covid lists.
2020-04-03 13:37:17 +02:00
c14634a031
new: added covid generators / lists
2020-04-01 12:00:55 +02:00
109470bf0d
Create generate_majestic-million.py
...
Pulls top 10K of the most referred to hosts from Majestic Million.
2020-02-03 01:01:22 -06:00
17952df91e
add script to generate warning list from Tranco ( https://tranco-list.eu/ )
2020-01-10 15:27:07 +01:00
622bd3510e
add: [wikimedia] adds a warning list for wikimedia infrastructure.
2019-09-12 11:09:16 +02:00
d22c6902b3
Added list and tool to generate list for cloudflare IP ranges.
2019-07-24 17:35:39 +02:00
Alexandre Dulaunoy
4d8ce3986f
Merge pull request #106 from SteveClement/tools
...
fix: [alexa] The generator wants to decode things ;)
2019-04-24 15:38:25 +02:00
Steve Clement
14a675a70a
fix: [moz500] Fix the confusion about Moz.com and Mozilla.com
2019-04-24 14:42:41 +09:00
Steve Clement
88ad201afd
fix: [alex] The generator wants to decode things ;)
2019-04-24 10:51:11 +09:00
Steve Clement
1be60b805a
fix: [tools] Made python scripts executable.
2019-04-24 10:38:42 +09:00
Jakub Onderka
Alexandre Dulaunoy
Steve Clement