Commit Graph

115 Commits (f84f3042545a901c558cfe53add287630b42f4ed)

Author SHA1 Message Date
Jakub Onderka c7d17cad9d chg: [feed] Use less memory when parsing CSV feeds 2020-07-26 21:26:08 +02:00
Jakub Onderka 3df94f868f chg: [freetext] Various code fixes and optimisations 2020-07-13 21:12:26 +02:00
Jakub Onderka a34141c099 chg: [internal] More tests for ComplexTypeTool::checkFreeText 2020-07-13 15:29:17 +02:00
Jakub Onderka 06c7af4f83 chg: [internal] Simplified ComplexTypeTool::checkFreeText 2020-07-13 12:24:52 +02:00
Jakub Onderka e908a80656 fix: [internal] Reduce number of regexp in refang table 2020-07-13 10:13:26 +02:00
Jakub Onderka 9150c2be85 new: [freetext] Convert `[at]` to `@` and `hxtp` and `htxp` to `http`
Fixes #4908 and #4805
2020-07-06 18:22:40 +02:00
Jakub Onderka 73ccbd1f83 fix: [freetext] Handle IPv6 and punycode domains when import 2020-07-06 16:25:24 +02:00
Jakub Onderka cfe18d8c39
chg: [internal] Removed unused function 2020-02-16 14:13:34 +01:00
Andras Iklody d10495f789
Relaxed defanging rules, fixes #5203
Removed multiple dot implosion for links/urls
2019-09-23 14:26:42 +02:00
iglocska 69c6562888 fix: [freetext] Also trim out no-break spaces
░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░
░░░░░░░░███████████████░░░░░░░░░░
░░░░░░███████████████████░░░░░░░░
░░░░░███░░░░░░░░░░░░░░████░░░░░░░
░░░░██░░░░░░░░░░░░░░░░░░░███░░░░░
░░░██░░░░░░░░░░░░░░░░░░░░░███░░░░
░░██░░███████░░░░░░██████░░██░░░░
░██░░██─────██░░░░██────██░░██░░░
░██░░█▄▄▄▄▄▄▄██░░░█▄▄▄▄▄▄██░░██░░
░██░░████─────█░░░████────█░░░██░
░██░░█────────█░░░█───────█░░░██░
██░░░██──────██░░░██─────██░░░░██
██░░░░████████░░░░░███████░░░░░██
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░███████████████░░░░░░░░░░░█
█░░░████░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░█
██░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██
░██░░░░░░░░░░░░░░░░░░░░░░░░░░░░█░
░░███░░░░░░░░░░░░░░░░░░░░░░░░░██░
░░░░██░░░░░░░░░░░░░░░░░░░░░░░██░░
2019-04-26 09:39:10 +02:00
iglocska 8076dbfad1 fix: [refanging] Removed invalid pattern 2019-04-09 15:51:13 +02:00
iglocska 5eecd75e5b fix: [bug] Typo causing "\" to be stripped from attributes where it shouldn't be stripped 2019-04-09 15:48:29 +02:00
iglocska a3381b8196 new: [refanging] Attributes automatically refanged in beforeValidate, fixes #4442 2019-04-09 14:53:39 +02:00
iglocska 510b781762 fix: [freetext] Stop parsing dates as phone numbers 2019-03-20 13:10:53 +01:00
iglocska 4dd53eb8e3 fix: [freetext import] Handle cases where a value can be both a hash and a btc address better 2019-01-22 08:58:02 +01:00
Daniel Roethlisberger 5b4079637a new: [attributes] Add cdhash attribute, 40+ digit hash, default Payload delivery, ids=1 (#3965) 2018-12-19 20:19:49 +01:00
iglocska a28909d366 new: [freetext] Added BTC recognition, fixes #3864 2018-11-13 12:06:00 +01:00
iglocska cefab3e01c new: [freetext import] Added detection for AS 2018-09-23 11:43:55 +02:00
iglocska 48c6150257 new: [Complex type tool] Detection of [1] style refanging 2018-09-21 15:08:33 +02:00
iglocska bc9524c712 fix: [feeds] Feed caching generates a lot of notices 2018-09-07 13:29:26 +02:00
iglocska a81894f14c chg: [CS] Changed to PSR-2
- to make contributions easier, adopted PSR-2
- used php-cs-fixer to rework the style
- *sniff sniff* Goodbye tab indentation
2018-07-19 11:48:22 +02:00
iglocska 90a8e9110b fix: [freetext] parser was detecting any number as a phone number, fixes #3469
- new requirement: must start with + or contain a -
2018-07-11 15:02:30 +02:00
iglocska 8cc1d86068 fix: Allow filename as an alternative for parsed domains/hostnames 2018-04-25 15:41:58 +02:00
iglocska 8c2dd6d00a fix: Fix to the invalid refanging (Third time's the charm) 2018-04-18 11:25:58 +02:00
iglocska 5b62965e98 fix: fixed invalid refanging 2018-04-18 11:23:57 +02:00
iglocska d04f263c95 chg: Added [:] to the refanging options 2018-04-18 09:50:04 +02:00
iglocska 9485dfe5e2 chg: Refactor of the complex type tool
- makes it more readable
2018-04-03 22:25:52 +02:00
Émilio Gonzalez bb8d4fa634 Fixed a bug regarding filename|ssdeep attributes importing using FreeTextImport. See Issue #2971 2018-02-28 18:34:46 -05:00
iglocska 9fd8a1c14f chg: Use <> as delimiters for the freetext import too, fixes #2978 2018-02-27 18:21:08 +01:00
iglocska 501b933a56 fix: Don't try to refang filepaths, fixes #2926 2018-02-25 23:24:54 +01:00
iglocska 57197f092b fix: Add alternative x509 fingerprint hashes to the freetext import tool, fixes #2821 2018-01-17 10:16:33 +01:00
iglocska 58c97d8263 chg: Tuned the freetext import tool, fixes #2822
- refang e-mail addresses
- add [@] refanging
2018-01-16 15:01:21 +01:00
Andras Iklody 9d6c20709e
chg: Add hybrid analysis to the freetext import tool, fixes #2797 2018-01-09 22:43:12 +01:00
iglocska 3a45410e10 fix: Naive fix for an issue with tab separated feeds being broken by the switch to str_getcsv 2017-12-29 10:40:03 +01:00
iglocska 0df15f03e1 fix: Fixed the invalid default TLDs if no warninglist is loaded 2017-12-08 12:28:28 +01:00
iglocska 67f0acb6c6 fix: Made CSV parser for freetext import tool / feed ingestion compatible with escaped CSVs
- "" now handled correctly
2017-11-30 16:52:22 +01:00
Thomas Gardner 93160d69c1 added target-email to FreeText Import types 2017-11-22 11:07:42 -07:00
iglocska 45a2d1a09b new: Added phone number recognition to the freetext import tool
- also, changed the massaging of phone number type attributes to replace 00 with +
2017-11-16 16:25:46 +01:00
iglocska fd45eed6c4 chg: Added .onion to the TLD list for the complext type tool 2017-09-26 09:14:00 +02:00
iglocska a931af7223 chg: Some tuning to the freetext import tool 2017-09-12 10:20:38 +02:00
iglocska 4c4f9a4dbb chg: Allow for \t to be used as a CSV feed delimiter 2017-05-11 14:46:20 +02:00
iglocska a59aab9b23 fix: Re-added the accidentally removed code in a merge, fixes #1965
- affects f0e1a27b7d
2017-02-20 18:43:36 +01:00
iglocska dc8a9707c0 Merge branch '2.4' into feature/enhanced_sightings 2017-02-16 22:52:53 +01:00
iglocska ab7aadb924 fix: Fixed a bug with the freetext import that broke the detection of IP addresses 2017-02-14 16:51:07 +01:00
iglocska e1f5463a82 fix: Added correct recognition of ip:port indicators to the freetext import tool, fixes #1919 2017-02-10 17:59:35 +01:00
iglocska ca22435831 fix: Added (dot) to the refanging 2017-02-10 10:32:43 +01:00
iglocska a229af43ae fix: Empty delimiter for CSV feeds causing grief 2017-01-25 06:02:55 +01:00
Alexandre Dulaunoy 16d31458a8
fix: whois-registrant-email added as type when an email is detected in freetext 2017-01-18 14:13:36 +01:00
iglocska 7dcc11f0f7 fix: Copy paste fail 2017-01-01 16:29:50 +01:00
iglocska 734ff59cb4 fix: Left off changes to the complextypetool
- oops
2017-01-01 16:28:23 +01:00