CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity

chg: [OVERVIEW] add doc termfred DB

pull/342/head
Terrtia 2019-04-15 11:46:20 +02:00
parent fc1a04336c
commit 0ce4489ed2
No known key found for this signature in database
GPG Key ID: 1E1B1F50D84613D0
1 changed files with 44 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
OVERVIEW.md
View File

@ -30,12 +30,54 @@ Redis and ARDB overview
# Database Map: # Database Map:
## Tags: ## DB2 - TermFreq: (TODO: update/refractor DB)
##### Set:
| Key | Value |
| ------ | ------ |
| TrackedSetTermSet | **tracked_term** |
| TrackedSetSet | **tracked_set** |
| TrackedRegexSet | **tracked_regex** |
| | |
| tracked_**tracked_term** | **item_path** |
| set_**tracked_set** | **item_path** |
| regex_**tracked_regex** | **item_path** |
| | |
| TrackedNotifications | **tracked_trem / set / regex** |
| | |
| TrackedNotificationTags_**tracked_trem / set / regex** | **tag** |
| | |
| TrackedNotificationEmails_**tracked_trem / set / regex** | **email** |
##### Zset:
| Key | Field | Value |
| ------ | ------ | ------ |
| per_paste_TopTermFreq_set_month | **term** | **nb_seen** |
| per_paste_TopTermFreq_set_week | **term** | **nb_seen** |
| per_paste_TopTermFreq_set_day_**epoch** | **term** | **nb_seen** |
| | | |
| TopTermFreq_set_month | **term** | **nb_seen** |
| TopTermFreq_set_week | **term** | **nb_seen** |
| TopTermFreq_set_day_**epoch** | **term** | **nb_seen** |
##### Hset: ##### Hset:
| Key | Field | Value | | Key | Field | Value |
| ------ | ------ | ------ | | ------ | ------ | ------ |
| daily_tags:**date** | **tag** | **nb tagged this day** | | TrackedTermDate | **tracked_term** | **epoch** |
| TrackedSetDate | **tracked_set** | **epoch** |
| TrackedRegexDate | **tracked_regex** | **epoch** |
| | | |
| BlackListTermDate | **blacklisted_term** | **epoch** |
| | | |
| **epoch** | **term** | **nb_seen** |
## DB6 - Tags:
##### Hset:
| Key | Field | Value |
| ------ | ------ | ------ |
| per_paste_**epoch** | **term** | **nb_seen** |
| | | | | |
| tag_metadata:**tag** | first_seen | **date** | | tag_metadata:**tag** | first_seen | **date** |
| tag_metadata:**tag** | last_seen | **date** | | tag_metadata:**tag** | last_seen | **date** |
@ -121,26 +163,6 @@ Redis and ARDB overview
ARDB overview ARDB overview
* DB 2 - TermFreq
----------------------------------------- TERM ----------------------------------------
SET - 'TrackedRegexSet' term
HSET - 'TrackedRegexDate' tracked_regex today_timestamp
SET - 'TrackedSetSet' set_to_add
HSET - 'TrackedSetDate' set_to_add today_timestamp
SET - 'TrackedSetTermSet' term
HSET - 'TrackedTermDate' tracked_regex today_timestamp
SET - 'TrackedNotificationEmails_'+term/set email
SET - 'TrackedNotifications' term/set
* DB 4 - Sentiment
----------------------------------------- SENTIMENT ------------------------------------ ----------------------------------------- SENTIMENT ------------------------------------
SET - 'Provider_set' Provider SET - 'Provider_set' Provider
@ -151,8 +173,6 @@ ARDB overview
SET - UniqID avg_score SET - UniqID avg_score
DB 5 - TermCred
* DB 7 - Metadata: * DB 7 - Metadata: