CIRCL
/
AIL-framework
mirror of https://github.com/CIRCL/AIL-framework
2
1
Fork 0
Code Issues Releases Wiki Activity
1,267 Commits
6 Branches
41 Tags
109 MiB
Commit Graph

1267 Commits (6fdf7c2123e69343efd145ec945ee43a5766c5b7)

Author SHA1 Message Date
Terrtia 6fdf7c2123
chg: [UI crawler] status/remove auto crawler 2019-04-18 16:57:51 +02:00
Terrtia ecca5b11d0
fix: [UI dashboard] typo 2019-04-18 11:01:11 +02:00
Terrtia de285ed64c
chg: [UI DOC] add dynamic update message + add bootstrap 4 migration message + add update doc (DB) 2019-04-18 10:56:00 +02:00
Terrtia 7a5fac9a43
fix: [UI show domain] fix domain/port unpacking 2019-04-18 09:54:25 +02:00
Terrtia ddaad5db93
chg: [v1.5 background update] create update stats 2019-04-17 17:07:09 +02:00
Terrtia 4bff603d4f
chg: [settings UI] add background update progress 2019-04-17 14:39:29 +02:00
Terrtia 68077316eb
chg: [git directory] add git helper 2019-04-16 17:26:35 +02:00
Terrtia 3b11df7f90
chg: [UI] add server Management, check if new update/version are available + background update status + link to new realease/commits 2019-04-16 17:24:59 +02:00
Terrtia ddf043059a
chg: [UI] update nav bar endpoint 2019-04-15 17:14:48 +02:00
Terrtia 346e26433a
fix: [Update tags + UI] use srandmember + display update warning 2019-04-15 14:30:28 +02:00
Terrtia 3f1e68f638
fix: [tracked terms] show mime type 2019-04-15 13:36:14 +02:00
Terrtia a43467a0f2
chg: [Update v1.5] update tracked terms path 2019-04-15 13:27:46 +02:00
Terrtia 0ce4489ed2
chg: [OVERVIEW] add doc termfred DB 2019-04-15 11:46:20 +02:00
Terrtia fc1a04336c
fix: [dynamic update v1.5] make sure updates are excuted in the correct order + fix nb_seen_hash dynamic update 2019-04-15 11:01:33 +02:00
Terrtia a3167a740a
fix: [Update DB] remove indexErrors 2019-04-12 17:32:17 +02:00
Terrtia bf92a2f22f
fix: [background update] avoid multiple update on the same key 2019-04-12 16:07:40 +02:00
Terrtia f2dc3bcbf4
fix: [Global] remove absolute path from item path 2019-04-12 15:13:36 +02:00
Terrtia 3a8631d44a
chg: [Update + Launcher] use background update + add new options in launcher 2019-04-12 15:06:24 +02:00
Terrtia 5c4db5f2f1
chg: [update] add update script Redis ARDB + add background update 2019-04-11 17:49:20 +02:00
Terrtia 1ed9468881
fix: [UI showpaste] min preview 2019-04-11 13:44:20 +02:00
Terrtia 3cc614a1ad
chg: [DB] update items tags metadata 2019-04-11 11:58:06 +02:00
Terrtia e83174327a
chg: [DB] remove absolute path 2019-04-10 17:47:40 +02:00
Terrtia d44acea04d
chg: [merge] merge update into advanced_crawler 2019-04-10 16:41:06 +02:00
Terrtia 2589fc2161
chg: [merge] merge master into update branche 2019-04-10 15:43:15 +02:00
Terrtia b4bee29a33
Merge branch 'master' into advanced_crawler 2019-04-10 15:37:42 +02:00
Terrtia 6165987ec3
chg: [DB Migration] add hash dynamic update 2019-04-10 15:36:41 +02:00
Terrtia 0519b4a437
chg: [update DB] add migration script 2019-04-08 17:04:09 +02:00
Terrtia 870fa1981c
fix: [update_thirdparty] detect and use virtualenv 2019-04-02 09:50:52 +02:00
Terrtia f6d7d2ae16
chg: [Update] add debug 2019-04-02 09:46:04 +02:00
Thirion Aurélien 976b97bcd1
Merge pull request #338 from CIRCL/pre_update 
Merge Pre update
 2019-03-26 16:51:26 +01:00
Terrtia 044933aa33
chg: [Launch] remove old launchers 2019-03-26 16:47:10 +01:00
Terrtia cee2b2486f
chg: [Launch] remove old launchers 2019-03-26 16:45:01 +01:00
Terrtia 59664efe45
Merge branch 'master' into advanced_crawler 2019-03-26 16:03:42 +01:00
Terrtia 5165a5de2f
fix: [Crawler] fix index + redis history key 2019-03-22 17:14:27 +01:00
Terrtia f64c385343
chg: [Crawler] handle port: crawling + history 2019-03-22 16:48:07 +01:00
Terrtia f4cdddbc7f
chg: [UI + item submit] remove browse paste from UI + fix pasteSubmit tagging 2019-03-21 11:50:53 +01:00
Terrtia 51eb951a47
fix: [UI terms] update tags endpoint 2019-03-21 11:34:21 +01:00
Terrtia c2bc72c72e
chg: [Tags alertHandler] remove alertHandler + add quick tag search sidebar + use tag last_seen to display tagged items 2019-03-21 11:11:53 +01:00
Terrtia ce6475e96e
chg: [tags] remove/add tags by daterange + improve item loading time 2019-03-20 15:57:55 +01:00
Terrtia 8b5266706e
fix: [Tags] cast nb_max_page to int 2019-03-20 13:33:12 +01:00
Terrtia a9e86231ad
chg: [Tags] add backend, daterange selector 2019-03-20 13:25:02 +01:00
Terrtia 65e6eae00d
chg: [UI preview items] b4 migration + new template + better display + decrease item loading time + add basic animation/transition 2019-03-19 13:34:41 +01:00
Terrtia e9bac098c9
fix: [UI Tags] paste preview 2019-03-15 11:16:44 +01:00
Terrtia 23d9cb2265
chg: [UI Tags search] bootstrap 4 migration 2019-03-14 17:04:55 +01:00
Thirion Aurélien c87f0ffcb1
Merge pull request #336 from 5thColumn/docker 
Add indexdir volume to docker-compose
 2019-03-14 09:35:19 +01:00
Jason Hedden 9279f56048 Add indexdir volume to docker-compose 
Make the indexdir data persistent and shared with Flask for searching pastes.
 2019-03-13 17:05:54 -05:00
Alexandre Dulaunoy 85d2ea578c
Merge pull request #335 from 5thColumn/docker 
Add docker-compose support
 2019-03-13 21:38:12 +01:00
Terrtia 9e4813ca4b
chg: [UI Tags] start b4 migration + add date range selector 2019-03-12 13:37:15 +01:00
Jason Hedden 23fb91d326 Add docker-compose support 
This commit separates the AIL Framework services into individual Docker
containers that can be managed with docker-compose. It's intended to ease the
installation, development and troubleshooting procedures for all-in-one
deployments.

No changes to architecture or base code are required in this commit.

Changes to existing files:
.gitignore
  * Ignore pystemon archive directory

Dockerfile:
  * Add AIL shell environment variables
  * Install pystemon and crawler pip requirements into AIL virtual environment

Docker-compose notes:
  * All containers are participating in a service network `network_mode:
  service:flask`.  This allows the containers to share the same IP namespace
  to accommodate hard-coded localhost entries.
  * By default persistent data is saved to the following directories in the
  local AIL framework git directory: PASTES, HASHS, CRAWLED_SCREENSHOTS,
  pystemon/archives

This was tested with docker-ce on Ubuntu 16.04 and MacOS. A typical deployment
would look like:
```
git clone https://github.com/CIRCL/AIL-framework.git
cd AIL-framework
cp bin/packages/config.cfg.docker-compose-sample bin/packages/config.cfg
	(optionally enable activate_crawler in config.cfg)
docker-compose build
docker-compose up -d
```
 2019-03-11 19:46:34 -05:00
Alexandre Dulaunoy d54c4e2d86
Merge pull request #334 from 5thColumn/pybgpranking 
Add pybgpranking to install deps
 2019-03-11 07:46:46 +01:00