Merge branch 'master' of github.com:ail-project/ail-framework

pull/569/head
Terrtia 2021-08-16 10:18:55 +02:00
commit 39579fb5a2
No known key found for this signature in database
GPG Key ID: 1E1B1F50D84613D0
1 changed files with 47 additions and 9 deletions

View File

@ -1,3 +1,4 @@
Feeding, adding new features and contributing Feeding, adding new features and contributing
============================================= =============================================
@ -8,7 +9,7 @@ For the moment, there are three different ways to feed AIL with data:
1. Be a collaborator of CIRCL and ask to access our feed. It will be sent to the static IP you are using for AIL. 1. Be a collaborator of CIRCL and ask to access our feed. It will be sent to the static IP you are using for AIL.
2. You can setup [pystemon](https://github.com/CIRCL/pystemon) and use the custom feeder provided by AIL (see below). 2. You can setup [pystemon](https://github.com/cvandeplas/pystemon) and use the custom feeder provided by AIL (see below).
3. You can feed your own data using the [./bin/import_dir.py](./bin/import_dir.py) script. 3. You can feed your own data using the [./bin/import_dir.py](./bin/import_dir.py) script.
@ -19,16 +20,53 @@ However, if you want to collect some pastes and feed them to AIL, the procedure
Feed data to AIL: Feed data to AIL:
1. Clone the [pystemon's git repository](https://github.com/CIRCL/pystemon) 1. Clone the [pystemon's git repository](https://github.com/cvandeplas/pystemon):
``` git clone https://github.com/cvandeplas/pystemon.git ```
2. Install its python dependencies inside your virtual environment 2. Edit configuration file for pystemon ```pystemon/pystemon.yaml```:
* Configuration of storage section (adapt to your needs):
```
storage:
archive:
storage-classname: FileStorage
save: yes
save-all: yes
dir: "alerts"
dir-all: "archive"
compress: yes
3. Launch pystemon ``` ./pystemon ``` redis:
storage-classname: RedisStorage
4. Edit your configuration file ```configs/core.cfg``` and modify the pystemonpath path accordingly save: yes
save-all: yes
5. Launch pystemon-feeder ``` ./bin/feeder/pystemon-feeder.py ``` server: "localhost"
port: 6379
database: 10
lookup: no
```
* Change configuration for paste-sites according to your needs (don't forget to throttle download time and/or update time).
3. Install python dependencies inside the virtual environment:
```
cd ail-framework/
. ./AILENV/bin/activate
cd pystemon/ #cd to pystemon folder
pip3 install -U -r requirements.txt
```
4. Edit configuration file ```ail-framework/configs/core.cfg```:
* Modify the "pystemonpath" path accordingly
5. Launch ail-framework, pystemon and pystemon-feeder.py (still inside virtual environment):
* Option 1 (recommended):
```
./ail-framework/bin/LAUNCH.py -l #starts ail-framework
./ail-framework/bin/LAUNCH.py -f #starts pystemon and the pystemon-feeder.py
```
* Option 2 (you may need two terminal windows):
```
./ail-framework/bin/LAUNCH.py -l #starts ail-framework
./pystemon/pystemon.py
./ail-framework/bin/feeder/pystemon-feeder.py
```
How to create a new module How to create a new module
-------------------------- --------------------------