2017-08-12 17:45:33 +02:00
|
|
|
# Lookyloo
|
2017-07-23 19:56:51 +02:00
|
|
|
|
|
|
|
*Lookyloo* is a web interface allowing to scrape a website and then displays a
|
|
|
|
tree of domains calling each other.
|
|
|
|
|
2017-08-12 17:45:33 +02:00
|
|
|
# What is that name?!
|
2017-07-23 19:56:51 +02:00
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
1. People who just come to look.
|
|
|
|
2. People who go out of their way to look at people or something often causing crowds and more disruption.
|
|
|
|
3. People who enjoy staring at watching other peoples misfortune. Oftentimes car onlookers to car accidents.
|
|
|
|
Same as Looky Lou; often spelled as Looky-loo (hyphen) or lookylou
|
|
|
|
In L.A. usually the lookyloo's cause more accidents by not paying full attention to what is ahead of them.
|
|
|
|
```
|
|
|
|
|
|
|
|
Source: Urban Dictionary
|
|
|
|
|
|
|
|
|
2017-08-12 17:45:33 +02:00
|
|
|
# Implementation details
|
2017-07-23 19:56:51 +02:00
|
|
|
|
|
|
|
This code is very heavily inspired by https://github.com/etetoolkit/webplugin and adapted to use flask as backend.
|
|
|
|
|
2017-08-12 17:45:33 +02:00
|
|
|
# Installation of har2tree
|
2017-07-23 19:56:51 +02:00
|
|
|
|
|
|
|
The core dependency is ETE Toolkit, which you can install following the guide
|
|
|
|
on the official website: http://etetoolkit.org/download/
|
|
|
|
|
2017-08-12 17:45:33 +02:00
|
|
|
## Protip
|
2017-07-23 19:56:51 +02:00
|
|
|
|
|
|
|
If you like using virtualenv and have `pew` installed you can also do it this way:
|
|
|
|
|
|
|
|
```bash
|
2017-08-12 17:45:33 +02:00
|
|
|
sudo apt-get install python3-pyqt4
|
|
|
|
```
|
|
|
|
|
|
|
|
# Installation of scrapysplashwrapper
|
|
|
|
|
|
|
|
You need a running splash instance, preferably on docker: https://splash.readthedocs.io/en/stable/install.html
|
|
|
|
|
|
|
|
```bash
|
|
|
|
sudo apt install docker.io
|
|
|
|
sudo docker pull scrapinghub/splash
|
|
|
|
sudo docker run -p 8050:8050 -p 5023:5023 scrapinghub/splash
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
# Installation of the whole thing
|
|
|
|
|
|
|
|
(assuming you already installed the dependencies ete3 and splash in docker)
|
|
|
|
|
|
|
|
```bash
|
2017-07-23 19:56:51 +02:00
|
|
|
pew toggleglobalsitepackages # PyQt4 is not easily installable in a virtualenv
|
2017-08-12 17:45:33 +02:00
|
|
|
pip install -r requirements.txt
|
2017-07-23 19:56:51 +02:00
|
|
|
pip install -e .
|
|
|
|
```
|