Merge pull request #61 from dputtick/dev

Release 2.3.1: fix bug with RPi 3
pull/68/head v2.3.1
Raphaël Vinot 2017-10-25 17:09:55 -04:00 committed by GitHub
commit 87aeb6a919
9 changed files with 51 additions and 72 deletions

View File

@ -1,3 +1,7 @@
Version 2.3.1 - 2017-10-25
- Bugfix for Raspberry Pi 3 that was causing rc.local to fail to boot
- Turned off several networking related services: startup is much faster
Version 2.3 - 2017-09-08
- Updated to the newest version of Raspbian Stretch lite (2017-08-16 release)
- Using the newest version of PyCIRCLean, including several vulnerability and bug fixes

View File

@ -23,8 +23,6 @@ echo "GROOMER: end of boot, running rc.local."
if [ -e /dev/sda ]; then
if [ -e /dev/sdb ]; then
# Avoid possible misuse - turn off eth0 (ethernet port)
/sbin/ifconfig eth0 down
trap clean EXIT TERM INT
cd /opt/groomer
/usr/sbin/led &

View File

@ -1,12 +0,0 @@
USB Ports
---------
If you connect multiple keys to the Rpi, they will be listed that way:
First: Top left
Second: Top right
Third: Bottom left
Forth: Bottom right
tl;dr: as long as the source key (sda) is connected to the top left port, the destination (sdb) can be connected on any other port.

View File

@ -1,15 +1,15 @@
Notes
=====
Usage Notes
===========
* don't plug in USB devices with a hub because there's no way to tell it which
is source and target - its the first drive enumerated (top port) that is the
* Don't plug in USB devices with a hub because there's no way to tell it which
is source and target - its the first drive detected (top port) that is the
source and the second (bottom port) is the target
* don't turn it off without shutting down the system, when grooming is done it
* Don't turn it off without shutting down the system, when grooming is done it
shuts down automatically: losing power while it's running can trash the OS
on the SD cards because SD cards don't always like dirty shutdowns (ie power loss)
* Using a target usb stick that has a status light as long as the device has
power is a really useful thing as there the other status lights on the groomer
are less than indicative at times: because the 'OK' led on the rPi toggles on activity
are less than indicative at times: because the 'OK' led on the RPi toggles on activity
it can be off for a long time while processing something and only comes back
on when that process finishes - hence why a USB that has some sort of LED activity
when just plugged in (even if not reading or writing but while the USB port is
@ -18,46 +18,32 @@ Notes
then be off on the USB device
* Use a larger target device as all zip files get unpacked and processed onto
the target
* if you have an hdmi monitor plugged in you can watch what's happening for about
* If you have an hdmi monitor plugged in you can watch what's happening for about
30 minutes until the rPI's power saving kicks in and turns off the monitor
* if only one usb stick is present at power up, it doesn't groom and looks like
* If only one usb stick is present at power up, it doesn't groom and looks like
a normal rPi
* if you want to ssh into the rPi username is 'pi' password 'raspberry' as per defaults
* If you want to ssh into the RPi username is 'pi' password 'raspberry' as per defaults
Technical notes
===============
* groomer script is in /opt/groomer/ with the other required files
* dependencies are libre-office and OpenJRE
* and the ip address is 192.168.1.89
* the groomer process is kicked off in /etc/rc.local
* the heavy lifting takes place or is dispatched from /opt/groomer/groomer.sh
in that script file is what file types get processed (or if not listed there,
get ignored)
* there are two ways pdf's can get handled -right now they have their text extracted
to the target device, the other way copies it and extracts the text
* the pdf text extraction isn't perfect and is the slowest part of it, but should
be able to handle unicode stuff and currently doesn't do image extraction from
pdf's but could do that too
* Groomer script is in /opt/groomer/ with the other required files and the ip
address is 192.168.1.89
* The groomer process is kicked off in /etc/rc.local
* The heavy lifting is dispatched from /opt/groomer/groomer.sh
* All files processing is in filecheck.py
Discussion
==========
USB Ports
=========
* however image exports of pdf pages only have the images and no text so it's not
like saving each page to a jpg which would be a really handy and safe way of
converting pdf's
* spread sheets and presentations get converted to pdfs to kill off any embedded
macros and it's assumed that it's not producing evil pdf's on export but does
nothing to sanitize any embedded links within those documents
* for spreadsheets, if they are longer than a page, only a page worth from that
sheet is exported right from the middle of the sheet (ie the top and bottom of
that sheet will get cut off and only the contents in the middle exported to pdf)
dumb but i figure if you want to go back to the source because it's interesting
enough on the groomed side of it, then you can take the extra precautions
* the groomed target only copies "safe" files, and does its best to convert any
potential unsafe files to a safer format
* safe files being one that I know of that can't contain malicious embedded macros
or other crap like that, and those than can get converted to something that wont
contain code after conversion
If you connect multiple keys to the RPi, they will be detected in this order:
First: Top left
Second: Top right
Third: Bottom left
Forth: Bottom right
* As long as the source key (sda) is connected to the top left port, the
destination (sdb) can be connected on any other port.

View File

@ -1,13 +0,0 @@
TODO
====
* strip exif data and leave it in a .txt file next to the image it came from
=> exiftool
* Scripts to generate a SD card automatically (win/mac/linux)
* Extract metadata from all the files => https://mat.boum.org/
HTML Files
==========
- disable JS?
- cleanup external imports (js/css/images)

View File

@ -38,6 +38,7 @@
* Symlink /proc/mounts to /etc/mtab
* Turn on rc-local.service `systemctl enable rc-local.service`
- If it doesn't work, read these instructions: https://www.linuxbabe.com/linux-server/how-to-enable-etcrc-local-with-systemd
* Disable networking.service, bluetooth.service, and dchpcd.service
* apt-get autoclean
* apt-get autoremove
* Exit from proot

View File

@ -185,6 +185,13 @@ This is what triggers CIRCLean to run.
```
systemctl enable rc-local.service
```
* Turn off several networking related services. This speeds up boot and reduces the
attack surface:
```
systemctl disable networking.service
systemctl disable bluetooth.service
systemctl disable dhcpcd.service
```
* Clean up:
```
apt-get clean

View File

@ -0,0 +1,14 @@
#!/bin/bash
# Filename: cp-metadata
myecho=echo
src_path="$1"
dst_path="$2"
find "$src_path" |
while read src_file; do
dst_file="$dst_path${src_file#$src_path}"
$myecho chmod --reference="$src_file" "$dst_file"
$myecho chown --reference="$src_file" "$dst_file"
$myecho touch --reference="$src_file" "$dst_file"
done

View File

@ -3,15 +3,9 @@
# change locales to en_US.UTF-8
dpkg-reconfigure locales
sed -i "s/wheezy/jessie/" /etc/apt/sources.list
apt-get update
apt-get dist-upgrade
apt-get autoremove
apt-get install libreoffice p7zip-full libfontforge1 timidity freepats pmount
dpkg -i pdf2htmlex*.deb
# Disable swap
dphys-swapfile uninstall
# enable rc.local
systemctl enable rc-local.service