mirror of https://github.com/CIRCL/Circlean
Raphaël Vinot
GitHub
commit
4e5b7b2815
26
CHANGELOG
26
CHANGELOG
|
|
@ -1,26 +0,0 @@
|
|||
Version 1.2 - 2015-03-10
|
||||
|
||||
- Rollback the migration to Jessie and use Wheezy again: the only important dependency from Jessie was poppler, which is available in the backports
|
||||
- Use the most recent security patches
|
||||
- Do not wait for user input in case of password protected archive
|
||||
|
||||
Version 1.1.1 - 2014-10-26
|
||||
|
||||
- General upgrade of Debian to avoid the system to fail in case there is no HDMI cable connected.
|
||||
|
||||
Version 1.1 - 2014-10-01
|
||||
|
||||
- NTFS support added for USB key
|
||||
- Updated to Debian Jessie including patches for [bash vulnerabilities CVE-2014-6271 - CVE-2014-7169](/pub/tr-27/)
|
||||
- CIRCLean user are now removed from the sudoer
|
||||
|
||||
Version 1.0 - 2014-05-20
|
||||
|
||||
- Based on Raspbian Jessie
|
||||
- Fully automated tests with Qemu
|
||||
- Mimetype: support of PDF, Office documents, archives, windows executables
|
||||
- Filesystem: USB keys have to be formated in vfat
|
||||
- Support of multiple partitions
|
||||
- Renaming of autorun.inf on the source key
|
||||
- Operating system is read only
|
||||
- Use pdf2htmlEX v0.11
|
||||
|
|
@ -0,0 +1,50 @@
|
|||
Version 2.1 - 2017-02-02
|
||||
- Updated to the newest version of Raspbian Jessie lite (January 11th 2017 release)
|
||||
- NTFS files can now be mounted as source or destination keys
|
||||
- Added udev rules that ensure the USB ports map deterministically to source and destination keys
|
||||
- New debug flag and debug logging functionality to make working on Circlean without a monitor easier
|
||||
- Turned off automatic display sleep
|
||||
|
||||
Version 2.0.2 - 2016-05-12
|
||||
- Improve filename encoding
|
||||
|
||||
Version 2.0.1 - 2016-04-26
|
||||
- Re-add [timidity](http://timidity.sourceforge.net/) so the MIDI files are played properly
|
||||
|
||||
Version 2.0 - 2016-04-26
|
||||
- No critical bugs have been identified, this release uses the latest version of Raspbian Jessie lite, with all system updates
|
||||
|
||||
Version 2.0-BETA - 2015-11-06
|
||||
- There a new beta version of CIRCLean which is a significant improvement from the latest version in term of speed and efficiency on low-end hardware like the first version of the Raspberry Pi. The new code base of CIRCLean is now based on [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean)
|
||||
|
||||
Version 1.3 - 2015-05-27
|
||||
- Fix a [critical security bug](https://www.circl.lu/projects/CIRCLean/security/advisory-01) related to [polyglot files](https://github.com/CIRCL/Circlean/issues/9) - thanks to the reporters ([Jann Horn](https://github.com/thejh), [seclab-solutions](http://www.seclab-solutions.com/))
|
||||
- Use [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean) for conversion
|
||||
- Convert PDF files to PDF/A before converting to HTML
|
||||
|
||||
Version 1.2 - 2015-03-10
|
||||
|
||||
- Rollback the migration to Jessie and use Wheezy again: the only important dependency from Jessie was poppler, which is available in the backports
|
||||
- Use the most recent security patches
|
||||
- Do not wait for user input in case of password protected archive
|
||||
|
||||
Version 1.1.1 - 2014-10-26
|
||||
|
||||
- General upgrade of Debian to avoid the system to fail in case there is no HDMI cable connected.
|
||||
|
||||
Version 1.1 - 2014-10-01
|
||||
|
||||
- NTFS support added for USB key
|
||||
- Updated to Debian Jessie including patches for [bash vulnerabilities CVE-2014-6271 - CVE-2014-7169](/pub/tr-27/)
|
||||
- CIRCLean user are now removed from the sudoer
|
||||
|
||||
Version 1.0 - 2014-05-20
|
||||
|
||||
- Based on Raspbian Jessie
|
||||
- Fully automated tests with Qemu
|
||||
- Mimetype: support of PDF, Office documents, archives, windows executables
|
||||
- Filesystem: USB keys have to be formated in vfat
|
||||
- Support of multiple partitions
|
||||
- Renaming of autorun.inf on the source key
|
||||
- Operating system is read only
|
||||
- Use pdf2htmlEX v0.11
|
||||
|
|
@ -1,17 +1,50 @@
|
|||
Building the project
|
||||
====================
|
||||
Getting started
|
||||
===============
|
||||
|
||||
To get started contributing to Circlean, first, fork the project and `git clone`
|
||||
your fork. Then, follow the instructions in [README_setup.md](README_setup.md)
|
||||
to build an image.
|
||||
If you'd like to work on the Python code that processes files for Circlean, you should
|
||||
take a look at [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean), specifically the
|
||||
filecheck.py script. To get started contributing to Circlean, first, fork the project and
|
||||
`git clone` your fork. Then, follow the instructions in [setup_with_proot.md](doc/setup_with_proot.md) to build an image. To make things easier, you can also download a
|
||||
prebuilt image as mentioned in the README, and then mount and make modifications to this
|
||||
image to test your changes.
|
||||
|
||||
The issue tracker
|
||||
=================
|
||||
|
||||
If you find a bug or see a problem with PyCIRCLean, please open an issue in the Github
|
||||
repo. We'll do our best to respond as quickly as possible. Also, feel free to contribute a solution
|
||||
to any of the open issues - we'll do our best to review your pull request in a timely manner.
|
||||
This project is in active development, so any contributions are welcome!
|
||||
repo. We'll do our best to respond as quickly as possible. Also, feel free to contribute a
|
||||
solution to any of the open issues - we'll do our best to review your pull request in a
|
||||
timely manner. This project is in active development, so any contributions are welcome!
|
||||
|
||||
Dependencies
|
||||
============
|
||||
* Timidity for playing midi files
|
||||
* Git for installing some Python dependencies
|
||||
* 7Zip for unpacking archives
|
||||
* Pmount and ntfs-3g for mounting usb key partitions
|
||||
* Python 3 and pip for installing and running Python dependencies
|
||||
* Python3-lxml for handling ooxml and other Office files in filecheck.py
|
||||
* libjpeg-dev, libtiff-dev, libwebp-dev, liblcms2-dev, tcl-dev, tk-dev, and python-tk for various image formats (dependencies for pillow)
|
||||
* Exifread for file metadata
|
||||
* Pillow for handling images
|
||||
* Olefile, oletools, and officedissector for handling various Office filetypes
|
||||
* PyCIRCLean for main file handling code
|
||||
|
||||
Helper scripts
|
||||
==============
|
||||
|
||||
Use the scripts in shell_utils/ as examples - do not run them blindly as you will most
|
||||
probably have to change some constants/paths accordingly to your configuration.
|
||||
|
||||
IN ALL CASES, PLEASE READ THE COMMENTS IN THE SCRIPTS AT LEAST ONCE.
|
||||
|
||||
* proper_chroot.sh: uses qemu to chroot into a raspbian instance (.img or SD Card)
|
||||
* prepare_rPI.sh: update the system, some configuration
|
||||
* create_user.sh: create the user who will run the scripts, assign the proper sudo rights.
|
||||
* copy_to_final.sh: populate the content of the directory fs/ in the image,
|
||||
contains a sample of dd command to write the image on the SD card.
|
||||
NOTE: TAKE CARE NOT TO USE THE WRONG DESTINATION
|
||||
|
||||
|
||||
Running the tests
|
||||
=================
|
||||
|
|
|
|||
46
README.md
46
README.md
|
|
@ -3,17 +3,19 @@ CIRCLean
|
|||
|
||||
|
||||
|
||||
How To
|
||||
======
|
||||
How To Install
|
||||
==============
|
||||
|
||||
[Graphical how-to and pre-built image](http://circl.lu/projects/CIRCLean/).
|
||||
[Graphical how-to and pre-built image download](http://circl.lu/projects/CIRCLean/).
|
||||
|
||||
To prepare the SD card on Windows, you can use [Win32DiskImager](http://sourceforge.net/projects/win32diskimager/).
|
||||
To prepare the SD card on Windows, you can use [Win32DiskImager](http://sourceforge.net/projects/win32diskimager/). On linux/macOS, use dd (see the how-to link for
|
||||
instructions).
|
||||
|
||||
And the linux way is in the command line, via dd (see in copy_to_final.sh)
|
||||
The current prebuilt image is based on the 1-11-17 release of Raspbian Jessie Lite.
|
||||
The smallest SD card that Circlean can fit on is currently 4GB.
|
||||
|
||||
If you'd like to contribute to the project or build the image yourself, see
|
||||
[contributing](CONTRIBUTING.md) and the [setup readme](README_setup.md).
|
||||
[contributing.md](CONTRIBUTING.md) and the [setup instructions](doc/setup_with_proot.md).
|
||||
This is a work in progress - contributions are welcome.
|
||||
|
||||
Why/What
|
||||
|
|
@ -21,21 +23,25 @@ Why/What
|
|||
|
||||
This project aims to be useful when you get/find a USB key that you can't trust,
|
||||
and you want to look at its contents without taking the risk of plugging it into
|
||||
your computer directly.
|
||||
your computer directly. The official project page can be found at [https://www.circl.lu/projects/CIRCLean/]
|
||||
|
||||
The Raspberry Pi Foundation has a [blog post](https://www.raspberrypi.org/blog/kittengroomercirclean-data-security-for-journalists-and-activists/) with more information
|
||||
about an older version of the project and details of the inspiration behind it.
|
||||
|
||||
CIRCLean is currently tested to work with USB keys that have FAT32, NTFS, or
|
||||
ext2/3/4 filesystems. Currently, exFAT is not supported due to lack of support for
|
||||
this format in pmount. The vast majority of USB keys will be FAT32 or NTFS.
|
||||
ext2/3/4 filesystems (ext* filesystems can only be used as source keys, not destination
|
||||
keys). Currently, exFAT is not supported due to lack of support for this format in pmount.
|
||||
The vast majority of USB keys will be FAT32 or NTFS.
|
||||
|
||||
The content of the untrusted key will be copied or/and converted to the second
|
||||
(blank) key following these rules (based on the mime type as determined bylibmagic):
|
||||
(blank) key following these rules (based on the mime type as determined by libmagic):
|
||||
- Direct copy of:
|
||||
- Plain text files (mime type: text/*)
|
||||
- Audio files (mime type: audio/*)
|
||||
- Video files (mime type: video/*)
|
||||
- Example files (mime type: example/*)
|
||||
- Multipart files (mime type: multipart/*)
|
||||
- *xml* files, after being converted to text files
|
||||
- xml files, after being converted to text files
|
||||
- Octet-stream files
|
||||
- Copied after verification:
|
||||
- Image files after verifying that they are not compression bombs (mime type: image/*)
|
||||
|
|
@ -55,7 +61,7 @@ Usage
|
|||
=====
|
||||
|
||||
0. Power off the device and unplug all connections.
|
||||
1. Plug the untrusted key in the top USB slot of the Raspberry Pi.
|
||||
1. Plug the untrusted key in the top left USB slot of the Raspberry Pi.
|
||||
2. Plug your own key in the bottom USB slot (or use any of the other slots if
|
||||
there are more than 2).
|
||||
|
||||
|
|
@ -71,19 +77,3 @@ there are more than 2).
|
|||
connected the HDMI cable, check the screen. The process is slow and can take
|
||||
30-60 minutes depending on how many document conversions take place.
|
||||
6. Power off the device and disconnect the drives.
|
||||
|
||||
Helper scripts
|
||||
==============
|
||||
|
||||
You should use them as examples when you are creating a new image and probably not
|
||||
run them blindly as you will most probably have to change parameters accordingly to
|
||||
your configuration.
|
||||
|
||||
IN ALL CASES, PLEASE READ THE COMMENTS IN THE SCRIPTS AT LEAST ONCE.
|
||||
|
||||
* proper_chroot.sh: uses qemu to chroot into a raspbian instance (.img or SD Card)
|
||||
* prepare_rPI.sh: update the system, some configuration
|
||||
* create_user.sh: create the user who will run the scripts, assign the proper sudo rights.
|
||||
* copy_to_final.sh: populate the content of the directory fs/ in the image,
|
||||
contains a sample of dd command to write the image on the SD card.
|
||||
NOTE: TAKE CARE NOT TO USE THE WRONG DESTINATION
|
||||
|
|
|
|||
196
README_setup.md
196
README_setup.md
|
|
@ -1,196 +0,0 @@
|
|||
Building the image from scratch
|
||||
===============================
|
||||
|
||||
There is always a prebuilt image available for download and installation as
|
||||
described in the [README](README.md). If you'd like to build the project yourself,
|
||||
there are several steps involved:
|
||||
|
||||
* Downloading a generic Raspbian Lite image
|
||||
* Adding space to the image
|
||||
* Downloading and building the dependencies
|
||||
* Copying the project filesystem into the image
|
||||
|
||||
This procedure will only work on Ubuntu or Debian Linux. If you use MacOS or
|
||||
Windows, the best option is to install Linux in a virtual machine using
|
||||
something like VirtualBox.
|
||||
|
||||
Downloading the Raspbian image
|
||||
==============================
|
||||
|
||||
* Get the most recent version of Raspbian Jessie Lite:
|
||||
|
||||
```
|
||||
wget https://downloads.raspberrypi.org/raspbian_lite_latest
|
||||
```
|
||||
|
||||
* Unpack it:
|
||||
|
||||
```
|
||||
unzip XXXX-XX-XX-raspbian-jessie-lite.zip
|
||||
```
|
||||
|
||||
Adding space to the image
|
||||
=========================
|
||||
|
||||
* Use dd to add 2GB (2048 blocks at 1024k each). Using /dev/zero as the input
|
||||
file yields an unlimited number of "0x00" bytes.
|
||||
|
||||
```
|
||||
> dd if=/dev/zero bs=1024k count=2048 >> XXXX-XX-XX-raspbian-jessie-lite.img
|
||||
```
|
||||
|
||||
* Grow the root partition using fdisk. The "p" command prints the current partition
|
||||
table. The first partition listed is the boot partition, which shouldn't be changed.
|
||||
The "d" command, when given the parameter "2", deletes the current root partition.
|
||||
The "n" command then makes a new partition. It can take the default for "type"
|
||||
and "number". The "First sector" should be the value that was the "start" sector of the root
|
||||
partition (131072 in the example below, but this varies depending on the version of the
|
||||
Raspbian image). The "Last sector" should be the default, and it should be significantly
|
||||
larger than it was before (6852607 vs. 2658303 in the example).
|
||||
|
||||
|
||||
```
|
||||
> fdisk XXXX-XX-XX-raspbian-jessie-lite.img
|
||||
|
||||
Command (m for help): *p*
|
||||
Disk XXXX-XX-XX-raspbian-jessie-lite.img: 3.3 GiB, 3508535296 bytes, 6852608 sectors
|
||||
Units: sectors of 1 * 512 = 512 bytes
|
||||
Sector size (logical/physical): 512 bytes / 512 bytes
|
||||
I/O size (minimum/optimal): 512 bytes / 512 bytes
|
||||
Disklabel type: dos
|
||||
Disk identifier: 0x6f92008e
|
||||
|
||||
Device Boot Start End Sectors Size Id Type
|
||||
XXXX-XX-XX-raspbian-jessie-lite.img1 8192 131071 122880 60M c W95 FAT32 (LBA)
|
||||
XXXX-XX-XX-raspbian-jessie-lite.img2 131072 2658303 2527232 1.2G 83 Linux
|
||||
|
||||
Command (m for help): *d*
|
||||
Partition number (1,2, default 2): *2*
|
||||
|
||||
Partition 2 has been deleted.
|
||||
|
||||
Command (m for help): *n*
|
||||
Partition type
|
||||
p primary (1 primary, 0 extended, 3 free)
|
||||
e extended (container for logical partitions)
|
||||
Select (default p):
|
||||
|
||||
Using default response p.
|
||||
Partition number (2-4, default 2):
|
||||
First sector (2048-6852607, default 2048): *131072*
|
||||
Last sector, +sectors or +size{K,M,G,T,P} (131072-6852607, default 6852607):
|
||||
|
||||
Created a new partition 2 of type 'Linux' and of size 3.2 GiB.
|
||||
|
||||
Command (m for help): *w*
|
||||
The partition table has been altered.
|
||||
Syncing disks.
|
||||
```
|
||||
|
||||
* Mount the image in loop mode: first, edit /mount_image.sh to use the proper values
|
||||
for $OFFSET_BOOT and $OFFSET_ROOTFS, which you can obtain using fdisk and "p" as
|
||||
shown above. You must also change $IMAGE to the correct path. Then run:
|
||||
|
||||
```
|
||||
sudo ./proper_chroot.sh
|
||||
```
|
||||
|
||||
* After mounting the image, the above script will chroot into the mounted image.
|
||||
While in a chroot, the / directory of the image appears as the system / directory
|
||||
(thus the name, change root). To exit the chroot, run "exit" in the root directory.
|
||||
Then, verify the path to the mounted partitions, and resize the filesystem
|
||||
to fill the new larger partition using resize2fs:
|
||||
|
||||
```
|
||||
> df | grep /mnt/arm
|
||||
|
||||
/dev/loop0 3927752 1955672 1794172 53% /mnt/arm_rPi
|
||||
/dev/loop1 57288 18960 38328 34% /mnt/arm_rPi/boot
|
||||
|
||||
> sudo resize2fs /dev/loop0
|
||||
```
|
||||
|
||||
Installing the dependencies
|
||||
===========================
|
||||
|
||||
* To install the dependencies, you'll have to reenter the chroot again:
|
||||
|
||||
```
|
||||
sudo chroot /mnt/arm_rPi
|
||||
```
|
||||
|
||||
* Change your user to root (your global variables may be broken as a result):
|
||||
|
||||
```
|
||||
su root
|
||||
```
|
||||
|
||||
* Change the locales (remove "en_GB.UTF-8 UTF-8", add "en_US.UTF-8 UTF-8"). The
|
||||
arrow keys move the cursor, spacebar selects/deselects a locale, tab moves the cursor
|
||||
to a different context, and enter lets you select "ok":
|
||||
|
||||
```
|
||||
dpkg-reconfigure locales
|
||||
```
|
||||
|
||||
* In the image, make sure everything is up-to-date and remove the old packages:
|
||||
|
||||
```
|
||||
apt-get update
|
||||
apt-get dist-upgrade
|
||||
apt-get autoremove
|
||||
apt-get install timidity git p7zip-full python-dev python-pip python-lxml pmount libjpeg-dev libtiff-dev libwebp-dev liblcms2-dev tcl-dev tk-dev python-tk libxml2-dev libxslt1-dev
|
||||
```
|
||||
|
||||
* Install the Python dependencies for PyCIRCLean. Currently, PyCIRCLean is
|
||||
Python 2.7 and 3.3+ compatible, but Python 2 support might be dropped at some point.
|
||||
|
||||
```
|
||||
pip install oletools olefile exifread Pillow
|
||||
pip install git+https://github.com/Rafiot/officedissector.git
|
||||
pip install git+https://github.com/CIRCL/PyCIRCLean.git
|
||||
```
|
||||
|
||||
* Create a new user and make mounting work with a read-only filesystem.
|
||||
|
||||
```
|
||||
useradd -m kitten
|
||||
chown -R kitten:kitten /home/kitten
|
||||
ln -s /proc/mounts /etc/mtab
|
||||
```
|
||||
|
||||
* Enable rc.local, which ensures that the code in /etc/rc.local is run on boot.
|
||||
This is what triggers CIRCLean to run.
|
||||
|
||||
```
|
||||
systemctl enable rc-local.service
|
||||
```
|
||||
|
||||
* Exit the chroot again, and copy the files from your repository into the mounted
|
||||
image.
|
||||
|
||||
```
|
||||
sudo ./copy_to_final.sh /mnt/arm_rPi/
|
||||
```
|
||||
|
||||
Write the image on a SD card
|
||||
============================
|
||||
|
||||
* Plug your SD card into the computer. Then, find where it is mounted using df:
|
||||
|
||||
```
|
||||
df -h
|
||||
```
|
||||
|
||||
* If it has been automatically mounted, unmount the SD card (use the path you
|
||||
found in the previous step):
|
||||
|
||||
```
|
||||
umount $PATH_TO_YOUR_SD
|
||||
```
|
||||
|
||||
* Write the image to the card:
|
||||
|
||||
```
|
||||
sudo dd bs=4M if=$PATH_TO_YOUR_IMAGE of=$PATH_TO_YOUR_SD
|
||||
```
|
||||
|
|
@ -0,0 +1 @@
|
|||
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait fastboot noswap ro consoleblank=0
|
||||
|
|
@ -19,7 +19,7 @@ cdrom:x:24:pi
|
|||
floppy:x:25:
|
||||
tape:x:26:
|
||||
sudo:x:27:pi
|
||||
audio:x:29:pi
|
||||
audio:x:29:pi,timidity
|
||||
dip:x:30:
|
||||
www-data:x:33:
|
||||
backup:x:34:
|
||||
|
|
@ -52,3 +52,4 @@ indiecity:x:1001:root
|
|||
spi:x:1002:pi
|
||||
gpio:x:1003:pi
|
||||
kitten:x:1004:
|
||||
timidity:x:110:
|
||||
|
|
@ -21,18 +21,13 @@ clean(){
|
|||
|
||||
echo "GROOMER: end of boot, running rc.local."
|
||||
|
||||
# Print the IP address (this doesn't work currently?)
|
||||
# _IP=$(hostname -I) || true
|
||||
# if [ "$_IP" ]; then
|
||||
# printf "My IP address is %s\n" "$_IP"
|
||||
# fi
|
||||
|
||||
if [ -e /dev/sda ]; then
|
||||
if [ -e /dev/sdb ]; then
|
||||
# Avoid possible misuse - turn off eth0 (ethernet port)
|
||||
/sbin/ifconfig eth0 down
|
||||
trap clean EXIT TERM INT
|
||||
cd /opt/groomer
|
||||
/usr/sbin/led &
|
||||
./init.sh
|
||||
fi
|
||||
fi
|
||||
|
|
@ -1,12 +1,14 @@
|
|||
[Unit]
|
||||
Description=/etc/rc.local Compatibility
|
||||
ConditionPathExists=/etc/rc.local
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/etc/rc.local
|
||||
Type=forking
|
||||
ExecStart=/etc/rc.local start
|
||||
TimeoutSec=0
|
||||
StandardInput=tty
|
||||
StandardOutput=tty
|
||||
RemainAfterExit=yes
|
||||
SysVStartPriority=99
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -14,7 +14,7 @@ DST_MNT="/media/dst"
|
|||
TEMP="${DST_MNT}/temp"
|
||||
ZIPTEMP="${DST_MNT}/ziptemp"
|
||||
LOGS="${DST_MNT}/logs"
|
||||
GROOM_LOG="/tmp/groom_log.txt"
|
||||
DEBUG_LOG="/tmp/groomer_debug_log.txt"
|
||||
MUSIC="/opt/midi/"
|
||||
|
||||
|
||||
|
|
@ -24,3 +24,6 @@ TIMIDITY="/usr/bin/timidity"
|
|||
MOUNT="/bin/mount"
|
||||
PMOUNT="/usr/bin/pmount -A -s"
|
||||
PUMOUNT="/usr/bin/pumount"
|
||||
|
||||
# Config flags
|
||||
DEBUG=false
|
||||
|
|
@ -12,7 +12,11 @@ if ! [ "${ID}" -ge "1000" ]; then
|
|||
exit
|
||||
fi
|
||||
|
||||
clean(){
|
||||
clean(){
|
||||
if [ ${DEBUG} = true ]; then
|
||||
sleep 20
|
||||
fi
|
||||
|
||||
# Write anything in memory to disk
|
||||
${SYNC}
|
||||
|
||||
|
|
@ -58,7 +62,7 @@ do
|
|||
# Create a directory on ${DST} named PARTION_$PARTCOUNT
|
||||
target_dir="/media/${DST}/FROM_PARTITION_${PARTCOUNT}"
|
||||
mkdir -p "${target_dir}"
|
||||
LOGFILE="${LOGS}/processing.txt"
|
||||
LOGFILE="${LOGS}/processing_log.txt"
|
||||
|
||||
# Run the current partition through filecheck.py
|
||||
echo "==== Starting processing of /media/${SRC} to ${target_dir}. ====" >> ${LOGFILE}
|
||||
|
|
@ -13,6 +13,9 @@ if [ ${ID} -ne 0 ]; then
|
|||
fi
|
||||
|
||||
clean(){
|
||||
if [ ${DEBUG} = true ]; then
|
||||
sleep 20
|
||||
fi
|
||||
echo "GROOMER: cleaning up after init.sh."
|
||||
${SYNC}
|
||||
# Stop the music from playing
|
||||
|
|
@ -22,15 +25,13 @@ clean(){
|
|||
|
||||
trap clean EXIT TERM INT
|
||||
|
||||
# Stop hdmi display from sleeping after a period of time
|
||||
setterm -powersave off -blank 0
|
||||
|
||||
# Start music
|
||||
./music.sh &
|
||||
echo $! > /tmp/music.pid
|
||||
|
||||
# List block storage devices for debugging
|
||||
# Make sure to set tee in append (-a) mode below if you uncomment
|
||||
# lsblk |& tee ${GROOM_LOG}
|
||||
if [ ${DEBUG} = true ]; then
|
||||
lsblk |& tee -a ${DEBUG_LOG}
|
||||
fi
|
||||
|
||||
su ${USERNAME} -c ./mount_dest.sh |& tee ${GROOM_LOG}
|
||||
su ${USERNAME} -c ./mount_dest.sh |& tee -a ${DEBUG_LOG}
|
||||
|
|
@ -13,10 +13,13 @@ if ! [ "${ID}" -ge "1000" ]; then
|
|||
fi
|
||||
|
||||
clean(){
|
||||
echo "GROOMER: Cleaning up in mount_keys.sh."
|
||||
if [ ${DEBUG} = true ]; then
|
||||
sleep 20
|
||||
# Copy the temporary logfile to the destination key
|
||||
cp ${DEBUG_LOG} "${DST_MNT}/groomer_debug_log.txt"
|
||||
fi
|
||||
|
||||
# Copy the temporary logfile to the destination key
|
||||
cp ${GROOM_LOG} "${DST_MNT}/groomer_log_dst.txt"
|
||||
echo "GROOMER: Cleaning up in mount_keys.sh."
|
||||
|
||||
# Write anything in memory to disk
|
||||
${SYNC}
|
||||
|
|
@ -17,5 +17,6 @@ amixer cset numid=3 1
|
|||
files=(${MUSIC}*)
|
||||
|
||||
while true; do
|
||||
$TIMIDITY ${files[RANDOM % ${#files[@]}]}
|
||||
# -id flags set interface to "dumb" and -qq silences most/all terminal output
|
||||
$TIMIDITY -idqq ${files[RANDOM % ${#files[@]}]}
|
||||
done
|
||||
0
fs_filecheck/usr/local/bin/pdfid.py → circlean_fs/root_partition/usr/local/bin/pdfid.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/pdfid.py → circlean_fs/root_partition/usr/local/bin/pdfid.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_embeddedfile.py → circlean_fs/root_partition/usr/local/bin/plugin_embeddedfile.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_embeddedfile.py → circlean_fs/root_partition/usr/local/bin/plugin_embeddedfile.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_list → circlean_fs/root_partition/usr/local/bin/plugin_list
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_list → circlean_fs/root_partition/usr/local/bin/plugin_list
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_nameobfuscation.py → circlean_fs/root_partition/usr/local/bin/plugin_nameobfuscation.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_nameobfuscation.py → circlean_fs/root_partition/usr/local/bin/plugin_nameobfuscation.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_triage.py → circlean_fs/root_partition/usr/local/bin/plugin_triage.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_triage.py → circlean_fs/root_partition/usr/local/bin/plugin_triage.py
Normal file → Executable file
|
|
@ -0,0 +1,2 @@
|
|||
led: led.c
|
||||
gcc -ggdb -o led led.c
|
||||
|
|
@ -0,0 +1,81 @@
|
|||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
#include <stdint.h>
|
||||
#include <sys/types.h>
|
||||
#include <sys/stat.h>
|
||||
#include <fcntl.h>
|
||||
#include <sys/mman.h>
|
||||
#include <unistd.h>
|
||||
|
||||
/* GPIO registers address */
|
||||
#define BCM2708_PERI_BASE 0x20000000
|
||||
#define GPIO_BASE (BCM2708_PERI_BASE + 0x200000) /* GPIO controller */
|
||||
#define BLOCK_SIZE (256)
|
||||
|
||||
/* GPIO setup macros. Always use GPIO_IN(x) before using GPIO_OUT(x) or GPIO_ALT(x,y) */
|
||||
#define GPIO_IN(g) *(gpio+((g)/10)) &= ~(7<<(((g)%10)*3))
|
||||
#define GPIO_OUT(g) *(gpio+((g)/10)) |= (1<<(((g)%10)*3))
|
||||
#define GPIO_ALT(g,a) *(gpio+(((g)/10))) |= (((a)<=3?(a)+4:(a)==4?3:2)<<(((g)%10)*3))
|
||||
|
||||
#define GPIO_SET(g) *(gpio+7) = 1<<(g) /* sets bit which are 1, ignores bit which are 0 */
|
||||
#define GPIO_CLR(g) *(gpio+10) = 1<<(g) /* clears bit which are 1, ignores bit which are 0 */
|
||||
#define GPIO_LEV(g) (*(gpio+13) >> (g)) & 0x00000001
|
||||
|
||||
|
||||
|
||||
#define GPIO_4 4
|
||||
|
||||
int mem_fd;
|
||||
void *gpio_map;
|
||||
volatile uint32_t *gpio;
|
||||
|
||||
int main(int argc, char* argv[])
|
||||
{
|
||||
int ret;
|
||||
int i;
|
||||
/* open /dev/mem */
|
||||
mem_fd = open("/dev/mem", O_RDWR|O_SYNC);
|
||||
if (mem_fd == -1) {
|
||||
perror("Cannot open /dev/mem");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
/* mmap GPIO */
|
||||
gpio_map = mmap(NULL, BLOCK_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, mem_fd, GPIO_BASE);
|
||||
if (gpio_map == MAP_FAILED) {
|
||||
perror("mmap() failed");
|
||||
exit(1);
|
||||
}
|
||||
/* Always use volatile pointer! */
|
||||
gpio = (volatile uint32_t *)gpio_map;
|
||||
|
||||
|
||||
GPIO_IN(GPIO_4); /* must use GPIO_IN before we can use GPIO_OUT */
|
||||
GPIO_OUT(GPIO_4);
|
||||
|
||||
//Turn on led
|
||||
while (1) {
|
||||
//printf("Enable LED\n");
|
||||
GPIO_SET(GPIO_4);
|
||||
usleep(1000000);
|
||||
//printf("Disable GPIO\n"); // Does not seem to work?
|
||||
//GPIO_CLR(GPIO_4);
|
||||
//usleep(1000000);
|
||||
}
|
||||
/* Free up ressources */
|
||||
/* munmap GPIO */
|
||||
ret = munmap(gpio_map, BLOCK_SIZE);
|
||||
if (ret == -1) {
|
||||
perror("munmap() failed");
|
||||
exit(1);
|
||||
}
|
||||
/* close /dev/mem */
|
||||
ret = close(mem_fd);
|
||||
if (ret == -1) {
|
||||
perror("Cannot close /dev/mem");
|
||||
exit(1);
|
||||
}
|
||||
|
||||
return EXIT_SUCCESS;
|
||||
}
|
||||
36
doc/TODO
36
doc/TODO
|
|
@ -1,36 +0,0 @@
|
|||
TODO
|
||||
====
|
||||
|
||||
* the script locations should be changed in the next version so they don't sit
|
||||
next to the rPi's example development code that ships with the stock rPi
|
||||
* the system isn't optimised and should be : cleanup and making it as close to
|
||||
stock as possible
|
||||
[Npot sure] Starting process should be more obfuscated
|
||||
* strip exif data and leave it in a .txt file next to the image it came from
|
||||
=> exiftool
|
||||
[Done with remount] set filesystem of OS in RO (physical switch and/or remount OS)
|
||||
[OK] mount source key in RO and noexec <= also nosuid and nodev
|
||||
[OK] mount target key with noexec <= also nosuid and nodev
|
||||
* convert spreadsheets in csv ?
|
||||
[done in HTML] convert documents (pdfs/*office/...) in images ?
|
||||
[Not Needed] Have a look at Ghostscript to work on PDFs (.pdf -> .eps -> .png?)
|
||||
[do everything as user] do not run the conversions as root -> run in chroot
|
||||
* take eth0 down in /etc/network/interfaces or in the groomer script disable the
|
||||
interface before anything happens
|
||||
* hdmi should stay up: solvable by poking the power management timer
|
||||
(better not to disable the PM completely)
|
||||
[Done] get rid of pdfbox. remove need for java
|
||||
[WIP] scripts to generate a SD card automatically (win/mac/linux)
|
||||
* move the scripts away from /opt/
|
||||
* strip back libreoffice to minimum required packages. in particular, if possible,
|
||||
remove libreoffice-java-common package
|
||||
* Write the groomer log on the destination key
|
||||
[Done] use /etc/mime.types and file -b --mime-type <filename> to find out the type of
|
||||
the file
|
||||
* Extract metadata from all the files => https://mat.boum.org/
|
||||
|
||||
HTML Files
|
||||
==========
|
||||
|
||||
- disable JS
|
||||
- cleanup external imports (js/css/images)
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
TODO
|
||||
====
|
||||
|
||||
* strip exif data and leave it in a .txt file next to the image it came from
|
||||
=> exiftool
|
||||
* Scripts to generate a SD card automatically (win/mac/linux)
|
||||
* Extract metadata from all the files => https://mat.boum.org/
|
||||
|
||||
HTML Files
|
||||
==========
|
||||
|
||||
- disable JS?
|
||||
- cleanup external imports (js/css/images)
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
* Download qemu, qemu-user-static, and proot if not already installed
|
||||
* Download the newest raspbian-lite image from raspberrypi.org
|
||||
* Verify the sha1 hash of the downloaded .zip file
|
||||
* Unzip the image
|
||||
* Expand the image by 2GB using dd
|
||||
* Expand the root partition using fdisk
|
||||
* Mount both partitions in loop mode using /shell_utils/basic_mount_image.sh
|
||||
* Use df to find the larger partition, and resize the filesystem to fill it
|
||||
* Copy circlean_fs/root_partition/etc/systemd/system/rc-local.service into the equivalent location
|
||||
* Use proot to enter a chroot in the image: sudo proot -q qemu-arm -S /mnt/rpi-root -b /mnt/rpi-boot:/boot /bin/bash
|
||||
* Run dpkg-reconfigure locales
|
||||
* apt-get update
|
||||
* apt-get dist-upgrade (might have to run this and autoremove several times)
|
||||
* apt-get autoremove
|
||||
* apt-get install the linux dependencies:
|
||||
- timidity
|
||||
- git
|
||||
- p7zip-full
|
||||
- pmount ntfs-3g
|
||||
- python3 python3-pip
|
||||
- python3-lxml
|
||||
- libjpeg-dev libtiff-dev libwebp-dev liblcms2-dev tcl-dev tk-dev python-tk
|
||||
* Compile p7zip-rar from source
|
||||
- Change your source.list file
|
||||
- Make a new directory and cd to it
|
||||
- apt-get build-dep p7zip-rar
|
||||
- dpkg -i <p7zip-rar .deb file path>
|
||||
* Make sure the right pip executable is called by `pip3`, change your path if necessary
|
||||
* Upgrade pip: pip3 install -U pip
|
||||
* pip3 install python dependencies
|
||||
- exifread
|
||||
- pillow
|
||||
- olefile
|
||||
- git+https://github.com/decalage2/oletools.git
|
||||
- git+https://github.com/grierforensics/officedissector.git
|
||||
- git+https://github.com/CIRCL/PyCIRCLean.git
|
||||
* Add a user named "kitten"
|
||||
* Symlink /proc/mounts to /etc/mtab
|
||||
* Turn on rc-local.service `systemctl enable rc-local.service`
|
||||
- If it doesn't work, read these instructions: https://www.linuxbabe.com/linux-server/how-to-enable-etcrc-local-with-systemd
|
||||
* apt-get autoclean
|
||||
* apt-get autoremove
|
||||
* Exit from proot
|
||||
* Copy all of the project files from circlean_fs/ into the two partitions:
|
||||
- rsync -vnri <source> <destination> will do a dry run of what will be copied, remove the -n to copy. See the rsync manpage for details.
|
||||
- diode_controller/ if you're using the led functionality and have an external led
|
||||
- midi/ files into /opt/midi/
|
||||
- you might want to double check all of the permissions of the new files/directories
|
||||
* Copy the image over to the SD card: sudo dd bs=4M if=<image> of=/dev/sd<letter>
|
||||
- In newer versions of dd, you can add status=progress
|
||||
* Optional: fsck the root partition (sudo e2fsck -f /dev/sd<letter>2).
|
||||
* Test with an rpi
|
||||
- FAT32 filesystem
|
||||
- NTFS filesystem
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
Modifying an already-built image
|
||||
================================
|
||||
One way to debug the project or test changes quickly is to modify an already built
|
||||
version of the project. Once you've got an image set up on an SD card, you can mount
|
||||
the image and make changes to the files directly or copy changes you've made locally
|
||||
onto the mounted image. The only requirement is a linux distro such as Debian or Ubuntu.
|
||||
If you're using MacOS, you can download and install VirtualBox.
|
||||
|
||||
Mounting an image
|
||||
=================
|
||||
* The steps listed in mount_image.sh are only necessary if you'd like to chroot
|
||||
into and run executables from the image locally.
|
||||
* To mount the image for the purpose of reading/writing to it, the process is much
|
||||
* Plug the SD card into the computer.
|
||||
* If you're on Virtualbox, you'll probably have to unmount the image on the host OS
|
||||
(on MacOS this involves ejecting it or using diskutil unmountDisk) and then mount it
|
||||
on the virtualized OS. You might have to select it under "Devices" first.
|
||||
* Then, in linux, use sudo fdisk -l to find the location of the image.
|
||||
* sudo mount $PATH_TO_IMAGE $PATH_TO_CHOSEN_MOUNT_POINT will mount the image.
|
||||
* The path to the image will need to be the path to the partition with the OS on it,
|
||||
which should be the second partition. So /dev/sdb2, not just dev/sdb.
|
||||
* When you're done, sudo umount $PATH_TO_MOUNT_POINT will unmount it.
|
||||
* If you get a warning about "No caching mode page found," it's safe to skip it
|
||||
by pressing enter.
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
Various qemu startup commands
|
||||
=============================
|
||||
|
||||
From https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=37386
|
||||
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-4.4.13-jessie -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1" -hda ~/qemu_vms/2016-09-23-raspbian-jessie-lite.img -redir tcp:5022::22
|
||||
|
||||
|
||||
From https://github.com/dhruvvyas90/qemu-rpi-kernel
|
||||
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-4.4.13-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ~/qemu_vms/2016-09-23-raspbian-jessie-lite.img
|
||||
|
||||
|
||||
From http://pub.phyks.me/respawn/mypersonaldata/public/2014-05-20-11-08-01/
|
||||
qemu-system-arm -kernel <<<path to kernel>>> -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw init=/bin/bash" -hda <<<path to disk image>>>
|
||||
|
||||
|
||||
Others:
|
||||
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-3.10.25-wheezy -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ~/qemu_vms/2015-02-16-raspbian-wheezy.img
|
||||
|
||||
qemu-system-arm -kernel qemu-rpi-kernel/kernel-qemu-3.10.25-wheezy -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda 2015-02-16-raspbian-wheezy.img
|
||||
|
||||
|
||||
|
||||
Places to get raspbian base images:
|
||||
===================================
|
||||
|
||||
For Raspbian Wheezy image:
|
||||
wget https://downloads.raspberrypi.org/raspbian/images/raspbian-2015-02-17/2015-02-16-raspbian-wheezy.zip
|
||||
|
||||
For Raspbian Jessie Lite image:
|
||||
wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2016-09-28/2016-09-23-raspbian-jessie-lite.zip
|
||||
|
||||
|
||||
|
||||
|
||||
Traceback of the qemu failure on digitalocean
|
||||
=============================================
|
||||
|
||||
pulseaudio: pa_context_connect() failed
|
||||
pulseaudio: Reason: Connection refused
|
||||
pulseaudio: Failed to initialize PA contextaudio: Could not init 'pa' audio driver
|
||||
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
|
||||
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
|
||||
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
|
||||
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
|
||||
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
|
||||
alsa: Could not initialize DAC
|
||||
alsa: Failed to open 'default':
|
||||
alsa: Reason: No such file or directory
|
||||
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
|
||||
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
|
||||
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
|
||||
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
|
||||
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
|
||||
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
|
||||
alsa: Could not initialize DAC
|
||||
alsa: Failed to open 'default':
|
||||
alsa: Reason: No such file or directory
|
||||
audio: Failed to create voice 'lm4549.out'
|
||||
Could not initialize SDL(No available video device) - exiting
|
||||
|
||||
|
||||
Notes
|
||||
=====
|
||||
- The error message: it is probably not a big deal - can make them not being blocking by modifying https://github.com/CIRCL/Circlean/blob/master/tests/run.exp#L10
|
||||
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760365
|
||||
- Could not initialize SDL(No available video device) - exiting <= this one is blocking
|
||||
- I guess it is the vnc switch - requires x11 installed
|
||||
- If you use a cloud instance, you will need to get qemu to open a port you can connect to with vnc
|
||||
- The good thing of having VNC is that you can see what explodes when you're running the image
|
||||
|
|
@ -0,0 +1,224 @@
|
|||
Building the image from scratch
|
||||
===============================
|
||||
|
||||
There is always a prebuilt image available for download and installation as
|
||||
described in the [README](../README.md). If you'd like to build the project yourself,
|
||||
there are several steps involved:
|
||||
|
||||
* Downloading a generic Raspbian Lite image
|
||||
* Resizing the image and partition
|
||||
* Downloading and building the dependencies
|
||||
* Modifying the image configuration
|
||||
* Copying the project filesystem into the image
|
||||
|
||||
This procedure will only work on Ubuntu or Debian Linux. If you use MacOS or
|
||||
Windows, the best option is to install Linux in a virtual machine using
|
||||
something like VirtualBox.
|
||||
|
||||
It is recommended that you make a copy of image_setup_checklist.md and √ items off
|
||||
on the list as you go.
|
||||
|
||||
Preparation
|
||||
===========
|
||||
|
||||
* Make sure your development environment is up to date:
|
||||
```
|
||||
apt-get update
|
||||
apt-get dist-upgrade
|
||||
```
|
||||
* Install qemu, qemu-user-static, and proot if not already installed:
|
||||
```
|
||||
apt-get install qemu qemu-user-static proot
|
||||
```
|
||||
|
||||
Download the Raspbian image
|
||||
==============================
|
||||
|
||||
* Get the most recent version of Raspbian Jessie Lite from https://downloads.raspberrypi.org/raspbian_lite/images/:
|
||||
|
||||
```
|
||||
wget https://downloads.raspberrypi.org/raspbian_lite_latest
|
||||
```
|
||||
* Verify the hash of the downloaded file and compare it to the hash on the server:
|
||||
```
|
||||
shasum XXXX-XX-XX-raspbian-jessie-lite.zip
|
||||
```
|
||||
* Unpack it:
|
||||
```
|
||||
unzip XXXX-XX-XX-raspbian-jessie-lite.zip
|
||||
```
|
||||
|
||||
Add space to the image
|
||||
=========================
|
||||
|
||||
* Use dd to add 2GB (2048 blocks of 1024k each). Using /dev/zero as the input
|
||||
file yields an unlimited number of "0x00" bytes.
|
||||
```
|
||||
> dd if=/dev/zero bs=1024k count=2048 >> XXXX-XX-XX-raspbian-jessie-lite.img
|
||||
```
|
||||
* Expand the root (second) partition using fdisk. The first partition listed is the boot
|
||||
partition, which shouldn't be changed. In the new partition, the "First sector" should be
|
||||
the value that was the "start" sector of the old root partition (137216 in the example
|
||||
below, but this varies depending on the version of the Raspbian image). The "Last sector"
|
||||
should be the default, and it should be significantly larger than it was before (6909951 vs.
|
||||
2715647 in the example).
|
||||
|
||||
```
|
||||
> fdisk XXXX-XX-XX-raspbian-jessie-lite.img
|
||||
|
||||
Command (m for help): *p*
|
||||
Disk XXXX-XX-XX-raspbian-jessie-lite.img: 3.3 GiB, 3537895424 bytes, 6909952 sectors
|
||||
Units: sectors of 1 * 512 = 512 bytes
|
||||
Sector size (logical/physical): 512 bytes / 512 bytes
|
||||
I/O size (minimum/optimal): 512 bytes / 512 bytes
|
||||
Disklabel type: dos
|
||||
Disk identifier: 0x244b8248
|
||||
|
||||
Device Boot Start End Sectors Size Id Type
|
||||
XXXX-XX-XX-raspbian-jessie-lite.img1 8192 137215 129024 63M c W95 FAT32 (LBA)
|
||||
XXXX-XX-XX-raspbian-jessie-lite.img2 137216 2715647 2578432 1.2G 83 Linux
|
||||
|
||||
Command (m for help): *d*
|
||||
Partition number (1,2, default 2): *2*
|
||||
|
||||
Partition 2 has been deleted.
|
||||
|
||||
Command (m for help): *n*
|
||||
Partition type
|
||||
p primary (1 primary, 0 extended, 3 free)
|
||||
e extended (container for logical partitions)
|
||||
Select (default p):
|
||||
|
||||
Using default response p.
|
||||
Partition number (2-4, default 2):
|
||||
First sector (2048-6852607, default 2048): *137216*
|
||||
Last sector, +sectors or +size{K,M,G,T,P} (131216-6909951, default 6909951):
|
||||
|
||||
Created a new partition 2 of type 'Linux' and of size 3.2 GiB.
|
||||
|
||||
Command (m for help): *w*
|
||||
The partition table has been altered.
|
||||
Syncing disks.
|
||||
```
|
||||
* Mount the image in loop mode: first, edit shell_utils/basic_mount_image.sh to use the
|
||||
proper values for $BOOT_START and $ROOT_START, which you can obtain using fdisk and "p"
|
||||
as in the previous step. You must also change $IMAGE to the correct path. Then run:
|
||||
```
|
||||
sudo ./shell_utils/basic_mount_image.md
|
||||
```
|
||||
* Verify the path to the mounted partitions in /dev, and resize the root (larger) filesystem
|
||||
to fill the new larger partition using resize2fs:
|
||||
```
|
||||
> df | grep /mnt/arm
|
||||
|
||||
/dev/loop0 3927752 1955672 1794172 53% /mnt/rpi-root
|
||||
/dev/loop1 57288 18960 38328 34% /mnt/rpi-boot
|
||||
|
||||
> sudo resize2fs /dev/loop0
|
||||
```
|
||||
|
||||
Installing the dependencies
|
||||
===========================
|
||||
|
||||
* Copy circlean_fs/root_partition/systemd/system/rc-local.service into the equivalent location in the image.
|
||||
```
|
||||
cp circlean_fs/root_partition/systemd/system/rc-local.service /mnt/rpi-root/etc/systemd/system/rc-local.service
|
||||
```
|
||||
* Use [proot](https://proot-me.github.io/) to enter the equivalent of a chroot inside
|
||||
the mounted image.
|
||||
```
|
||||
sudo proot -q qemu-arm -S /mnt/rpi-root -b /mnt/rpi-boot:/boot /bin/bash
|
||||
```
|
||||
* Change your locales (remove "en_GB.UTF-8 UTF-8", add "en_US.UTF-8 UTF-8"). The
|
||||
arrow keys move the cursor, spacebar selects/deselects a locale, tab moves the cursor
|
||||
to a different context, and enter lets you select "ok". This step might take some time,
|
||||
be patient:
|
||||
```
|
||||
dpkg-reconfigure locales
|
||||
```
|
||||
* In the image, make sure everything is up-to-date and remove old packages. You may have to
|
||||
run dist-upgrade and autoremove several times for everything to be installed, and a few
|
||||
raspbian-sys-mods related installs may fail - you can ignore them:
|
||||
```
|
||||
apt-get update
|
||||
apt-get dist-upgrade
|
||||
apt-get autoremove
|
||||
```
|
||||
* Install the linux dependencies (see CONTRIBUTING.md for more details):
|
||||
```
|
||||
apt-get install timidity git p7zip-full python3 python3-pip python3-lxml pmount ntfs-3g libjpeg-dev libtiff-dev libwebp-dev tk-dev python-tk liblcms2-dev tcl-dev
|
||||
```
|
||||
* Compile p7zip-rar from source. First, uncomment out the second line in /etc/apt/sources.list. Then:
|
||||
```
|
||||
cd /home/pi
|
||||
mkdir rar && cd rar/
|
||||
apt-get build-dep p7zip-rar
|
||||
dpkg -i ${path to p7zip-rar .deb file}
|
||||
```
|
||||
* Install the Python dependencies for PyCIRCLean/filecheck.py. PyCIRCLean is 3.3+
|
||||
compatible, so use pip -V to make sure you're using the right version of pip. You might
|
||||
have to edit your PATH variable or use pip3 to get the correct pip. You also might want to
|
||||
verify that these dependencies are current by checking in the PyCIRCLean git repo.
|
||||
```
|
||||
pip install -U pip
|
||||
pip install oletools exifread Pillow
|
||||
pip install git+https://github.com/decalage2/oletools.git
|
||||
pip install git+https://github.com/Rafiot/officedissector.git
|
||||
pip install git+https://github.com/CIRCL/PyCIRCLean.git
|
||||
```
|
||||
* Create a new user named "kitten":
|
||||
```
|
||||
useradd -m kitten
|
||||
chown -R kitten:kitten /home/kitten
|
||||
```
|
||||
* Symlinking /proc/mounts to /etc/mtab is necessary because /etc/mtab cannot be edited by
|
||||
pmount if root is read-only. /proc/mounts is maintained by the kernel and is guaranteed to
|
||||
be accurate.
|
||||
```
|
||||
ln -s /proc/mounts /etc/mtab
|
||||
```
|
||||
* Enable rc.local, which ensures that the code in /etc/rc.local is run on boot.
|
||||
This is what triggers CIRCLean to run.
|
||||
```
|
||||
systemctl enable rc-local.service
|
||||
```
|
||||
* Clean up:
|
||||
```
|
||||
apt-get clean
|
||||
apt-get autoremove
|
||||
apt-get autoclean
|
||||
```
|
||||
* Exit proot, and copy the files from your repository into the mounted
|
||||
image. Adding a -n flag will make rsync do a dry run instead of copying. See the rsync
|
||||
manpage for more details. Make sure to include the trailing slashes on the paths:
|
||||
```
|
||||
exit
|
||||
sudo rsync -vri circlean_fs/boot/ /mnt/rpi-boot/
|
||||
sudo rsync -vri circlean_fs/root_partition/ /mnt/rpi-root/
|
||||
cp -rf midi /mnt/rpi-root/opt/
|
||||
```
|
||||
* If have an external hardware led and you're using the led functionality, copy
|
||||
the led files from diode_controller/ as well.
|
||||
|
||||
Write the image on a SD card
|
||||
============================
|
||||
|
||||
* Plug your SD card into the computer. Then, find where it is mounted using lsblk or df:
|
||||
```
|
||||
lsblk
|
||||
df -h
|
||||
```
|
||||
* If it has been automatically mounted, unmount the SD card (use the path you
|
||||
found in the previous step):
|
||||
```
|
||||
umount $PATH_TO_YOUR_SD
|
||||
```
|
||||
* Write the image to the card. Newer versions of dd include a status option to monitor the
|
||||
copying process:
|
||||
```
|
||||
sudo dd bs=4M if=$PATH_TO_YOUR_IMAGE of=$PATH_TO_YOUR_SD status=progress
|
||||
```
|
||||
* Use fsck to verify the root partition:
|
||||
```
|
||||
sudo e2fsck -f /dev/sd<number>2
|
||||
```
|
||||
|
|
@ -16,14 +16,14 @@ Ideas
|
|||
=====
|
||||
|
||||
Source keys:
|
||||
[DONE] Working documents, one / multiple partitions
|
||||
- Working documents, one / multiple partitions
|
||||
- Non working documents: one / multiple partitions
|
||||
- different FS on different partitions
|
||||
- Non working FS
|
||||
- Malicious documents (very slow, might break the conversions)
|
||||
|
||||
Destinations keys
|
||||
[DONE] empty, big enough
|
||||
- empty, big enough
|
||||
- empty, too small
|
||||
- broken
|
||||
- not empty
|
||||
|
|
|
|||
|
|
@ -0,0 +1,39 @@
|
|||
#!/bin/bash
|
||||
|
||||
# This script will mount a given image in loop mode.
|
||||
# Make sure to change the path and offsets for the image you use. You can get
|
||||
# the correct offsets using `file $PATH_TO_IMAGE` or fdisk.
|
||||
|
||||
# To make debugging easier
|
||||
echo "KittenGroomer: in mount_image.sh" 1>&2
|
||||
|
||||
if [ "$(id -u)" != "0" ]; then
|
||||
echo "This script must be run as root" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
set -e
|
||||
set -x
|
||||
|
||||
# Double check the path and offsets as noted above!
|
||||
# Path to the image
|
||||
IMAGE='2017-02-02_CIRCLean.img'
|
||||
# Start sector of boot (first) partition
|
||||
BOOT_START=8192
|
||||
# Start sector of root (second) partition
|
||||
ROOT_START=137216
|
||||
# Locations you'd like the partitions mounted
|
||||
BOOT_PATH='/mnt/rpi-boot'
|
||||
ROOTFS_PATH='/mnt/rpi-root'
|
||||
|
||||
# Calculate offsets for each partition
|
||||
offset_boot=$((${BOOT_START} * 512))
|
||||
offset_rootfs=$((${ROOT_START} * 512))
|
||||
# TODO: add logic for creating directories if they aren't already there
|
||||
mkdir -p ${BOOT_PATH}
|
||||
mkdir -p ${ROOTFS_PATH}
|
||||
# Mount each partition in loop mode
|
||||
mount -o loop,offset=${offset_boot} ${IMAGE} ${BOOT_PATH}
|
||||
mount -o loop,offset=${offset_rootfs} ${IMAGE} ${ROOTFS_PATH}
|
||||
|
||||
echo "Image mounted" 1>&2
|
||||
|
|
@ -1,12 +0,0 @@
|
|||
#!/bin/bash
|
||||
|
||||
set -x
|
||||
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/init.sh /mnt/rpi/opt/groomer/init.sh
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/groomer.sh /mnt/rpi/opt/groomer/groomer.sh
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/config.sh /mnt/rpi/opt/groomer/config.sh
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/mount_dest.sh /mnt/rpi/opt/groomer/mount_dest.sh
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/music.sh /mnt/rpi/opt/groomer/music.sh
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/rc.local /mnt/rpi/etc/rc.local
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/pmount.allow /mnt/rpi/etc/pmount.allow
|
||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/udev/rules.d/10-usb.rules /mnt/rpi/etc/udev/rules.d/10-usb.rules
|
||||
Loading…
Reference in New Issue