mirror of https://github.com/CIRCL/Circlean
commit
4e5b7b2815
26
CHANGELOG
26
CHANGELOG
|
@ -1,26 +0,0 @@
|
||||||
Version 1.2 - 2015-03-10
|
|
||||||
|
|
||||||
- Rollback the migration to Jessie and use Wheezy again: the only important dependency from Jessie was poppler, which is available in the backports
|
|
||||||
- Use the most recent security patches
|
|
||||||
- Do not wait for user input in case of password protected archive
|
|
||||||
|
|
||||||
Version 1.1.1 - 2014-10-26
|
|
||||||
|
|
||||||
- General upgrade of Debian to avoid the system to fail in case there is no HDMI cable connected.
|
|
||||||
|
|
||||||
Version 1.1 - 2014-10-01
|
|
||||||
|
|
||||||
- NTFS support added for USB key
|
|
||||||
- Updated to Debian Jessie including patches for [bash vulnerabilities CVE-2014-6271 - CVE-2014-7169](/pub/tr-27/)
|
|
||||||
- CIRCLean user are now removed from the sudoer
|
|
||||||
|
|
||||||
Version 1.0 - 2014-05-20
|
|
||||||
|
|
||||||
- Based on Raspbian Jessie
|
|
||||||
- Fully automated tests with Qemu
|
|
||||||
- Mimetype: support of PDF, Office documents, archives, windows executables
|
|
||||||
- Filesystem: USB keys have to be formated in vfat
|
|
||||||
- Support of multiple partitions
|
|
||||||
- Renaming of autorun.inf on the source key
|
|
||||||
- Operating system is read only
|
|
||||||
- Use pdf2htmlEX v0.11
|
|
|
@ -0,0 +1,50 @@
|
||||||
|
Version 2.1 - 2017-02-02
|
||||||
|
- Updated to the newest version of Raspbian Jessie lite (January 11th 2017 release)
|
||||||
|
- NTFS files can now be mounted as source or destination keys
|
||||||
|
- Added udev rules that ensure the USB ports map deterministically to source and destination keys
|
||||||
|
- New debug flag and debug logging functionality to make working on Circlean without a monitor easier
|
||||||
|
- Turned off automatic display sleep
|
||||||
|
|
||||||
|
Version 2.0.2 - 2016-05-12
|
||||||
|
- Improve filename encoding
|
||||||
|
|
||||||
|
Version 2.0.1 - 2016-04-26
|
||||||
|
- Re-add [timidity](http://timidity.sourceforge.net/) so the MIDI files are played properly
|
||||||
|
|
||||||
|
Version 2.0 - 2016-04-26
|
||||||
|
- No critical bugs have been identified, this release uses the latest version of Raspbian Jessie lite, with all system updates
|
||||||
|
|
||||||
|
Version 2.0-BETA - 2015-11-06
|
||||||
|
- There a new beta version of CIRCLean which is a significant improvement from the latest version in term of speed and efficiency on low-end hardware like the first version of the Raspberry Pi. The new code base of CIRCLean is now based on [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean)
|
||||||
|
|
||||||
|
Version 1.3 - 2015-05-27
|
||||||
|
- Fix a [critical security bug](https://www.circl.lu/projects/CIRCLean/security/advisory-01) related to [polyglot files](https://github.com/CIRCL/Circlean/issues/9) - thanks to the reporters ([Jann Horn](https://github.com/thejh), [seclab-solutions](http://www.seclab-solutions.com/))
|
||||||
|
- Use [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean) for conversion
|
||||||
|
- Convert PDF files to PDF/A before converting to HTML
|
||||||
|
|
||||||
|
Version 1.2 - 2015-03-10
|
||||||
|
|
||||||
|
- Rollback the migration to Jessie and use Wheezy again: the only important dependency from Jessie was poppler, which is available in the backports
|
||||||
|
- Use the most recent security patches
|
||||||
|
- Do not wait for user input in case of password protected archive
|
||||||
|
|
||||||
|
Version 1.1.1 - 2014-10-26
|
||||||
|
|
||||||
|
- General upgrade of Debian to avoid the system to fail in case there is no HDMI cable connected.
|
||||||
|
|
||||||
|
Version 1.1 - 2014-10-01
|
||||||
|
|
||||||
|
- NTFS support added for USB key
|
||||||
|
- Updated to Debian Jessie including patches for [bash vulnerabilities CVE-2014-6271 - CVE-2014-7169](/pub/tr-27/)
|
||||||
|
- CIRCLean user are now removed from the sudoer
|
||||||
|
|
||||||
|
Version 1.0 - 2014-05-20
|
||||||
|
|
||||||
|
- Based on Raspbian Jessie
|
||||||
|
- Fully automated tests with Qemu
|
||||||
|
- Mimetype: support of PDF, Office documents, archives, windows executables
|
||||||
|
- Filesystem: USB keys have to be formated in vfat
|
||||||
|
- Support of multiple partitions
|
||||||
|
- Renaming of autorun.inf on the source key
|
||||||
|
- Operating system is read only
|
||||||
|
- Use pdf2htmlEX v0.11
|
|
@ -1,17 +1,50 @@
|
||||||
Building the project
|
Getting started
|
||||||
====================
|
===============
|
||||||
|
|
||||||
To get started contributing to Circlean, first, fork the project and `git clone`
|
If you'd like to work on the Python code that processes files for Circlean, you should
|
||||||
your fork. Then, follow the instructions in [README_setup.md](README_setup.md)
|
take a look at [PyCIRCLean](https://github.com/CIRCL/PyCIRCLean), specifically the
|
||||||
to build an image.
|
filecheck.py script. To get started contributing to Circlean, first, fork the project and
|
||||||
|
`git clone` your fork. Then, follow the instructions in [setup_with_proot.md](doc/setup_with_proot.md) to build an image. To make things easier, you can also download a
|
||||||
|
prebuilt image as mentioned in the README, and then mount and make modifications to this
|
||||||
|
image to test your changes.
|
||||||
|
|
||||||
The issue tracker
|
The issue tracker
|
||||||
=================
|
=================
|
||||||
|
|
||||||
If you find a bug or see a problem with PyCIRCLean, please open an issue in the Github
|
If you find a bug or see a problem with PyCIRCLean, please open an issue in the Github
|
||||||
repo. We'll do our best to respond as quickly as possible. Also, feel free to contribute a solution
|
repo. We'll do our best to respond as quickly as possible. Also, feel free to contribute a
|
||||||
to any of the open issues - we'll do our best to review your pull request in a timely manner.
|
solution to any of the open issues - we'll do our best to review your pull request in a
|
||||||
This project is in active development, so any contributions are welcome!
|
timely manner. This project is in active development, so any contributions are welcome!
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
* Timidity for playing midi files
|
||||||
|
* Git for installing some Python dependencies
|
||||||
|
* 7Zip for unpacking archives
|
||||||
|
* Pmount and ntfs-3g for mounting usb key partitions
|
||||||
|
* Python 3 and pip for installing and running Python dependencies
|
||||||
|
* Python3-lxml for handling ooxml and other Office files in filecheck.py
|
||||||
|
* libjpeg-dev, libtiff-dev, libwebp-dev, liblcms2-dev, tcl-dev, tk-dev, and python-tk for various image formats (dependencies for pillow)
|
||||||
|
* Exifread for file metadata
|
||||||
|
* Pillow for handling images
|
||||||
|
* Olefile, oletools, and officedissector for handling various Office filetypes
|
||||||
|
* PyCIRCLean for main file handling code
|
||||||
|
|
||||||
|
Helper scripts
|
||||||
|
==============
|
||||||
|
|
||||||
|
Use the scripts in shell_utils/ as examples - do not run them blindly as you will most
|
||||||
|
probably have to change some constants/paths accordingly to your configuration.
|
||||||
|
|
||||||
|
IN ALL CASES, PLEASE READ THE COMMENTS IN THE SCRIPTS AT LEAST ONCE.
|
||||||
|
|
||||||
|
* proper_chroot.sh: uses qemu to chroot into a raspbian instance (.img or SD Card)
|
||||||
|
* prepare_rPI.sh: update the system, some configuration
|
||||||
|
* create_user.sh: create the user who will run the scripts, assign the proper sudo rights.
|
||||||
|
* copy_to_final.sh: populate the content of the directory fs/ in the image,
|
||||||
|
contains a sample of dd command to write the image on the SD card.
|
||||||
|
NOTE: TAKE CARE NOT TO USE THE WRONG DESTINATION
|
||||||
|
|
||||||
|
|
||||||
Running the tests
|
Running the tests
|
||||||
=================
|
=================
|
||||||
|
|
46
README.md
46
README.md
|
@ -3,17 +3,19 @@ CIRCLean
|
||||||
![CIRCLean logo](https://www.circl.lu/assets/images/logos/circlean.png)
|
![CIRCLean logo](https://www.circl.lu/assets/images/logos/circlean.png)
|
||||||
![Cleaner in action](http://www.circl.lu/assets/images/CIRCLean/CIRCLean.png)
|
![Cleaner in action](http://www.circl.lu/assets/images/CIRCLean/CIRCLean.png)
|
||||||
|
|
||||||
How To
|
How To Install
|
||||||
======
|
==============
|
||||||
|
|
||||||
[Graphical how-to and pre-built image](http://circl.lu/projects/CIRCLean/).
|
[Graphical how-to and pre-built image download](http://circl.lu/projects/CIRCLean/).
|
||||||
|
|
||||||
To prepare the SD card on Windows, you can use [Win32DiskImager](http://sourceforge.net/projects/win32diskimager/).
|
To prepare the SD card on Windows, you can use [Win32DiskImager](http://sourceforge.net/projects/win32diskimager/). On linux/macOS, use dd (see the how-to link for
|
||||||
|
instructions).
|
||||||
|
|
||||||
And the linux way is in the command line, via dd (see in copy_to_final.sh)
|
The current prebuilt image is based on the 1-11-17 release of Raspbian Jessie Lite.
|
||||||
|
The smallest SD card that Circlean can fit on is currently 4GB.
|
||||||
|
|
||||||
If you'd like to contribute to the project or build the image yourself, see
|
If you'd like to contribute to the project or build the image yourself, see
|
||||||
[contributing](CONTRIBUTING.md) and the [setup readme](README_setup.md).
|
[contributing.md](CONTRIBUTING.md) and the [setup instructions](doc/setup_with_proot.md).
|
||||||
This is a work in progress - contributions are welcome.
|
This is a work in progress - contributions are welcome.
|
||||||
|
|
||||||
Why/What
|
Why/What
|
||||||
|
@ -21,21 +23,25 @@ Why/What
|
||||||
|
|
||||||
This project aims to be useful when you get/find a USB key that you can't trust,
|
This project aims to be useful when you get/find a USB key that you can't trust,
|
||||||
and you want to look at its contents without taking the risk of plugging it into
|
and you want to look at its contents without taking the risk of plugging it into
|
||||||
your computer directly.
|
your computer directly. The official project page can be found at [https://www.circl.lu/projects/CIRCLean/]
|
||||||
|
|
||||||
|
The Raspberry Pi Foundation has a [blog post](https://www.raspberrypi.org/blog/kittengroomercirclean-data-security-for-journalists-and-activists/) with more information
|
||||||
|
about an older version of the project and details of the inspiration behind it.
|
||||||
|
|
||||||
CIRCLean is currently tested to work with USB keys that have FAT32, NTFS, or
|
CIRCLean is currently tested to work with USB keys that have FAT32, NTFS, or
|
||||||
ext2/3/4 filesystems. Currently, exFAT is not supported due to lack of support for
|
ext2/3/4 filesystems (ext* filesystems can only be used as source keys, not destination
|
||||||
this format in pmount. The vast majority of USB keys will be FAT32 or NTFS.
|
keys). Currently, exFAT is not supported due to lack of support for this format in pmount.
|
||||||
|
The vast majority of USB keys will be FAT32 or NTFS.
|
||||||
|
|
||||||
The content of the untrusted key will be copied or/and converted to the second
|
The content of the untrusted key will be copied or/and converted to the second
|
||||||
(blank) key following these rules (based on the mime type as determined bylibmagic):
|
(blank) key following these rules (based on the mime type as determined by libmagic):
|
||||||
- Direct copy of:
|
- Direct copy of:
|
||||||
- Plain text files (mime type: text/*)
|
- Plain text files (mime type: text/*)
|
||||||
- Audio files (mime type: audio/*)
|
- Audio files (mime type: audio/*)
|
||||||
- Video files (mime type: video/*)
|
- Video files (mime type: video/*)
|
||||||
- Example files (mime type: example/*)
|
- Example files (mime type: example/*)
|
||||||
- Multipart files (mime type: multipart/*)
|
- Multipart files (mime type: multipart/*)
|
||||||
- *xml* files, after being converted to text files
|
- xml files, after being converted to text files
|
||||||
- Octet-stream files
|
- Octet-stream files
|
||||||
- Copied after verification:
|
- Copied after verification:
|
||||||
- Image files after verifying that they are not compression bombs (mime type: image/*)
|
- Image files after verifying that they are not compression bombs (mime type: image/*)
|
||||||
|
@ -55,7 +61,7 @@ Usage
|
||||||
=====
|
=====
|
||||||
|
|
||||||
0. Power off the device and unplug all connections.
|
0. Power off the device and unplug all connections.
|
||||||
1. Plug the untrusted key in the top USB slot of the Raspberry Pi.
|
1. Plug the untrusted key in the top left USB slot of the Raspberry Pi.
|
||||||
2. Plug your own key in the bottom USB slot (or use any of the other slots if
|
2. Plug your own key in the bottom USB slot (or use any of the other slots if
|
||||||
there are more than 2).
|
there are more than 2).
|
||||||
|
|
||||||
|
@ -71,19 +77,3 @@ there are more than 2).
|
||||||
connected the HDMI cable, check the screen. The process is slow and can take
|
connected the HDMI cable, check the screen. The process is slow and can take
|
||||||
30-60 minutes depending on how many document conversions take place.
|
30-60 minutes depending on how many document conversions take place.
|
||||||
6. Power off the device and disconnect the drives.
|
6. Power off the device and disconnect the drives.
|
||||||
|
|
||||||
Helper scripts
|
|
||||||
==============
|
|
||||||
|
|
||||||
You should use them as examples when you are creating a new image and probably not
|
|
||||||
run them blindly as you will most probably have to change parameters accordingly to
|
|
||||||
your configuration.
|
|
||||||
|
|
||||||
IN ALL CASES, PLEASE READ THE COMMENTS IN THE SCRIPTS AT LEAST ONCE.
|
|
||||||
|
|
||||||
* proper_chroot.sh: uses qemu to chroot into a raspbian instance (.img or SD Card)
|
|
||||||
* prepare_rPI.sh: update the system, some configuration
|
|
||||||
* create_user.sh: create the user who will run the scripts, assign the proper sudo rights.
|
|
||||||
* copy_to_final.sh: populate the content of the directory fs/ in the image,
|
|
||||||
contains a sample of dd command to write the image on the SD card.
|
|
||||||
NOTE: TAKE CARE NOT TO USE THE WRONG DESTINATION
|
|
||||||
|
|
196
README_setup.md
196
README_setup.md
|
@ -1,196 +0,0 @@
|
||||||
Building the image from scratch
|
|
||||||
===============================
|
|
||||||
|
|
||||||
There is always a prebuilt image available for download and installation as
|
|
||||||
described in the [README](README.md). If you'd like to build the project yourself,
|
|
||||||
there are several steps involved:
|
|
||||||
|
|
||||||
* Downloading a generic Raspbian Lite image
|
|
||||||
* Adding space to the image
|
|
||||||
* Downloading and building the dependencies
|
|
||||||
* Copying the project filesystem into the image
|
|
||||||
|
|
||||||
This procedure will only work on Ubuntu or Debian Linux. If you use MacOS or
|
|
||||||
Windows, the best option is to install Linux in a virtual machine using
|
|
||||||
something like VirtualBox.
|
|
||||||
|
|
||||||
Downloading the Raspbian image
|
|
||||||
==============================
|
|
||||||
|
|
||||||
* Get the most recent version of Raspbian Jessie Lite:
|
|
||||||
|
|
||||||
```
|
|
||||||
wget https://downloads.raspberrypi.org/raspbian_lite_latest
|
|
||||||
```
|
|
||||||
|
|
||||||
* Unpack it:
|
|
||||||
|
|
||||||
```
|
|
||||||
unzip XXXX-XX-XX-raspbian-jessie-lite.zip
|
|
||||||
```
|
|
||||||
|
|
||||||
Adding space to the image
|
|
||||||
=========================
|
|
||||||
|
|
||||||
* Use dd to add 2GB (2048 blocks at 1024k each). Using /dev/zero as the input
|
|
||||||
file yields an unlimited number of "0x00" bytes.
|
|
||||||
|
|
||||||
```
|
|
||||||
> dd if=/dev/zero bs=1024k count=2048 >> XXXX-XX-XX-raspbian-jessie-lite.img
|
|
||||||
```
|
|
||||||
|
|
||||||
* Grow the root partition using fdisk. The "p" command prints the current partition
|
|
||||||
table. The first partition listed is the boot partition, which shouldn't be changed.
|
|
||||||
The "d" command, when given the parameter "2", deletes the current root partition.
|
|
||||||
The "n" command then makes a new partition. It can take the default for "type"
|
|
||||||
and "number". The "First sector" should be the value that was the "start" sector of the root
|
|
||||||
partition (131072 in the example below, but this varies depending on the version of the
|
|
||||||
Raspbian image). The "Last sector" should be the default, and it should be significantly
|
|
||||||
larger than it was before (6852607 vs. 2658303 in the example).
|
|
||||||
|
|
||||||
|
|
||||||
```
|
|
||||||
> fdisk XXXX-XX-XX-raspbian-jessie-lite.img
|
|
||||||
|
|
||||||
Command (m for help): *p*
|
|
||||||
Disk XXXX-XX-XX-raspbian-jessie-lite.img: 3.3 GiB, 3508535296 bytes, 6852608 sectors
|
|
||||||
Units: sectors of 1 * 512 = 512 bytes
|
|
||||||
Sector size (logical/physical): 512 bytes / 512 bytes
|
|
||||||
I/O size (minimum/optimal): 512 bytes / 512 bytes
|
|
||||||
Disklabel type: dos
|
|
||||||
Disk identifier: 0x6f92008e
|
|
||||||
|
|
||||||
Device Boot Start End Sectors Size Id Type
|
|
||||||
XXXX-XX-XX-raspbian-jessie-lite.img1 8192 131071 122880 60M c W95 FAT32 (LBA)
|
|
||||||
XXXX-XX-XX-raspbian-jessie-lite.img2 131072 2658303 2527232 1.2G 83 Linux
|
|
||||||
|
|
||||||
Command (m for help): *d*
|
|
||||||
Partition number (1,2, default 2): *2*
|
|
||||||
|
|
||||||
Partition 2 has been deleted.
|
|
||||||
|
|
||||||
Command (m for help): *n*
|
|
||||||
Partition type
|
|
||||||
p primary (1 primary, 0 extended, 3 free)
|
|
||||||
e extended (container for logical partitions)
|
|
||||||
Select (default p):
|
|
||||||
|
|
||||||
Using default response p.
|
|
||||||
Partition number (2-4, default 2):
|
|
||||||
First sector (2048-6852607, default 2048): *131072*
|
|
||||||
Last sector, +sectors or +size{K,M,G,T,P} (131072-6852607, default 6852607):
|
|
||||||
|
|
||||||
Created a new partition 2 of type 'Linux' and of size 3.2 GiB.
|
|
||||||
|
|
||||||
Command (m for help): *w*
|
|
||||||
The partition table has been altered.
|
|
||||||
Syncing disks.
|
|
||||||
```
|
|
||||||
|
|
||||||
* Mount the image in loop mode: first, edit /mount_image.sh to use the proper values
|
|
||||||
for $OFFSET_BOOT and $OFFSET_ROOTFS, which you can obtain using fdisk and "p" as
|
|
||||||
shown above. You must also change $IMAGE to the correct path. Then run:
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo ./proper_chroot.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
* After mounting the image, the above script will chroot into the mounted image.
|
|
||||||
While in a chroot, the / directory of the image appears as the system / directory
|
|
||||||
(thus the name, change root). To exit the chroot, run "exit" in the root directory.
|
|
||||||
Then, verify the path to the mounted partitions, and resize the filesystem
|
|
||||||
to fill the new larger partition using resize2fs:
|
|
||||||
|
|
||||||
```
|
|
||||||
> df | grep /mnt/arm
|
|
||||||
|
|
||||||
/dev/loop0 3927752 1955672 1794172 53% /mnt/arm_rPi
|
|
||||||
/dev/loop1 57288 18960 38328 34% /mnt/arm_rPi/boot
|
|
||||||
|
|
||||||
> sudo resize2fs /dev/loop0
|
|
||||||
```
|
|
||||||
|
|
||||||
Installing the dependencies
|
|
||||||
===========================
|
|
||||||
|
|
||||||
* To install the dependencies, you'll have to reenter the chroot again:
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo chroot /mnt/arm_rPi
|
|
||||||
```
|
|
||||||
|
|
||||||
* Change your user to root (your global variables may be broken as a result):
|
|
||||||
|
|
||||||
```
|
|
||||||
su root
|
|
||||||
```
|
|
||||||
|
|
||||||
* Change the locales (remove "en_GB.UTF-8 UTF-8", add "en_US.UTF-8 UTF-8"). The
|
|
||||||
arrow keys move the cursor, spacebar selects/deselects a locale, tab moves the cursor
|
|
||||||
to a different context, and enter lets you select "ok":
|
|
||||||
|
|
||||||
```
|
|
||||||
dpkg-reconfigure locales
|
|
||||||
```
|
|
||||||
|
|
||||||
* In the image, make sure everything is up-to-date and remove the old packages:
|
|
||||||
|
|
||||||
```
|
|
||||||
apt-get update
|
|
||||||
apt-get dist-upgrade
|
|
||||||
apt-get autoremove
|
|
||||||
apt-get install timidity git p7zip-full python-dev python-pip python-lxml pmount libjpeg-dev libtiff-dev libwebp-dev liblcms2-dev tcl-dev tk-dev python-tk libxml2-dev libxslt1-dev
|
|
||||||
```
|
|
||||||
|
|
||||||
* Install the Python dependencies for PyCIRCLean. Currently, PyCIRCLean is
|
|
||||||
Python 2.7 and 3.3+ compatible, but Python 2 support might be dropped at some point.
|
|
||||||
|
|
||||||
```
|
|
||||||
pip install oletools olefile exifread Pillow
|
|
||||||
pip install git+https://github.com/Rafiot/officedissector.git
|
|
||||||
pip install git+https://github.com/CIRCL/PyCIRCLean.git
|
|
||||||
```
|
|
||||||
|
|
||||||
* Create a new user and make mounting work with a read-only filesystem.
|
|
||||||
|
|
||||||
```
|
|
||||||
useradd -m kitten
|
|
||||||
chown -R kitten:kitten /home/kitten
|
|
||||||
ln -s /proc/mounts /etc/mtab
|
|
||||||
```
|
|
||||||
|
|
||||||
* Enable rc.local, which ensures that the code in /etc/rc.local is run on boot.
|
|
||||||
This is what triggers CIRCLean to run.
|
|
||||||
|
|
||||||
```
|
|
||||||
systemctl enable rc-local.service
|
|
||||||
```
|
|
||||||
|
|
||||||
* Exit the chroot again, and copy the files from your repository into the mounted
|
|
||||||
image.
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo ./copy_to_final.sh /mnt/arm_rPi/
|
|
||||||
```
|
|
||||||
|
|
||||||
Write the image on a SD card
|
|
||||||
============================
|
|
||||||
|
|
||||||
* Plug your SD card into the computer. Then, find where it is mounted using df:
|
|
||||||
|
|
||||||
```
|
|
||||||
df -h
|
|
||||||
```
|
|
||||||
|
|
||||||
* If it has been automatically mounted, unmount the SD card (use the path you
|
|
||||||
found in the previous step):
|
|
||||||
|
|
||||||
```
|
|
||||||
umount $PATH_TO_YOUR_SD
|
|
||||||
```
|
|
||||||
|
|
||||||
* Write the image to the card:
|
|
||||||
|
|
||||||
```
|
|
||||||
sudo dd bs=4M if=$PATH_TO_YOUR_IMAGE of=$PATH_TO_YOUR_SD
|
|
||||||
```
|
|
|
@ -0,0 +1 @@
|
||||||
|
dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait fastboot noswap ro consoleblank=0
|
|
@ -19,7 +19,7 @@ cdrom:x:24:pi
|
||||||
floppy:x:25:
|
floppy:x:25:
|
||||||
tape:x:26:
|
tape:x:26:
|
||||||
sudo:x:27:pi
|
sudo:x:27:pi
|
||||||
audio:x:29:pi
|
audio:x:29:pi,timidity
|
||||||
dip:x:30:
|
dip:x:30:
|
||||||
www-data:x:33:
|
www-data:x:33:
|
||||||
backup:x:34:
|
backup:x:34:
|
||||||
|
@ -52,3 +52,4 @@ indiecity:x:1001:root
|
||||||
spi:x:1002:pi
|
spi:x:1002:pi
|
||||||
gpio:x:1003:pi
|
gpio:x:1003:pi
|
||||||
kitten:x:1004:
|
kitten:x:1004:
|
||||||
|
timidity:x:110:
|
|
@ -21,18 +21,13 @@ clean(){
|
||||||
|
|
||||||
echo "GROOMER: end of boot, running rc.local."
|
echo "GROOMER: end of boot, running rc.local."
|
||||||
|
|
||||||
# Print the IP address (this doesn't work currently?)
|
|
||||||
# _IP=$(hostname -I) || true
|
|
||||||
# if [ "$_IP" ]; then
|
|
||||||
# printf "My IP address is %s\n" "$_IP"
|
|
||||||
# fi
|
|
||||||
|
|
||||||
if [ -e /dev/sda ]; then
|
if [ -e /dev/sda ]; then
|
||||||
if [ -e /dev/sdb ]; then
|
if [ -e /dev/sdb ]; then
|
||||||
# Avoid possible misuse - turn off eth0 (ethernet port)
|
# Avoid possible misuse - turn off eth0 (ethernet port)
|
||||||
/sbin/ifconfig eth0 down
|
/sbin/ifconfig eth0 down
|
||||||
trap clean EXIT TERM INT
|
trap clean EXIT TERM INT
|
||||||
cd /opt/groomer
|
cd /opt/groomer
|
||||||
|
/usr/sbin/led &
|
||||||
./init.sh
|
./init.sh
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
|
@ -1,12 +1,14 @@
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=/etc/rc.local Compatibility
|
Description=/etc/rc.local Compatibility
|
||||||
|
ConditionPathExists=/etc/rc.local
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
Type=oneshot
|
Type=forking
|
||||||
ExecStart=/etc/rc.local
|
ExecStart=/etc/rc.local start
|
||||||
TimeoutSec=0
|
TimeoutSec=0
|
||||||
StandardInput=tty
|
StandardOutput=tty
|
||||||
RemainAfterExit=yes
|
RemainAfterExit=yes
|
||||||
|
SysVStartPriority=99
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
|
@ -14,7 +14,7 @@ DST_MNT="/media/dst"
|
||||||
TEMP="${DST_MNT}/temp"
|
TEMP="${DST_MNT}/temp"
|
||||||
ZIPTEMP="${DST_MNT}/ziptemp"
|
ZIPTEMP="${DST_MNT}/ziptemp"
|
||||||
LOGS="${DST_MNT}/logs"
|
LOGS="${DST_MNT}/logs"
|
||||||
GROOM_LOG="/tmp/groom_log.txt"
|
DEBUG_LOG="/tmp/groomer_debug_log.txt"
|
||||||
MUSIC="/opt/midi/"
|
MUSIC="/opt/midi/"
|
||||||
|
|
||||||
|
|
||||||
|
@ -24,3 +24,6 @@ TIMIDITY="/usr/bin/timidity"
|
||||||
MOUNT="/bin/mount"
|
MOUNT="/bin/mount"
|
||||||
PMOUNT="/usr/bin/pmount -A -s"
|
PMOUNT="/usr/bin/pmount -A -s"
|
||||||
PUMOUNT="/usr/bin/pumount"
|
PUMOUNT="/usr/bin/pumount"
|
||||||
|
|
||||||
|
# Config flags
|
||||||
|
DEBUG=false
|
|
@ -12,7 +12,11 @@ if ! [ "${ID}" -ge "1000" ]; then
|
||||||
exit
|
exit
|
||||||
fi
|
fi
|
||||||
|
|
||||||
clean(){
|
clean(){
|
||||||
|
if [ ${DEBUG} = true ]; then
|
||||||
|
sleep 20
|
||||||
|
fi
|
||||||
|
|
||||||
# Write anything in memory to disk
|
# Write anything in memory to disk
|
||||||
${SYNC}
|
${SYNC}
|
||||||
|
|
||||||
|
@ -58,7 +62,7 @@ do
|
||||||
# Create a directory on ${DST} named PARTION_$PARTCOUNT
|
# Create a directory on ${DST} named PARTION_$PARTCOUNT
|
||||||
target_dir="/media/${DST}/FROM_PARTITION_${PARTCOUNT}"
|
target_dir="/media/${DST}/FROM_PARTITION_${PARTCOUNT}"
|
||||||
mkdir -p "${target_dir}"
|
mkdir -p "${target_dir}"
|
||||||
LOGFILE="${LOGS}/processing.txt"
|
LOGFILE="${LOGS}/processing_log.txt"
|
||||||
|
|
||||||
# Run the current partition through filecheck.py
|
# Run the current partition through filecheck.py
|
||||||
echo "==== Starting processing of /media/${SRC} to ${target_dir}. ====" >> ${LOGFILE}
|
echo "==== Starting processing of /media/${SRC} to ${target_dir}. ====" >> ${LOGFILE}
|
|
@ -13,6 +13,9 @@ if [ ${ID} -ne 0 ]; then
|
||||||
fi
|
fi
|
||||||
|
|
||||||
clean(){
|
clean(){
|
||||||
|
if [ ${DEBUG} = true ]; then
|
||||||
|
sleep 20
|
||||||
|
fi
|
||||||
echo "GROOMER: cleaning up after init.sh."
|
echo "GROOMER: cleaning up after init.sh."
|
||||||
${SYNC}
|
${SYNC}
|
||||||
# Stop the music from playing
|
# Stop the music from playing
|
||||||
|
@ -22,15 +25,13 @@ clean(){
|
||||||
|
|
||||||
trap clean EXIT TERM INT
|
trap clean EXIT TERM INT
|
||||||
|
|
||||||
# Stop hdmi display from sleeping after a period of time
|
|
||||||
setterm -powersave off -blank 0
|
|
||||||
|
|
||||||
# Start music
|
# Start music
|
||||||
./music.sh &
|
./music.sh &
|
||||||
echo $! > /tmp/music.pid
|
echo $! > /tmp/music.pid
|
||||||
|
|
||||||
# List block storage devices for debugging
|
# List block storage devices for debugging
|
||||||
# Make sure to set tee in append (-a) mode below if you uncomment
|
if [ ${DEBUG} = true ]; then
|
||||||
# lsblk |& tee ${GROOM_LOG}
|
lsblk |& tee -a ${DEBUG_LOG}
|
||||||
|
fi
|
||||||
|
|
||||||
su ${USERNAME} -c ./mount_dest.sh |& tee ${GROOM_LOG}
|
su ${USERNAME} -c ./mount_dest.sh |& tee -a ${DEBUG_LOG}
|
|
@ -13,10 +13,13 @@ if ! [ "${ID}" -ge "1000" ]; then
|
||||||
fi
|
fi
|
||||||
|
|
||||||
clean(){
|
clean(){
|
||||||
echo "GROOMER: Cleaning up in mount_keys.sh."
|
if [ ${DEBUG} = true ]; then
|
||||||
|
sleep 20
|
||||||
|
# Copy the temporary logfile to the destination key
|
||||||
|
cp ${DEBUG_LOG} "${DST_MNT}/groomer_debug_log.txt"
|
||||||
|
fi
|
||||||
|
|
||||||
# Copy the temporary logfile to the destination key
|
echo "GROOMER: Cleaning up in mount_keys.sh."
|
||||||
cp ${GROOM_LOG} "${DST_MNT}/groomer_log_dst.txt"
|
|
||||||
|
|
||||||
# Write anything in memory to disk
|
# Write anything in memory to disk
|
||||||
${SYNC}
|
${SYNC}
|
|
@ -17,5 +17,6 @@ amixer cset numid=3 1
|
||||||
files=(${MUSIC}*)
|
files=(${MUSIC}*)
|
||||||
|
|
||||||
while true; do
|
while true; do
|
||||||
$TIMIDITY ${files[RANDOM % ${#files[@]}]}
|
# -id flags set interface to "dumb" and -qq silences most/all terminal output
|
||||||
|
$TIMIDITY -idqq ${files[RANDOM % ${#files[@]}]}
|
||||||
done
|
done
|
0
fs_filecheck/usr/local/bin/pdfid.py → circlean_fs/root_partition/usr/local/bin/pdfid.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/pdfid.py → circlean_fs/root_partition/usr/local/bin/pdfid.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_embeddedfile.py → circlean_fs/root_partition/usr/local/bin/plugin_embeddedfile.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_embeddedfile.py → circlean_fs/root_partition/usr/local/bin/plugin_embeddedfile.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_list → circlean_fs/root_partition/usr/local/bin/plugin_list
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_list → circlean_fs/root_partition/usr/local/bin/plugin_list
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_nameobfuscation.py → circlean_fs/root_partition/usr/local/bin/plugin_nameobfuscation.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_nameobfuscation.py → circlean_fs/root_partition/usr/local/bin/plugin_nameobfuscation.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_triage.py → circlean_fs/root_partition/usr/local/bin/plugin_triage.py
Normal file → Executable file
0
fs_filecheck/usr/local/bin/plugin_triage.py → circlean_fs/root_partition/usr/local/bin/plugin_triage.py
Normal file → Executable file
|
@ -0,0 +1,2 @@
|
||||||
|
led: led.c
|
||||||
|
gcc -ggdb -o led led.c
|
|
@ -0,0 +1,81 @@
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <stdint.h>
|
||||||
|
#include <sys/types.h>
|
||||||
|
#include <sys/stat.h>
|
||||||
|
#include <fcntl.h>
|
||||||
|
#include <sys/mman.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
/* GPIO registers address */
|
||||||
|
#define BCM2708_PERI_BASE 0x20000000
|
||||||
|
#define GPIO_BASE (BCM2708_PERI_BASE + 0x200000) /* GPIO controller */
|
||||||
|
#define BLOCK_SIZE (256)
|
||||||
|
|
||||||
|
/* GPIO setup macros. Always use GPIO_IN(x) before using GPIO_OUT(x) or GPIO_ALT(x,y) */
|
||||||
|
#define GPIO_IN(g) *(gpio+((g)/10)) &= ~(7<<(((g)%10)*3))
|
||||||
|
#define GPIO_OUT(g) *(gpio+((g)/10)) |= (1<<(((g)%10)*3))
|
||||||
|
#define GPIO_ALT(g,a) *(gpio+(((g)/10))) |= (((a)<=3?(a)+4:(a)==4?3:2)<<(((g)%10)*3))
|
||||||
|
|
||||||
|
#define GPIO_SET(g) *(gpio+7) = 1<<(g) /* sets bit which are 1, ignores bit which are 0 */
|
||||||
|
#define GPIO_CLR(g) *(gpio+10) = 1<<(g) /* clears bit which are 1, ignores bit which are 0 */
|
||||||
|
#define GPIO_LEV(g) (*(gpio+13) >> (g)) & 0x00000001
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
#define GPIO_4 4
|
||||||
|
|
||||||
|
int mem_fd;
|
||||||
|
void *gpio_map;
|
||||||
|
volatile uint32_t *gpio;
|
||||||
|
|
||||||
|
int main(int argc, char* argv[])
|
||||||
|
{
|
||||||
|
int ret;
|
||||||
|
int i;
|
||||||
|
/* open /dev/mem */
|
||||||
|
mem_fd = open("/dev/mem", O_RDWR|O_SYNC);
|
||||||
|
if (mem_fd == -1) {
|
||||||
|
perror("Cannot open /dev/mem");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* mmap GPIO */
|
||||||
|
gpio_map = mmap(NULL, BLOCK_SIZE, PROT_READ|PROT_WRITE, MAP_SHARED, mem_fd, GPIO_BASE);
|
||||||
|
if (gpio_map == MAP_FAILED) {
|
||||||
|
perror("mmap() failed");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
/* Always use volatile pointer! */
|
||||||
|
gpio = (volatile uint32_t *)gpio_map;
|
||||||
|
|
||||||
|
|
||||||
|
GPIO_IN(GPIO_4); /* must use GPIO_IN before we can use GPIO_OUT */
|
||||||
|
GPIO_OUT(GPIO_4);
|
||||||
|
|
||||||
|
//Turn on led
|
||||||
|
while (1) {
|
||||||
|
//printf("Enable LED\n");
|
||||||
|
GPIO_SET(GPIO_4);
|
||||||
|
usleep(1000000);
|
||||||
|
//printf("Disable GPIO\n"); // Does not seem to work?
|
||||||
|
//GPIO_CLR(GPIO_4);
|
||||||
|
//usleep(1000000);
|
||||||
|
}
|
||||||
|
/* Free up ressources */
|
||||||
|
/* munmap GPIO */
|
||||||
|
ret = munmap(gpio_map, BLOCK_SIZE);
|
||||||
|
if (ret == -1) {
|
||||||
|
perror("munmap() failed");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
/* close /dev/mem */
|
||||||
|
ret = close(mem_fd);
|
||||||
|
if (ret == -1) {
|
||||||
|
perror("Cannot close /dev/mem");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
return EXIT_SUCCESS;
|
||||||
|
}
|
36
doc/TODO
36
doc/TODO
|
@ -1,36 +0,0 @@
|
||||||
TODO
|
|
||||||
====
|
|
||||||
|
|
||||||
* the script locations should be changed in the next version so they don't sit
|
|
||||||
next to the rPi's example development code that ships with the stock rPi
|
|
||||||
* the system isn't optimised and should be : cleanup and making it as close to
|
|
||||||
stock as possible
|
|
||||||
[Npot sure] Starting process should be more obfuscated
|
|
||||||
* strip exif data and leave it in a .txt file next to the image it came from
|
|
||||||
=> exiftool
|
|
||||||
[Done with remount] set filesystem of OS in RO (physical switch and/or remount OS)
|
|
||||||
[OK] mount source key in RO and noexec <= also nosuid and nodev
|
|
||||||
[OK] mount target key with noexec <= also nosuid and nodev
|
|
||||||
* convert spreadsheets in csv ?
|
|
||||||
[done in HTML] convert documents (pdfs/*office/...) in images ?
|
|
||||||
[Not Needed] Have a look at Ghostscript to work on PDFs (.pdf -> .eps -> .png?)
|
|
||||||
[do everything as user] do not run the conversions as root -> run in chroot
|
|
||||||
* take eth0 down in /etc/network/interfaces or in the groomer script disable the
|
|
||||||
interface before anything happens
|
|
||||||
* hdmi should stay up: solvable by poking the power management timer
|
|
||||||
(better not to disable the PM completely)
|
|
||||||
[Done] get rid of pdfbox. remove need for java
|
|
||||||
[WIP] scripts to generate a SD card automatically (win/mac/linux)
|
|
||||||
* move the scripts away from /opt/
|
|
||||||
* strip back libreoffice to minimum required packages. in particular, if possible,
|
|
||||||
remove libreoffice-java-common package
|
|
||||||
* Write the groomer log on the destination key
|
|
||||||
[Done] use /etc/mime.types and file -b --mime-type <filename> to find out the type of
|
|
||||||
the file
|
|
||||||
* Extract metadata from all the files => https://mat.boum.org/
|
|
||||||
|
|
||||||
HTML Files
|
|
||||||
==========
|
|
||||||
|
|
||||||
- disable JS
|
|
||||||
- cleanup external imports (js/css/images)
|
|
|
@ -0,0 +1,13 @@
|
||||||
|
TODO
|
||||||
|
====
|
||||||
|
|
||||||
|
* strip exif data and leave it in a .txt file next to the image it came from
|
||||||
|
=> exiftool
|
||||||
|
* Scripts to generate a SD card automatically (win/mac/linux)
|
||||||
|
* Extract metadata from all the files => https://mat.boum.org/
|
||||||
|
|
||||||
|
HTML Files
|
||||||
|
==========
|
||||||
|
|
||||||
|
- disable JS?
|
||||||
|
- cleanup external imports (js/css/images)
|
|
@ -0,0 +1,54 @@
|
||||||
|
* Download qemu, qemu-user-static, and proot if not already installed
|
||||||
|
* Download the newest raspbian-lite image from raspberrypi.org
|
||||||
|
* Verify the sha1 hash of the downloaded .zip file
|
||||||
|
* Unzip the image
|
||||||
|
* Expand the image by 2GB using dd
|
||||||
|
* Expand the root partition using fdisk
|
||||||
|
* Mount both partitions in loop mode using /shell_utils/basic_mount_image.sh
|
||||||
|
* Use df to find the larger partition, and resize the filesystem to fill it
|
||||||
|
* Copy circlean_fs/root_partition/etc/systemd/system/rc-local.service into the equivalent location
|
||||||
|
* Use proot to enter a chroot in the image: sudo proot -q qemu-arm -S /mnt/rpi-root -b /mnt/rpi-boot:/boot /bin/bash
|
||||||
|
* Run dpkg-reconfigure locales
|
||||||
|
* apt-get update
|
||||||
|
* apt-get dist-upgrade (might have to run this and autoremove several times)
|
||||||
|
* apt-get autoremove
|
||||||
|
* apt-get install the linux dependencies:
|
||||||
|
- timidity
|
||||||
|
- git
|
||||||
|
- p7zip-full
|
||||||
|
- pmount ntfs-3g
|
||||||
|
- python3 python3-pip
|
||||||
|
- python3-lxml
|
||||||
|
- libjpeg-dev libtiff-dev libwebp-dev liblcms2-dev tcl-dev tk-dev python-tk
|
||||||
|
* Compile p7zip-rar from source
|
||||||
|
- Change your source.list file
|
||||||
|
- Make a new directory and cd to it
|
||||||
|
- apt-get build-dep p7zip-rar
|
||||||
|
- dpkg -i <p7zip-rar .deb file path>
|
||||||
|
* Make sure the right pip executable is called by `pip3`, change your path if necessary
|
||||||
|
* Upgrade pip: pip3 install -U pip
|
||||||
|
* pip3 install python dependencies
|
||||||
|
- exifread
|
||||||
|
- pillow
|
||||||
|
- olefile
|
||||||
|
- git+https://github.com/decalage2/oletools.git
|
||||||
|
- git+https://github.com/grierforensics/officedissector.git
|
||||||
|
- git+https://github.com/CIRCL/PyCIRCLean.git
|
||||||
|
* Add a user named "kitten"
|
||||||
|
* Symlink /proc/mounts to /etc/mtab
|
||||||
|
* Turn on rc-local.service `systemctl enable rc-local.service`
|
||||||
|
- If it doesn't work, read these instructions: https://www.linuxbabe.com/linux-server/how-to-enable-etcrc-local-with-systemd
|
||||||
|
* apt-get autoclean
|
||||||
|
* apt-get autoremove
|
||||||
|
* Exit from proot
|
||||||
|
* Copy all of the project files from circlean_fs/ into the two partitions:
|
||||||
|
- rsync -vnri <source> <destination> will do a dry run of what will be copied, remove the -n to copy. See the rsync manpage for details.
|
||||||
|
- diode_controller/ if you're using the led functionality and have an external led
|
||||||
|
- midi/ files into /opt/midi/
|
||||||
|
- you might want to double check all of the permissions of the new files/directories
|
||||||
|
* Copy the image over to the SD card: sudo dd bs=4M if=<image> of=/dev/sd<letter>
|
||||||
|
- In newer versions of dd, you can add status=progress
|
||||||
|
* Optional: fsck the root partition (sudo e2fsck -f /dev/sd<letter>2).
|
||||||
|
* Test with an rpi
|
||||||
|
- FAT32 filesystem
|
||||||
|
- NTFS filesystem
|
|
@ -0,0 +1,24 @@
|
||||||
|
Modifying an already-built image
|
||||||
|
================================
|
||||||
|
One way to debug the project or test changes quickly is to modify an already built
|
||||||
|
version of the project. Once you've got an image set up on an SD card, you can mount
|
||||||
|
the image and make changes to the files directly or copy changes you've made locally
|
||||||
|
onto the mounted image. The only requirement is a linux distro such as Debian or Ubuntu.
|
||||||
|
If you're using MacOS, you can download and install VirtualBox.
|
||||||
|
|
||||||
|
Mounting an image
|
||||||
|
=================
|
||||||
|
* The steps listed in mount_image.sh are only necessary if you'd like to chroot
|
||||||
|
into and run executables from the image locally.
|
||||||
|
* To mount the image for the purpose of reading/writing to it, the process is much
|
||||||
|
* Plug the SD card into the computer.
|
||||||
|
* If you're on Virtualbox, you'll probably have to unmount the image on the host OS
|
||||||
|
(on MacOS this involves ejecting it or using diskutil unmountDisk) and then mount it
|
||||||
|
on the virtualized OS. You might have to select it under "Devices" first.
|
||||||
|
* Then, in linux, use sudo fdisk -l to find the location of the image.
|
||||||
|
* sudo mount $PATH_TO_IMAGE $PATH_TO_CHOSEN_MOUNT_POINT will mount the image.
|
||||||
|
* The path to the image will need to be the path to the partition with the OS on it,
|
||||||
|
which should be the second partition. So /dev/sdb2, not just dev/sdb.
|
||||||
|
* When you're done, sudo umount $PATH_TO_MOUNT_POINT will unmount it.
|
||||||
|
* If you get a warning about "No caching mode page found," it's safe to skip it
|
||||||
|
by pressing enter.
|
|
@ -0,0 +1,74 @@
|
||||||
|
Various qemu startup commands
|
||||||
|
=============================
|
||||||
|
|
||||||
|
From https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=37386
|
||||||
|
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-4.4.13-jessie -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1" -hda ~/qemu_vms/2016-09-23-raspbian-jessie-lite.img -redir tcp:5022::22
|
||||||
|
|
||||||
|
|
||||||
|
From https://github.com/dhruvvyas90/qemu-rpi-kernel
|
||||||
|
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-4.4.13-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ~/qemu_vms/2016-09-23-raspbian-jessie-lite.img
|
||||||
|
|
||||||
|
|
||||||
|
From http://pub.phyks.me/respawn/mypersonaldata/public/2014-05-20-11-08-01/
|
||||||
|
qemu-system-arm -kernel <<<path to kernel>>> -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw init=/bin/bash" -hda <<<path to disk image>>>
|
||||||
|
|
||||||
|
|
||||||
|
Others:
|
||||||
|
qemu-system-arm -kernel ~/qemu_vms/kernel-qemu-3.10.25-wheezy -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ~/qemu_vms/2015-02-16-raspbian-wheezy.img
|
||||||
|
|
||||||
|
qemu-system-arm -kernel qemu-rpi-kernel/kernel-qemu-3.10.25-wheezy -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda 2015-02-16-raspbian-wheezy.img
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Places to get raspbian base images:
|
||||||
|
===================================
|
||||||
|
|
||||||
|
For Raspbian Wheezy image:
|
||||||
|
wget https://downloads.raspberrypi.org/raspbian/images/raspbian-2015-02-17/2015-02-16-raspbian-wheezy.zip
|
||||||
|
|
||||||
|
For Raspbian Jessie Lite image:
|
||||||
|
wget https://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2016-09-28/2016-09-23-raspbian-jessie-lite.zip
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Traceback of the qemu failure on digitalocean
|
||||||
|
=============================================
|
||||||
|
|
||||||
|
pulseaudio: pa_context_connect() failed
|
||||||
|
pulseaudio: Reason: Connection refused
|
||||||
|
pulseaudio: Failed to initialize PA contextaudio: Could not init 'pa' audio driver
|
||||||
|
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
|
||||||
|
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
|
||||||
|
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
|
||||||
|
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
|
||||||
|
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
|
||||||
|
alsa: Could not initialize DAC
|
||||||
|
alsa: Failed to open 'default':
|
||||||
|
alsa: Reason: No such file or directory
|
||||||
|
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
|
||||||
|
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
|
||||||
|
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
|
||||||
|
ALSA lib conf.c:4259:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
|
||||||
|
ALSA lib conf.c:4738:(snd_config_expand) Evaluate error: No such file or directory
|
||||||
|
ALSA lib pcm.c:2239:(snd_pcm_open_noupdate) Unknown PCM default
|
||||||
|
alsa: Could not initialize DAC
|
||||||
|
alsa: Failed to open 'default':
|
||||||
|
alsa: Reason: No such file or directory
|
||||||
|
audio: Failed to create voice 'lm4549.out'
|
||||||
|
Could not initialize SDL(No available video device) - exiting
|
||||||
|
|
||||||
|
|
||||||
|
Notes
|
||||||
|
=====
|
||||||
|
- The error message: it is probably not a big deal - can make them not being blocking by modifying https://github.com/CIRCL/Circlean/blob/master/tests/run.exp#L10
|
||||||
|
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=760365
|
||||||
|
- Could not initialize SDL(No available video device) - exiting <= this one is blocking
|
||||||
|
- I guess it is the vnc switch - requires x11 installed
|
||||||
|
- If you use a cloud instance, you will need to get qemu to open a port you can connect to with vnc
|
||||||
|
- The good thing of having VNC is that you can see what explodes when you're running the image
|
|
@ -0,0 +1,224 @@
|
||||||
|
Building the image from scratch
|
||||||
|
===============================
|
||||||
|
|
||||||
|
There is always a prebuilt image available for download and installation as
|
||||||
|
described in the [README](../README.md). If you'd like to build the project yourself,
|
||||||
|
there are several steps involved:
|
||||||
|
|
||||||
|
* Downloading a generic Raspbian Lite image
|
||||||
|
* Resizing the image and partition
|
||||||
|
* Downloading and building the dependencies
|
||||||
|
* Modifying the image configuration
|
||||||
|
* Copying the project filesystem into the image
|
||||||
|
|
||||||
|
This procedure will only work on Ubuntu or Debian Linux. If you use MacOS or
|
||||||
|
Windows, the best option is to install Linux in a virtual machine using
|
||||||
|
something like VirtualBox.
|
||||||
|
|
||||||
|
It is recommended that you make a copy of image_setup_checklist.md and √ items off
|
||||||
|
on the list as you go.
|
||||||
|
|
||||||
|
Preparation
|
||||||
|
===========
|
||||||
|
|
||||||
|
* Make sure your development environment is up to date:
|
||||||
|
```
|
||||||
|
apt-get update
|
||||||
|
apt-get dist-upgrade
|
||||||
|
```
|
||||||
|
* Install qemu, qemu-user-static, and proot if not already installed:
|
||||||
|
```
|
||||||
|
apt-get install qemu qemu-user-static proot
|
||||||
|
```
|
||||||
|
|
||||||
|
Download the Raspbian image
|
||||||
|
==============================
|
||||||
|
|
||||||
|
* Get the most recent version of Raspbian Jessie Lite from https://downloads.raspberrypi.org/raspbian_lite/images/:
|
||||||
|
|
||||||
|
```
|
||||||
|
wget https://downloads.raspberrypi.org/raspbian_lite_latest
|
||||||
|
```
|
||||||
|
* Verify the hash of the downloaded file and compare it to the hash on the server:
|
||||||
|
```
|
||||||
|
shasum XXXX-XX-XX-raspbian-jessie-lite.zip
|
||||||
|
```
|
||||||
|
* Unpack it:
|
||||||
|
```
|
||||||
|
unzip XXXX-XX-XX-raspbian-jessie-lite.zip
|
||||||
|
```
|
||||||
|
|
||||||
|
Add space to the image
|
||||||
|
=========================
|
||||||
|
|
||||||
|
* Use dd to add 2GB (2048 blocks of 1024k each). Using /dev/zero as the input
|
||||||
|
file yields an unlimited number of "0x00" bytes.
|
||||||
|
```
|
||||||
|
> dd if=/dev/zero bs=1024k count=2048 >> XXXX-XX-XX-raspbian-jessie-lite.img
|
||||||
|
```
|
||||||
|
* Expand the root (second) partition using fdisk. The first partition listed is the boot
|
||||||
|
partition, which shouldn't be changed. In the new partition, the "First sector" should be
|
||||||
|
the value that was the "start" sector of the old root partition (137216 in the example
|
||||||
|
below, but this varies depending on the version of the Raspbian image). The "Last sector"
|
||||||
|
should be the default, and it should be significantly larger than it was before (6909951 vs.
|
||||||
|
2715647 in the example).
|
||||||
|
|
||||||
|
```
|
||||||
|
> fdisk XXXX-XX-XX-raspbian-jessie-lite.img
|
||||||
|
|
||||||
|
Command (m for help): *p*
|
||||||
|
Disk XXXX-XX-XX-raspbian-jessie-lite.img: 3.3 GiB, 3537895424 bytes, 6909952 sectors
|
||||||
|
Units: sectors of 1 * 512 = 512 bytes
|
||||||
|
Sector size (logical/physical): 512 bytes / 512 bytes
|
||||||
|
I/O size (minimum/optimal): 512 bytes / 512 bytes
|
||||||
|
Disklabel type: dos
|
||||||
|
Disk identifier: 0x244b8248
|
||||||
|
|
||||||
|
Device Boot Start End Sectors Size Id Type
|
||||||
|
XXXX-XX-XX-raspbian-jessie-lite.img1 8192 137215 129024 63M c W95 FAT32 (LBA)
|
||||||
|
XXXX-XX-XX-raspbian-jessie-lite.img2 137216 2715647 2578432 1.2G 83 Linux
|
||||||
|
|
||||||
|
Command (m for help): *d*
|
||||||
|
Partition number (1,2, default 2): *2*
|
||||||
|
|
||||||
|
Partition 2 has been deleted.
|
||||||
|
|
||||||
|
Command (m for help): *n*
|
||||||
|
Partition type
|
||||||
|
p primary (1 primary, 0 extended, 3 free)
|
||||||
|
e extended (container for logical partitions)
|
||||||
|
Select (default p):
|
||||||
|
|
||||||
|
Using default response p.
|
||||||
|
Partition number (2-4, default 2):
|
||||||
|
First sector (2048-6852607, default 2048): *137216*
|
||||||
|
Last sector, +sectors or +size{K,M,G,T,P} (131216-6909951, default 6909951):
|
||||||
|
|
||||||
|
Created a new partition 2 of type 'Linux' and of size 3.2 GiB.
|
||||||
|
|
||||||
|
Command (m for help): *w*
|
||||||
|
The partition table has been altered.
|
||||||
|
Syncing disks.
|
||||||
|
```
|
||||||
|
* Mount the image in loop mode: first, edit shell_utils/basic_mount_image.sh to use the
|
||||||
|
proper values for $BOOT_START and $ROOT_START, which you can obtain using fdisk and "p"
|
||||||
|
as in the previous step. You must also change $IMAGE to the correct path. Then run:
|
||||||
|
```
|
||||||
|
sudo ./shell_utils/basic_mount_image.md
|
||||||
|
```
|
||||||
|
* Verify the path to the mounted partitions in /dev, and resize the root (larger) filesystem
|
||||||
|
to fill the new larger partition using resize2fs:
|
||||||
|
```
|
||||||
|
> df | grep /mnt/arm
|
||||||
|
|
||||||
|
/dev/loop0 3927752 1955672 1794172 53% /mnt/rpi-root
|
||||||
|
/dev/loop1 57288 18960 38328 34% /mnt/rpi-boot
|
||||||
|
|
||||||
|
> sudo resize2fs /dev/loop0
|
||||||
|
```
|
||||||
|
|
||||||
|
Installing the dependencies
|
||||||
|
===========================
|
||||||
|
|
||||||
|
* Copy circlean_fs/root_partition/systemd/system/rc-local.service into the equivalent location in the image.
|
||||||
|
```
|
||||||
|
cp circlean_fs/root_partition/systemd/system/rc-local.service /mnt/rpi-root/etc/systemd/system/rc-local.service
|
||||||
|
```
|
||||||
|
* Use [proot](https://proot-me.github.io/) to enter the equivalent of a chroot inside
|
||||||
|
the mounted image.
|
||||||
|
```
|
||||||
|
sudo proot -q qemu-arm -S /mnt/rpi-root -b /mnt/rpi-boot:/boot /bin/bash
|
||||||
|
```
|
||||||
|
* Change your locales (remove "en_GB.UTF-8 UTF-8", add "en_US.UTF-8 UTF-8"). The
|
||||||
|
arrow keys move the cursor, spacebar selects/deselects a locale, tab moves the cursor
|
||||||
|
to a different context, and enter lets you select "ok". This step might take some time,
|
||||||
|
be patient:
|
||||||
|
```
|
||||||
|
dpkg-reconfigure locales
|
||||||
|
```
|
||||||
|
* In the image, make sure everything is up-to-date and remove old packages. You may have to
|
||||||
|
run dist-upgrade and autoremove several times for everything to be installed, and a few
|
||||||
|
raspbian-sys-mods related installs may fail - you can ignore them:
|
||||||
|
```
|
||||||
|
apt-get update
|
||||||
|
apt-get dist-upgrade
|
||||||
|
apt-get autoremove
|
||||||
|
```
|
||||||
|
* Install the linux dependencies (see CONTRIBUTING.md for more details):
|
||||||
|
```
|
||||||
|
apt-get install timidity git p7zip-full python3 python3-pip python3-lxml pmount ntfs-3g libjpeg-dev libtiff-dev libwebp-dev tk-dev python-tk liblcms2-dev tcl-dev
|
||||||
|
```
|
||||||
|
* Compile p7zip-rar from source. First, uncomment out the second line in /etc/apt/sources.list. Then:
|
||||||
|
```
|
||||||
|
cd /home/pi
|
||||||
|
mkdir rar && cd rar/
|
||||||
|
apt-get build-dep p7zip-rar
|
||||||
|
dpkg -i ${path to p7zip-rar .deb file}
|
||||||
|
```
|
||||||
|
* Install the Python dependencies for PyCIRCLean/filecheck.py. PyCIRCLean is 3.3+
|
||||||
|
compatible, so use pip -V to make sure you're using the right version of pip. You might
|
||||||
|
have to edit your PATH variable or use pip3 to get the correct pip. You also might want to
|
||||||
|
verify that these dependencies are current by checking in the PyCIRCLean git repo.
|
||||||
|
```
|
||||||
|
pip install -U pip
|
||||||
|
pip install oletools exifread Pillow
|
||||||
|
pip install git+https://github.com/decalage2/oletools.git
|
||||||
|
pip install git+https://github.com/Rafiot/officedissector.git
|
||||||
|
pip install git+https://github.com/CIRCL/PyCIRCLean.git
|
||||||
|
```
|
||||||
|
* Create a new user named "kitten":
|
||||||
|
```
|
||||||
|
useradd -m kitten
|
||||||
|
chown -R kitten:kitten /home/kitten
|
||||||
|
```
|
||||||
|
* Symlinking /proc/mounts to /etc/mtab is necessary because /etc/mtab cannot be edited by
|
||||||
|
pmount if root is read-only. /proc/mounts is maintained by the kernel and is guaranteed to
|
||||||
|
be accurate.
|
||||||
|
```
|
||||||
|
ln -s /proc/mounts /etc/mtab
|
||||||
|
```
|
||||||
|
* Enable rc.local, which ensures that the code in /etc/rc.local is run on boot.
|
||||||
|
This is what triggers CIRCLean to run.
|
||||||
|
```
|
||||||
|
systemctl enable rc-local.service
|
||||||
|
```
|
||||||
|
* Clean up:
|
||||||
|
```
|
||||||
|
apt-get clean
|
||||||
|
apt-get autoremove
|
||||||
|
apt-get autoclean
|
||||||
|
```
|
||||||
|
* Exit proot, and copy the files from your repository into the mounted
|
||||||
|
image. Adding a -n flag will make rsync do a dry run instead of copying. See the rsync
|
||||||
|
manpage for more details. Make sure to include the trailing slashes on the paths:
|
||||||
|
```
|
||||||
|
exit
|
||||||
|
sudo rsync -vri circlean_fs/boot/ /mnt/rpi-boot/
|
||||||
|
sudo rsync -vri circlean_fs/root_partition/ /mnt/rpi-root/
|
||||||
|
cp -rf midi /mnt/rpi-root/opt/
|
||||||
|
```
|
||||||
|
* If have an external hardware led and you're using the led functionality, copy
|
||||||
|
the led files from diode_controller/ as well.
|
||||||
|
|
||||||
|
Write the image on a SD card
|
||||||
|
============================
|
||||||
|
|
||||||
|
* Plug your SD card into the computer. Then, find where it is mounted using lsblk or df:
|
||||||
|
```
|
||||||
|
lsblk
|
||||||
|
df -h
|
||||||
|
```
|
||||||
|
* If it has been automatically mounted, unmount the SD card (use the path you
|
||||||
|
found in the previous step):
|
||||||
|
```
|
||||||
|
umount $PATH_TO_YOUR_SD
|
||||||
|
```
|
||||||
|
* Write the image to the card. Newer versions of dd include a status option to monitor the
|
||||||
|
copying process:
|
||||||
|
```
|
||||||
|
sudo dd bs=4M if=$PATH_TO_YOUR_IMAGE of=$PATH_TO_YOUR_SD status=progress
|
||||||
|
```
|
||||||
|
* Use fsck to verify the root partition:
|
||||||
|
```
|
||||||
|
sudo e2fsck -f /dev/sd<number>2
|
||||||
|
```
|
|
@ -16,14 +16,14 @@ Ideas
|
||||||
=====
|
=====
|
||||||
|
|
||||||
Source keys:
|
Source keys:
|
||||||
[DONE] Working documents, one / multiple partitions
|
- Working documents, one / multiple partitions
|
||||||
- Non working documents: one / multiple partitions
|
- Non working documents: one / multiple partitions
|
||||||
- different FS on different partitions
|
- different FS on different partitions
|
||||||
- Non working FS
|
- Non working FS
|
||||||
- Malicious documents (very slow, might break the conversions)
|
- Malicious documents (very slow, might break the conversions)
|
||||||
|
|
||||||
Destinations keys
|
Destinations keys
|
||||||
[DONE] empty, big enough
|
- empty, big enough
|
||||||
- empty, too small
|
- empty, too small
|
||||||
- broken
|
- broken
|
||||||
- not empty
|
- not empty
|
||||||
|
|
|
@ -0,0 +1,39 @@
|
||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
# This script will mount a given image in loop mode.
|
||||||
|
# Make sure to change the path and offsets for the image you use. You can get
|
||||||
|
# the correct offsets using `file $PATH_TO_IMAGE` or fdisk.
|
||||||
|
|
||||||
|
# To make debugging easier
|
||||||
|
echo "KittenGroomer: in mount_image.sh" 1>&2
|
||||||
|
|
||||||
|
if [ "$(id -u)" != "0" ]; then
|
||||||
|
echo "This script must be run as root" 1>&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
set -e
|
||||||
|
set -x
|
||||||
|
|
||||||
|
# Double check the path and offsets as noted above!
|
||||||
|
# Path to the image
|
||||||
|
IMAGE='2017-02-02_CIRCLean.img'
|
||||||
|
# Start sector of boot (first) partition
|
||||||
|
BOOT_START=8192
|
||||||
|
# Start sector of root (second) partition
|
||||||
|
ROOT_START=137216
|
||||||
|
# Locations you'd like the partitions mounted
|
||||||
|
BOOT_PATH='/mnt/rpi-boot'
|
||||||
|
ROOTFS_PATH='/mnt/rpi-root'
|
||||||
|
|
||||||
|
# Calculate offsets for each partition
|
||||||
|
offset_boot=$((${BOOT_START} * 512))
|
||||||
|
offset_rootfs=$((${ROOT_START} * 512))
|
||||||
|
# TODO: add logic for creating directories if they aren't already there
|
||||||
|
mkdir -p ${BOOT_PATH}
|
||||||
|
mkdir -p ${ROOTFS_PATH}
|
||||||
|
# Mount each partition in loop mode
|
||||||
|
mount -o loop,offset=${offset_boot} ${IMAGE} ${BOOT_PATH}
|
||||||
|
mount -o loop,offset=${offset_rootfs} ${IMAGE} ${ROOTFS_PATH}
|
||||||
|
|
||||||
|
echo "Image mounted" 1>&2
|
|
@ -1,12 +0,0 @@
|
||||||
#!/bin/bash
|
|
||||||
|
|
||||||
set -x
|
|
||||||
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/init.sh /mnt/rpi/opt/groomer/init.sh
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/groomer.sh /mnt/rpi/opt/groomer/groomer.sh
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/config.sh /mnt/rpi/opt/groomer/config.sh
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/mount_dest.sh /mnt/rpi/opt/groomer/mount_dest.sh
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/opt/groomer/music.sh /mnt/rpi/opt/groomer/music.sh
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/rc.local /mnt/rpi/etc/rc.local
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/pmount.allow /mnt/rpi/etc/pmount.allow
|
|
||||||
# cp /media/sf_ubuntu-shared/Circlean-Ubuntu/fs_filecheck/etc/udev/rules.d/10-usb.rules /mnt/rpi/etc/udev/rules.d/10-usb.rules
|
|
Loading…
Reference in New Issue