chg: Use udisksctl to mount source keys, support more FS types

CONTRIBUTING.md

@@ -21,7 +21,7 @@ Dependencies
 * Timidity for playing midi files
 * Git for installing some Python dependencies
 * 7Zip for unpacking archives
-* Pmount and ntfs-3g for mounting usb key partitions
+* ntfs-3g, exfat-fuse for mounting usb key partitions
 * Python 3 and pip for installing and running Python dependencies
 * Python3-lxml for handling ooxml and other Office files in filecheck.py
 * libjpeg-dev, libtiff-dev, libwebp-dev, liblcms2-dev, tcl-dev, tk-dev, and python-tk for various image formats (dependencies for pillow)

README.md

@@ -42,31 +42,31 @@ your computer directly. The official project page can be found at [https://www.c
 The Raspberry Pi Foundation has a [blog post](https://www.raspberrypi.org/blog/kittengroomercirclean-data-security-for-journalists-and-activists/) with more information
 about an older version of the project and details of the inspiration behind it.
 
-CIRCLean is currently tested to work with USB keys that have FAT32, NTFS, or
-ext2/3/4 filesystems (ext* filesystems can only be used as source keys, not destination
-keys). Currently, exFAT is not supported due to lack of support for this format in pmount.
-The vast majority of USB keys will be FAT32 or NTFS.
+CIRCLean is currently tested to work with USB keys that have FAT32, NTFS, exFAT or
+ext2/3/4 filesystems (ext\* filesystems can only be used as source keys, not destination
+keys).
+The vast majority of USB keys will be FAT32, NTFS, and exFAT.
 
 The content of the untrusted key will be copied or/and converted to the second
 (blank) key following these rules (based on the mime type as determined by libmagic):
 - Direct copy of:
-  - Plain text files (mime type: text/*)
-  - Audio files (mime type: audio/*)
-  - Video files (mime type: video/*)
-  - Example files (mime type: example/*)
-  - Multipart files (mime type: multipart/*)
+  - Plain text files (mime type: text/\*)
+  - Audio files (mime type: audio/\*)
+  - Video files (mime type: video/\*)
+  - Example files (mime type: example/\*)
+  - Multipart files (mime type: multipart/\*)
   - xml files, after being converted to text files
   - Octet-stream files
 - Copied after verification:
-  - Image files after verifying that they are not compression bombs (mime type: image/*)
+  - Image files after verifying that they are not compression bombs (mime type: image/\*)
   - PDF files, after marking as dangerous if they contain malicious content
-  - msword|vnd.openxmlformats-officedocument.*|vnd.ms-*|vnd.oasis.opendocument*, after
+  - msword|vnd.openxmlformats-officedocument.\*|vnd.ms-\*|vnd.oasis.opendocument\*, after
     parsing with oletools/olefile and marking as dangerous if the parsing fails.
-- Copied but marked as dangerous (DANGEROUS_filename_DANGEROUS)
-  - Message files (mime type: message/*)
-  - Model files (mime type: model/*)
+- Copied but marked as dangerous (DANGEROUS\_filename\_DANGEROUS)
+  - Message files (mime type: message/\*)
+  - Model files (mime type: model/\*)
   - x-dosexec (executable)
-- Compressed files (zip|x-rar|x-bzip2|x-lzip|x-lzma|x-lzop|x-xz|x-compress|x-gzip|x-tar|*compressed):
+- Compressed files (zip|x-rar|x-bzip2|x-lzip|x-lzma|x-lzop|x-xz|x-compress|x-gzip|x-tar|\*compressed):
   - Archives are unpacked, with the unpacking process stopped after 2 levels of archives
     to prevent archive bombs.
   - The above rules are applied recursively to the unpacked files.

circlean_fs/root_partition/etc/fstab

@@ -4,3 +4,4 @@ proc            /proc           proc    defaults          0       0
 tmpfs   /tmp    tmpfs  rw,size=64M,noexec,nodev,nosuid,mode=1777   0  0
 tmpfs   /media  tmpfs  rw,size=64M,noexec,nodev,nosuid,mode=1777   0  0
 # a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that
+/dev/dest_key1 /media/kitten/dest_key auto rw,user,noauto,uid=1001 0 2

circlean_fs/root_partition/etc/pmount.allow

@@ -1,5 +0,0 @@
-# /etc/pmount.allow
-# pmount will allow users to additionally mount all devices that are
-# listed here.
-/dev/sdb*
-/dev/sda*

circlean_fs/root_partition/etc/polkit-1/localauthority/50-local.d/10-automount.pkla

@@ -0,0 +1,14 @@
+[udisks1]
+Identity=unix-group:plugdev
+Action=org.freedesktop.udisks.filesystem-mount;org.freedesktop.udisks.luks-unlock;org.freedesktop.udisks.drive-eject;org.freedesktop.udisks.drive-detach
+ResultAny=yes
+
+[udisks2]
+Identity=unix-group:plugdev
+Action=org.freedesktop.udisks2.filesystem-mount;org.freedesktop.udisks2.encrypted-unlock;org.freedesktop.udisks2.eject-media;org.freedesktop.udisks2.power-off-drive
+ResultAny=yes
+
+[udisks2-other-seat]
+Identity=unix-group:plugdev
+Action=org.freedesktop.udisks2.filesystem-mount-other-seat;org.freedesktop.udisks2.filesystem-unmount-others;org.freedesktop.udisks2.encrypted-unlock-other-seat;org.freedesktop.udisks2.eject-media-other-seat;org.freedesktop.udisks2.power-off-drive-other-seat
+ResultAny=yes

circlean_fs/root_partition/etc/polkit-1/rules.d/50-udisksctl.rules

@@ -0,0 +1,25 @@
+polkit.addRule(function(action, subject) {
+  var YES = polkit.Result.YES;
+  // NOTE: there must be a comma at the end of each line except for the last:
+  var permission = {
+    // required for udisks1:
+    "org.freedesktop.udisks.filesystem-mount": YES,
+    "org.freedesktop.udisks.luks-unlock": YES,
+    "org.freedesktop.udisks.drive-eject": YES,
+    "org.freedesktop.udisks.drive-detach": YES,
+    // required for udisks2:
+    "org.freedesktop.udisks2.filesystem-mount": YES,
+    "org.freedesktop.udisks2.encrypted-unlock": YES,
+    "org.freedesktop.udisks2.eject-media": YES,
+    "org.freedesktop.udisks2.power-off-drive": YES,
+    // required for udisks2 if using udiskie from another seat (e.g. systemd):
+    "org.freedesktop.udisks2.filesystem-mount-other-seat": YES,
+    "org.freedesktop.udisks2.filesystem-unmount-others": YES,
+    "org.freedesktop.udisks2.encrypted-unlock-other-seat": YES,
+    "org.freedesktop.udisks2.eject-media-other-seat": YES,
+    "org.freedesktop.udisks2.power-off-drive-other-seat": YES
+  };
+  if (subject.isInGroup("plugdev")) {
+    return permission[action.id];
+  }
+});

circlean_fs/root_partition/etc/udev/rules.d/10-usb.rules

@@ -1,3 +1,3 @@
 # The purpose of this rules file is to ensure that the top left usb port and its partitions have a symlink to /dev/source_key[num], and the other ports to /dev/dest_key[num]
-KERNEL=="sd*", KERNELS=="1-1.2", SUBSYSTEMS=="usb", SYMLINK+="source_key%n"
-KERNEL=="sd*", KERNELS=="1-1.[3-5]", SUBSYSTEMS=="usb", SYMLINK+="dest_key%n"
+KERNEL=="sd*", KERNELS=="1-1.2", SUBSYSTEMS=="usb", SYMLINK+="source_key%n" GROUP="kitten" OWNER="kitten" MODE="0666"
+KERNEL=="sd*", KERNELS=="1-1.[3-5]", SUBSYSTEMS=="usb", SYMLINK+="dest_key%n" GROUP="kitten" OWNER="kitten" MODE="0666"

circlean_fs/root_partition/opt/groomer/config.sh

@@ -7,10 +7,9 @@ readonly ID=$(/usr/bin/id -u)
 
 # Paths used in multiple scripts
 readonly SRC_DEV="/dev/source_key"
-readonly SRC_MNT="/media/src"
 
 readonly DST_DEV="/dev/dest_key"
-readonly DST_MNT="/media/dst"
+readonly DST_MNT="/media/kitten/dest_key"
 
 readonly TEMP="${DST_MNT}/temp"
 readonly LOGS_DIR="${DST_MNT}/logs"
@@ -20,9 +19,8 @@ readonly MUSIC_DIR="/opt/midi/"
 # Commands
 readonly SYNC="/bin/sync"
 readonly TIMIDITY="/usr/bin/timidity"
-readonly MOUNT="/bin/mount"
-readonly PMOUNT="/usr/bin/pmount -A -s"
-readonly PUMOUNT="/usr/bin/pumount"
+readonly MOUNT="udisksctl mount"
+readonly UMOUNT="udisksctl unmount"
 
 # Config flags
 readonly DEBUG=false

circlean_fs/root_partition/opt/groomer/groomer.sh

@@ -27,13 +27,6 @@ check_has_partitions () {
     fi
 }
 
-unmount_source_partition() {
-    # Unmount anything that is mounted on /media/src
-    if [ "$(${MOUNT} | grep -c "${SRC_MNT}")" -ne 0 ]; then
-            ${PUMOUNT} "${SRC_MNT}"
-    fi
-}
-
 run_groomer() {
     local dev_partitions
     # Find the partition names on the device
@@ -44,15 +37,18 @@ run_groomer() {
     for partition in ${dev_partitions}
     do
         echo "GROOMER: Processing partition ${partition}"
-        unmount_source_partition
         # Mount the current partition in write mode
-        ${PMOUNT} -w ${partition} "${SRC_MNT}"
+        SRC_MNT=`${MOUNT} -o rw -b ${partition}| sed -ne 's/Mounted \(.*\) at \(\/media\/kitten\/.*\).$/\2/p'`
+        if [ -z "$SRC_MNT" ]; then
+            echo "Unable to mount source partition (${partition})."
+            continue
+        fi
         # Mark any autorun.inf files as dangerous on the source device to be extra careful
         ls "${SRC_MNT}" | grep -i autorun.inf | xargs -I {} mv "${SRC_MNT}"/{} "${SRC_MNT}"/DANGEROUS_{}_DANGEROUS || true
         # Unmount and remount the current partition in read-only mode
-        ${PUMOUNT} "${SRC_MNT}"
+        ${UMOUNT} -b "${partition}"
 
-        if ${PMOUNT} -r "${partition}" "${SRC_MNT}"; then
+        if ${MOUNT} -o ro -b "${partition}"; then
             echo "GROOMER: ${partition} mounted at ${SRC_MNT}"
 
             # Create a directory on ${DST_MNT} named PARTION_$PARTCOUNT
@@ -66,6 +62,7 @@ run_groomer() {
             if [ "${DEBUG}" = true ]; then
                 ls -lR "${target_dir}"
             fi
+            ${UMOUNT} -b "${partition}"
         else
             # Previous command (mounting current partition) failed
             echo "GROOMER: Unable to mount ${partition} on ${SRC_MNT}"

circlean_fs/root_partition/opt/groomer/mount_dest.sh

@@ -9,9 +9,8 @@ clean(){
         echo "GROOMER: Cleaning up in mount_keys.sh."
         rm -rf "${DST_MNT}/IN_PROGRESS.txt"*
         ${SYNC}  # Write anything in memory to disk
-        # Unmount source and destination
-        pumount "${SRC_MNT}"
-        pumount "${DST_MNT}"
+        # Unmount destination
+        ${UMOUNT} -b "${DST_DEV}"
         exit
 }
 
@@ -37,13 +36,13 @@ check_dest_exists() {
 }
 
 unmount_dest_if_mounted() {
-    if ${MOUNT}|grep "${DST_MNT}"; then
-        ${PUMOUNT} "${DST_MNT}" || true
+    if /bin/mount|grep "${DST_MNT}"; then
+        ${UMOUNT} -b "${DST_DEV}" || true
     fi
 }
 
 mount_dest_partition() {
-    if ${PMOUNT} -w "${DST_DEV}1" "${DST_MNT}"; then  # pmount automatically mounts on /media/ (at /media/dst in this case).
+    if ${MOUNT} -o rw -b "${DST_DEV}1"; then  # mount automatically mounts on /media/ (at /media/dst in this case).
         echo "GROOMER: Destination USB device (${DST_DEV}1) mounted at ${DST_MNT}"
     else
         echo "GROOMER: Unable to mount ${DST_DEV}1 on ${DST_MNT}"

doc/image_setup_checklist.md

@@ -16,7 +16,7 @@
     - timidity
     - git
     - p7zip-full
-    - pmount ntfs-3g
+    - ntfs-3g exfat-fuse exfat-utils
     - python3 python3-pip
     - python3-lxml
     - libjpeg-dev libtiff-dev libwebp-dev liblcms2-dev tcl-dev tk-dev python3-tk

doc/setup_with_proot.md

@@ -41,11 +41,11 @@ Download the Raspbian image
 ```
 * Verify the hash of the downloaded file and compare it to the hash on the server:
 ```
-    shasum XXXX-XX-XX-raspbian-jessie-lite.zip
+    shasum XXXX-XX-XX-raspbian-buster-lite.zip
 ```
 * Unpack it:
 ```
-    unzip XXXX-XX-XX-raspbian-jessie-lite.zip
+    unzip XXXX-XX-XX-raspbian-buster-lite.zip
 ```
 
 Add space to the image
@@ -96,6 +96,8 @@ sudo shell_utils/basic_mount_image.sh
 
 * Resize the filesystem
 
+Find the loop device of the root filesystem by running `losetup`, and it is the biggest one related to the image you mounted
+
 ```
 sudo resize2fs /dev/loop<ID of the loop FS mounted as /mnt/rpi-root>
 ```
@@ -138,9 +140,9 @@ raspbian-sys-mods related installs may fail - you can ignore them:
 from qemu about "Unsupported syscall: 384", you can ignore them. `getrandom(2)` was implemented in
 kernel 3.17 and apt will use /dev/urandom when it fails:
 ```
-    apt-get install timidity git p7zip-full python3 python3-pip pmount ntfs-3g libjpeg-dev libtiff-dev \
+    apt-get install timidity git p7zip-full python3 python3-pip ntfs-3g libjpeg-dev libtiff-dev \
                     libwebp-dev tk-dev python3-tk liblcms2-dev tcl-dev libopenjp2-7 libxml2-dev \
-                    libssl-dev libffi-dev libxslt1-dev
+                    libssl-dev libffi-dev libxslt1-dev exfat-fuse exfat-utils udisks2
 ```
 * Compile p7zip-rar from source. First, uncomment out the second line in /etc/apt/sources.list. Then:
 ```
@@ -161,7 +163,7 @@ verify that these dependencies are current by checking in the PyCIRCLean git rep
     cd /home/pi
     git clone https://github.com/CIRCL/PyCIRCLean.git
     cd PyCIRCLean
-    pip install -r requirements.txt
+    pip3 install -r requirements.txt
 ```
 * Create a new user named "kitten":
 ```

shell_utils/basic_mount_image.sh

@@ -17,7 +17,7 @@ set -x
 
 # Double check the path and offsets as noted above!
 # Path to the image
-IMAGE='2019-07-10-raspbian-buster-lite.img'
+IMAGE='2019-09-26-raspbian-buster-lite.img'
 # Start sector of boot (first) partition
 BOOT_START=`sfdisk -J ${IMAGE} | grep img1 | sed -n 's/.*"start":*\([[:digit:]]*\).*/\1/p'`
 # Amount of sectors of boot (first) partition