From 963f0a8da6d36eb52c0ddafb096c42e3ad009953 Mon Sep 17 00:00:00 2001 From: Kevin Veen-Birkenbach Date: Tue, 21 Jul 2026 18:53:17 +0200 Subject: [PATCH] docs: document Tor unlock, e2e tests, and make targets README: Tor onion unlock feature + security note, the rootless and QEMU end-to-end tests and how to run them, project-structure entries, and a correction that PyYAML>=6 is required (not stdlib-only) on the package-manager/symlink install path. Makefile: test-tor, test-qemu and test-all targets, with an overridable PYTHON interpreter. Co-Authored-By: Claude Opus 4.8 --- Makefile | 22 ++++++++++++++-- README.md | 75 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 93 insertions(+), 4 deletions(-) diff --git a/Makefile b/Makefile index 9a1ae27..b2724e2 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,9 @@ -.PHONY: install test +.PHONY: install test test-tor test-qemu test-all PREFIX ?= $(HOME)/.local +# Interpreter used for the tests. Override if your active venv lacks pytest, +# e.g. make test PYTHON=/usr/bin/python3 +PYTHON ?= python3 install: chmod +x main.py @@ -8,5 +11,20 @@ install: ln -sf $(CURDIR)/main.py $(PREFIX)/bin/lim @echo "Installed lim to $(PREFIX)/bin/lim" +# Default suite: fully mocked, no root/QEMU/network. The opt-in e2e tests skip. test: - python3 -m pytest + $(PYTHON) -m pytest + +# Rootless Tor onion unlock e2e (needs the `tor` binary and network access). +test-tor: + LIM_E2E_TOR=1 $(PYTHON) -m pytest tests/e2e/test_tor_unlock_e2e.py -v + +# Full virtualized build+boot+unlock e2e. Needs qemu, arch-install-scripts, +# cryptsetup, tor and ncat; the build stage runs under sudo. Set CHUTNEY_PATH +# to use a private, offline Tor network instead of the public one. +test-qemu: + LIM_E2E_QEMU=1 $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s + +# Everything: mocked suite plus both opt-in e2e stages, in one run. +test-all: + LIM_E2E_TOR=1 LIM_E2E_QEMU=1 $(PYTHON) -m pytest -v -s diff --git a/README.md b/README.md index fdc8589..12a3c24 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ Linux Image Manager (lim) is a Python tool for downloading, configuring, and man - **Image Download & Setup:** Automatically download, verify (checksum + GPG signature) and prepare Linux distributions. - **Encrypted Storage:** Configure LUKS encryption for secure image management. +- **Tor Onion Unlock:** Optionally bake a Tor onion service into the initramfs so the dropbear LUKS unlock shell stays reachable behind NAT or dynamic IPs. - **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy. - **Backup & Restore:** Create image backups from devices using dd. - **Chroot Environment:** Easily enter a chroot shell to maintain or modify Linux images. @@ -26,7 +27,7 @@ package-manager install lim This command makes Linux Image Manager globally available as `lim` in your terminal. The `lim` alias points to the **main.py** entry point. -There are no Python dependencies beyond the standard library. The commands call the usual system tools (`cryptsetup`, `fdisk`, `dd`, `rsync`, `wget`, `gpg`, `encfs`, `pv`, `bsdtar`, ...), so those need to be installed for the command you use. +The only Python dependency is **PyYAML** (`>=6`), used to read the image catalog. The pip/wheel install pulls it in automatically; on the `package-manager`/symlink install path make sure it is present (`python-yaml` on Arch). The commands also call the usual system tools (`cryptsetup`, `fdisk`, `dd`, `rsync`, `wget`, `gpg`, `encfs`, `pv`, `bsdtar`, ...), so those need to be installed for the command you use. ## Usage โš™๏ธ @@ -94,6 +95,8 @@ lim/ distributions.yml # single point of truth for the image catalog configuration/ # package collections used during image setup tests/unit/ # unit tests (all external commands mocked) +tests/e2e/ # Tor onion unlock end-to-end tests (rootless + QEMU, opt-in) +tests/e2e/qemu/ # full virtualized build+boot+unlock harness tests/lint/ # architecture guards (e.g. max file length) ``` @@ -102,15 +105,83 @@ tests/lint/ # architecture guards (e.g. max file length) Customize your environment in the `lim/configuration/` folder: - **General Packages:** Contains common packages for all setup scripts. - **Server LUKS Packages:** Contains packages needed for setting up LUKS encryption on servers. +- **initcpio Hooks:** The mkinitcpio install/runtime hooks and torrc baked into images for the Tor onion unlock. + +## Remote LUKS Unlock via Tor ๐Ÿง… + +When you answer yes to *"Should the system be remotely unlockable via a Tor +onion service?"* during an encrypted `lim --type image` setup, the image gets: + +- `tor` and `busybox` installed, plus a custom `tor` mkinitcpio hook ordered + between `netconf` and `dropbear`. +- A v3 onion service key generated **offline** inside the image chroot; the + resulting `.onion` address is printed at the end of the setup and stays + stable across reboots and IP changes. +- An NTP clock sync in early userspace (boards like the Raspberry Pi have no + RTC and would otherwise boot with a clock Tor rejects). Override the server + with the kernel parameter `tor_ntp=`. + +After booting the device, unlock it from any machine with a running Tor +client: + +```bash +torsocks ssh root@ +``` + +Typing the LUKS passphrase into that shell resumes the boot. The direct +`ssh root@` unlock keeps working as before; Tor is an additional path, +useful behind NAT/CGNAT where no port forwarding is possible. + +**Security note:** the onion private key is stored in the initramfs on the +unencrypted boot partition. Anyone with physical access to the SD card can +read it and impersonate the onion service โ€” treat the address as +non-secret. Access control remains the SSH public key +(`/etc/dropbear/root_key`), exactly as with the direct unlock. ## Development & Tests ๐Ÿงช -The test suite mocks all external commands, so it runs safely on any machine: +The unit suite mocks all external commands, so it runs safely on any machine: ```bash pytest ``` +### Tor unlock end-to-end test + +`tests/e2e/test_tor_unlock_e2e.py` reproduces the operator-visible half of the +Tor decryption process **rootless** โ€” no block devices, no `cryptsetup`, no +`chroot`. It generates the v3 onion keys offline (exactly as `lim.image.tor` +does inside the image), starts a real Tor onion service from a torrc mirroring +the baked-in one, then connects to the `.onion` address through Tor and +delivers a passphrase to a dropbear stand-in, asserting it arrives and the +endpoint "unlocks". + +It needs the real `tor` binary and live Tor network access (onion bootstrap is +slow), so it is opt-in and skipped otherwise: + +```bash +LIM_E2E_TOR=1 pytest tests/e2e/test_tor_unlock_e2e.py -v +``` + +The offline-only checks in that file (onion keygen, production-flag guard) run +without network. The physical flashing half (`dd`, `cryptsetup luksFormat`, +`mount`, `chroot`, `mkinitcpio`) requires root and a matching CPU/qemu setup and +is not covered by this rootless test. + +### Full virtualized unlock (QEMU) + +For the *whole* process โ€” build an encrypted image, boot it, run the real +initramfs Tor+dropbear chain, and unlock it โ€” `tests/e2e/qemu/` drives a QEMU +virtual machine end to end and asserts the real system booted only after the +passphrase was delivered over Tor. It is opt-in (`LIM_E2E_QEMU=1`) and needs +QEMU, `arch-install-scripts`, `cryptsetup`, `tor` and `ncat`; the build stage +needs root (ideally inside a throwaway VM to keep the host rootless). A private, +offline Tor network via [chutney](https://gitlab.torproject.org/tpo/core/chutney) +is supported. See [tests/e2e/qemu/README.md](tests/e2e/qemu/README.md). + +The harness's pure logic and drift guards run in the normal suite +(`test_qemu_harness_unit.py`) โ€” no QEMU, root, or network needed. + ## License ๐Ÿ“œ This project is licensed under the GNU General Public License Version 3. See the [LICENSE.txt](./LICENSE.txt) file for details.