docs: document Tor unlock, e2e tests, and make targets
README: Tor onion unlock feature + security note, the rootless and QEMU end-to-end tests and how to run them, project-structure entries, and a correction that PyYAML>=6 is required (not stdlib-only) on the package-manager/symlink install path. Makefile: test-tor, test-qemu and test-all targets, with an overridable PYTHON interpreter. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
22
Makefile
22
Makefile
@@ -1,6 +1,9 @@
|
|||||||
.PHONY: install test
|
.PHONY: install test test-tor test-qemu test-all
|
||||||
|
|
||||||
PREFIX ?= $(HOME)/.local
|
PREFIX ?= $(HOME)/.local
|
||||||
|
# Interpreter used for the tests. Override if your active venv lacks pytest,
|
||||||
|
# e.g. make test PYTHON=/usr/bin/python3
|
||||||
|
PYTHON ?= python3
|
||||||
|
|
||||||
install:
|
install:
|
||||||
chmod +x main.py
|
chmod +x main.py
|
||||||
@@ -8,5 +11,20 @@ install:
|
|||||||
ln -sf $(CURDIR)/main.py $(PREFIX)/bin/lim
|
ln -sf $(CURDIR)/main.py $(PREFIX)/bin/lim
|
||||||
@echo "Installed lim to $(PREFIX)/bin/lim"
|
@echo "Installed lim to $(PREFIX)/bin/lim"
|
||||||
|
|
||||||
|
# Default suite: fully mocked, no root/QEMU/network. The opt-in e2e tests skip.
|
||||||
test:
|
test:
|
||||||
python3 -m pytest
|
$(PYTHON) -m pytest
|
||||||
|
|
||||||
|
# Rootless Tor onion unlock e2e (needs the `tor` binary and network access).
|
||||||
|
test-tor:
|
||||||
|
LIM_E2E_TOR=1 $(PYTHON) -m pytest tests/e2e/test_tor_unlock_e2e.py -v
|
||||||
|
|
||||||
|
# Full virtualized build+boot+unlock e2e. Needs qemu, arch-install-scripts,
|
||||||
|
# cryptsetup, tor and ncat; the build stage runs under sudo. Set CHUTNEY_PATH
|
||||||
|
# to use a private, offline Tor network instead of the public one.
|
||||||
|
test-qemu:
|
||||||
|
LIM_E2E_QEMU=1 $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
|
|
||||||
|
# Everything: mocked suite plus both opt-in e2e stages, in one run.
|
||||||
|
test-all:
|
||||||
|
LIM_E2E_TOR=1 LIM_E2E_QEMU=1 $(PYTHON) -m pytest -v -s
|
||||||
|
|||||||
75
README.md
75
README.md
@@ -10,6 +10,7 @@ Linux Image Manager (lim) is a Python tool for downloading, configuring, and man
|
|||||||
|
|
||||||
- **Image Download & Setup:** Automatically download, verify (checksum + GPG signature) and prepare Linux distributions.
|
- **Image Download & Setup:** Automatically download, verify (checksum + GPG signature) and prepare Linux distributions.
|
||||||
- **Encrypted Storage:** Configure LUKS encryption for secure image management.
|
- **Encrypted Storage:** Configure LUKS encryption for secure image management.
|
||||||
|
- **Tor Onion Unlock:** Optionally bake a Tor onion service into the initramfs so the dropbear LUKS unlock shell stays reachable behind NAT or dynamic IPs.
|
||||||
- **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy.
|
- **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy.
|
||||||
- **Backup & Restore:** Create image backups from devices using dd.
|
- **Backup & Restore:** Create image backups from devices using dd.
|
||||||
- **Chroot Environment:** Easily enter a chroot shell to maintain or modify Linux images.
|
- **Chroot Environment:** Easily enter a chroot shell to maintain or modify Linux images.
|
||||||
@@ -26,7 +27,7 @@ package-manager install lim
|
|||||||
|
|
||||||
This command makes Linux Image Manager globally available as `lim` in your terminal. The `lim` alias points to the **main.py** entry point.
|
This command makes Linux Image Manager globally available as `lim` in your terminal. The `lim` alias points to the **main.py** entry point.
|
||||||
|
|
||||||
There are no Python dependencies beyond the standard library. The commands call the usual system tools (`cryptsetup`, `fdisk`, `dd`, `rsync`, `wget`, `gpg`, `encfs`, `pv`, `bsdtar`, ...), so those need to be installed for the command you use.
|
The only Python dependency is **PyYAML** (`>=6`), used to read the image catalog. The pip/wheel install pulls it in automatically; on the `package-manager`/symlink install path make sure it is present (`python-yaml` on Arch). The commands also call the usual system tools (`cryptsetup`, `fdisk`, `dd`, `rsync`, `wget`, `gpg`, `encfs`, `pv`, `bsdtar`, ...), so those need to be installed for the command you use.
|
||||||
|
|
||||||
## Usage ⚙️
|
## Usage ⚙️
|
||||||
|
|
||||||
@@ -94,6 +95,8 @@ lim/
|
|||||||
distributions.yml # single point of truth for the image catalog
|
distributions.yml # single point of truth for the image catalog
|
||||||
configuration/ # package collections used during image setup
|
configuration/ # package collections used during image setup
|
||||||
tests/unit/ # unit tests (all external commands mocked)
|
tests/unit/ # unit tests (all external commands mocked)
|
||||||
|
tests/e2e/ # Tor onion unlock end-to-end tests (rootless + QEMU, opt-in)
|
||||||
|
tests/e2e/qemu/ # full virtualized build+boot+unlock harness
|
||||||
tests/lint/ # architecture guards (e.g. max file length)
|
tests/lint/ # architecture guards (e.g. max file length)
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -102,15 +105,83 @@ tests/lint/ # architecture guards (e.g. max file length)
|
|||||||
Customize your environment in the `lim/configuration/` folder:
|
Customize your environment in the `lim/configuration/` folder:
|
||||||
- **General Packages:** Contains common packages for all setup scripts.
|
- **General Packages:** Contains common packages for all setup scripts.
|
||||||
- **Server LUKS Packages:** Contains packages needed for setting up LUKS encryption on servers.
|
- **Server LUKS Packages:** Contains packages needed for setting up LUKS encryption on servers.
|
||||||
|
- **initcpio Hooks:** The mkinitcpio install/runtime hooks and torrc baked into images for the Tor onion unlock.
|
||||||
|
|
||||||
|
## Remote LUKS Unlock via Tor 🧅
|
||||||
|
|
||||||
|
When you answer yes to *"Should the system be remotely unlockable via a Tor
|
||||||
|
onion service?"* during an encrypted `lim --type image` setup, the image gets:
|
||||||
|
|
||||||
|
- `tor` and `busybox` installed, plus a custom `tor` mkinitcpio hook ordered
|
||||||
|
between `netconf` and `dropbear`.
|
||||||
|
- A v3 onion service key generated **offline** inside the image chroot; the
|
||||||
|
resulting `.onion` address is printed at the end of the setup and stays
|
||||||
|
stable across reboots and IP changes.
|
||||||
|
- An NTP clock sync in early userspace (boards like the Raspberry Pi have no
|
||||||
|
RTC and would otherwise boot with a clock Tor rejects). Override the server
|
||||||
|
with the kernel parameter `tor_ntp=<host>`.
|
||||||
|
|
||||||
|
After booting the device, unlock it from any machine with a running Tor
|
||||||
|
client:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
torsocks ssh root@<onion-address>
|
||||||
|
```
|
||||||
|
|
||||||
|
Typing the LUKS passphrase into that shell resumes the boot. The direct
|
||||||
|
`ssh root@<ip>` unlock keeps working as before; Tor is an additional path,
|
||||||
|
useful behind NAT/CGNAT where no port forwarding is possible.
|
||||||
|
|
||||||
|
**Security note:** the onion private key is stored in the initramfs on the
|
||||||
|
unencrypted boot partition. Anyone with physical access to the SD card can
|
||||||
|
read it and impersonate the onion service — treat the address as
|
||||||
|
non-secret. Access control remains the SSH public key
|
||||||
|
(`/etc/dropbear/root_key`), exactly as with the direct unlock.
|
||||||
|
|
||||||
## Development & Tests 🧪
|
## Development & Tests 🧪
|
||||||
|
|
||||||
The test suite mocks all external commands, so it runs safely on any machine:
|
The unit suite mocks all external commands, so it runs safely on any machine:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pytest
|
pytest
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Tor unlock end-to-end test
|
||||||
|
|
||||||
|
`tests/e2e/test_tor_unlock_e2e.py` reproduces the operator-visible half of the
|
||||||
|
Tor decryption process **rootless** — no block devices, no `cryptsetup`, no
|
||||||
|
`chroot`. It generates the v3 onion keys offline (exactly as `lim.image.tor`
|
||||||
|
does inside the image), starts a real Tor onion service from a torrc mirroring
|
||||||
|
the baked-in one, then connects to the `.onion` address through Tor and
|
||||||
|
delivers a passphrase to a dropbear stand-in, asserting it arrives and the
|
||||||
|
endpoint "unlocks".
|
||||||
|
|
||||||
|
It needs the real `tor` binary and live Tor network access (onion bootstrap is
|
||||||
|
slow), so it is opt-in and skipped otherwise:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
LIM_E2E_TOR=1 pytest tests/e2e/test_tor_unlock_e2e.py -v
|
||||||
|
```
|
||||||
|
|
||||||
|
The offline-only checks in that file (onion keygen, production-flag guard) run
|
||||||
|
without network. The physical flashing half (`dd`, `cryptsetup luksFormat`,
|
||||||
|
`mount`, `chroot`, `mkinitcpio`) requires root and a matching CPU/qemu setup and
|
||||||
|
is not covered by this rootless test.
|
||||||
|
|
||||||
|
### Full virtualized unlock (QEMU)
|
||||||
|
|
||||||
|
For the *whole* process — build an encrypted image, boot it, run the real
|
||||||
|
initramfs Tor+dropbear chain, and unlock it — `tests/e2e/qemu/` drives a QEMU
|
||||||
|
virtual machine end to end and asserts the real system booted only after the
|
||||||
|
passphrase was delivered over Tor. It is opt-in (`LIM_E2E_QEMU=1`) and needs
|
||||||
|
QEMU, `arch-install-scripts`, `cryptsetup`, `tor` and `ncat`; the build stage
|
||||||
|
needs root (ideally inside a throwaway VM to keep the host rootless). A private,
|
||||||
|
offline Tor network via [chutney](https://gitlab.torproject.org/tpo/core/chutney)
|
||||||
|
is supported. See [tests/e2e/qemu/README.md](tests/e2e/qemu/README.md).
|
||||||
|
|
||||||
|
The harness's pure logic and drift guards run in the normal suite
|
||||||
|
(`test_qemu_harness_unit.py`) — no QEMU, root, or network needed.
|
||||||
|
|
||||||
## License 📜
|
## License 📜
|
||||||
|
|
||||||
This project is licensed under the GNU General Public License Version 3. See the [LICENSE.txt](./LICENSE.txt) file for details.
|
This project is licensed under the GNU General Public License Version 3. See the [LICENSE.txt](./LICENSE.txt) file for details.
|
||||||
|
|||||||
Reference in New Issue
Block a user