30 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
b3bd5b6740 Release version 2.3.0
Some checks failed
tests / lint (push) Has been cancelled
tests / pytest (push) Has been cancelled
tests / tor-network-e2e (push) Has been cancelled
tests / qemu-e2e-debian (push) Has been cancelled
2026-07-24 18:16:57 +02:00
Kevin Veen-Birkenbach
496268fa39 feat(cli): numbered selection menus for distributions and versions
Some checks failed
tests / lint (push) Has been cancelled
tests / pytest (push) Has been cancelled
tests / tor-network-e2e (push) Has been cancelled
tests / qemu-e2e-debian (push) Has been cancelled
Replace the free-text distribution/version prompts with numbered menus via a
shared _choose helper. Name-first resolution keeps typing the name working, so
numeric catalog keys (e.g. arch "4") still select by name, not by position.
Applied to the distribution, arch, manjaro, retropie and raspios choosers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:57 +02:00
Kevin Veen-Birkenbach
919651277e fix(image): keep the Tor onion reachable on RTC-less boards
An RTC-less Pi boots at 1970; Tor then rejects the consensus and the unlock
onion never publishes. Bake a clock floor (build epoch) into the initramfs and
jump the clock forward to it before Tor starts, and pass a numeric tor_ntp= in
the cmdline so busybox ntpd syncs without DNS. Proven on hardware: the onion
unlock succeeded on a Pi 3 after this.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:57 +02:00
Kevin Veen-Birkenbach
0c621f60ae fix(session): mount the boot FAT at /boot/firmware on Bookworm
Raspberry Pi OS Bookworm keeps the FAT boot partition at /boot/firmware, where
the raspi-firmware post-update hook syncs the initramfs. Binding it at /boot in
the chroot sent update-initramfs's output to the ext4 root, so the firmware
booted a stock initramfs without cryptsetup -> (initramfs) emergency shell, no
LUKS prompt. boot_bind_target() detects the layout and binds (and unmounts) the
FAT at the right place; legacy single-FAT layouts still use /boot.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:56 +02:00
Kevin Veen-Birkenbach
30ee1101ce feat(cli): guided encrypted-image wizard (default) + remote-unlock
Add a guided setup: one interactive command that asks everything up front then
builds an encrypted, Tor-remote-unlockable image unattended (distribution,
target device, hostname, login user + key, password), creating or renaming the
login user and installing the SSH key for both unlock and post-boot login.

- wizard.py: _collect (all prompts) + _execute (autonomous build); renames a
  stock pi/alarm user or creates one, grants sudo, installs the login key.
- unlock.py + `lim --type remote-unlock`: reach the initramfs over Tor (onion,
  torsocks) or plain SSH (host/IP), run cryptroot-unlock or the passphrase
  prompt; the wizard persists a target record under ~/.config/lim/unlocks.
- cli.py: register guided (default --type) and remote-unlock; drop the
  deprecated --extra argument.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:56 +02:00
Kevin Veen-Birkenbach
14094e9436 feat(image): auto-enable cross-arch chroot via qemu binfmt
Building a foreign-arch image (e.g. arm64 Raspberry Pi OS on an x86 host) needs
a qemu binfmt handler, or the chroot fails with "Exec format error". crossarch
detects this before the target is erased and, on confirmation, installs
qemu-user-static via the host package manager (apt-get/dnf/zypper/pacman) and
registers binfmt; otherwise it aborts with per-distro instructions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:56 +02:00
Kevin Veen-Birkenbach
5bef6177ac feat(image): flash full disk images into LUKS; add Raspberry Pi OS
Support distributions that ship a full .img (Raspberry Pi OS, moode, RetroPie,
Manjaro ARM) rather than a rootfs tarball, by loop-mounting the image and
copying its boot + root partitions into a fresh LUKS container.

- distributions.yml/catalog.py/choosers.py: raspios catalog (lite64/desktop64/
  lite32 via the stable _latest redirects) + choose_raspios.
- plan.py: source_url override so a _latest redirect downloads under an .img.xz
  name that decompress_command recognises.
- transfer.py: transfer_disk_image (loop-mount -> repartition -> LUKS -> rsync
  copy with progress, cp fallback -> fix boot fstab); transfer_image gains
  interactive= and routes encrypted non-arch images here; download_image gains
  force_prompt.
- loopimg.py: losetup attach/detach/partition helper.
- fsutil.drop_fstab_mount + register_encrypted_root: replace a stock image's
  existing / fstab line instead of colliding with it.
- encryption.configure_encryption returns the onion address.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:56 +02:00
Kevin Veen-Birkenbach
68ea39b784 chore(session): harden chroot package installs (PATH + apt-get update)
- chroot_bash exports a Debian-safe PATH so /usr/sbin tools (update-initramfs,
  useradd, chpasswd, ...) resolve inside the chroot instead of failing with
  code 127.
- install_packages runs apt-get update before install (a stock image ships
  stale lists whose superseded .deb URLs 404) and non-interactive
  apt-get install -y; pacman gains -Sy for the same index-refresh reason.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:56 +02:00
Kevin Veen-Birkenbach
c3a41db796 style: apply ruff format across the tree; refresh moved-module doc refs
Bring 14 files that predated the ruff-format run into line with the configured
formatter (line-length 100); provably formatting-only (ruff format of HEAD ==
working tree, and ruff format is semantics-preserving). Also refresh two
lim.image.tor._KEYGEN_SCRIPT doc references in tor_harness.py to
lim.image.initramfs.keygen after the module moved.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:55 +02:00
Kevin Veen-Birkenbach
336e68845e ci: run the Debian QEMU e2e on push; add make target and docs
- test.yml: add a qemu-e2e-debian job (build -> boot -> LUKS-unlock in QEMU
  via the deterministic direct transport) running on push/PR, plus a
  workflow_dispatch trigger for manual runs.
- Makefile: test-qemu-debian target (LIM_E2E_OS=debian); document the direct
  vs tor transport on test-qemu.
- README: document the two build scripts and the direct transport.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:55 +02:00
Kevin Veen-Birkenbach
e7712880f1 test(e2e): Debian QEMU build+boot+unlock harness and deterministic direct transport
Extend the QEMU e2e to cover the initramfs-tools backend and add a
deterministic unlock transport that avoids the flaky public-Tor onion
round-trip inside QEMU.

- build_image_debian.sh: debootstrap Bookworm, install the real
  lim/configuration/initramfs-tools hooks, LUKS + cryptsetup-initramfs +
  dropbear-initramfs, offline onion keys, boot-ok marker; same image.env
  contract as build_image.sh.
- config.py: QemuSpec gains os_family / unlock_command / direct_ssh_port;
  Debian cmdline uses root=/dev/mapper (crypttab-baked, no cryptdevice=);
  direct_ssh_port adds hostfwd to guest dropbear and a plain-SSH target.
- harness.py: unlock_transport="direct" default, _NullNet, per-OS build
  script + unlock command, up-front sudo priming with keepalive.
- boot_unlock.py: background delivery worker holds the SSH session open;
  direct vs tor target and initial delay.
- test_qemu_harness_unit.py: always-on guards for the Debian/direct branches;
  test_qemu_unlock_e2e.py parameterized by ARCH/OS/TRANSPORT env.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:55 +02:00
Kevin Veen-Birkenbach
9b7a34989d feat(image): distro-agnostic remote unlock via initramfs backends + Debian support
Split the mkinitcpio-only remote-LUKS-unlock path into an InitramfsBackend
ABC with a get_backend() dispatch, and add the initramfs-tools backend for
Debian / Raspberry Pi OS.

- base.py: six-step backend contract; encryption.py becomes a thin,
  distro-neutral sequencer (get_backend by distribution).
- initramfs_tools.py: crypttab `none luks,initramfs`, cmdline rewritten to
  root=/dev/mapper + ip=::::host:eth0:dhcp, dropbear-initramfs
  authorized_keys, update-initramfs -k all (no build-host uname leak).
- shipped hooks (configuration/initramfs-tools/*): single-hop non-anonymous
  onion, libnss DNS baking, sed-not-source DHCP, kill-tor-before-pivot.
- shared offline onion keygen in keygen.py; tor.py removed (logic moved to
  mkinitcpio.py).
- raspios added to the apt distro family (session.py, raspberry.py).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 18:15:54 +02:00
Kevin Veen-Birkenbach
2a4b606cdb Release version 2.2.0
Some checks failed
tests / lint (push) Has been cancelled
tests / pytest (push) Has been cancelled
tests / tor-network-e2e (push) Has been cancelled
2026-07-21 19:12:36 +02:00
Kevin Veen-Birkenbach
8044cdafcd ci: run the offline Tor keygen in CI, add a non-blocking network e2e job
Split the rootless Tor e2e skip so the deterministic offline checks (the
real offline onion keygen and the production-flag guard) run whenever the
tor binary is present, while only the live onion round-trip stays gated on
LIM_E2E_TOR=1. The pytest job now installs tor so the keygen — which
validates the exact production keygen path — runs on every push. A separate
continue-on-error tor-network-e2e job exercises the full round-trip without
making the public Tor network a blocking merge gate.

The full QEMU build/boot/unlock e2e still needs root + KVM + Arch tooling
and stays local/opt-in.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:12:36 +02:00
Kevin Veen-Birkenbach
c58dfaeaa8 docs: document Tor unlock, e2e tests, and make targets
README: Tor onion unlock feature + security note, the rootless and QEMU
end-to-end tests and how to run them, project-structure entries, and a
correction that PyYAML>=6 is required (not stdlib-only) on the
package-manager/symlink install path. Makefile: test-tor, test-qemu and
test-all targets, with an overridable PYTHON interpreter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:12:36 +02:00
Kevin Veen-Birkenbach
2c3b29ee98 test(e2e): full virtualized build+boot+unlock QEMU harness
Opt-in via LIM_E2E_QEMU=1. Models the whole process on a virtio VM (the
Pi's USB-gadget net can't be emulated, so it models the software stack,
not the board): builds a LUKS image carrying the real lim initcpio Tor
artifacts, boots it in QEMU rootless, lets the real netconf/tor/dropbear/
encryptssh chain publish the onion, delivers the passphrase over Tor, and
asserts the boot-ok marker on the serial console. Supports a private
offline Tor network via chutney.

The pure command builders (config.qemu_argv/kernel_cmdline/ssh_argv,
qemu_binary), the env parser, and drift guards that keep build_image.sh
aligned with the harness run in the normal suite — no QEMU/root/network.

The build stage needs root; harness.py primes sudo up front, keeps the
credential warm, and reclaims work-dir ownership on every exit path so
the pytest tmp cleanup never trips on root-owned files. QEMU stderr is
captured so an early exit is debuggable, and each teardown step is
fault-isolated so none masks the real error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:12:35 +02:00
Kevin Veen-Birkenbach
6295c09b8f test(e2e): rootless Tor onion unlock end-to-end test
Opt-in via LIM_E2E_TOR=1. Generates the v3 onion keys offline exactly as
the image build does, stands up a real Tor onion service from a torrc
mirroring the baked-in one, and delivers a passphrase through Tor to a
dropbear stand-in — asserting it arrives and the endpoint "unlocks".
Needs the real tor binary and network, so it is skipped otherwise; the
offline keygen and production-flag guard run without network.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:12:35 +02:00
Kevin Veen-Birkenbach
1e711f5c96 feat(image): remote LUKS unlock via a Tor onion service in the initramfs
Encrypted image setups can now bake a Tor onion service into the initramfs
so the dropbear unlock shell stays reachable behind NAT or a dynamic IP.
When the user opts in, configure_encryption installs tor + busybox, drops
the mkinitcpio hooks (ordered `netconf tor dropbear encryptssh`), generates
the v3 onion keys offline in the image chroot, and prints the stable
.onion address. Unlock with `torsocks ssh root@<onion-address>`.

The runtime hook syncs the clock via NTP first (RTC-less boards boot at
1970, which Tor's consensus checks reject) and starts the onion service
pointing at dropbear on 127.0.0.1:22.

Hardening baked in from an adversarial review of the shipped path:
- cmdline.txt boot path (RPi4-class firmware boot) now sets the same
  ip=::::<host>:eth0:dhcp net.ifnames=0 params as the boot.txt path, so
  the initramfs actually gets a network and the onion can publish.
- the initramfs bakes in libnss_dns.so.2 so the NTP hostname resolves.
- the hook extracts DHCP DNS with sed instead of sourcing the lease files,
  which would run attacker-controlled DHCP option strings as root pre-boot.
- NTP is attempted unconditionally (bounded), not gated on DHCP-provided DNS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:12:35 +02:00
Kevin Veen-Birkenbach
b38303b72f Release version 2.1.0
Some checks failed
tests / lint (push) Has been cancelled
tests / pytest (push) Has been cancelled
2026-07-14 12:06:25 +02:00
Kevin Veen-Birkenbach
557fe159d5 Optimized Python package building 2026-07-14 12:05:08 +02:00
Kevin Veen-Birkenbach
a8c321fca1 build: rename dist to linux-image-manager, add MIRRORS
Some checks failed
tests / lint (push) Has been cancelled
tests / pytest (push) Has been cancelled
"lim" is taken on PyPI (a GLMM genomics package); linux-image-manager
is free and matches the repo name. Import package and CLI alias stay
"lim". MIRRORS follows the hetzner-arch-luks schema.
2026-07-14 11:57:02 +02:00
Kevin Veen-Birkenbach
5cf23fcdd4 Release version 2.0.0 2026-07-14 11:42:13 +02:00
Kevin Veen-Birkenbach
a9a5340ba0 chore(claude): ask on every git commit/push variant 2026-07-14 11:37:19 +02:00
Kevin Veen-Birkenbach
ccdef065df refactor!: port shell scripts to Python package
Bash scripts were untestable and duplicated device/LUKS/mount logic;
the lim/ package centralizes it behind one subprocess wrapper and a
YAML image catalog (single point of truth).

BREAKING CHANGE: scripts/*.sh removed. Use `lim --type <cmd>`; new
types mount/umount/single-boot/raid1-boot/lock/unlock/import/export
replace direct script calls. --extra is deprecated and ignored.

- distributions.yml + lim/catalog.py hold the image catalog (PyYAML)
- pytest suite: 102 tests with mocked subprocess (tests/unit) and a
  250-line max file-length guard (tests/lint)
- ruff strict (select ALL), GitHub Actions CI, Dependabot; Travis gone
- Makefile: install (symlink ~/.local/bin/lim) and test targets
- fixes over bash: SUDO_USER-aware chown, mmcblk/nvme partition paths,
  sha512 checksum support, whole-pipeline failure detection, blkid
  UUID fallback for pre-mounted images, conditional fstab seeding for
  PARTUUID/LABEL images, clean errors for missing binaries
2026-07-14 11:37:19 +02:00
Kevin Veen-Birkenbach
c420dd164d Release version 1.0.0 2026-07-14 10:19:30 +02:00
Kevin Veen-Birkenbach
509ed85fb2 chore(claude): sandbox bash, deny git push/commit 2026-07-07 09:21:59 +02:00
Kevin Veen-Birkenbach
d89a433695 Added Manjaro GNOME 25 2025-10-17 14:03:09 +02:00
Kevin Veen-Birkenbach
e60b3cf2a7 Overall optimations 2025-03-22 11:42:13 +01:00
Kevin Veen-Birkenbach
94fdcf5758 Implemented wrapper for pkgmgr and more sync for safe image transfer and manjaro gnome 24 2025-03-22 10:28:14 +01:00
Kevin Veen-Birkenbach
b4a0b50e1f Added Funding 2025-03-12 20:52:47 +01:00
119 changed files with 7211 additions and 1412 deletions

25
.claude/settings.json Normal file
View File

@@ -0,0 +1,25 @@
{
"env": {
"GIT_CONFIG_COUNT": "1",
"GIT_CONFIG_KEY_0": "commit.gpgsign",
"GIT_CONFIG_VALUE_0": "false"
},
"permissions": {
"defaultMode": "dontAsk",
"ask": [
"Bash(git commit)",
"Bash(git commit:*)",
"Bash(git * commit)",
"Bash(git * commit:*)",
"Bash(git push)",
"Bash(git push:*)",
"Bash(git * push)",
"Bash(git * push:*)"
]
},
"sandbox": {
"enabled": true,
"autoAllowBashIfSandboxed": true,
"allowUnsandboxedCommands": false
}
}

7
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,7 @@
github: kevinveenbirkenbach
patreon: kevinveenbirkenbach
buy_me_a_coffee: kevinveenbirkenbach
custom: https://s.veen.world/paypaldonate

10
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"

55
.github/workflows/test.yml vendored Normal file
View File

@@ -0,0 +1,55 @@
name: tests
on:
push:
pull_request:
workflow_dispatch:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/ruff-action@v3
pytest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
# tor lets the deterministic offline onion-keygen test run (it validates
# the real production keygen path); that test needs no network.
- run: sudo apt-get update && sudo apt-get install -y tor
- run: pip install pytest pyyaml
- run: make test
tor-network-e2e:
# The live onion round-trip depends on the public Tor network, so it is
# informational (continue-on-error) rather than a blocking merge gate.
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: sudo apt-get update && sudo apt-get install -y tor
- run: pip install pytest pyyaml
- run: LIM_E2E_TOR=1 make test-tor PYTHON=python
qemu-e2e-debian:
# Full build -> boot -> LUKS-unlock in QEMU, deterministic via the direct
# transport (debootstrap works on Ubuntu; no Tor). Heavy (root, loop,
# cryptsetup, and TCG when the runner has no /dev/kvm).
runs-on: ubuntu-latest
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: sudo apt-get update && sudo apt-get install -y qemu-system-x86 debootstrap cryptsetup
- run: pip install pytest pyyaml
- run: make test-qemu-debian PYTHON=python

6
.gitignore vendored
View File

@@ -2,3 +2,9 @@ decrypted/
.encrypted/ .encrypted/
*package-lock.json *package-lock.json
log.txt log.txt
__pycache__/
*.pyc
.pytest_cache/
dist/
build/
*.egg-info/

View File

@@ -1,2 +0,0 @@
language: shell
script: shellcheck $(find . -type f -name '*.sh')

109
CHANGELOG.md Normal file
View File

@@ -0,0 +1,109 @@
# Changelog
## [2.3.0] - 2026-07-24
**✨ New**
- Guided setup wizard, now the default command: *lim* with no *--type* (or
*lim --type guided*) asks everything up front — distribution, target device,
hostname, login user + SSH key, password — then builds an encrypted,
Tor-remote-unlockable image unattended.
- Debian / Raspberry Pi OS remote LUKS unlock. The initramfs unlock stack is
now distro-agnostic behind an *InitramfsBackend* abstraction: *mkinitcpio*
for Arch/Manjaro, *initramfs-tools* (cryptsetup-initramfs + dropbear-initramfs
+ a Tor onion) for Raspberry Pi OS / moode / RetroPie.
- Flash full disk images into LUKS. Distributions that ship a *.img* rather
than a rootfs tarball (Raspberry Pi OS, moode, RetroPie, Manjaro ARM) are
loop-mounted and their boot + root partitions copied into a fresh LUKS
container. New Raspberry Pi OS catalog (*lite64* / *desktop64* / *lite32*).
- New *remote-unlock* command: reach a waiting initramfs over Tor (onion, via
*torsocks*) or plain SSH (host/IP), running *cryptroot-unlock* or presenting
the passphrase prompt. The wizard saves a target record under
*~/.config/lim/unlocks* so *lim --type remote-unlock* offers it directly.
- Cross-architecture builds. Building a foreign-arch image (e.g. arm64
Raspberry Pi OS on an x86 host) auto-installs *qemu-user-static* and registers
binfmt, or aborts before the target is erased with per-distro instructions.
- RTC-less clock hardening. A clock floor (build epoch baked into the initramfs)
plus a numeric *tor_ntp=* cmdline let the clock sync without DNS, so the Tor
onion still publishes on boards without a real-time clock (which boot at 1970).
- Numbered selection menus for distribution and version prompts; typing the
name still works.
- CI: a full Debian *build → boot → LUKS-unlock* QEMU end-to-end job runs on
push (*make test-qemu-debian*).
**🐛 Fixes**
- Raspberry Pi OS Bookworm boots the encrypted image: the initramfs is placed on
the */boot/firmware* FAT partition instead of the ext4 root, so cryptsetup is
present at boot (no more *(initramfs)* emergency shell).
- Chroot package installs are robust cross-host: a Debian-safe *PATH* (so
*update-initramfs* / *useradd* resolve) and *apt-get update* before install
(stock images ship stale lists whose superseded *.deb* URLs 404).
- Stock-image *fstab* is normalized (the existing */* line is replaced, not
duplicated) when copying a distro image into LUKS.
- Large copies show progress (*rsync --info=progress2*, *cp* fallback).
**💥 Breaking / behavior**
- *lim* with no *--type* now runs the guided wizard (previously *--type* was
required).
- The long-deprecated *--extra* argument is removed.
## [2.2.0] - 2026-07-21
- Optional remote LUKS unlock via a Tor onion service in the initramfs.
Encrypted image setups can bake *tor* into the initramfs (mkinitcpio hooks
ordered *netconf tor dropbear encryptssh*), generate the v3 onion keys
offline in the image chroot, and print the stable *.onion* address. Unlock
with *torsocks ssh root@<onion-address>*. Useful behind NAT/CGNAT or a
dynamic IP, where the direct *ssh root@<ip>* unlock cannot reach the host.
- End-to-end tests: a rootless one (*LIM_E2E_TOR=1*) that round-trips a
passphrase through a real Tor onion service, and a full virtualized one
(*LIM_E2E_QEMU=1*) that builds a LUKS image, boots it in QEMU, and unlocks
the root over Tor. The offline onion keygen runs in CI; *make test-tor* /
*make test-qemu* / *make test-all* run the opt-in stages.
## [2.1.0] - 2026-07-14
Initial PyPI release 🥳
## [2.0.0] - 2026-07-14
All bash scripts were ported to the *lim/* Python package with a unified CLI, a mocked test suite, and CI.
**💥 Breaking changes**
- *scripts/*.sh* removed. Use *lim --type <command>* instead.
- *--extra* is deprecated and ignored.
- Root commands re-execute themselves with sudo automatically.
**✨ New**
- New command types besides *image*, *single*, *raid1*, *backup* and
*chroot*: *mount*, *umount*, *single-boot*, *raid1-boot*, *lock*,
*unlock*, *import*, *export*.
- *distributions.yml* as single point of truth for the image catalog.
- 102 pytest tests with mocked system commands plus a max-250-lines
architecture guard. Run with *make test*.
- Strict ruff linting, GitHub Actions CI, Dependabot, *Makefile* with
*install* and *test* targets.
**🐛 Fixes over the bash version**
- Correct partition paths for *mmcblk* and *nvme* devices.
- *chown* targets the invoking user instead of root.
- sha512 checksums are verified; every pipeline stage is checked.
- fstab UUID seeding skips PARTUUID and LABEL based images.
- Pre-mounted images no longer get empty UUIDs in fstab and crypttab.
- Missing system tools abort with a clear message, not a traceback.
**📦 Dependencies**
- Python 3.10 or newer, PyYAML 6 or newer.
- System tools per command: *cryptsetup*, *fdisk*, *rsync*, *wget*,
*gpg*, *encfs*, *pv*, *bsdtar*.
## [1.0.0] - 2026-07-14
Official Release 🥳

4
MIRRORS Normal file
View File

@@ -0,0 +1,4 @@
git@github.com:kevinveenbirkenbach/linux-image-manager.git
ssh://git@code.infinito.nexus:2201/kevinveenbirkenbach/linux-image-manager.git
ssh://git@git.veen.world:2201/kevinveenbirkenbach/linux-image-manager.git
https://pypi.org/project/linux-image-manager/

37
Makefile Normal file
View File

@@ -0,0 +1,37 @@
.PHONY: install test test-tor test-qemu test-qemu-debian test-all
PREFIX ?= $(HOME)/.local
# Interpreter used for the tests. Override if your active venv lacks pytest,
# e.g. make test PYTHON=/usr/bin/python3
PYTHON ?= python3
install:
chmod +x main.py
install -d $(PREFIX)/bin
ln -sf $(CURDIR)/main.py $(PREFIX)/bin/lim
@echo "Installed lim to $(PREFIX)/bin/lim"
# Default suite: fully mocked, no root/QEMU/network. The opt-in e2e tests skip.
test:
$(PYTHON) -m pytest
# Rootless Tor onion unlock e2e (needs the `tor` binary and network access).
test-tor:
LIM_E2E_TOR=1 $(PYTHON) -m pytest tests/e2e/test_tor_unlock_e2e.py -v
# Full virtualized build+boot+unlock e2e. Needs qemu, arch-install-scripts,
# cryptsetup and ssh; the build stage runs under sudo. The default "direct"
# transport delivers the passphrase deterministically via a QEMU port-forward;
# set LIM_E2E_TRANSPORT=tor for the full onion path (also needs tor + ncat, and
# is flaky over public Tor inside QEMU).
test-qemu:
LIM_E2E_QEMU=1 $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
# Same, but builds a Debian image with debootstrap (initramfs-tools backend)
# instead of Arch. Needs debootstrap + cryptsetup + ssh; runs under sudo.
test-qemu-debian:
LIM_E2E_QEMU=1 LIM_E2E_OS=debian $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
# Everything: mocked suite plus both opt-in e2e stages, in one run.
test-all:
LIM_E2E_TOR=1 LIM_E2E_QEMU=1 $(PYTHON) -m pytest -v -s

177
README.md
View File

@@ -1,59 +1,186 @@
# Linux Image Manager 🖥️🛠️ # Linux Image Manager 🖥️🛠️
[![GitHub Sponsors](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-blue?logo=github)](https://github.com/sponsors/kevinveenbirkenbach) [![Patreon](https://img.shields.io/badge/Support-Patreon-orange?logo=patreon)](https://www.patreon.com/c/kevinveenbirkenbach) [![Buy Me a Coffee](https://img.shields.io/badge/Buy%20me%20a%20Coffee-Funding-yellow?logo=buymeacoffee)](https://buymeacoffee.com/kevinveenbirkenbach) [![PayPal](https://img.shields.io/badge/Donate-PayPal-blue?logo=paypal)](https://s.veen.world/paypaldonate)
[![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](./LICENSE.txt) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/linux-image-manager.svg?style=social)](https://github.com/kevinveenbirkenbach/linux-image-manager/stargazers) [![License: GPL v3](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](./LICENSE.txt) [![GitHub stars](https://img.shields.io/github/stars/kevinveenbirkenbach/linux-image-manager.svg?style=social)](https://github.com/kevinveenbirkenbach/linux-image-manager/stargazers)
Linux Image Manager(lim) is a powerful collection of shell scripts for downloading, configuring, and managing Linux images. Whether you're setting up encrypted storage, configuring a virtual Btrfs RAID1, performing backups, or chrooting into an image, this tool makes Linux image administration simple and efficient. 🚀 Linux Image Manager (lim) is a Python tool for downloading, configuring, and managing Linux images. Whether you're setting up encrypted storage, configuring a virtual Btrfs RAID1, performing backups, or chrooting into an image, this tool makes Linux image administration simple and efficient. 🚀
## Features ✨ ## Features ✨
- **Image Download & Setup:** Automatically download and prepare Linux distributions. - **Image Download & Setup:** Automatically download, verify (checksum + GPG signature) and prepare Linux distributions.
- **Encrypted Storage:** Configure LUKS encryption for secure image management. - **Encrypted Storage:** Configure LUKS encryption for secure image management.
- **Tor Onion Unlock:** Optionally bake a Tor onion service into the initramfs so the dropbear LUKS unlock shell stays reachable behind NAT or dynamic IPs.
- **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy. - **Virtual RAID1:** Easily set up virtual Btrfs RAID1 for data redundancy.
- **Chroot Environment:** Seamlessly chroot into your Linux image for system maintenance. - **Backup & Restore:** Create image backups from devices using dd.
- **Backup & Restore:** Comprehensive backup and restore options for system images. - **Chroot Environment:** Easily enter a chroot shell to maintain or modify Linux images.
- **Data Import/Export:** Sync personal data into an encfs-encrypted store and back.
- **Automated Procedures:** Simplify partitioning, formatting, mounting, and more. - **Automated Procedures:** Simplify partitioning, formatting, mounting, and more.
## Installation 📦 ## Installation 📦
Install Linux Image Manager quickly with [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager) under the alias `lim`. Just run: Install Linux Image Manager quickly using [Kevin's Package Manager](https://github.com/kevinveenbirkenbach/package-manager) under the alias `lim`. Just run:
```bash ```bash
package-manager install lim package-manager install lim
``` ```
This command makes Linux Image Manager globally available as `lim` in your terminal. 🔧 This command makes Linux Image Manager globally available as `lim` in your terminal. The `lim` alias points to the **main.py** entry point.
The only Python dependency is **PyYAML** (`>=6`), used to read the image catalog. The pip/wheel install pulls it in automatically; on the `package-manager`/symlink install path make sure it is present (`python-yaml` on Arch). The commands also call the usual system tools (`cryptsetup`, `fdisk`, `dd`, `rsync`, `wget`, `gpg`, `encfs`, `pv`, `bsdtar`, ...), so those need to be installed for the command you use.
## Usage ⚙️ ## Usage ⚙️
Linux Image Manager comes with a variety of scripts tailored for different tasks. Here are a few examples: `lim` provides a unified interface for all image and storage operations. Commands that need root privileges re-execute themselves with `sudo` automatically. The built-in `--help` option displays detailed usage information.
### Available Command Types
| `--type` | Description |
|---------------|-----------------------------------------------------------------------------|
| `image` | Download, verify and transfer a Linux image to a device, incl. optional LUKS encryption and Raspberry Pi configuration. |
| `single` | Set up LUKS encryption with Btrfs on a single drive. |
| `raid1` | Set up an encrypted virtual Btrfs RAID1 across two drives. |
| `backup` | Create an image backup from a memory device using dd. |
| `chroot` | Mount an image and open a shell inside it. |
| `mount` | Unlock and mount an encrypted drive. |
| `umount` | Unmount an encrypted drive and close the mapper. |
| `single-boot` | Register a single encrypted drive for automount on boot (keyfile, crypttab, fstab). |
| `raid1-boot` | Register an encrypted RAID1 for automount on boot. |
| `unlock` | Decrypt the encfs data store. |
| `lock` | Lock the encfs data store. |
| `import` | Import personal data from the system into the encrypted store. |
| `export` | Export personal data from the encrypted store back to the system. |
### Command-Line Options
- **`--type`** *(required)*: Choose the command to execute (see table above).
- **`--auto-confirm`** *(optional)*: Bypass the confirmation prompt before execution.
- **`--help`** *(optional)*: Display detailed help information.
### Example Commands
### Virtual Btrfs RAID1 Setup
```bash ```bash
lim raid1/setup.sh # Display help
lim --help
# Execute the Linux image setup
lim --type image
# Run the single drive encryption setup without a confirmation prompt
lim --type single --auto-confirm
# Set up an encrypted RAID1
lim --type raid1
# Back up a memory device to an image file
lim --type backup
# Enter a chroot environment for a Linux image
lim --type chroot
``` ```
### Linux Image Setup ## Project Structure 🗂️
```bash
lim image/setup.sh
```
### Chroot into Linux Image
```bash
lim image/chroot.sh
``` ```
main.py # entry point (the `lim` alias)
### Backup Image lim/
```bash cli.py # argument parsing and command dispatch
lim image/backup.sh catalog.py # read-only access to distributions.yml
device.py # block device selection, dd, blkid helpers
luks.py # LUKS keyfiles, crypttab/fstab bookkeeping
runner.py # subprocess wrapper used by all modules
storage/ # single drive and RAID1 encryption setups
image/ # image setup, backup, chroot, verification
data/ # encfs lock/unlock and data import/export
distributions.yml # single point of truth for the image catalog
configuration/ # package collections used during image setup
tests/unit/ # unit tests (all external commands mocked)
tests/e2e/ # Tor onion unlock end-to-end tests (rootless + QEMU, opt-in)
tests/e2e/qemu/ # full virtualized build+boot+unlock harness
tests/lint/ # architecture guards (e.g. max file length)
``` ```
Explore the `scripts/` directory for more functionalities and detailed usage instructions.
## Configuration & Customization 🔧 ## Configuration & Customization 🔧
Customize your environment in the `configuration/` folder: Customize your environment in the `lim/configuration/` folder:
- **General Packages:** Common packages for all setup scripts. - **General Packages:** Contains common packages for all setup scripts.
- **Server LUKS Packages:** Packages needed for setting up LUKS encryption on servers. - **Server LUKS Packages:** Contains packages needed for setting up LUKS encryption on servers.
- **initcpio Hooks:** The mkinitcpio install/runtime hooks and torrc baked into images for the Tor onion unlock.
## Remote LUKS Unlock via Tor 🧅
When you answer yes to *"Should the system be remotely unlockable via a Tor
onion service?"* during an encrypted `lim --type image` setup, the image gets:
- `tor` and `busybox` installed, plus a custom `tor` mkinitcpio hook ordered
between `netconf` and `dropbear`.
- A v3 onion service key generated **offline** inside the image chroot; the
resulting `.onion` address is printed at the end of the setup and stays
stable across reboots and IP changes.
- An NTP clock sync in early userspace (boards like the Raspberry Pi have no
RTC and would otherwise boot with a clock Tor rejects). Override the server
with the kernel parameter `tor_ntp=<host>`.
After booting the device, unlock it from any machine with a running Tor
client:
```bash
torsocks ssh root@<onion-address>
```
Typing the LUKS passphrase into that shell resumes the boot. The direct
`ssh root@<ip>` unlock keeps working as before; Tor is an additional path,
useful behind NAT/CGNAT where no port forwarding is possible.
**Security note:** the onion private key is stored in the initramfs on the
unencrypted boot partition. Anyone with physical access to the SD card can
read it and impersonate the onion service — treat the address as
non-secret. Access control remains the SSH public key
(`/etc/dropbear/root_key`), exactly as with the direct unlock.
## Development & Tests 🧪
The unit suite mocks all external commands, so it runs safely on any machine:
```bash
pytest
```
### Tor unlock end-to-end test
`tests/e2e/test_tor_unlock_e2e.py` reproduces the operator-visible half of the
Tor decryption process **rootless** — no block devices, no `cryptsetup`, no
`chroot`. It generates the v3 onion keys offline (exactly as `lim.image.tor`
does inside the image), starts a real Tor onion service from a torrc mirroring
the baked-in one, then connects to the `.onion` address through Tor and
delivers a passphrase to a dropbear stand-in, asserting it arrives and the
endpoint "unlocks".
It needs the real `tor` binary and live Tor network access (onion bootstrap is
slow), so it is opt-in and skipped otherwise:
```bash
LIM_E2E_TOR=1 pytest tests/e2e/test_tor_unlock_e2e.py -v
```
The offline-only checks in that file (onion keygen, production-flag guard) run
without network. The physical flashing half (`dd`, `cryptsetup luksFormat`,
`mount`, `chroot`, `mkinitcpio`) requires root and a matching CPU/qemu setup and
is not covered by this rootless test.
### Full virtualized unlock (QEMU)
For the *whole* process — build an encrypted image, boot it, run the real
initramfs Tor+dropbear chain, and unlock it — `tests/e2e/qemu/` drives a QEMU
virtual machine end to end and asserts the real system booted only after the
passphrase was delivered over Tor. It is opt-in (`LIM_E2E_QEMU=1`) and needs
QEMU, `arch-install-scripts`, `cryptsetup`, `tor` and `ncat`; the build stage
needs root (ideally inside a throwaway VM to keep the host rootless). A private,
offline Tor network via [chutney](https://gitlab.torproject.org/tpo/core/chutney)
is supported. See [tests/e2e/qemu/README.md](tests/e2e/qemu/README.md).
The harness's pure logic and drift guards run in the normal suite
(`test_qemu_harness_unit.py`) — no QEMU, root, or network needed.
## License 📜 ## License 📜

3
lim/__init__.py Normal file
View File

@@ -0,0 +1,3 @@
"""Linux Image Manager — administration tool for Linux images and encrypted storage."""
__version__ = "1.0.0"

35
lim/catalog.py Normal file
View File

@@ -0,0 +1,35 @@
"""Read-only access to the image catalog (distributions.yml in the package)."""
from functools import cache
import yaml
from lim import config
CATALOG_PATH = config.CATALOG_PATH
@cache
def _load() -> dict:
with CATALOG_PATH.open() as handle:
return yaml.safe_load(handle)
def arch_rpi_images() -> dict[str, dict[str, str]]:
return _load()["arch_rpi_images"]
def manjaro_gnome_releases() -> dict[str, dict[str, str]]:
return _load()["manjaro_gnome_releases"]
def retropie_images() -> dict[str, dict[str, str]]:
return _load()["retropie_images"]
def raspios_images() -> dict[str, dict[str, str]]:
return _load()["raspios_images"]
def mkinitcpio_modules_by_rpi() -> dict[str, str]:
return _load()["mkinitcpio_modules_by_rpi"]

169
lim/cli.py Normal file
View File

@@ -0,0 +1,169 @@
"""Command line interface: `lim --type <command>`."""
import argparse
import sys
from collections.abc import Callable
from dataclasses import dataclass
from lim import system, ui
from lim.data import crypt, sync
from lim.errors import LimError
from lim.image import backup, chroot
from lim.image import setup as image_setup
from lim.image import unlock as image_unlock
from lim.image import wizard as image_wizard
from lim.storage import raid1, single_drive
@dataclass(frozen=True)
class Command:
func: Callable[[], None]
description: str
needs_root: bool
COMMANDS: dict[str, Command] = {
"image": Command(
image_setup.run_setup,
"Linux Image Setup:\n"
" - Creates partitions and formats them.\n"
" - Transfers the Linux image file to the device.\n"
" - Configures boot and root partitions.",
needs_root=True,
),
"guided": Command(
image_wizard.run_guided,
"Guided Encrypted Linux Image Setup (default command):\n"
" - Walks through flashing a Linux image (Arch, Raspberry Pi OS, ...) onto a device.\n"
" - Encrypts the root with LUKS and enables remote unlock over Tor.",
needs_root=True,
),
"remote-unlock": Command(
image_unlock.remote_unlock,
"Remote LUKS Boot Unlock:\n"
" - Reaches a waiting initramfs over Tor (onion) or plain SSH.\n"
" - Runs cryptroot-unlock or presents the passphrase prompt.",
needs_root=False,
),
"single": Command(
single_drive.setup,
"Single Drive Encryption Setup:\n"
" - Sets up disk encryption using LUKS on one drive.\n"
" - Configures a Btrfs file system for secure storage.",
needs_root=True,
),
"raid1": Command(
raid1.setup,
"RAID1 Encryption Setup:\n"
" - Configures a virtual RAID1 with two drives.\n"
" - Uses LUKS encryption and a Btrfs RAID1 file system for redundancy.",
needs_root=True,
),
"backup": Command(
backup.run_backup,
"Backup Image Setup:\n"
" - Creates an image backup from a memory device to a file.\n"
" - Uses dd to transfer the image from the specified device to an image file.",
needs_root=True,
),
"chroot": Command(
chroot.run_chroot,
"Chroot Environment Setup:\n"
" - Mounts partitions and configures the chroot environment for a Linux image.\n"
" - Provides a shell within the Linux image for system maintenance.",
needs_root=True,
),
"mount": Command(
single_drive.mount,
"Mount Encrypted Storage:\n - Unlocks a LUKS partition and mounts it.",
needs_root=True,
),
"umount": Command(
single_drive.umount,
"Unmount Encrypted Storage:\n - Unmounts a LUKS partition and closes the mapper.",
needs_root=True,
),
"single-boot": Command(
single_drive.mount_on_boot,
"Single Drive Automount Setup:\n"
" - Creates a LUKS keyfile and registers it in crypttab and fstab.",
needs_root=True,
),
"raid1-boot": Command(
raid1.mount_on_boot,
"RAID1 Automount Setup:\n"
" - Creates LUKS keyfiles for both drives and registers them in crypttab/fstab.",
needs_root=True,
),
"unlock": Command(
crypt.unlock,
"Unlock Data:\n - Decrypts the encfs data store into the decrypted folder.",
needs_root=False,
),
"lock": Command(
crypt.lock,
"Lock Data:\n - Unmounts the decrypted encfs folder.",
needs_root=False,
),
"import": Command(
sync.import_from_system,
"Import Data:\n - Copies personal data from the system into the encrypted store.",
needs_root=False,
),
"export": Command(
sync.export_to_system,
"Export Data:\n - Copies personal data from the encrypted store back to the system.",
needs_root=False,
),
}
def build_parser() -> argparse.ArgumentParser:
epilog_lines = ["Available script types:"]
epilog_lines += [
f" {name:<12} - {command.description.splitlines()[0].rstrip(':')}"
for name, command in COMMANDS.items()
]
parser = argparse.ArgumentParser(
prog="lim",
description="Linux Image Manager — manages Linux images and encrypted storage.",
epilog="\n".join(epilog_lines),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--type",
default="guided",
choices=list(COMMANDS.keys()),
help="Command to execute (default: guided).",
)
parser.add_argument(
"--auto-confirm",
action="store_true",
help="Automatically confirm execution without prompting the user.",
)
return parser
def main(argv: list[str] | None = None) -> None:
args = build_parser().parse_args(argv)
command = COMMANDS[args.type]
if command.needs_root:
system.ensure_root()
ui.header()
ui.info(f"Selected command: {args.type}")
ui.info("Description:")
print(command.description)
try:
if not args.auto_confirm:
input("Press Enter to execute or Ctrl+C to cancel...")
command.func()
except LimError as exc:
ui.error(f"{exc} -> Leaving program.")
sys.exit(1)
except KeyboardInterrupt:
print()
ui.error("Execution aborted by user.")
sys.exit(1)

18
lim/config.py Normal file
View File

@@ -0,0 +1,18 @@
"""Package and repository paths shared by all modules.
Configuration and the image catalog ship inside the package so that
wheel installs work; the encfs data store stays repository-relative.
"""
from pathlib import Path
PACKAGE_ROOT = Path(__file__).resolve().parent
CONFIGURATION_PATH = PACKAGE_ROOT / "configuration"
PACKAGE_PATH = CONFIGURATION_PATH / "packages"
CATALOG_PATH = PACKAGE_ROOT / "distributions.yml"
REPOSITORY_PATH = PACKAGE_ROOT.parent
ENCRYPTED_PATH = REPOSITORY_PATH / ".encrypted"
DECRYPTED_PATH = REPOSITORY_PATH / "decrypted"
DATA_PATH = DECRYPTED_PATH / "data"
BACKUP_PATH = DECRYPTED_PATH / "backup"

View File

@@ -0,0 +1,47 @@
#!/usr/bin/ash
# mkinitcpio runtime hook: start Tor so the dropbear unlock shell is
# reachable as an onion service while encryptssh waits for the passphrase.
# Installed to /etc/initcpio/hooks/tor by linux-image-manager.
# netconf's ipconfig drops its DHCP lease data (incl. DNS) into
# /tmp/net-*.conf; busybox's resolver only reads /etc/resolv.conf.
_tor_write_resolv_conf() {
[ -s /etc/resolv.conf ] && return 0
local conf dns
for conf in /tmp/net-*.conf; do
[ -f "$conf" ] || continue
# Extract ONLY the DNS fields with sed; never source these files —
# they hold attacker-controllable DHCP option strings (hostname,
# domain, rootpath), and sourcing would run them as root pre-boot.
for dns in $(sed -n 's/^IPV4DNS[01]=//p' "$conf"); do
[ -n "$dns" ] && [ "$dns" != "0.0.0.0" ] \
&& echo "nameserver $dns" >> /etc/resolv.conf
done
done
[ -s /etc/resolv.conf ]
}
run_hook() {
# Tor rejects consensus documents when the clock is far off; boards
# without an RTC boot in 1970, so sync before starting Tor. Bounded:
# a failed sync must never block the boot.
msg "tor: syncing clock via NTP..."
# Best-effort DNS; a tor_ntp IP literal needs none, so never gate on it.
_tor_write_resolv_conf || msg "tor: no DNS from DHCP (fine if tor_ntp is an IP)"
/usr/local/bin/busybox timeout 30 \
/usr/local/bin/busybox ntpd -n -q -p "${tor_ntp:-pool.ntp.org}" \
|| msg "tor: NTP sync failed, keeping current clock"
msg "tor: starting onion service for remote unlock..."
mkdir -p /var/lib/tor
chmod 0700 /var/lib/tor /etc/tor/onion
chmod 0600 /etc/tor/onion/hs_ed25519_secret_key
tor -f /etc/tor/torrc --RunAsDaemon 1 --Log "notice file /tmp/tor.log" \
|| msg "tor: failed to start, unlock stays reachable via direct IP"
}
run_cleanuphook() {
# Nothing from the initramfs may keep running after the pivot.
/usr/local/bin/busybox killall tor 2>/dev/null
return 0
}

View File

@@ -0,0 +1,37 @@
#!/bin/bash
# mkinitcpio install hook: Tor onion service for remote LUKS unlock.
# Installed to /etc/initcpio/install/tor by linux-image-manager.
build() {
add_binary /usr/bin/tor
# Full busybox for the ntpd applet: boards without an RTC (e.g. most
# Raspberry Pis) wake up in 1970, which Tor's consensus checks reject.
# A different target path keeps mkinitcpio's own busybox untouched.
add_binary /usr/bin/busybox /usr/local/bin/busybox
# glibc resolves the NTP server hostname via getaddrinfo(), which dlopen()s
# these NSS modules at runtime — add_binary only follows NEEDED libs, so
# without them DNS silently fails, ntpd never syncs, and the clock stays
# at 1970. glibc's built-in default (no nsswitch.conf) is "dns ... files".
add_binary /usr/lib/libnss_dns.so.2
add_binary /usr/lib/libnss_files.so.2
add_file /etc/tor/initramfs-torrc /etc/tor/torrc
add_file /etc/tor/initramfs-onion/hostname /etc/tor/onion/hostname
add_file /etc/tor/initramfs-onion/hs_ed25519_public_key /etc/tor/onion/hs_ed25519_public_key
add_file /etc/tor/initramfs-onion/hs_ed25519_secret_key /etc/tor/onion/hs_ed25519_secret_key
add_runscript
}
help() {
cat <<HELPEOF
Starts a Tor onion service in early userspace so the dropbear unlock
shell stays reachable under the .onion address baked into the image,
even behind NAT or a dynamic IP.
Place it between netconf and dropbear:
HOOKS=(... netconf tor dropbear encryptssh ...)
Optional kernel parameter:
tor_ntp=<server> NTP server used to set the clock before Tor starts
(default: pool.ntp.org)
HELPEOF
}

View File

@@ -0,0 +1,7 @@
# Tor configuration for the initramfs onion unlock service.
# Baked into the initramfs as /etc/tor/torrc by the "tor" mkinitcpio hook;
# the onion keys come from /etc/tor/initramfs-onion on the system.
DataDirectory /var/lib/tor
HiddenServiceDir /etc/tor/onion
HiddenServicePort 22 127.0.0.1:22
SocksPort 0

View File

@@ -0,0 +1,19 @@
#!/bin/sh
# initramfs-tools cleanup hook: stop Tor before the pivot to the real root.
# Installed to /etc/initramfs-tools/scripts/init-bottom/tor by lim.
# Nothing from the initramfs may keep running once run-init replaces it.
PREREQ=""
prereqs() { echo "$PREREQ"; }
case "$1" in
prereqs) prereqs; exit 0 ;;
esac
# pidof/killall may be absent; scan /proc for the tor binary (no extra tools).
for pid in $(ls /proc 2>/dev/null); do
case "$pid" in
*[!0-9]*) continue ;;
esac
[ "$(readlink "/proc/$pid/exe" 2>/dev/null)" = "/usr/bin/tor" ] \
&& kill "$pid" 2>/dev/null
done
exit 0

View File

@@ -0,0 +1,37 @@
#!/bin/sh
# initramfs-tools build hook: bake the Tor onion service into the initramfs.
# Installed to /etc/initramfs-tools/hooks/tor by linux-image-manager.
PREREQ="dropbear"
prereqs() { echo "$PREREQ"; }
case "$1" in
prereqs) prereqs; exit 0 ;;
esac
. /usr/share/initramfs-tools/hook-functions
copy_exec /usr/bin/tor /usr/bin
# glibc resolves the NTP server hostname via getaddrinfo(), which dlopen()s
# these NSS modules at runtime; without them DNS silently fails and the clock
# (RTC-less boards) stays at 1970, so Tor rejects the consensus.
for nss in /usr/lib/*/libnss_dns.so.2 /usr/lib/*/libnss_files.so.2 \
/lib/*/libnss_dns.so.2 /lib/*/libnss_files.so.2; do
[ -e "$nss" ] && copy_exec "$nss"
done
# Full busybox for the ntpd applet (the initramfs busybox may lack it); a
# distinct path keeps the initramfs's own busybox untouched.
for bb in /bin/busybox /usr/bin/busybox; do
[ -x "$bb" ] && { copy_exec "$bb" /usr/local/bin/busybox; break; }
done
date -u +%s > "${DESTDIR}/etc/tor-clock-floor"
# Onion keys + torrc, staged on the system by lim.
mkdir -p "${DESTDIR}/etc/tor/onion"
for key in hostname hs_ed25519_public_key hs_ed25519_secret_key; do
cp -a "/etc/tor/initramfs-onion/${key}" "${DESTDIR}/etc/tor/onion/${key}"
done
chmod 0700 "${DESTDIR}/etc/tor/onion"
chmod 0600 "${DESTDIR}/etc/tor/onion/hs_ed25519_secret_key"
cp -a /etc/tor/initramfs-torrc "${DESTDIR}/etc/tor/torrc"

View File

@@ -0,0 +1,55 @@
#!/bin/sh
# initramfs-tools runtime hook: bring up networking, sync the clock and start
# the Tor onion service so the dropbear unlock shell is reachable while
# cryptsetup waits. Installed to /etc/initramfs-tools/scripts/init-premount/tor.
PREREQ=""
prereqs() { echo "$PREREQ"; }
case "$1" in
prereqs) prereqs; exit 0 ;;
esac
. /scripts/functions
log_begin_msg "tor: bringing up networking and starting the onion service"
# init-premount runs BEFORE the cryptroot/dropbear networking, so Tor would
# start with no network and never publish the onion. Bring the interface up
# ourselves from the ip= cmdline (idempotent; the later dropbear setup reuses
# the lease in /run/net-*.conf).
configure_networking
# initramfs-tools does not export arbitrary cmdline params as shell vars.
tor_ntp=$(sed -n 's/.*\btor_ntp=\([^ ]*\).*/\1/p' /proc/cmdline)
# Best-effort DNS from the DHCP lease; extract ONLY the DNS fields with sed,
# never source the files (attacker-controllable DHCP option strings would run
# as root pre-boot).
if [ ! -s /etc/resolv.conf ]; then
for conf in /run/net-*.conf /tmp/net-*.conf; do
[ -f "$conf" ] || continue
for dns in $(sed -n 's/^IPV4DNS[01]=//p' "$conf"); do
[ -n "$dns" ] && [ "$dns" != "0.0.0.0" ] \
&& echo "nameserver $dns" >> /etc/resolv.conf
done
done
fi
floor=$(cat /etc/tor-clock-floor 2>/dev/null || echo 0)
if [ "$(date +%s)" -lt "$floor" ]; then
/usr/local/bin/busybox date -s "@$floor" >/dev/null 2>&1 \
|| date -s "@$floor" >/dev/null 2>&1 || true
fi
if [ -x /usr/local/bin/busybox ]; then
/usr/local/bin/busybox timeout 30 \
/usr/local/bin/busybox ntpd -n -q -p "${tor_ntp:-pool.ntp.org}" \
|| log_warning_msg "tor: NTP sync failed, keeping current clock"
fi
mkdir -p /var/lib/tor
chmod 0700 /var/lib/tor /etc/tor/onion
chmod 0600 /etc/tor/onion/hs_ed25519_secret_key
tor -f /etc/tor/torrc --RunAsDaemon 1 --Log "notice file /run/tor.log" \
|| log_warning_msg "tor: failed to start, unlock stays reachable via direct IP"
log_end_msg

View File

@@ -0,0 +1,16 @@
# Tor configuration for the initramfs onion unlock service (initramfs-tools).
# Baked into the initramfs as /etc/tor/torrc by the "tor" hook; the onion keys
# come from /etc/tor/initramfs-onion on the system.
DataDirectory /var/lib/tor
HiddenServiceDir /etc/tor/onion
HiddenServicePort 22 127.0.0.1:22
SocksPort 0
# Single-hop, non-anonymous onion service. A boot-unlock onion needs
# reachability (behind NAT / a dynamic IP), NOT server-location anonymity, so
# it can skip the multi-hop, vanguards-restricted circuits that a fresh Tor
# with a limited relay view fails to build ("Giving up on launching a
# rendezvous circuit"). Single-hop builds trivial 1-hop circuits that publish
# and rendezvous reliably. Entry guards are likewise unnecessary here.
HiddenServiceNonAnonymousMode 1
HiddenServiceSingleHopMode 1
UseEntryGuards 0

View File

@@ -0,0 +1,5 @@
# Packages for LUKS remote unlock on Debian / Raspberry Pi OS (initramfs-tools)
cryptsetup
cryptsetup-initramfs
dropbear-initramfs
busybox

View File

@@ -0,0 +1,3 @@
# Packages for the Tor onion unlock service in the initramfs (Debian)
tor
busybox # full applet set: ntpd for clock sync on RTC-less boards

View File

@@ -0,0 +1,3 @@
# Packages for the Tor onion unlock service in the initramfs
tor
busybox # full applet set: ntpd for clock sync on RTC-less boards

0
lim/data/__init__.py Normal file
View File

24
lim/data/crypt.py Normal file
View File

@@ -0,0 +1,24 @@
"""Lock/unlock the encfs-encrypted data directory."""
from lim import config, runner, ui
def unlock() -> None:
ui.info(f"Unlocking directory {config.DECRYPTED_PATH}...")
if not config.DECRYPTED_PATH.is_dir():
ui.info(f"Creating directory {config.DECRYPTED_PATH}...")
config.DECRYPTED_PATH.mkdir()
ui.info(f"Decrypting directory {config.ENCRYPTED_PATH} to {config.DECRYPTED_PATH}...")
runner.run(["encfs", str(config.ENCRYPTED_PATH), str(config.DECRYPTED_PATH)])
print("ATTENTION: DATA IS NOW DECRYPTED!")
def lock() -> None:
ui.info(f"Locking directory {config.DECRYPTED_PATH}...")
runner.run(
["fusermount", "-u", str(config.DECRYPTED_PATH)],
error_msg="Unmounting failed.",
)
ui.info("Data is now encrypted.")
ui.info(f"Removing directory {config.DECRYPTED_PATH}...")
config.DECRYPTED_PATH.rmdir()

126
lim/data/sync.py Normal file
View File

@@ -0,0 +1,126 @@
"""Import personal data from the system into the encrypted store, or export it back."""
import time
from dataclasses import dataclass
from pathlib import Path
from lim import config, runner, system, ui
from lim.data import crypt
from lim.errors import LimError
# Paths relative to $HOME; trailing "/" marks a directory tree.
BACKUP_ITEMS = [
".ssh/",
".gitconfig",
".atom/config.cson",
".projectlibre/projectlibre.conf",
".local/share/rhythmbox/rhythmdb.xml",
".config/keepassxc/keepassxc.ini",
"Documents/certificates/",
"Documents/security/",
"Documents/identity/",
"Documents/health/",
"Documents/licenses/",
]
@dataclass(frozen=True)
class SyncOperation:
source: str
destination: str
backup_dir: str
is_directory: bool
def build_sync_plan(
mode: str,
home: Path,
data_path: Path,
backup_folder: Path,
) -> list[SyncOperation]:
"""One rsync operation per existing backup item; missing sources are skipped."""
operations = []
for item in BACKUP_ITEMS:
is_directory = item.endswith("/")
system_path = home / item
# The store mirrors the absolute system path below DATA_PATH.
data_path_item = Path(f"{data_path}{system_path}")
if mode == "export":
source, destination = data_path_item, system_path
elif mode == "import":
source, destination = system_path, data_path_item
else:
raise LimError(f"Unknown sync mode: {mode}")
ui.info(f"{mode.capitalize()} data from {source} to {destination}...")
if not (source.is_file() or source.is_dir()):
ui.warning(f"{source} doesn't exist. Copying data is not possible.")
continue
if is_directory:
backup_dir = Path(f"{backup_folder}{system_path}")
else:
backup_dir = Path(f"{backup_folder}{system_path}").parent
operations.append(
SyncOperation(
source=f"{source}/" if is_directory else str(source),
destination=f"{destination}/" if is_directory else str(destination),
backup_dir=str(backup_dir),
is_directory=is_directory,
)
)
return operations
def rsync_command(operation: SyncOperation) -> list[str]:
command = ["rsync", "-abcEPuvW"]
if operation.is_directory:
command.append("--delete")
command += [f"--backup-dir={operation.backup_dir}", operation.source, operation.destination]
return command
def execute_sync_plan(operations: list[SyncOperation]) -> None:
for operation in operations:
Path(operation.backup_dir).mkdir(parents=True, exist_ok=True)
destination_dir = (
Path(operation.destination)
if operation.is_directory
else Path(operation.destination).parent
)
destination_dir.mkdir(parents=True, exist_ok=True)
if not operation.is_directory and Path(operation.destination).is_file():
ui.info("The destination file already exists!")
ui.info("Difference:")
runner.run(["diff", operation.destination, operation.source], check=False)
runner.run(rsync_command(operation))
def ensure_unlocked() -> None:
mounts = runner.output(["mount"], check=False)
if str(config.DECRYPTED_PATH) not in mounts:
ui.info(f"The decrypted folder {config.DECRYPTED_PATH} is locked. You need to unlock it!")
crypt.unlock()
def import_from_system(mode: str = "import") -> None:
ensure_unlocked()
backup_folder = config.BACKUP_PATH / mode / time.strftime("%Y%m%d%H%M%S")
backup_folder.mkdir(parents=True, exist_ok=True)
operations = build_sync_plan(mode, system.real_home(), config.DATA_PATH, backup_folder)
execute_sync_plan(operations)
def export_to_system() -> None:
import_from_system(mode="export")
user = system.real_user()
home = system.real_home()
ui.info("Setting right permissions for imported files...")
try:
(home / ".ssh").chmod(0o700)
for child in (home / ".ssh").rglob("*"):
child.chmod(0o700 if child.is_dir() else 0o600)
except OSError as exc:
raise LimError(f"Failed to set correct ssh permissions: {exc}") from exc
# Best effort without privilege escalation, like the original script.
chowned = runner.run(["chown", "-R", f"{user}:{user}", str(home)], check=False)
if chowned.returncode != 0:
ui.warning(f'Not all files could be owned by user "{user}"...')

128
lim/device.py Normal file
View File

@@ -0,0 +1,128 @@
"""Block-device selection and low-level device helpers."""
import stat as stat_module
from dataclasses import dataclass
from pathlib import Path
from lim import runner, ui
from lim.errors import LimError
SYS_BLOCK_PATH = Path("/sys/block")
PARTITION_TABLE_INFO = """\
##########################################################################################
Note on Partition Table Deletion:
---------------------------------------------
• MBR (Master Boot Record):
- Typically occupies the first sector (512 bytes), i.e., 1 block.
• GPT (GUID Partition Table):
- Uses a protective MBR (1 block), a GPT header (1 block),
and usually a partition entry array that takes up about 32 blocks.
- Total: approximately 34 blocks (assuming a 512-byte block size).
Recommendation: For deleting a GPT partition table, use a block size of 512 bytes
and overwrite at least 34 blocks to ensure the entire table is cleared.
##########################################################################################"""
def optimal_blocksize(name: str, sys_block_path: Path = SYS_BLOCK_PATH) -> str:
"""64 * physical block size, or "4K" when the size cannot be read.
See https://www.heise.de/ct/hotline/Optimale-Blockgroesse-fuer-dd-2056768.html
"""
size_path = sys_block_path / name / "queue" / "physical_block_size"
try:
return str(64 * int(size_path.read_text().strip()))
except (OSError, ValueError):
return "4K"
@dataclass(frozen=True)
class Device:
name: str # e.g. "sda" or "mmcblk0"
@property
def path(self) -> str:
return f"/dev/{self.name}"
@property
def optimal_blocksize(self) -> str:
return optimal_blocksize(self.name)
def partition(self, number: int) -> str:
"""/dev/sda -> /dev/sda1, /dev/mmcblk0 -> /dev/mmcblk0p1."""
if self.name[-1].isdigit():
return f"{self.path}p{number}"
return f"{self.path}{number}"
def is_block_device(path: str) -> bool:
try:
return stat_module.S_ISBLK(Path(path).stat().st_mode)
except OSError:
return False
def select_device() -> Device:
ui.info("Available devices:")
runner.run(["lsblk", "-o", "NAME,SIZE,TYPE,MODEL"], check=False)
name = ui.ask("Please type in the name of the device: /dev/")
device = Device(name)
if not name or not is_block_device(device.path):
raise LimError(f"{device.path} is not a valid device.")
ui.info(f"Device path set to: {device.path}")
ui.info(f"Optimal blocksize set to: {device.optimal_blocksize}")
return device
def overwrite_device(device: Device) -> None:
"""Optionally overwrite the device (or its first blocks) with zeros."""
print(PARTITION_TABLE_INFO)
answer = ui.ask(
f"Should {device.path} be overwritten with zeros before copying? (y/N/block count)"
)
if answer == "y":
ui.info("Overwriting entire device...")
runner.run(
[
"dd",
"if=/dev/zero",
f"of={device.path}",
f"bs={device.optimal_blocksize}",
"status=progress",
],
sudo=True,
error_msg=f"Overwriting {device.path} failed.",
)
runner.sync_disks()
elif answer in ("", "N"):
ui.info("Skipping Overwriting...")
elif answer.isdigit():
ui.info(f"Overwriting {answer} blocks...")
runner.run(
[
"dd",
"if=/dev/zero",
f"of={device.path}",
f"bs={device.optimal_blocksize}",
f"count={answer}",
"status=progress",
],
sudo=True,
error_msg=f"Overwriting {device.path} failed.",
)
runner.sync_disks()
else:
raise LimError("Invalid input. Block count must be a number.")
def blkid_value(path: str, tag: str) -> str:
"""Return a blkid tag value ("TYPE", "UUID", ...) or "" when unavailable."""
return runner.output(["blkid", path, "-s", tag, "-o", "value"], sudo=True, check=False)
def is_mounted(path_fragment: str) -> bool:
"""Whether any current mount line contains the given fragment."""
mounts = runner.output(["mount"], check=False)
return any(path_fragment in line for line in mounts.splitlines())

80
lim/distributions.yml Normal file
View File

@@ -0,0 +1,80 @@
# Single point of truth for the downloadable image catalog.
# Consumed by lim/catalog.py.
# Image and LUKS --pbkdf-memory cost per Raspberry Pi version.
arch_rpi_images:
"1":
image: ArchLinuxARM-rpi-armv7-latest.tar.gz
luks_memory_cost: "64000"
"2":
image: ArchLinuxARM-rpi-armv7-latest.tar.gz
luks_memory_cost: "128000"
"3b":
image: ArchLinuxARM-rpi-aarch64-latest.tar.gz
luks_memory_cost: "128000"
"3b+":
image: ArchLinuxARM-rpi-aarch64-latest.tar.gz
luks_memory_cost: "128000"
"4":
image: ArchLinuxARM-rpi-aarch64-latest.tar.gz
luks_memory_cost: "256000"
manjaro_gnome_releases:
"20":
checksum: 2df3697908483550d4a473815b08c1377e6b6892
url: https://osdn.net/projects/manjaro-archive/storage/gnome/20.0/
image: manjaro-gnome-20.0-200426-linux56.iso
"21":
url: https://download.manjaro.org/gnome/21.3.7/
image: manjaro-gnome-21.3.7-220816-linux515.iso
"22":
url: https://download.manjaro.org/gnome/22.1.3/
image: manjaro-gnome-22.1.3-230529-linux61.iso
"24":
url: https://download.manjaro.org/gnome/24.2.1/
image: manjaro-gnome-24.2.1-241216-linux612.iso
"25":
url: https://download.manjaro.org/gnome/25.0.10/
image: manjaro-gnome-25.0.10-251013-linux612.iso
# At the moment just optimized for Raspberry Pi 4.
raspberrypi:
url: https://github.com/manjaro-arm/rpi4-images/releases/download/23.02/
image: Manjaro-ARM-gnome-rpi4-23.02.img.xz
luks_memory_cost: "256000"
raspberry_pi_version: "4"
retropie_images:
"1":
checksum: 95a6f84453df36318830de7e8507170e
image: retropie-buster-4.8-rpi1_zero.img.gz
"2":
checksum: 224e64d8820fc64046ba3850f481c87e
image: retropie-buster-4.8-rpi2_3_zero2w.img.gz
"3":
checksum: 224e64d8820fc64046ba3850f481c87e
image: retropie-buster-4.8-rpi2_3_zero2w.img.gz
"4":
checksum: b5daa6e7660a99c246966f3f09b4014b
image: retropie-buster-4.8-rpi4_400.img.gz
# Raspberry Pi OS. The "_latest" endpoints 302-redirect to the current release;
# image is only the local filename (must end .img.xz for decompress_command).
raspios_images:
lite64:
source_url: https://downloads.raspberrypi.com/raspios_lite_arm64_latest
image: raspios_lite_arm64_latest.img.xz
desktop64:
source_url: https://downloads.raspberrypi.com/raspios_arm64_latest
image: raspios_arm64_latest.img.xz
lite32:
source_url: https://downloads.raspberrypi.com/raspios_lite_armhf_latest
image: raspios_lite_armhf_latest.img.xz
# Extra kernel modules the initramfs needs for early network access,
# see https://raspberrypi.stackexchange.com/questions/67051
mkinitcpio_modules_by_rpi:
"1": ""
"2": ""
"3b": smsc95xx
"3b+": lan78xx
"4": lan78xx

2
lim/errors.py Normal file
View File

@@ -0,0 +1,2 @@
class LimError(Exception):
"""Fatal error that aborts the current lim command."""

49
lim/fsutil.py Normal file
View File

@@ -0,0 +1,49 @@
"""Small file manipulation helpers."""
from pathlib import Path
from lim import ui
from lim.errors import LimError
def replace_in_file(search: str, replace: str, path: str | Path) -> None:
"""Replace every literal occurrence of ``search``; fail when absent."""
path = Path(path)
text = path.read_text()
new_text = text.replace(search, replace)
if new_text == text:
raise LimError(f"Search string '{search}' not found in {path}.")
path.write_text(new_text)
def drop_fstab_mount(mount_point: str, path: str | Path) -> None:
"""Remove any (non-comment) fstab line whose mount point field equals mount_point.
Lets a fresh mapper/boot entry replace a stock image's existing one instead
of colliding with it. No-op when the file or a matching line is absent.
"""
path = Path(path)
if not path.exists():
return
kept = [
line
for line in path.read_text().splitlines()
if line.lstrip().startswith("#") or line.split()[1:2] != [mount_point]
]
path.write_text("".join(f"{line}\n" for line in kept))
def ensure_line_in_file(line: str, path: str | Path) -> bool:
"""Append ``line`` unless already present. Returns True when appended."""
path = Path(path)
content = path.read_text() if path.exists() else ""
if line in content.splitlines():
ui.warning(f"File {path} already contains the following entry:")
print(line)
ui.info("Skipped.")
return False
with path.open("a") as handle:
if content and not content.endswith("\n"):
handle.write("\n")
handle.write(line + "\n")
return True

0
lim/image/__init__.py Normal file
View File

32
lim/image/backup.py Normal file
View File

@@ -0,0 +1,32 @@
"""Create an image backup from a memory device into a file."""
from pathlib import Path
from lim import device as device_module
from lim import runner, ui
def run_backup() -> None:
ui.info("Backupscript for memory devices started...")
print()
device = device_module.select_device()
working_dir = Path.cwd()
path = ""
while not path:
path = ui.ask(f"Please type in backup image path+name relative to {working_dir}:")
output_file = f"{path}.img" if path.startswith("/") else f"{working_dir}/{path}.img"
ui.info(f"Input file: {device.path}")
ui.info(f"Output file: {output_file}")
ui.ask('Please confirm by pushing "Enter". To cancel use "Ctrl + C"')
ui.info("Imagetransfer starts. This can take a while...")
runner.run(
["dd", f"if={device.path}", f"of={output_file}", "bs=1M", "status=progress"],
sudo=True,
error_msg='"dd" failed.',
)
runner.sync_disks()
ui.success("Imagetransfer successfull.")

126
lim/image/choosers.py Normal file
View File

@@ -0,0 +1,126 @@
"""Interactive selection of the distribution image to install."""
from lim import catalog, runner, ui
from lim.errors import LimError
from lim.image.plan import ImagePlan
def _choose(prompt: str, options: list[str]) -> str:
"""List options numbered; accept either a number or the name itself."""
ui.info(f"{prompt}:")
for index, name in enumerate(options, 1):
ui.info(f" {index}) {name}")
answer = ui.ask("Pick a number or name:").strip()
if answer in options: # name wins, so numeric names (e.g. arch "4") still work
return answer
if answer.isdigit() and 1 <= int(answer) <= len(options):
return options[int(answer) - 1]
return answer
def choose_arch(plan: ImagePlan) -> None:
version = _choose("Which Raspberry Pi version", list(catalog.arch_rpi_images()))
entry = catalog.arch_rpi_images().get(version)
if entry is None:
raise LimError(f"Version {version} isn't supported.")
plan.raspberry_pi_version = version
plan.boot_size = "+500M"
plan.base_download_url = "http://os.archlinuxarm.org/os/"
plan.image_name = entry["image"]
plan.luks_memory_cost = entry["luks_memory_cost"]
def choose_manjaro(plan: ImagePlan) -> None:
plan.boot_size = "+500M"
flavour = _choose("Which Manjaro version", ["architect", "gnome"])
if flavour == "architect":
plan.image_checksum = "6b1c2fce12f244c1e32212767a9d3af2cf8263b2"
plan.base_download_url = (
"https://osdn.net/frs/redir.php?m=dotsrc&f=%2Fstorage%2Fg%2Fm%2Fma"
"%2Fmanjaro%2Farchitect%2F20.0%2F"
)
plan.image_name = "manjaro-architect-20.0-200426-linux56.iso"
elif flavour == "gnome":
release = _choose("Which Gnome release", list(catalog.manjaro_gnome_releases()))
entry = catalog.manjaro_gnome_releases().get(release)
if entry is None:
raise LimError(f"Gnome release {release} isn't supported.")
plan.image_checksum = entry.get("checksum")
plan.base_download_url = entry["url"]
plan.image_name = entry["image"]
plan.luks_memory_cost = entry.get("luks_memory_cost")
plan.raspberry_pi_version = entry.get("raspberry_pi_version")
else:
raise LimError(f"Manjaro version {flavour} isn't supported.")
def choose_moode(plan: ImagePlan) -> None:
plan.boot_size = "+200M"
plan.image_checksum = "185cbc9a4994534bb7a4bc2744c78197"
plan.base_download_url = "https://github.com/moode-player/moode/releases/download/r651prod/"
plan.image_name = "moode-r651-iso.zip"
def choose_retropie(plan: ImagePlan) -> None:
plan.boot_size = "+500M"
version = _choose("Which RetroPie version", list(catalog.retropie_images()))
entry = catalog.retropie_images().get(version)
if entry is None:
raise LimError(f"Version {version} isn't supported.")
plan.raspberry_pi_version = version
plan.base_download_url = "https://github.com/RetroPie/RetroPie-Setup/releases/download/4.8/"
plan.image_checksum = entry["checksum"]
plan.image_name = entry["image"]
def choose_raspios(plan: ImagePlan) -> None:
plan.boot_size = "+512M"
plan.root_filesystem = "ext4"
variant = _choose("Which Raspberry Pi OS image", list(catalog.raspios_images()))
entry = catalog.raspios_images().get(variant)
if entry is None:
raise LimError(f"Raspberry Pi OS variant {variant} isn't supported.")
plan.source_url = entry["source_url"]
plan.image_name = entry["image"]
def choose_torbox(plan: ImagePlan) -> None:
plan.base_download_url = "https://www.torbox.ch/data/"
plan.image_name = "torbox-20220102-v050.gz"
plan.image_checksum = "0E1BA7FFD14AAAE5F0462C8293D95B62C3BF1D9E726E26977BD04772C55680D3"
plan.boot_size = "+200M"
def choose_android_x86(plan: ImagePlan) -> None:
plan.base_download_url = (
"https://www.fosshub.com/Android-x86.html?dwl=android-x86_64-9.0-r2.iso"
)
plan.image_name = "android-x86_64-9.0-r2.iso"
plan.image_checksum = "f7eb8fc56f29ad5432335dc054183acf086c539f3990f0b6e9ff58bd6df4604e"
plan.boot_size = "+500M"
DISTRIBUTION_CHOOSERS = {
"android-x86": choose_android_x86,
"torbox": choose_torbox,
"arch": choose_arch,
"manjaro": choose_manjaro,
"moode": choose_moode,
"retropie": choose_retropie,
"raspios": choose_raspios,
}
def choose_linux_image(plan: ImagePlan) -> None:
distribution = _choose("Which distribution", list(DISTRIBUTION_CHOOSERS))
chooser = DISTRIBUTION_CHOOSERS.get(distribution)
if chooser is None:
raise LimError(f"Distribution {distribution} isn't supported.")
plan.distribution = distribution
chooser(plan)
def choose_local_image(plan: ImagePlan) -> None:
ui.info("Available images:")
runner.run(["ls", "-l", str(plan.image_folder)], check=False)
plan.image_name = ui.ask("Which image would you like to use?")

23
lim/image/chroot.py Normal file
View File

@@ -0,0 +1,23 @@
"""Mount an image and open an interactive shell inside it."""
from lim import device as device_module
from lim import runner, ui
from lim.image.session import ImageSession
def run_chroot() -> None:
ui.info("Starting chroot...")
device = device_module.select_device()
session = ImageSession(device)
try:
session.make_working_folder()
session.make_mount_folders()
session.decrypt_root()
session.mount_partitions()
session.mount_chroot_binds()
session.copy_qemu()
session.copy_resolv_conf()
ui.info("Bash shell starts...")
runner.run(["chroot", str(session.root_mount_path), "/bin/bash"], sudo=True)
finally:
session.destructor()

71
lim/image/crossarch.py Normal file
View File

@@ -0,0 +1,71 @@
"""Ensure the host can chroot into a foreign-architecture image (qemu binfmt)."""
import platform
import shutil
from pathlib import Path
from lim import runner, ui
from lim.errors import LimError
# Building natively needs no qemu; on these the chroot runs directly.
_NATIVE_ARM = ("aarch64", "armv7l", "armv6l", "arm64")
# Repo-based installs only; AUR helpers refuse to run from this root process.
_BINFMT_INSTALLERS = {
"apt-get": ["apt-get", "install", "-y", "qemu-user-static", "binfmt-support"],
"dnf": ["dnf", "install", "-y", "qemu-user-static"],
"zypper": ["zypper", "--non-interactive", "install", "qemu-linux-user"],
"pacman": ["pacman", "-S", "--needed", "--noconfirm", "qemu-user-static-binfmt"],
}
def _entry_enabled(entry: Path) -> bool:
try:
return "enabled" in entry.read_text()
except OSError:
return False
def qemu_binfmt_enabled() -> bool:
binfmt = Path("/proc/sys/fs/binfmt_misc")
return binfmt.is_dir() and any(_entry_enabled(e) for e in binfmt.glob("qemu-*"))
def auto_enable() -> bool:
"""Install qemu-user-static via the host package manager and register binfmt."""
command = next((cmd for tool, cmd in _BINFMT_INSTALLERS.items() if shutil.which(tool)), None)
if command is None:
ui.warning("No supported host package manager found (apt-get/dnf/zypper/pacman).")
return False
runner.run(command, sudo=True, check=False)
runner.run(["systemctl", "restart", "systemd-binfmt"], sudo=True, check=False)
return qemu_binfmt_enabled()
def ensure_ready() -> None:
"""Make foreign-arch chroot work, or fail with actionable guidance.
Called before the target is erased, so a host that cannot run the image's
binaries aborts early instead of mid-build with a cryptic "Exec format error".
"""
if platform.machine() in _NATIVE_ARM:
return
if qemu_binfmt_enabled():
ui.info("Cross-architecture build: qemu binfmt handlers are registered.")
return
ui.warning(
f"Cross-architecture build: this host ({platform.machine()}) cannot run the "
"image's ARM binaries in chroot yet."
)
if ui.confirm("Install qemu-user-static and register binfmt now?"):
if auto_enable():
ui.success("qemu binfmt handlers registered.")
return
ui.warning("Automatic setup did not register a handler.")
raise LimError(
"Cross-architecture chroot is not available. Install qemu-user-static + binfmt "
"manually, or build on the target itself (e.g. the Pi booted from USB).\n"
" Arch/Manjaro : qemu-user-static + qemu-user-static-binfmt (AUR), "
"then: sudo systemctl restart systemd-binfmt\n"
" Debian/Ubuntu: sudo apt install qemu-user-static binfmt-support"
)

42
lim/image/encryption.py Normal file
View File

@@ -0,0 +1,42 @@
"""Remote-unlockable LUKS boot configuration.
Distro-agnostic orchestration; the init-system-specific steps live in the
initramfs backends (mkinitcpio for Arch/Manjaro, initramfs-tools for
Debian/Raspberry Pi OS). See lim/image/initramfs/.
"""
from pathlib import Path
from lim import packages, ui
from lim.image.initramfs import get_backend
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession, install_packages
def configure_encryption(
plan: ImagePlan, session: ImageSession, authorized_keys: Path
) -> str | None:
"""Configure the remote-unlock LUKS stack; return the onion address, if any."""
root = session.root_mount_path
backend = get_backend(plan.distribution)
ui.info("Setup encryption...")
ui.info("Installing neccessary software...")
install_packages(
plan.distribution,
root,
" ".join(packages.get_packages(backend.luks_package_collection())),
)
backend.install_authorized_key(root, authorized_keys)
onion_address = None
if plan.tor_unlock:
# Hook files and onion keys must exist before the initramfs is baked.
onion_address = backend.install_tor_unlock(plan, root)
# crypttab must be written before the initramfs is (re)generated so the
# Debian cryptsetup hook can bake the mapping in.
backend.register_encrypted_root(plan, session, root)
backend.configure_initramfs(plan, root)
backend.configure_bootloader(plan, session, root)
return onion_address

View File

@@ -0,0 +1,21 @@
"""Distro-specific initramfs backends for encrypted remote unlock."""
from lim.errors import LimError
from lim.image.initramfs.base import InitramfsBackend
from lim.image.initramfs.initramfs_tools import InitramfsToolsBackend
from lim.image.initramfs.mkinitcpio import MkinitcpioBackend
_MKINITCPIO = frozenset({"arch", "manjaro"})
_INITRAMFS_TOOLS = frozenset({"raspios", "moode", "retropie"})
def get_backend(distribution: str | None) -> InitramfsBackend:
"""Pick the initramfs backend for a distribution's init system."""
if distribution in _MKINITCPIO:
return MkinitcpioBackend()
if distribution in _INITRAMFS_TOOLS:
return InitramfsToolsBackend()
raise LimError(f"No initramfs backend for distribution {distribution!r}.")
__all__ = ["InitramfsBackend", "get_backend"]

View File

@@ -0,0 +1,42 @@
"""The initramfs backend interface.
Encrypted remote-unlock images differ by init system: Arch/Manjaro use
mkinitcpio, Debian/Raspberry Pi OS use initramfs-tools. Each backend owns the
distro-specific bits (package set, crypttab syntax, initramfs generation,
bootloader cmdline, and how the Tor hook is baked in); the encryption flow
orchestrates them the same way for every distro.
"""
from abc import ABC, abstractmethod
from pathlib import Path
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession
class InitramfsBackend(ABC):
"""Distro-specific steps to build an encrypted, remote-unlockable image."""
@abstractmethod
def luks_package_collection(self) -> str:
"""Name of the package collection providing the LUKS/unlock stack."""
@abstractmethod
def install_authorized_key(self, root: Path, authorized_keys: Path) -> None:
"""Place the SSH public key where the initramfs dropbear reads it."""
@abstractmethod
def install_tor_unlock(self, plan: ImagePlan, root: Path) -> str:
"""Bake the Tor onion service into the initramfs; return the address."""
@abstractmethod
def register_encrypted_root(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
"""Write fstab and crypttab in the distro's syntax."""
@abstractmethod
def configure_initramfs(self, plan: ImagePlan, root: Path) -> None:
"""Configure the initramfs modules/hooks and (re)generate it."""
@abstractmethod
def configure_bootloader(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
"""Point the bootloader at the encrypted root with early networking."""

View File

@@ -0,0 +1,117 @@
"""initramfs-tools backend (Debian / Raspberry Pi OS).
Debian does not use mkinitcpio; the LUKS unlock stack is cryptsetup-initramfs
(reads /etc/crypttab, option ``initramfs``) + dropbear-initramfs (serves the
passphrase prompt over SSH). Raspberry Pi OS ships without an initramfs, so we
enable one in config.txt and regenerate with update-initramfs.
NOTE: the exact dropbear-initramfs key path, the boot-partition mount point
(/boot vs /boot/firmware) and the initramfs enable knob vary across Raspberry
Pi OS releases; verify on the target release. This path is not exercised by the
QEMU harness yet.
"""
import re
from pathlib import Path
from lim import config, fsutil, packages, runner, ui
from lim.image.initramfs import keygen
from lim.image.initramfs.base import InitramfsBackend
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession, chroot_bash, install_packages
# Numeric IP, not a hostname: the initramfs has no DNS resolver.
_DEFAULT_NTP = "162.159.200.123"
class InitramfsToolsBackend(InitramfsBackend):
def luks_package_collection(self) -> str:
return "server/luks-debian"
def install_authorized_key(self, root: Path, authorized_keys: Path) -> None:
# dropbear-initramfs (bookworm) reads /etc/dropbear/initramfs/authorized_keys.
target = root / "etc/dropbear/initramfs/authorized_keys"
ui.info(f"Adding {authorized_keys} to {target}...")
runner.run(
["install", "-D", "-m", "0600", str(authorized_keys), str(target)],
sudo=True,
)
def install_tor_unlock(self, plan: ImagePlan, root: Path) -> str:
ui.info("Setting up remote unlock via Tor onion service (initramfs-tools)...")
install_packages(
plan.distribution, root, " ".join(packages.get_packages("server/tor-debian"))
)
source_dir = config.CONFIGURATION_PATH / "initramfs-tools"
for source, target, mode in (
(source_dir / "tor_hook", root / "etc/initramfs-tools/hooks/tor", "0755"),
(
source_dir / "tor_premount",
root / "etc/initramfs-tools/scripts/init-premount/tor",
"0755",
),
(
source_dir / "tor_bottom",
root / "etc/initramfs-tools/scripts/init-bottom/tor",
"0755",
),
(source_dir / "torrc", root / keygen.TORRC_STAGING_PATH, "0644"),
):
ui.info(f"Installing {target}...")
runner.run(["install", "-D", "-m", mode, str(source), str(target)], sudo=True)
onion_address = keygen.generate_onion_keys(root)
ui.success(f"Onion unlock address: {onion_address}")
ui.info(f"Unlock later with: torsocks ssh root@{onion_address}")
return onion_address
def register_encrypted_root(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
fstab_path = root / "etc/fstab"
fstab_line = (
f"/dev/mapper/{session.root_mapper_name} / {plan.root_filesystem}"
" defaults,noatime 0 1"
)
ui.info(f"Configuring {fstab_path}...")
# A stock image already mounts / by PARTUUID; drop it so the mapper wins.
fsutil.drop_fstab_mount("/", fstab_path)
fsutil.ensure_line_in_file(fstab_line, fstab_path)
crypttab_path = root / "etc/crypttab"
# `initramfs` bakes the mapping into the initramfs so cryptsetup-initramfs
# unlocks the root before the pivot.
crypttab_line = (
f"{session.root_mapper_name} UUID={session.root_partition_uuid} none luks,initramfs"
)
ui.info(f"Configuring {crypttab_path}...")
fsutil.ensure_line_in_file(crypttab_line, crypttab_path)
def configure_initramfs(self, plan: ImagePlan, root: Path) -> None: # noqa: ARG002
ui.info("Regenerating the Debian initramfs (update-initramfs)...")
# Create for all installed kernels when none exists yet (Raspberry Pi OS
# ships without one), else update the existing one. `-k all` avoids the
# build host's `uname -r` leaking into the chroot.
chroot_bash(
root,
"update-initramfs -c -k all 2>/dev/null || update-initramfs -u -k all",
)
def configure_bootloader(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
hostname = (root / "etc/hostname").read_text().strip()
boot = session.boot_mount_path
config_txt = boot / "config.txt"
if config_txt.is_file():
ui.info(f"Enabling an initramfs in {config_txt}...")
fsutil.ensure_line_in_file("auto_initramfs=1", config_txt)
cmdline_txt = boot / "cmdline.txt"
ui.info(f"Configuring {cmdline_txt}...")
text = cmdline_txt.read_text().strip()
text = re.sub(r"root=\S+", f"root=/dev/mapper/{session.root_mapper_name}", text)
if "ip=" not in text:
text = f"{text} ip=::::{hostname}:eth0:dhcp"
if "net.ifnames=0" not in text:
text = f"{text} net.ifnames=0"
if plan.tor_unlock and "tor_ntp=" not in text:
text = f"{text} tor_ntp={_DEFAULT_NTP}"
cmdline_txt.write_text(text + "\n")
ui.info(f"Content of {cmdline_txt}:{cmdline_txt.read_text()}")

View File

@@ -0,0 +1,54 @@
"""Offline Tor onion key generation, shared by every initramfs backend.
``tor --DisableNetwork 1`` writes a v3 onion service's keys without touching
the network, so the same stable ``.onion`` address can be baked into the
initramfs whether the distro uses mkinitcpio or initramfs-tools.
"""
from pathlib import Path
from lim import ui
from lim.errors import LimError
from lim.image.session import chroot_bash
# Staged relative to the mounted image root.
ONION_STAGING_DIR = "etc/tor/initramfs-onion"
TORRC_STAGING_PATH = "etc/tor/initramfs-torrc"
# An empty -f torrc keeps the image's /etc/tor/torrc (User tor, ...) out of the
# keygen run, which would otherwise fail as non-root inside the chroot.
_KEYGEN_SCRIPT = f"""
mkdir -p /{ONION_STAGING_DIR}
chmod 0700 /{ONION_STAGING_DIR}
: > /tmp/tor-keygen-torrc
tor -f /tmp/tor-keygen-torrc --DisableNetwork 1 \\
--DataDirectory /tmp/tor-keygen-data \\
--HiddenServiceDir /{ONION_STAGING_DIR} \\
--HiddenServicePort "22 127.0.0.1:22" \\
--SocksPort 0 --RunAsDaemon 0 --Log "notice stderr" &
tor_pid=$!
for _ in $(seq 1 30); do
[ -s /{ONION_STAGING_DIR}/hostname ] && break
sleep 1
done
kill "$tor_pid" 2>/dev/null || true
rm -rf /tmp/tor-keygen-data /tmp/tor-keygen-torrc
[ -s /{ONION_STAGING_DIR}/hostname ]
"""
def generate_onion_keys(root: Path) -> str:
"""Generate the onion keys offline in the chroot; return the .onion address.
Idempotent: existing keys are kept so the address stays stable across
re-runs.
"""
hostname_path = root / ONION_STAGING_DIR / "hostname"
if hostname_path.is_file():
ui.info("Onion keys already exist, keeping the existing address.")
else:
ui.info("Generating onion service keys (offline, inside the chroot)...")
chroot_bash(root, _KEYGEN_SCRIPT, error_msg="Onion key generation failed.")
if not hostname_path.is_file():
raise LimError(f"Onion key generation produced no {hostname_path}.")
return hostname_path.read_text().strip()

View File

@@ -0,0 +1,122 @@
"""mkinitcpio backend (Arch Linux ARM / Manjaro ARM).
See https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38 and
https://gist.github.com/EnigmaCurry/2f9bed46073da8e38057fe78a61e7994
"""
from pathlib import Path
from lim import catalog, config, fsutil, packages, runner, ui
from lim.image.initramfs import keygen
from lim.image.initramfs.base import InitramfsBackend
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession, chroot_bash, install_packages
MKINITCPIO_HOOKS_PREFIX = (
"base udev autodetect microcode modconf kms keyboard keymap consolefont block"
)
MKINITCPIO_HOOKS_SUFFIX = "filesystems fsck"
class MkinitcpioBackend(InitramfsBackend):
def luks_package_collection(self) -> str:
return "server/luks"
def install_authorized_key(self, root: Path, authorized_keys: Path) -> None:
dropbear_root_key_path = root / "etc/dropbear/root_key"
ui.info(f"Adding {authorized_keys} to dropbear...")
runner.run(["cp", "-v", str(authorized_keys), str(dropbear_root_key_path)], sudo=True)
def install_tor_unlock(self, plan: ImagePlan, root: Path) -> str:
ui.info("Setting up remote unlock via Tor onion service...")
install_packages(plan.distribution, root, " ".join(packages.get_packages("server/tor")))
source_dir = config.CONFIGURATION_PATH / "initcpio"
for source, target in (
(source_dir / "tor_install", root / "etc/initcpio/install/tor"),
(source_dir / "tor_hook", root / "etc/initcpio/hooks/tor"),
(source_dir / "torrc", root / keygen.TORRC_STAGING_PATH),
):
ui.info(f"Installing {target}...")
runner.run(["install", "-D", "-m", "0644", str(source), str(target)], sudo=True)
onion_address = keygen.generate_onion_keys(root)
ui.success(f"Onion unlock address: {onion_address}")
ui.info(f"Unlock later with: torsocks ssh root@{onion_address}")
return onion_address
def register_encrypted_root(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
fstab_path = root / "etc/fstab"
fstab_line = (
f"UUID={session.root_partition_uuid} / {plan.root_filesystem}"
" defaults,noatime 0 1"
)
ui.info(f"Configuring {fstab_path}...")
fsutil.ensure_line_in_file(fstab_line, fstab_path)
ui.info(f"Content of {fstab_path}:{fstab_path.read_text()}")
crypttab_path = root / "etc/crypttab"
crypttab_line = f"{session.root_mapper_name} UUID={session.root_partition_uuid} none luks"
ui.info(f"Configuring {crypttab_path}...")
fsutil.ensure_line_in_file(crypttab_line, crypttab_path)
ui.info(f"Content of {crypttab_path}:{crypttab_path.read_text()}")
def configure_initramfs(self, plan: ImagePlan, root: Path) -> None:
# Concerning mkinitcpio warnings, see
# https://gist.github.com/imrvelj/c65cd5ca7f5505a65e59204f5a3f7a6d
mkinitcpio_path = root / "etc/mkinitcpio.conf"
ui.info(f"Configuring {mkinitcpio_path}...")
additional_modules = catalog.mkinitcpio_modules_by_rpi().get(plan.raspberry_pi_version)
if additional_modules is None:
ui.warning(f"Version {plan.raspberry_pi_version} isn't supported.")
additional_modules = ""
modules = f"g_cdc usb_f_acm usb_f_ecm {additional_modules} g_ether".replace(" ", " ")
fsutil.replace_in_file("MODULES=()", f"MODULES=({modules})", mkinitcpio_path)
fsutil.replace_in_file("BINARIES=()", "BINARIES=(/usr/lib/libgcc_s.so.1)", mkinitcpio_path)
tor_hook = "tor " if plan.tor_unlock else ""
fsutil.replace_in_file(
f"HOOKS=({MKINITCPIO_HOOKS_PREFIX} {MKINITCPIO_HOOKS_SUFFIX})",
f"HOOKS=({MKINITCPIO_HOOKS_PREFIX} sleep netconf {tor_hook}dropbear "
f"encryptssh {MKINITCPIO_HOOKS_SUFFIX})",
mkinitcpio_path,
)
ui.info(f"Content of {mkinitcpio_path}:{mkinitcpio_path.read_text()}")
ui.info("Generating mkinitcpio...")
chroot_bash(root, "mkinitcpio -vP")
def configure_bootloader(self, plan: ImagePlan, session: ImageSession, root: Path) -> None:
cryptdevice = (
f"cryptdevice=UUID={session.root_partition_uuid}:{session.root_mapper_name} "
f"root={session.root_mapper_path}"
)
boot_txt_path = session.boot_mount_path / "boot.txt"
if boot_txt_path.is_file():
ui.info(f"Configuring {boot_txt_path}...")
hostname = (root / "etc/hostname").read_text().strip()
fsutil.replace_in_file("part uuid ${devtype} ${devnum}:2 uuid", "", boot_txt_path)
fsutil.replace_in_file(
"setenv bootargs console=ttyS1,115200 console=tty0 root=PARTUUID=${uuid} "
'rw rootwait smsc95xx.macaddr="${usbethaddr}"',
f"setenv bootargs console=ttyS1,115200 console=tty0 "
f"ip=::::{hostname}:eth0:dhcp {cryptdevice} rw rootwait "
# Concerning issues with network adapter names, see
# https://forum.iobroker.net/topic/40542/raspberry-pi4-kein-eth0-mehr/16
'smsc95xx.macaddr="${usbethaddr}" net.ifnames=0 biosdevname=0',
boot_txt_path,
)
ui.info(f"Content of {boot_txt_path}:{boot_txt_path.read_text()}")
ui.info("Generating...")
chroot_bash(root, "cd /boot/ && ./mkscr || exit 1")
else:
cmdline_txt_path = session.boot_mount_path / "cmdline.txt"
ui.info(f"Configuring {cmdline_txt_path}...")
# Firmware-boot boards (e.g. RPi4 cmdline.txt) need the same early
# networking as the boot.txt branch, or the initramfs netconf hook
# brings up no interface and remote unlock (dropbear + tor) is
# unreachable — the onion never even publishes.
hostname = (root / "etc/hostname").read_text().strip()
fsutil.replace_in_file(
"root=/dev/mmcblk0p2",
f"{cryptdevice} rootfstype={plan.root_filesystem} "
f"ip=::::{hostname}:eth0:dhcp net.ifnames=0 biosdevname=0",
cmdline_txt_path,
)
ui.info(f"Content of {cmdline_txt_path}:{cmdline_txt_path.read_text()}")

28
lim/image/loopimg.py Normal file
View File

@@ -0,0 +1,28 @@
"""Attach a stock disk image as a loop device to read its partitions."""
from pathlib import Path
from lim import runner, ui
from lim.errors import LimError
def attach(image_path: Path) -> str:
"""Attach the image via losetup with partition scanning; return /dev/loopN."""
ui.info(f"Attaching {image_path} as a loop device...")
loop = runner.output(
["losetup", "-Pf", "--show", str(image_path)],
sudo=True,
error_msg=f"Attaching {image_path} as a loop device failed.",
).strip()
if not loop:
raise LimError(f"losetup returned no device for {image_path}.")
return loop
def detach(loop: str) -> None:
runner.run(["losetup", "-d", loop], sudo=True, check=False)
def partition(loop: str, number: int) -> str:
"""/dev/loop0 -> /dev/loop0p1 (losetup -P names partitions with a p suffix)."""
return f"{loop}p{number}"

38
lim/image/plan.py Normal file
View File

@@ -0,0 +1,38 @@
"""The image setup plan collected from the user's answers."""
from dataclasses import dataclass, field
from pathlib import Path
DEFAULT_BOOT_SIZE = "+500M"
@dataclass
class ImagePlan:
operation_system: str = "linux"
distribution: str | None = None
base_download_url: str | None = None
image_name: str | None = None
# Direct download URL when it cannot be derived as base_download_url + image_name
# (Raspberry Pi OS ships behind a "_latest" redirect but must land under a
# .img.xz name so decompress_command recognises it).
source_url: str | None = None
image_checksum: str | None = None
boot_size: str = ""
luks_memory_cost: str | None = None
raspberry_pi_version: str | None = None
encrypt_system: bool = False
tor_unlock: bool = False
root_filesystem: str = ""
image_folder: Path = field(default_factory=Path)
@property
def download_url(self) -> str | None:
if self.source_url is not None:
return self.source_url
if self.base_download_url is None or self.image_name is None:
return None
return f"{self.base_download_url}{self.image_name}"
@property
def image_path(self) -> Path:
return self.image_folder / self.image_name

232
lim/image/raspberry.py Normal file
View File

@@ -0,0 +1,232 @@
"""Raspberry-Pi-specific configuration of a freshly transferred image."""
from pathlib import Path
from lim import device as device_module
from lim import fsutil, runner, ui
from lim.errors import LimError
from lim.image.encryption import configure_encryption
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession, chroot_bash, install_packages
ADMINISTRATOR_USERNAME = "administrator"
def configure_sudoers(root_mount_path: Path, username: str) -> None:
sudo_config_dir = root_mount_path / "etc/sudoers.d"
sudo_config_file = sudo_config_dir / username
sudo_config_dir.mkdir(parents=True, exist_ok=True)
try:
sudo_config_file.write_text(f"{username} ALL=(ALL:ALL) ALL\n")
sudo_config_file.chmod(0o440)
except OSError as exc:
raise LimError(f"Failed to create sudoers file for {username}: {exc}") from exc
def configure_ssh_key(public_key_path: str, target_ssh_folder: Path, authorized_keys: Path) -> None:
source = Path(public_key_path)
if not source.is_file():
raise LimError(
f'The ssh key "{public_key_path}" can\'t be copied to '
f'"{authorized_keys}" because it doesn\'t exist.'
)
ui.info("Copy ssh key to target...")
target_ssh_folder.mkdir(parents=True, exist_ok=True)
authorized_keys.write_text(source.read_text())
ui.info(f"{authorized_keys} contains the following: {authorized_keys.read_text()}")
ui.info("Set permissions with chmod...")
target_ssh_folder.chmod(0o700)
authorized_keys.chmod(0o600)
def _ensure_image_mounted(session: ImageSession) -> None:
ui.info("Start regular mounting procedure...")
if device_module.is_mounted(session.boot_partition_path):
ui.info(f"{session.boot_partition_path} is allready mounted...")
elif device_module.is_mounted(session.root_mapper_path):
ui.info(f"{session.root_mapper_path} is allready mounted...")
else:
session.decrypt_root()
session.mount_partitions()
if session.boot_partition_uuid is None:
# Partitions were mounted by an earlier run; fetch what mount_partitions
# would have provided so fstab/crypttab never see a None UUID.
session.read_partition_uuids()
def _seed_boot_uuid(session: ImageSession) -> None:
fstab_path = session.root_mount_path / "etc/fstab"
if "/dev/mmcblk0p1" not in fstab_path.read_text():
# PARTUUID=/LABEL= based images (e.g. RetroPie, Manjaro ARM) have
# nothing to seed — that is a valid state, not an error.
ui.info(f"{fstab_path} does not reference /dev/mmcblk0p1. Skipping UUID seeding.")
return
ui.info(f"Seeding UUID to {fstab_path} to avoid path conflicts...")
fsutil.replace_in_file("/dev/mmcblk0p1", f"UUID={session.boot_partition_uuid}", fstab_path)
ui.info(f"Content of {fstab_path}:{fstab_path.read_text()}")
def _select_target_user(root: Path) -> tuple[str, str]:
"""Return (default username, target username), renaming the home on request."""
target_home_path = root / "home"
home_entries = sorted(path.name for path in target_home_path.iterdir())
if not home_entries:
raise LimError(f"No home directory found under {target_home_path}.")
default_username = home_entries[0]
rename = default_username != ADMINISTRATOR_USERNAME and ui.confirm(
f"Should the {default_username} be renamed to {ADMINISTRATOR_USERNAME}? "
)
if not rename:
return default_username, default_username
ui.info(
f"Rename home directory from {target_home_path / default_username} "
f"to {target_home_path / ADMINISTRATOR_USERNAME}..."
)
runner.run(
[
"mv",
"-v",
str(target_home_path / default_username),
str(target_home_path / ADMINISTRATOR_USERNAME),
],
sudo=True,
error_msg="Failed to rename home directory",
)
return default_username, ADMINISTRATOR_USERNAME
def _change_passwords(root: Path, target_username: str) -> None:
password = ui.ask(
f"Type in new password for user root and {target_username} (leave empty to skip): "
)
if not password:
ui.info("No password change requested, skipped password change...")
return
repeated = ui.ask(f'Repeat new password for "{target_username}": ')
if password != repeated:
raise LimError("Passwords didn't match.")
ui.info("Changing passwords on target system...")
chroot_bash(
root,
f"( echo '{password}'; echo '{password}' ) | passwd {target_username}\n"
f"( echo '{password}'; echo '{password}' ) | passwd\n",
error_msg="Failed to change password.",
)
def _configure_hostname(root: Path) -> None:
hostname_path = root / "etc/hostname"
hostname = ui.ask("Type in the hostname (leave empty to skip): ")
if hostname:
hostname_path.write_text(hostname + "\n")
else:
hostname = hostname_path.read_text().strip()
ui.info("No hostname change requested, skipped hostname change...")
ui.info(f"Used hostname is: {hostname}")
def _update_system(plan: ImagePlan, root: Path) -> None:
if not ui.confirm("Should the system be updated?"):
return
ui.info("Updating system...")
if plan.distribution in ("arch", "manjaro"):
chroot_bash(root, "pacman --noconfirm -Syyu")
elif plan.distribution in ("moode", "retropie", "raspios"):
chroot_bash(root, "yes | apt update\nyes | apt upgrade\n")
else:
ui.warning(
f'System update for operation system "{plan.distribution}" '
"is not supported yet. Skipped."
)
def _run_retropie_procedures(session: ImageSession, public_key_path: str) -> None:
if public_key_path:
(session.boot_mount_path / "ssh").write_text("\n")
if not ui.confirm("Should the RetroFlag specific procedures be executed?"):
return
ui.info("Executing RetroFlag specific procedures...")
chroot_bash(
session.root_mount_path,
'wget -O - "https://raw.githubusercontent.com/RetroFlag/'
'retroflag-picase/master/install_gpi.sh" | bash\n',
)
def _configure_users_and_keys(session: ImageSession) -> tuple[str, Path]:
"""Handle user rename, sudo rights and SSH key; return (public key path, authorized_keys)."""
root = session.root_mount_path
ui.info("Define target paths...")
default_username, target_username = _select_target_user(root)
renamed = default_username != target_username
target_user_ssh_folder = root / "home" / target_username / ".ssh"
target_authorized_keys = target_user_ssh_folder / "authorized_keys"
if ui.confirm(f"Should the {target_username} have sudo rights? "):
configure_sudoers(root, target_username)
public_key_path = ui.ask(
"Enter the path to the SSH key to be added to the image (default: none):"
)
if public_key_path:
configure_ssh_key(public_key_path, target_user_ssh_folder, target_authorized_keys)
else:
ui.info("Skipped SSH-key copying..")
ui.info("Start chroot procedures...")
session.mount_chroot_binds()
session.copy_qemu()
session.copy_resolv_conf()
chroot_user_home = f"/home/{target_username}/"
if renamed:
ui.info("Delete old user and create new user")
chroot_bash(
root,
f"userdel -r {default_username}\n"
f"useradd -m -d {chroot_user_home} -s /bin/bash {target_username}\n"
f"chown -R {target_username}:{target_username} {chroot_user_home}\n",
error_msg="Failed to delete old user and create new user",
)
if public_key_path:
ui.info("Chroot to set ownership...")
chroot_bash(
root,
f"chown -vR {target_username}:{target_username} {chroot_user_home}.ssh\n",
)
_change_passwords(root, target_username)
return public_key_path, target_authorized_keys
def configure_raspberry_image(plan: ImagePlan, session: ImageSession) -> None:
_ensure_image_mounted(session)
root = session.root_mount_path
_seed_boot_uuid(session)
public_key_path, target_authorized_keys = _configure_users_and_keys(session)
_configure_hostname(root)
if plan.distribution in ("arch", "manjaro"):
ui.info("Populating keys...")
chroot_bash(
root,
"yes | pacman-key --init\nyes | pacman-key --populate archlinuxarm\n",
)
_update_system(plan, root)
ui.info(f"Installing software for filesystem {plan.root_filesystem}...")
if plan.root_filesystem == "btrfs":
install_packages(plan.distribution, root, "btrfs-progs")
else:
ui.info("Skipped.")
if plan.encrypt_system:
configure_encryption(plan, session, target_authorized_keys)
ui.info("Running system specific procedures...")
if plan.distribution == "retropie":
_run_retropie_procedures(session, public_key_path)

206
lim/image/session.py Normal file
View File

@@ -0,0 +1,206 @@
"""Mount/unmount lifecycle for working on an image on a block device."""
import time
from pathlib import Path
from lim import device as device_module
from lim import runner, ui
from lim.device import Device
from lim.errors import LimError
class ImageSession:
"""Working folder, partition paths and (chroot) mounts for one device.
Call ``destructor()`` in a finally block: it unmounts and removes
everything best-effort, mirroring partially completed setups.
"""
def __init__(self, device: Device) -> None:
self.device = device
self.working_folder: Path | None = None
self.boot_mount_path: Path | None = None
self.root_mount_path: Path | None = None
self.boot_partition_path = device.partition(1)
self.root_partition_path = device.partition(2)
# Overridden by decrypt_root() when the root partition is LUKS.
self.root_mapper_path = self.root_partition_path
self.root_mapper_name: str | None = None
self.boot_partition_uuid: str | None = None
self.root_partition_uuid: str | None = None
def make_working_folder(self) -> None:
# Mount points must live on the root fs; mkdir() fails on collisions.
self.working_folder = Path(f"/tmp/linux-image-manager-{int(time.time())}") # noqa: S108
ui.info(f"Create temporary working folder in {self.working_folder}")
self.working_folder.mkdir(parents=True)
def make_mount_folders(self) -> None:
if self.working_folder is None:
self.make_working_folder()
ui.info("Preparing mount paths...")
self.boot_mount_path = self.working_folder / "boot"
self.root_mount_path = self.working_folder / "root"
self.boot_mount_path.mkdir()
self.root_mount_path.mkdir()
def root_is_luks(self) -> bool:
return device_module.blkid_value(self.root_partition_path, "TYPE") == "crypto_LUKS"
def read_partition_uuids(self) -> None:
"""Fetch partition UUIDs via blkid; derive the LUKS mapper name if unset."""
self.root_partition_uuid = device_module.blkid_value(self.root_partition_path, "UUID")
self.boot_partition_uuid = device_module.blkid_value(self.boot_partition_path, "UUID")
if self.root_mapper_name is None and self.root_is_luks():
# Same deterministic name decrypt_root() would have used.
self.root_mapper_name = f"linux-image-manager-{self.root_partition_uuid}"
self.root_mapper_path = f"/dev/mapper/{self.root_mapper_name}"
def decrypt_root(self) -> None:
if not self.root_is_luks():
return
self.root_partition_uuid = device_module.blkid_value(self.root_partition_path, "UUID")
self.root_mapper_name = f"linux-image-manager-{self.root_partition_uuid}"
self.root_mapper_path = f"/dev/mapper/{self.root_mapper_name}"
ui.info(f"Decrypting of {self.root_partition_path} is neccessary...")
runner.run(
[
"cryptsetup",
"-v",
"luksOpen",
self.root_partition_path,
self.root_mapper_name,
],
sudo=True,
)
def mount_partitions(self) -> None:
if self.boot_mount_path is None:
raise LimError("Mount folders are not prepared yet.")
ui.info("Mount boot and root partition...")
runner.run(
["mount", "-v", self.boot_partition_path, str(self.boot_mount_path)],
sudo=True,
)
runner.run(["mount", "-v", self.root_mapper_path, str(self.root_mount_path)], sudo=True)
ui.info("Setting uuid variables...")
self.read_partition_uuids()
ui.info("The following mounts refering this setup exist:")
runner.run(["findmnt", "-R", str(self.working_folder)], check=False)
def boot_bind_target(self) -> str:
"""Where the FAT partition belongs inside the chroot.
Raspberry Pi OS Bookworm keeps it at /boot/firmware (the raspi-firmware
post-update hook syncs the initramfs there); older layouts use /boot.
Binding it wrong sends update-initramfs's output to the ext4 root, so the
firmware boots a stock initramfs without cryptsetup -> initramfs shell.
"""
if (self.root_mount_path / "boot/firmware").is_dir():
return f"{self.root_mount_path}/boot/firmware"
return f"{self.root_mount_path}/boot"
def mount_chroot_binds(self) -> None:
ui.info("Mount chroot environments...")
root = self.root_mount_path
runner.run(
["mount", "--bind", str(self.boot_mount_path), self.boot_bind_target()], sudo=True
)
runner.run(["mount", "--bind", "/dev", f"{root}/dev"], sudo=True)
runner.run(["mount", "--bind", "/sys", f"{root}/sys"], sudo=True)
runner.run(["mount", "--bind", "/proc", f"{root}/proc"], sudo=True)
runner.run(["mount", "--bind", "/dev/pts", f"{root}/dev/pts"], sudo=True)
def copy_qemu(self) -> None:
ui.info("Copy qemu binary...")
runner.run(
["cp", "-v", "/usr/bin/qemu-arm-static", f"{self.root_mount_path}/usr/bin/"],
sudo=True,
)
def copy_resolv_conf(self) -> None:
ui.info("Copy resolv.conf...")
copied = runner.run(
[
"cp",
"--remove-destination",
"-v",
"/etc/resolv.conf",
f"{self.root_mount_path}/etc/",
],
sudo=True,
check=False,
)
if copied.returncode != 0:
ui.warning("Failed. Probably there is no internet connection available.")
def _umount(self, path: str, *, lazy: bool = False) -> None:
flags = ["-lv"] if lazy else ["-v"]
result = runner.run(["umount", *flags, path], sudo=True, check=False)
if result.returncode != 0:
ui.warning(f"Umounting {path} failed!")
def _rmdir(self, path: Path | None) -> None:
if path is None:
return
result = runner.run(["rmdir", "-v", str(path)], sudo=True, check=False)
if result.returncode != 0:
ui.warning(f"Removing {path} failed!")
def destructor(self) -> None:
ui.info("Cleaning up...")
ui.info("Unmounting everything...")
root = self.root_mount_path
if root is not None:
self._umount(f"{root}/dev/pts", lazy=True)
self._umount(f"{root}/dev", lazy=True)
self._umount(f"{root}/proc")
self._umount(f"{root}/sys")
self._umount(self.boot_bind_target())
self._umount(str(root))
if self.boot_mount_path is not None:
self._umount(str(self.boot_mount_path))
ui.info("Deleting mount folders...")
self._rmdir(self.root_mount_path)
self._rmdir(self.boot_mount_path)
self._rmdir(self.working_folder)
if self.root_mapper_name and self.root_is_luks():
ui.info(f"Trying to close decrypted {self.root_mapper_name}...")
closed = runner.run(
["cryptsetup", "-v", "luksClose", self.root_mapper_name],
sudo=True,
check=False,
)
if closed.returncode != 0:
ui.warning("Failed.")
# The host PATH leaks into the chroot; force one that includes the sbin dirs
# where Debian keeps update-initramfs, useradd, chpasswd, ... (else code 127).
_CHROOT_PATH = "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
def chroot_bash(root_mount_path: Path | str, script: str, error_msg: str | None = None) -> None:
"""Run a bash script inside the image via chroot (with a Debian-safe PATH)."""
runner.run(
["chroot", str(root_mount_path), "/bin/bash"],
input_text=f"export PATH={_CHROOT_PATH}\n{script}",
sudo=True,
error_msg=error_msg,
)
def install_packages(distribution: str, root_mount_path: Path, package_names: str) -> None:
"""Install packages inside the image with the distribution's package manager."""
ui.info(f"Installing {package_names}...")
if distribution in ("arch", "manjaro"):
chroot_bash(root_mount_path, f"pacman --noconfirm -Sy --needed {package_names}")
elif distribution in ("moode", "retropie", "raspios"):
# apt-get update first: a stock image ships stale lists whose old .deb
# URLs 404 once the mirror has superseded them.
chroot_bash(
root_mount_path,
f"apt-get update\nDEBIAN_FRONTEND=noninteractive apt-get install -y {package_names}",
)
else:
raise LimError("Package manager not supported.")

97
lim/image/setup.py Normal file
View File

@@ -0,0 +1,97 @@
"""Interactive Linux image setup: download, verify, transfer, configure.
Reference for encrypted Raspberry Pi images:
https://wiki.polaire.nl/doku.php?id=archlinux-raspberry-encrypted
"""
import pwd
from pathlib import Path
from lim import device as device_module
from lim import system, ui
from lim.errors import LimError
from lim.image import choosers, transfer, verify
from lim.image.plan import ImagePlan
from lim.image.raspberry import configure_raspberry_image
from lim.image.session import ImageSession
def _prepare_image_folder(plan: ImagePlan) -> None:
ui.info("Configure user...")
origin_username = ui.ask("Please type in a valid working username:")
try:
pwd.getpwnam(origin_username)
except KeyError:
raise LimError(f"User {origin_username} doesn't exist.") from None
ui.info("Image routine starts...")
plan.image_folder = Path(f"/home/{origin_username}/Software/Images")
ui.info(f'The images will be stored in "{plan.image_folder}".')
if not plan.image_folder.is_dir():
ui.info(f'Folder "{plan.image_folder}" doesn\'t exist. It will be created now.')
plan.image_folder.mkdir(parents=True)
def _select_unmounted_device() -> device_module.Device:
device = device_module.select_device()
if device_module.is_mounted(device.path):
raise LimError(
f'Device {device.path} is allready mounted. Umount with "umount {device.path}*".'
)
return device
def _choose_and_verify_image(plan: ImagePlan) -> None:
plan.operation_system = ui.ask(
"Which operation system would you like to use [linux,windows,...]?"
)
if plan.operation_system == "linux":
choosers.choose_linux_image(plan)
plan.encrypt_system = ui.confirm("Should the system be encrypted?")
if plan.encrypt_system:
plan.tor_unlock = ui.confirm(
"Should the system be remotely unlockable via a Tor onion service?"
)
ui.info("Generating os-image...")
transfer.download_image(plan)
else:
choosers.choose_local_image(plan)
ui.info("Verifying image...")
ui.info("Verifying checksum...")
if plan.image_checksum is None and plan.download_url is not None:
plan.image_checksum = verify.resolve_checksum(plan.download_url)
verify.verify_checksum(plan.image_path, plan.image_checksum)
if plan.download_url is not None:
verify.verify_signature(plan.download_url, plan.image_path, plan.image_folder)
def run_setup() -> None:
ui.info("Setupscript for images started...")
ui.info("Checking if root...")
if not system.is_root():
raise LimError("This script must be executed as root!")
plan = ImagePlan()
_prepare_image_folder(plan)
device = _select_unmounted_device()
_choose_and_verify_image(plan)
session = ImageSession(device)
try:
session.make_working_folder()
session.make_mount_folders()
plan.root_filesystem = ui.ask("Which filesystem should be used? E.g.:btrfs,ext4... (none):")
if ui.confirm(f"Should the image be transfered to {device.path}?"):
transfer.transfer_image(plan, session)
else:
ui.info("Skipping image transfer...")
if plan.raspberry_pi_version:
configure_raspberry_image(plan, session)
finally:
session.destructor()
ui.success("Setup successfull :)")

236
lim/image/transfer.py Normal file
View File

@@ -0,0 +1,236 @@
"""Download the selected image and transfer it onto the target device."""
import shutil
from pathlib import Path
from lim import device as device_module
from lim import runner, ui
from lim.errors import LimError
from lim.image import loopimg
from lim.image.plan import DEFAULT_BOOT_SIZE, ImagePlan
from lim.image.session import ImageSession
def download_image(plan: ImagePlan, *, force_prompt: bool = True) -> None:
if force_prompt and ui.confirm("Should the image download be forced?"):
if plan.image_path.is_file():
ui.info(f"Removing image {plan.image_path}.")
plan.image_path.unlink()
else:
ui.info(f"Forcing download wasn't neccessary. File {plan.image_path} doesn't exist.")
ui.info("Start Download procedure...")
if plan.image_path.is_file():
ui.info("Image exist local. Download skipped.")
return
ui.info(f'Image "{plan.image_name}" doesn\'t exist under local path "{plan.image_path}".')
ui.info(f'Image "{plan.image_name}" gets downloaded from "{plan.download_url}"...')
runner.run(
["wget", plan.download_url, "-O", str(plan.image_path)],
error_msg=f'Download from "{plan.download_url}" failed.',
)
def arch_partition_input(boot_size: str) -> str:
"""Fdisk answers: FAT32 boot partition of boot_size plus root partition."""
return (
"o\n" # clear out any partitions on the drive
"p\n" # list partitions (should be empty)
"n\np\n1\n\n" # new primary partition 1, default start sector
f"{boot_size}\n"
"t\nc\n" # set partition 1 to type W95 FAT32 (LBA)
"n\np\n2\n\n\n" # new primary partition 2 over the remaining space
"w\n" # write partition table
)
def decompress_command(image_path: Path) -> list[str]:
"""Build the command that streams the raw image to stdout for dd."""
suffix = image_path.suffix
if suffix == ".zip":
return ["unzip", "-p", str(image_path)]
if suffix == ".gz":
return ["gunzip", "-c", str(image_path)]
if suffix == ".iso":
return ["pv", str(image_path)]
if suffix == ".xz":
return ["unxz", "-c", str(image_path)]
raise LimError(f'Image transfer for "{image_path.name}" is not supported yet!')
def _luks_format_root(plan: ImagePlan, session: ImageSession) -> None:
luks_format = [
"cryptsetup",
"-v",
"luksFormat",
"-c",
"aes-xts-plain64",
"-s",
"512",
"-h",
"sha512",
"--use-random",
"-i",
"1000",
]
if plan.luks_memory_cost:
ui.info(
f"Formating {session.root_partition_path} with LUKS "
f"with --pbkdf-memory set to {plan.luks_memory_cost}"
)
luks_format += ["--pbkdf-memory", plan.luks_memory_cost]
else:
ui.info(f"Formating {session.root_partition_path} with LUKS")
runner.run([*luks_format, session.root_partition_path], sudo=True)
def transfer_arch_image(plan: ImagePlan, session: ImageSession) -> None:
boot_size = plan.boot_size or DEFAULT_BOOT_SIZE
ui.info(f"The boot partition will be set to {boot_size}.")
ui.info("Creating partitions...")
runner.run(
["fdisk", session.device.path],
input_text=arch_partition_input(boot_size),
sudo=True,
)
ui.info("Format boot partition...")
runner.run(["mkfs.vfat", session.boot_partition_path], sudo=True)
if plan.encrypt_system:
_luks_format_root(plan, session)
session.decrypt_root()
ui.info("Format root partition...")
runner.run([f"mkfs.{plan.root_filesystem}", "-f", session.root_mapper_path], sudo=True)
session.mount_partitions()
ui.info("Root files will be transfered to device...")
runner.run(
["bsdtar", "-xpf", str(plan.image_path), "-C", str(session.root_mount_path)],
sudo=True,
)
runner.sync_disks()
ui.info("Boot files will be transfered to device...")
boot_source = session.root_mount_path / "boot"
for entry in sorted(boot_source.iterdir()):
runner.run(["mv", "-v", str(entry), str(session.boot_mount_path)], sudo=True)
def _create_encrypted_layout(plan: ImagePlan, session: ImageSession) -> None:
"""Partition the target, LUKS-format the root, format and mount both."""
boot_size = plan.boot_size or DEFAULT_BOOT_SIZE
ui.info(f"Creating partitions (boot {boot_size})...")
runner.run(
["fdisk", session.device.path],
input_text=arch_partition_input(boot_size),
sudo=True,
)
runner.run(["mkfs.vfat", session.boot_partition_path], sudo=True)
_luks_format_root(plan, session)
session.decrypt_root()
runner.run([f"mkfs.{plan.root_filesystem}", "-F", session.root_mapper_path], sudo=True)
session.mount_partitions()
def _copy_tree(source: str, target: str, flags: list[str]) -> None:
"""Copy a directory's contents, showing progress when rsync is available."""
if shutil.which("rsync"):
runner.run(["rsync", *flags, "--info=progress2", f"{source}/", f"{target}/"], sudo=True)
else:
runner.run(["cp", "-a", f"{source}/.", target], sudo=True)
def _fix_boot_fstab(session: ImageSession) -> None:
"""Repoint the stock boot fstab line at the freshly created boot partition.
The copied image mounts /boot(/firmware) by the stock PARTUUID, which no
longer exists after repartitioning; swap in the new boot UUID, mount point
unchanged. No-op when the image carries no boot line.
"""
fstab = session.root_mount_path / "etc/fstab"
if not fstab.is_file():
return
lines = []
for line in fstab.read_text().splitlines():
fields = line.split()
if not line.lstrip().startswith("#") and fields[1:2] and fields[1].startswith("/boot"):
fields[0] = f"UUID={session.boot_partition_uuid}"
lines.append(" ".join(fields))
else:
lines.append(line)
fstab.write_text("".join(f"{entry}\n" for entry in lines))
def transfer_disk_image(plan: ImagePlan, session: ImageSession) -> None:
"""Copy a full disk image (boot + root partitions) into a fresh encrypted layout.
Distributions that ship a partitioned .img rather than a rootfs tarball
(Raspberry Pi OS, moode, RetroPie, Manjaro ARM) are loop-mounted and their
first two partitions copied into a new LUKS container.
"""
raw = plan.image_path.with_suffix("")
ui.info(f"Decompressing {plan.image_path} to {raw}...")
runner.pipeline(
decompress_command(plan.image_path),
["dd", f"of={raw}", "bs=4M", "conv=fsync", "status=progress"],
error_msg=f"Decompressing {plan.image_path} failed.",
)
loop = loopimg.attach(raw)
source_boot = session.working_folder / "source-boot"
source_root = session.working_folder / "source-root"
try:
source_boot.mkdir()
source_root.mkdir()
runner.run(["mount", "-o", "ro", loopimg.partition(loop, 1), str(source_boot)], sudo=True)
runner.run(["mount", "-o", "ro", loopimg.partition(loop, 2), str(source_root)], sudo=True)
_create_encrypted_layout(plan, session)
ui.info("Copying the root filesystem into the encrypted container...")
_copy_tree(str(source_root), str(session.root_mount_path), ["-aHAX"])
ui.info("Copying the boot partition...")
_copy_tree(str(source_boot), str(session.boot_mount_path), ["-a"])
runner.sync_disks()
_fix_boot_fstab(session)
finally:
runner.run(["umount", "-v", str(source_boot)], sudo=True, check=False)
runner.run(["umount", "-v", str(source_root)], sudo=True, check=False)
loopimg.detach(loop)
for path in (source_boot, source_root):
runner.run(["rmdir", str(path)], sudo=True, check=False)
def transfer_image(plan: ImagePlan, session: ImageSession, *, interactive: bool = True) -> None:
if interactive:
if ui.confirm(f"Should the partition table of {session.device.path} be deleted?"):
ui.info("Deleting...")
runner.run(["wipefs", "-a", session.device.path], sudo=True)
else:
ui.info("Skipping partition table deletion...")
device_module.overwrite_device(session.device)
ui.info("Starting image transfer...")
if plan.distribution == "arch":
transfer_arch_image(plan, session)
return
if plan.encrypt_system:
# Non-tarball distros ship a full .img; copy it into a LUKS container.
transfer_disk_image(plan, session)
return
blocksize = session.device.optimal_blocksize
ui.info(f"Transfering {plan.image_path.suffix} file...")
runner.pipeline(
decompress_command(plan.image_path),
[
"dd",
f"of={session.device.path}",
f"bs={blocksize}",
"conv=fsync",
"status=progress",
],
sudo_last=True,
error_msg=f"DD {plan.image_path} to {session.device.path} failed.",
)
runner.sync_disks()

100
lim/image/unlock.py Normal file
View File

@@ -0,0 +1,100 @@
"""Remote LUKS boot unlock over the initramfs SSH/dropbear session.
Reaches the waiting initramfs either through a Tor onion address (wrapped in
torsocks) or a plain host/IP, then either runs cryptroot-unlock (initramfs-tools
targets) or drops into the passphrase prompt (mkinitcpio encryptssh).
"""
import json
import os
from pathlib import Path
from lim import runner, ui
from lim.errors import LimError
RECORDS_SUBPATH = ".config/lim/unlocks"
def records_dir(home: Path | None = None) -> Path:
return (home or Path.home()) / RECORDS_SUBPATH
def build_unlock_argv(target: str, key: str, unlock_command: str) -> list[str]:
"""SSH invocation for the unlock; torsocks-wrapped for .onion targets.
The initramfs dropbear key differs from the booted host key, so host-key
checking is disabled; a v3 onion address authenticates the endpoint itself.
"""
ssh = ["ssh", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null"]
if key:
ssh += ["-i", str(Path(key).expanduser())]
if unlock_command:
ssh += ["-t"] # PTY so the remote passphrase prompt is interactive
ssh += [f"root@{target}"]
if unlock_command:
ssh += [unlock_command]
if target.endswith(".onion"):
return ["torsocks", *ssh]
return ssh
def save_record(home: Path, uid: int, gid: int, name: str, record: dict) -> None:
"""Persist an unlock target under the target user's home, owned by them."""
directory = records_dir(home)
directory.mkdir(parents=True, exist_ok=True)
path = directory / f"{name}.json"
path.write_text(json.dumps(record, indent=2) + "\n")
try:
for entry in (path, directory, directory.parent, directory.parent.parent):
os.chown(entry, uid, gid)
except OSError:
pass
def _read_record(path: Path) -> dict | None:
try:
return json.loads(path.read_text())
except (OSError, ValueError):
return None
def _load_records() -> list[tuple[str, dict]]:
directory = records_dir()
if not directory.is_dir():
return []
loaded = ((path.stem, _read_record(path)) for path in sorted(directory.glob("*.json")))
return [(name, record) for name, record in loaded if record is not None]
def _select_record() -> dict | None:
records = _load_records()
if not records:
return None
ui.info("Saved unlock targets:")
for index, (name, record) in enumerate(records, 1):
ui.info(f" {index}) {name} -> {record.get('target')}")
answer = ui.ask("Pick a number, or Enter for manual entry:")
if answer.isdigit() and 1 <= int(answer) <= len(records):
return records[int(answer) - 1][1]
return None
def remote_unlock() -> None:
record = _select_record()
if record:
target = record["target"]
key = record.get("key", "")
unlock_command = record.get("unlock_command", "")
else:
target = ui.ask("Onion address or host/IP to unlock:")
if not target:
raise LimError("No target given.")
key = ui.ask("Path to the SSH private key (empty for ssh defaults):")
unlock_command = (
"cryptroot-unlock"
if ui.confirm("Debian / Raspberry Pi OS target (run cryptroot-unlock)?")
else ""
)
if target.endswith(".onion"):
ui.info("Routing through Tor (torsocks); a local tor must be running.")
runner.run(build_unlock_argv(target, key, unlock_command), check=False)

98
lim/image/verify.py Normal file
View File

@@ -0,0 +1,98 @@
"""Image integrity (checksums) and authenticity (GPG signatures) checks."""
import hashlib
import re
from pathlib import Path
from lim import runner, ui
from lim.errors import LimError
ALGORITHM_BY_DIGEST_LENGTH = {
32: "md5",
40: "sha1",
64: "sha256",
128: "sha512",
}
def url_exists(url: str) -> bool:
return runner.succeeds(["wget", "-q", "--method=HEAD", url])
def resolve_checksum(download_url: str) -> str | None:
"""Try to fetch a published checksum next to the image download."""
for extension in ("sha1", "sha512", "md5"):
checksum_url = f"{download_url}.{extension}"
ui.info(
f"Image Checksum is not defined. Try to download image signature from {checksum_url}."
)
if url_exists(checksum_url):
content = runner.output(["wget", checksum_url, "-q", "-O", "-"])
checksum = content.split()[0] if content.split() else ""
if checksum:
ui.info(f"Defined image_checksum as {checksum}")
return checksum
ui.warning(f"No checksum found under {checksum_url}.")
return None
def verify_checksum(image_path: str | Path, checksum: str | None) -> None:
"""Verify the image against an md5/sha1/sha256/sha512 hex checksum."""
if not checksum:
ui.warning("No checksum is defined. Skipping checksum verification.")
return
algorithm = ALGORITHM_BY_DIGEST_LENGTH.get(len(checksum))
if algorithm is None:
raise LimError(
f"Checksum '{checksum}' has no recognized digest length "
"(expected md5, sha1, sha256 or sha512)."
)
ui.info(f"Checking {algorithm} checksum...")
digest = hashlib.new(algorithm)
with Path(image_path).open("rb") as handle:
while chunk := handle.read(1024 * 1024):
digest.update(chunk)
if digest.hexdigest() != checksum.lower():
raise LimError("Verification failed. HINT: Force the download of the image.")
ui.info(f"{algorithm} checksum verified.")
def verify_signature(download_url: str, image_path: str | Path, image_folder: Path) -> None:
"""Best-effort GPG signature verification of the downloaded image."""
ui.info("Note: Checksums verify integrity but do not confirm authenticity.")
ui.info(
"Proceeding to signature verification, which ensures the file comes from a trusted source."
)
signature_url = f"{download_url}.sig"
ui.info(f"Attempting to download the image signature from: {signature_url}")
if not url_exists(signature_url):
ui.warning(f"No signature found under {signature_url}.")
return
signature_path = image_folder / (Path(str(image_path)).name + ".sig")
if not runner.succeeds(["wget", "-q", "-O", str(signature_path), signature_url]):
ui.warning("Failed to download the signature file.")
return
ui.info("Extract the key ID from the signature file")
verification = runner.output(
["gpg", "--status-fd", "1", "--verify", str(signature_path), str(image_path)],
check=False,
)
missing_keys = re.findall(r"NO_PUBKEY (\S+)", verification)
if missing_keys:
key_id = missing_keys[-1]
if runner.succeeds(["gpg", "--list-keys", key_id]):
ui.info(f"Key {key_id} already in keyring.")
else:
ui.info("Import the public key")
runner.run(
["gpg", "--keyserver", "keyserver.ubuntu.com", "--recv-keys", key_id],
check=False,
)
ui.info("Verify the signature")
if runner.succeeds(["gpg", "--verify", str(signature_path), str(image_path)]):
ui.info("Signature verification succeeded.")
else:
ui.warning("Signature verification failed.")

206
lim/image/wizard.py Normal file
View File

@@ -0,0 +1,206 @@
"""Guided setup: an encrypted, Tor-remote-unlockable Linux image on a device.
All questions are asked up front (``_collect``); the build then runs unattended
(``_execute``): pick a distribution and target block device (the internal SSD of
a USB-booted Pi, or a spare stick to clone later), download the image, copy it
into a LUKS container, create the login user, and bake the initramfs Tor onion
unlock. Works for any distribution with an initramfs backend (Arch/Manjaro via
mkinitcpio, Raspberry Pi OS/moode/RetroPie via initramfs-tools).
"""
import os
import pwd
import shlex
from dataclasses import dataclass
from pathlib import Path
from lim import device as device_module
from lim import ui
from lim.errors import LimError
from lim.image import choosers, crossarch, transfer, unlock
from lim.image.encryption import configure_encryption
from lim.image.plan import ImagePlan
from lim.image.session import ImageSession, chroot_bash
@dataclass
class _Answers:
host_user: str
device: device_module.Device
pubkey: Path
hostname: str
login_user: str
login_password: str
def _intro() -> None:
ui.info("Guided encrypted Linux image setup with Tor remote unlock.")
ui.info("The TARGET must be an attached block device:")
ui.info(" - internal SSD: boot the Pi from USB, then target /dev/sda or /dev/nvme0n1")
ui.info(" - golden image: target a spare USB stick, clone it onto the SSD afterwards")
ui.warning("The target device will be COMPLETELY ERASED.")
def _prepare_image_folder(plan: ImagePlan) -> str:
"""Cache downloads under the invoking (sudo) user's home; return that user."""
host_user = os.environ.get("SUDO_USER") or "root"
try:
home = Path(pwd.getpwnam(host_user).pw_dir)
except KeyError:
host_user, home = "root", Path("/root")
plan.image_folder = home / "Software/Images"
plan.image_folder.mkdir(parents=True, exist_ok=True)
ui.info(f"Images are cached in {plan.image_folder}.")
return host_user
def _select_target() -> device_module.Device:
device = device_module.select_device()
if device_module.is_mounted(device.path):
raise LimError(f"{device.path} is mounted. Unmount it first (umount {device.path}*).")
ui.warning(f"Everything on {device.path} will be destroyed.")
answer = ui.ask(f"Retype the device name ({device.name}) to confirm ERASE of {device.path}:")
if answer.strip().removeprefix("/dev/") != device.name:
raise LimError("Confirmation did not match. Aborting.")
return device
def _ask_pubkey() -> Path:
answer = ui.ask("Path to the SSH PUBLIC key (unlocks AND logs in):").strip()
if not answer:
raise LimError("No SSH public key path given.")
pubkey = Path(answer).expanduser()
if not pubkey.is_file():
raise LimError(f"SSH public key {pubkey} not found.")
return pubkey
def _collect(plan: ImagePlan) -> _Answers | None:
"""Ask everything up front so ``_execute`` can run without prompts."""
_intro()
if not ui.confirm("Continue?"):
return None
choosers.choose_linux_image(plan)
plan.root_filesystem = plan.root_filesystem or "ext4"
device = _select_target()
hostname = ui.ask("Hostname for the system:").strip()
login_user = ui.ask("Login username to create on the system (default pi):").strip() or "pi"
login_password = ui.ask(f"Login password for {login_user} (empty for key-only login):")
pubkey = _ask_pubkey()
host_user = _prepare_image_folder(plan)
crossarch.ensure_ready() # may install qemu; done here, before the erase in _execute
ui.success("All questions answered — the build now runs unattended.")
return _Answers(host_user, device, pubkey, hostname, login_user, login_password)
def _build_authorized_keys(session: ImageSession, pubkey: Path) -> Path:
authorized_keys = session.working_folder / "authorized_keys"
authorized_keys.write_text(pubkey.read_text())
return authorized_keys
def _apply_system_config(session: ImageSession, hostname: str) -> str:
"""Write the hostname and enable the post-boot SSH server; return the hostname."""
root = session.root_mount_path
if hostname:
(root / "etc/hostname").write_text(hostname + "\n")
# RPi OS enables sshd on first boot when this marker exists on the boot fs.
(session.boot_mount_path / "ssh").write_text("")
return (root / "etc/hostname").read_text().strip()
def _configure_login_user(session: ImageSession, user: str, password: str, pubkey: Path) -> None:
"""Rename a stock default user to the chosen one (or create it); add sudo + key.
Renaming pi/alarm reuses the account (uid, sudoers) and leaves no lingering
default; a fresh useradd covers images without one (e.g. RPi OS Bookworm).
"""
key = shlex.quote(pubkey.read_text().strip())
ui.info(f"Configuring login user {user}...")
script = f"""\
target={shlex.quote(user)}
for old in pi alarm; do
id "$old" >/dev/null 2>&1 || continue
[ "$old" = "$target" ] && continue
id "$target" >/dev/null 2>&1 && continue
usermod -l "$target" "$old"
usermod -d "/home/$target" -m "$target"
groupmod -n "$target" "$old" 2>/dev/null || true
break
done
id "$target" >/dev/null 2>&1 || useradd -m -s /bin/bash "$target"
usermod -aG sudo "$target" 2>/dev/null || true
install -d -m 700 "/home/$target/.ssh"
printf '%s\\n' {key} > "/home/$target/.ssh/authorized_keys"
chmod 600 "/home/$target/.ssh/authorized_keys"
chown -R "$target:$target" "/home/$target/.ssh"
"""
if password:
script += f"printf '%s' {shlex.quote(f'{user}:{password}')} | chpasswd\n"
chroot_bash(session.root_mount_path, script)
def _save_unlock_record(
plan: ImagePlan, host_user: str, hostname: str, onion: str | None, pubkey: Path
) -> None:
if not onion:
return
private_key = str(pubkey).removesuffix(".pub")
unlock_command = "" if plan.distribution in ("arch", "manjaro") else "cryptroot-unlock"
record = {
"target": onion,
"key": private_key,
"unlock_command": unlock_command,
"hostname": hostname,
}
entry = pwd.getpwnam(host_user)
try:
unlock.save_record(
Path(entry.pw_dir), entry.pw_uid, entry.pw_gid, hostname or "device", record
)
ui.info("Saved unlock target; run later with: lim --type remote-unlock")
except OSError as exc:
ui.warning(f"Could not save unlock record: {exc}")
def _print_summary(answers: _Answers, hostname: str, onion: str | None) -> None:
ui.success("Encrypted Linux image is ready.")
ui.info(f"Target device : {answers.device.path} (hostname: {hostname})")
if onion:
ui.info(f"Onion address : {onion}")
ui.info(f"Unlock later : lim --type remote-unlock (or: torsocks ssh root@{onion})")
ui.info(f"After unlock, log in: ssh {answers.login_user}@<the box>")
ui.info("Golden image? Clone this device onto the SSD, then grow it:")
ui.info(" parted <ssd> resizepart 2 100% && cryptsetup resize <mapper> && resize2fs <mapper>")
def _execute(plan: ImagePlan, answers: _Answers) -> None:
transfer.download_image(plan, force_prompt=False)
session = ImageSession(answers.device)
authorized_keys = None
try:
session.make_working_folder()
session.make_mount_folders()
transfer.transfer_image(plan, session, interactive=False)
authorized_keys = _build_authorized_keys(session, answers.pubkey)
hostname = _apply_system_config(session, answers.hostname)
session.mount_chroot_binds()
session.copy_resolv_conf()
onion = configure_encryption(plan, session, authorized_keys)
_configure_login_user(session, answers.login_user, answers.login_password, answers.pubkey)
_save_unlock_record(plan, answers.host_user, hostname, onion, answers.pubkey)
_print_summary(answers, hostname, onion)
finally:
if authorized_keys is not None:
authorized_keys.unlink(missing_ok=True)
session.destructor()
def run_guided() -> None:
ui.header()
plan = ImagePlan(encrypt_system=True, tor_unlock=True)
answers = _collect(plan)
if answers is None:
ui.info("Aborted.")
return
_execute(plan, answers)

74
lim/luks.py Normal file
View File

@@ -0,0 +1,74 @@
"""LUKS key management plus crypttab/fstab bookkeeping."""
import re
from pathlib import Path
from lim import fsutil, runner, ui
from lim.errors import LimError
LUKS_KEY_DIRECTORY = Path("/etc/luks-keys")
CRYPTTAB_PATH = Path("/etc/crypttab")
FSTAB_PATH = Path("/etc/fstab")
def luks_uuid(partition_path: str) -> str:
dump = runner.output(["cryptsetup", "luksDump", partition_path], sudo=True)
match = re.search(r"UUID:\s*(\S+)", dump)
if not match:
raise LimError(f"Could not read LUKS UUID of {partition_path}.")
return match.group(1)
def create_luks_key_and_update_crypttab(
mapper_name: str,
partition_path: str,
*,
key_directory: Path = LUKS_KEY_DIRECTORY,
crypttab_path: Path = CRYPTTAB_PATH,
) -> None:
"""Generate a random keyfile, register it with LUKS and /etc/crypttab."""
ui.info("Creating luks-key-directory...")
key_directory.mkdir(parents=True, exist_ok=True)
secret_key_path = key_directory / f"{mapper_name}.keyfile"
ui.info(f"Generate secret key under: {secret_key_path}")
if secret_key_path.exists():
ui.warning("File already exists. Overwriting!")
runner.run(
["dd", "if=/dev/urandom", f"of={secret_key_path}", "bs=512", "count=8"],
sudo=True,
)
runner.sync_disks()
ui.info("Opening and closing device to verify that everything works fine...")
closed = runner.run(["cryptsetup", "-v", "luksClose", mapper_name], sudo=True, check=False)
if closed.returncode != 0:
ui.info(f"No need to luksClose {mapper_name}. Device isn't open.")
runner.run(["cryptsetup", "luksAddKey", partition_path, str(secret_key_path)], sudo=True)
runner.run(
[
"cryptsetup",
"-v",
"luksOpen",
partition_path,
mapper_name,
f"--key-file={secret_key_path}",
],
sudo=True,
)
runner.run(["cryptsetup", "-v", "luksClose", mapper_name], sudo=True)
ui.info("Reading UUID...")
uuid = luks_uuid(partition_path)
entry = f"{mapper_name} UUID={uuid} {secret_key_path} luks"
ui.info("Adding crypttab entry...")
fsutil.ensure_line_in_file(entry, crypttab_path)
ui.info(f"The file {crypttab_path} contains now the following:")
print(crypttab_path.read_text())
def update_fstab(mapper_path: str, mount_path: str, *, fstab_path: Path = FSTAB_PATH) -> None:
entry = f"{mapper_path} {mount_path} btrfs defaults 0 2"
ui.info("Adding fstab entry...")
fsutil.ensure_line_in_file(entry, fstab_path)
ui.info(f"The file {fstab_path} contains now the following:")
print(fstab_path.read_text())

18
lim/packages.py Normal file
View File

@@ -0,0 +1,18 @@
"""Reads package collections from configuration/packages/."""
from lim import config
from lim.errors import LimError
def get_packages(*collections: str) -> list[str]:
"""Package names from the given collections, comments stripped."""
names: list[str] = []
for collection in collections:
path = config.PACKAGE_PATH / f"{collection}.txt"
if not path.is_file():
raise LimError(f"Package collection {path} does not exist.")
for line in path.read_text().splitlines():
name = line.split("#", 1)[0].strip()
if name:
names.append(name)
return names

132
lim/runner.py Normal file
View File

@@ -0,0 +1,132 @@
"""Thin wrapper around subprocess for all external commands.
Every module calls external tools through this module so tests can
replace ``run``/``output``/``succeeds``/``pipeline`` with fakes.
Missing binaries never surface as tracebacks: actions (``check=True``,
``pipeline``) abort with a clear LimError, tolerated calls
(``check=False``, ``succeeds``) degrade to a warning plus shell-style
exit code 127.
"""
import os
import shlex
import subprocess
from lim import ui
from lim.errors import LimError
COMMAND_NOT_FOUND = 127
def _with_sudo(cmd: list[str], *, sudo: bool) -> list[str]:
parts = [str(part) for part in cmd]
if sudo and os.geteuid() != 0:
return ["sudo", *parts]
return parts
def _not_found_message(cmd: list[str]) -> str:
return f"Command not found: {cmd[0]} — please install it."
def run(
cmd: list[str],
*,
sudo: bool = False,
input_text: str | None = None,
check: bool = True,
error_msg: str | None = None,
) -> subprocess.CompletedProcess:
"""Run a command inheriting stdio so progress output stays visible."""
cmd = _with_sudo(cmd, sudo=sudo)
ui.info(f"Running: {shlex.join(cmd)}")
try:
result = subprocess.run(cmd, input=input_text, text=True, check=False)
except FileNotFoundError:
if check:
raise LimError(error_msg or _not_found_message(cmd)) from None
ui.warning(_not_found_message(cmd))
return subprocess.CompletedProcess(cmd, COMMAND_NOT_FOUND)
if check and result.returncode != 0:
raise LimError(
error_msg or f"Command failed with code {result.returncode}: {shlex.join(cmd)}"
)
return result
def output(
cmd: list[str],
*,
sudo: bool = False,
check: bool = True,
error_msg: str | None = None,
) -> str:
"""Run a command and return its stripped stdout."""
cmd = _with_sudo(cmd, sudo=sudo)
try:
result = subprocess.run(cmd, text=True, capture_output=True, check=False)
except FileNotFoundError:
if check:
raise LimError(error_msg or _not_found_message(cmd)) from None
ui.warning(_not_found_message(cmd))
return ""
if check and result.returncode != 0:
raise LimError(
error_msg
or f"Command failed with code {result.returncode}: {shlex.join(cmd)}"
+ (f"\n{result.stderr.strip()}" if result.stderr else "")
)
return result.stdout.strip()
def succeeds(cmd: list[str], *, sudo: bool = False) -> bool:
"""Run a command silently, report whether it exited with 0."""
cmd = _with_sudo(cmd, sudo=sudo)
try:
return subprocess.run(cmd, capture_output=True, check=False).returncode == 0
except FileNotFoundError:
ui.warning(_not_found_message(cmd))
return False
def pipeline(
*cmds: list[str],
sudo_last: bool = False,
error_msg: str | None = None,
) -> None:
"""Run commands as a shell-style pipeline (cmd1 | cmd2 | ...)."""
prepared = [
_with_sudo(cmd, sudo=sudo_last and index == len(cmds) - 1) for index, cmd in enumerate(cmds)
]
pretty = " | ".join(shlex.join(cmd) for cmd in prepared)
ui.info(f"Running: {pretty}")
processes: list[subprocess.Popen] = []
previous_stdout = None
try:
for index, cmd in enumerate(prepared):
last = index == len(prepared) - 1
process = subprocess.Popen(
cmd,
stdin=previous_stdout,
stdout=None if last else subprocess.PIPE,
)
if previous_stdout is not None:
previous_stdout.close()
previous_stdout = process.stdout
processes.append(process)
except FileNotFoundError as exc:
if previous_stdout is not None:
previous_stdout.close()
for process in processes:
process.kill()
process.wait()
raise LimError(f"Command not found: {exc.filename} — please install it.") from None
for process in processes:
process.wait()
if any(process.returncode != 0 for process in processes):
raise LimError(error_msg or f"Pipeline failed: {pretty}")
def sync_disks() -> None:
run(["sync"])

0
lim/storage/__init__.py Normal file
View File

38
lim/storage/common.py Normal file
View File

@@ -0,0 +1,38 @@
"""Shared path derivation for encrypted storage setups."""
from dataclasses import dataclass
from lim import device as device_module
from lim import ui
from lim.device import Device
@dataclass(frozen=True)
class StorageTarget:
"""An encrypted drive plus all derived mapper/mount/partition paths."""
device: Device
@property
def mapper_name(self) -> str:
return f"encrypteddrive-{self.device.name}"
@property
def mapper_path(self) -> str:
return f"/dev/mapper/{self.mapper_name}"
@property
def mount_path(self) -> str:
return f"/media/{self.mapper_name}"
@property
def partition_path(self) -> str:
return self.device.partition(1)
def select_storage_target() -> StorageTarget:
target = StorageTarget(device_module.select_device())
ui.info(f"mapper name set to : {target.mapper_name}")
ui.info(f"mapper path set to : {target.mapper_path}")
ui.info(f"mount path set to : {target.mount_path}")
return target

66
lim/storage/raid1.py Normal file
View File

@@ -0,0 +1,66 @@
"""Encrypted Btrfs RAID1 across two drives.
See https://balaskas.gr/btrfs/raid1.html and
https://mutschler.eu/linux/install-guides/ubuntu-btrfs-raid1/
"""
from lim import luks, runner, ui
from lim.storage.common import StorageTarget, select_storage_target
def _select_pair() -> tuple[StorageTarget, StorageTarget]:
ui.info("RAID1 partition 1...")
first = select_storage_target()
ui.info("RAID1 partition 2...")
second = select_storage_target()
return first, second
def setup() -> None:
first, second = _select_pair()
ui.info(f"Encrypting {first.device.path}...")
runner.run(["cryptsetup", "luksFormat", first.device.path], sudo=True)
ui.info(f"Encrypting {second.device.path}...")
runner.run(["cryptsetup", "luksFormat", second.device.path], sudo=True)
runner.run(["cryptsetup", "luksOpen", first.device.path, first.mapper_name], sudo=True)
runner.run(["cryptsetup", "luksOpen", second.device.path, second.mapper_name], sudo=True)
runner.run(["cryptsetup", "status", first.mapper_path], sudo=True)
runner.run(["cryptsetup", "status", second.mapper_path], sudo=True)
runner.run(
[
"mkfs.btrfs",
"-m",
"raid1",
"-d",
"raid1",
first.mapper_path,
second.mapper_path,
],
sudo=True,
)
ui.success("Encryption successfull :)")
def _show_luks_devices() -> None:
for name in runner.output(["lsblk", "-dno", "NAME"], check=False).split():
if runner.succeeds(["cryptsetup", "isLuks", f"/dev/{name}"], sudo=True):
ui.info(f"/dev/{name} is a LUKS encrypted storage device.")
def mount_on_boot() -> None:
ui.info("Activate Automount raid1 encrypted storages...")
_show_luks_devices()
first, second = _select_pair()
luks.create_luks_key_and_update_crypttab(first.mapper_name, first.device.path)
ui.info(f'Creating mount folder under "{first.mount_path}"...')
runner.run(["mkdir", "-vp", first.mount_path], sudo=True)
luks.create_luks_key_and_update_crypttab(second.mapper_name, second.device.path)
luks.update_fstab(first.mapper_path, first.mount_path)
ui.success("Installation finished. Please restart :)")

View File

@@ -0,0 +1,84 @@
"""Single-drive LUKS + Btrfs storage: setup, mount, umount, mount-on-boot."""
from lim import device as device_module
from lim import luks, runner, system, ui
from lim.storage.common import select_storage_target
# fdisk answer sequences; empty lines accept the defaults.
CREATE_GPT_TABLE_INPUT = "g\nw\n"
CREATE_PARTITION_INPUT = "n\n\n\n\np\nw\n"
def setup() -> None:
print("Setups disk encryption")
target = select_storage_target()
device_module.overwrite_device(target.device)
ui.info("Creating new GPT partition table...")
runner.run(
["fdisk", "--wipe", "always", target.device.path],
input_text=CREATE_GPT_TABLE_INPUT,
sudo=True,
)
ui.info("Creating partition table...")
runner.run(
["fdisk", "--wipe", "always", target.device.path],
input_text=CREATE_PARTITION_INPUT,
sudo=True,
)
ui.info(f"Encrypt {target.device.path}...")
runner.run(["cryptsetup", "-v", "-y", "luksFormat", target.partition_path], sudo=True)
ui.info("Unlock partition...")
runner.run(["cryptsetup", "luksOpen", target.partition_path, target.mapper_name], sudo=True)
ui.info("Create btrfs file system...")
runner.run(["mkfs.btrfs", target.mapper_path], sudo=True)
ui.info(f'Creating mount folder under "{target.mount_path}"...')
runner.run(["mkdir", "-p", target.mount_path], sudo=True)
ui.info("Mount partition...")
runner.run(["mount", target.mapper_path, target.mount_path], sudo=True)
user = system.real_user()
ui.info("Own partition by user...")
runner.run(["chown", "-R", f"{user}:{user}", target.mount_path], sudo=True)
ui.success("Encryption successfull :)")
def mount() -> None:
print("Mounts encrypted storages")
target = select_storage_target()
ui.info("Unlock partition...")
runner.run(["cryptsetup", "luksOpen", target.partition_path, target.mapper_name], sudo=True)
ui.info("Mount partition...")
runner.run(["mount", target.mapper_path, target.mount_path], sudo=True)
ui.success("Mounting successfull :)")
def umount() -> None:
print("Unmount encrypted storages")
target = select_storage_target()
ui.info(f"Unmount {target.mapper_path}...")
runner.run(["umount", target.mapper_path], sudo=True)
runner.run(["cryptsetup", "luksClose", target.mapper_name], sudo=True)
ui.success("Successfull :)")
def mount_on_boot() -> None:
print("Automount encrypted storages")
target = select_storage_target()
luks.create_luks_key_and_update_crypttab(target.mapper_name, target.partition_path)
luks.update_fstab(target.mapper_path, target.mount_path)
ui.success("Installation finished. Please restart :)")

35
lim/system.py Normal file
View File

@@ -0,0 +1,35 @@
"""Process-level helpers: privilege handling and user resolution."""
import getpass
import os
import sys
from pathlib import Path
from lim import ui
def is_root() -> bool:
return os.geteuid() == 0
def ensure_root() -> None:
"""Re-execute the current command with sudo when not running as root."""
if is_root():
return
ui.info("Root privileges required. Re-executing with sudo...")
script = str(Path(sys.argv[0]).resolve())
# Deliberate privilege escalation: replace this process with sudo.
os.execvp("sudo", ["sudo", sys.executable, script, *sys.argv[1:]]) # noqa: S606
def real_user() -> str:
"""Return the invoking user, even when running under sudo."""
return os.environ.get("SUDO_USER") or getpass.getuser()
def real_home() -> Path:
"""Home directory of the invoking user, even when running under sudo."""
sudo_user = os.environ.get("SUDO_USER")
if sudo_user:
return Path("/home") / sudo_user
return Path.home()

62
lim/ui.py Normal file
View File

@@ -0,0 +1,62 @@
"""Colored console messages and interactive prompts."""
import sys
_USE_COLOR = sys.stdout.isatty()
def _color(code: str) -> str:
return f"\033[{code}m" if _USE_COLOR else ""
COLOR_RED = _color("31")
COLOR_GREEN = _color("32")
COLOR_YELLOW = _color("33")
COLOR_BLUE = _color("34")
COLOR_MAGENTA = _color("35")
COLOR_CYAN = _color("36")
COLOR_WHITE = _color("37")
COLOR_RESET = _color("0")
def message(color: str, tag: str, text: str) -> None:
print(f"{color}[{tag}]:{COLOR_RESET} {text}")
def question(text: str) -> None:
message(COLOR_MAGENTA, "QUESTION", text)
def info(text: str) -> None:
message(COLOR_BLUE, "INFO", text)
def warning(text: str) -> None:
message(COLOR_YELLOW, "WARNING", text)
def success(text: str) -> None:
message(COLOR_GREEN, "SUCCESS", text)
def error(text: str) -> None:
message(COLOR_RED, "ERROR", text)
def ask(prompt: str) -> str:
question(prompt)
return input().strip()
def confirm(prompt: str) -> bool:
return ask(f"{prompt}(y/N)") == "y"
def header() -> None:
print(
f"\n{COLOR_YELLOW}The\n"
"LINUX IMAGE MANAGER\n"
"is an administration tool designed from and for Kevin Veen-Birkenbach.\n\n"
"Licensed under GNU GENERAL PUBLIC LICENSE Version 3"
f"{COLOR_RESET}\n"
)

12
main.py Executable file
View File

@@ -0,0 +1,12 @@
#!/usr/bin/env python3
"""Entry point for the Linux Image Manager (lim)."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from lim.cli import main
if __name__ == "__main__":
main()

70
pyproject.toml Normal file
View File

@@ -0,0 +1,70 @@
[build-system]
# 77+ for PEP 639 SPDX `license = "..."` + `license-files`.
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
[project]
name = "linux-image-manager"
version = "2.3.0"
description = "CLI (`lim`) for downloading, verifying and flashing Linux images, LUKS/Btrfs encrypted storage (single drive and RAID1), image backups and chroot maintenance"
readme = "README.md"
requires-python = ">=3.10"
authors = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
maintainers = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
license = "GPL-3.0-only"
license-files = ["LICENSE.txt"]
urls = { Homepage = "https://veen.world", Repository = "https://github.com/kevinveenbirkenbach/linux-image-manager" }
classifiers = [
"Environment :: Console",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
]
dependencies = ["PyYAML>=6"]
[project.scripts]
lim = "lim.cli:main"
[tool.setuptools.packages.find]
include = ["lim*"]
[tool.setuptools.package-data]
lim = [
"distributions.yml",
"configuration/packages/**/*.txt",
"configuration/initcpio/*",
"configuration/initramfs-tools/*",
]
[tool.pytest.ini_options]
testpaths = ["tests"]
[tool.ruff]
target-version = "py310"
line-length = 100
[tool.ruff.lint]
select = ["ALL"]
ignore = [
# This tool's UI is print/prompt based and its purpose is orchestrating
# system commands — these rules contradict the design:
"T201", # print used as console UI
"S603", # subprocess without shell is exactly what runner.py wraps
"S607", # system tools are resolved via PATH on purpose
# Style choices:
"D203", # conflicts with D211 (blank line before class docstring)
"D213", # conflicts with D212 (multi-line docstring summary position)
"D1", # docstrings are not required on every symbol
"COM812", # trailing commas are handled by the formatter
"TRY003", # exception messages are written inline
"EM101", # literal exception messages are fine
"EM102", # f-string exception messages are fine
]
[tool.ruff.lint.per-file-ignores]
"tests/*" = [
"S101", # pytest asserts
"ANN", # test signatures need no annotations
"PLR2004", # literal expectations in assertions
"ARG", # fixtures are injected even when unused
"SLF001", # tests may exercise internal helpers directly
]

2
requirements.yml Normal file
View File

@@ -0,0 +1,2 @@
pacman:
- pv

View File

@@ -1,141 +0,0 @@
#!/bin/bash
#
# This script contains the global program variables and functions
#
# shellcheck disable=SC2034 #Deactivate checking of unused variables
# shellcheck disable=SC2003 #Deactivate "expr is antiquated"
# shellcheck disable=SC2015 #Deactivate bool hint
# shellcheck disable=SC2005 #Remove useless echo hint
# shellcheck disable=SC2010 #Deactivate ls | grep hint
REPOSITORY_PATH="$(readlink -f "${0}" | sed -e 's/\/scripts\/.*//g')"
CONFIGURATION_PATH="$REPOSITORY_PATH""/configuration/"
PACKAGE_PATH="$CONFIGURATION_PATH""packages/"
TEMPLATE_PATH="$CONFIGURATION_PATH""templates/";
HOME_TEMPLATE_PATH="$TEMPLATE_PATH""home/";
ENCRYPTED_PATH="$REPOSITORY_PATH/.encrypted";
DECRYPTED_PATH="$REPOSITORY_PATH/decrypted";
SCRIPT_PATH="$REPOSITORY_PATH/scripts/";
DATA_PATH="$DECRYPTED_PATH/data";
BACKUP_PATH="$DECRYPTED_PATH/backup";
COLOR_RED=$(tput setaf 1)
COLOR_GREEN=$(tput setaf 2)
COLOR_YELLOW=$(tput setaf 3)
COLOR_BLUE=$(tput setaf 4)
COLOR_MAGENTA=$(tput setaf 5)
COLOR_CYAN=$(tput setaf 6)
COLOR_WHITE=$(tput setaf 7)
COLOR_RESET=$(tput sgr0)
# FUNCTIONS
message(){
echo "$1[$2]:${COLOR_RESET} $3 ";
}
question(){
message "${COLOR_MAGENTA}" "QUESTION" "$1";
}
info(){
message "${COLOR_BLUE}" "INFO" "$1";
}
warning(){
message "${COLOR_YELLOW}" "WARNING" "$1";
}
success(){
message "${COLOR_GREEN}" "SUCCESS" "$1";
}
error(){
if [ -z "$1" ]
then
message="Failed."
else
message="$1"
fi
message "${COLOR_RED}" "ERROR" "$message -> Leaving program."
if declare -f "destructor" > /dev/null
then
info "Calling destructor..."
destructor
else
warning "No destructor defined."
info "Can be that this script left some waste."
fi
exit 1;
}
# Routine to echo the full sd-card-path
set_device_path(){
info "Available devices:"
ls -lasi /dev/ | grep -E "sd|mm"
question "Please type in the name of the device: /dev/" && read -r device
device_path="/dev/$device"
if [ ! -b "$device_path" ]
then
error "$device_path is not valid device."
fi
info "Device path set to: $device_path"
# @see https://www.heise.de/ct/hotline/Optimale-Blockgroesse-fuer-dd-2056768.html
PHYSICAL_BLOCK_SIZE_PATH="/sys/block/$device/queue/physical_block_size"
if [ -f "$PHYSICAL_BLOCK_SIZE_PATH" ]; then
PHYSICAL_BLOCK_SIZE=$(sudo cat $PHYSICAL_BLOCK_SIZE_PATH)
if [ $? -eq 0 ]; then
OPTIMAL_BLOCKSIZE=$((64 * PHYSICAL_BLOCK_SIZE)) || error
else
echo "Unable to read $PHYSICAL_BLOCK_SIZE_PATH"
OPTIMAL_BLOCKSIZE="4K"
fi
else
OPTIMAL_BLOCKSIZE="4K"
fi
info "Optimal blocksize set to: $OPTIMAL_BLOCKSIZE" || error
}
overwrite_device() {
question "Should $device_path be overwritten with zeros before copying? (y/N/block count)" && read -r copy_zeros_to_device
case "$copy_zeros_to_device" in
y)
info "Overwriting entire device..." &&
dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" status=progress || error "Overwriting $device_path failed."
;;
N|'')
info "Skipping Overwriting..."
;;
''|*[!0-9]*)
error "Invalid input."
;;
*)
if [[ "$copy_zeros_to_device" =~ ^[0-9]+$ ]]; then
info "Overwriting $copy_zeros_to_device blocks..." &&
dd if=/dev/zero of="$device_path" bs="$OPTIMAL_BLOCKSIZE" count="$copy_zeros_to_device" status=progress || error "Overwriting $device_path failed."
else
error "Invalid input. Block count must be a number."
fi
;;
esac
}
get_packages(){
for package_collection in "$@"
do
package_collection_path="$PACKAGE_PATH""$package_collection.txt" &&
echo "$(sed -e "/^#/d" -e "s/#.*//" "$package_collection_path" | tr '\n' ' ')" ||
error
done
}
HEADER(){
echo
echo "${COLOR_YELLOW}The"
echo "LINUX IMAGE MANAGER"
echo "is an administration tool designed from and for Kevin Veen-Birkenbach."
echo
echo "Licensed under GNU GENERAL PUBLIC LICENSE Version 3"
echo "${COLOR_RESET}"
}
HEADER

View File

@@ -1,12 +0,0 @@
#!/bin/bash
# Executes the import script in reverse mode
# @author Kevin Veen-Birkenbach [aka. Frantz]
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2015 # Deactivating bool hint
source "$(dirname "$(readlink -f "${0}")")/../base.sh" || (echo "Loading base.sh failed." && exit 1)
bash "$SCRIPT_PATH""data/import-from-system.sh" reverse
info "Setting right permissions for importet files..." &&
chmod -R 700 ~/.ssh &&
chmod 600 ~/.ssh/id_rsa &&
chmod 600 ~/.ssh/id_rsa.pub || error "Failed to set correct ssh permissions"
chown -R "$USER":"$USER" ~ || warning "Not all files could be owned by user \"$USER\"..."

View File

@@ -1,75 +0,0 @@
#!/bin/bash
#
# Imports data from the system
# @param $1 If the first parameter is "reverse" the data will be exported to the system
#
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2143 # Comparing with -z allowed
# shellcheck disable=SC2015 # Deactivating bool hint
source "$(dirname "$(readlink -f "${0}")")/../base.sh" || (echo "Loading base.sh failed." && exit 1)
declare -a BACKUP_LIST=("$HOME/.ssh/" \
"$HOME/.gitconfig" \
"$HOME/.atom/config.cson" \
"$HOME/.projectlibre/projectlibre.conf" \
"$HOME/.local/share/rhythmbox/rhythmdb.xml" \
"$HOME/.config/keepassxc/keepassxc.ini" \
"$HOME/Documents/certificates/" \
"$HOME/Documents/security/" \
"$HOME/Documents/identity/" \
"$HOME/Documents/health/" \
"$HOME/Documents/licenses/");
if [ -z "$(mount | grep "$DECRYPTED_PATH")" ]
then
info "The decrypted folder $DECRYPTED_PATH is locked. You need to unlock it!" &&
bash "$SCRIPT_PATH""encryption/data/unlock.sh" || error "Unlocking failed.";
fi
if [ "$1" = "reverse" ]
then
MODE="export"
else
MODE="import"
fi
CONCRETE_BACKUP_FOLDER="$BACKUP_PATH/$MODE/$(date '+%Y%m%d%H%M%S')"
mkdir -p "$CONCRETE_BACKUP_FOLDER" || error "Failed to create \"$CONCRETE_BACKUP_FOLDER\"."
for system_item_path in "${BACKUP_LIST[@]}";
do
data_item_path="$DATA_PATH$system_item_path"
if [ "$MODE" = "export" ]
then
destination="$system_item_path"
source="$data_item_path"
info "Export data from $source to $destination..."
else
source="$system_item_path"
destination="$data_item_path"
info "Import data from $source to $destination..."
fi
if [ -f "$destination" ]
then
info "The destination file allready exists!" &&
info "Difference:" &&
diff "$destination" "$source"
fi
destination_dir=$(dirname "$destination")
mkdir -p "$destination_dir" || error "Failed to create \"$destination_dir\"."
if [ -f "$source" ]
then
backup_dir=$(dirname "$CONCRETE_BACKUP_FOLDER/$system_item_path");
mkdir -p "$backup_dir" || error "Failed to create \"$backup_dir\"."
info "Copy data from $source to $destination..."
rsync -abcEPuvW --backup-dir="$backup_dir" "$source" "$destination" || error
else
if [ -d "$source" ]
then
mkdir -p "$destination" || error "Failed to create \"$destination\"."
backup_dir="$CONCRETE_BACKUP_FOLDER/$system_item_path";
mkdir -p "$backup_dir" || error "Failed to create \"$backup_dir\"."
info "Copy data from directory $source to directory $destination..."
rsync -abcEPuvW --delete --backup-dir="$backup_dir" "$source" "$destination" || error
else
warning "$source doesn't exist. Copying data is not possible."
fi
fi
done

View File

@@ -1,14 +0,0 @@
#!/bin/bash
#
# Locks the data
# @author Kevin Veen-Birkenbach [aka. Frantz]
#
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck source=/dev/null # Deactivate SC1090
source "$(dirname "$(readlink -f "${0}")")/../../base.sh" || (echo "Loading base.sh failed." && exit 1)
info "Locking directory $DECRYPTED_PATH..." &&
fusermount -u "$DECRYPTED_PATH" || error "Unmounting failed."
info "Data is now encrypted."
info "Removing directory $DECRYPTED_PATH..." &&
rmdir "$DECRYPTED_PATH" || error

View File

@@ -1,17 +0,0 @@
#!/bin/bash
#
# Unlocks the data
# @author Kevin Veen-Birkenbach [aka. Frantz]
#
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2015 # Deactivating bool hint
source "$(dirname "$(readlink -f "${0}")")/../../base.sh" || (echo "Loading base.sh failed." && exit 1)
info "Unlocking directory $DECRYPTED_PATH..."
if [ ! -d "$DECRYPTED_PATH" ]
then
info "Creating directory $DECRYPTED_PATH..." &&
mkdir "$DECRYPTED_PATH" || error
fi
info "Decrypting directory $DECRYPTED_PATH to $DECRYPTED_PATH..." &&
encfs "$ENCRYPTED_PATH" "$DECRYPTED_PATH" || error
echo "ATTENTION: DATA IS NOW DECRYPTED!"

View File

@@ -1,2 +0,0 @@
# Storage
For security reasons storages **SHOULD** be encrypted with [LUKS](https://de.wikipedia.org/wiki/Dm-crypt#Erweiterung_mit_LUKS). To keep it standardized and easy this scripts will use [btrfs](https://de.wikipedia.org/wiki/Btrfs) as file system.

View File

@@ -1,86 +0,0 @@
#!/bin/bash
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2034 # Unused variables
# shellcheck disable=SC2154 # Referenced but not assigned
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2001 # See if you can use ${variable//search/replace} instead
source "$(dirname "$(readlink -f "${0}")")/../../../base.sh" || (echo "Loading base.sh failed." && exit 1)
set_device_mount_partition_and_mapper_paths(){
set_device_path &&
mapper_name="encrypteddrive-$device" &&
mapper_path="/dev/mapper/$mapper_name" &&
mount_path="/media/$mapper_name" &&
partition_path="$device_path""1" &&
info "mapper name set to : $mapper_name" &&
info "mapper path set to : $mapper_path" &&
info "mount path set to : $mount_path" ||
error
}
# @var $1 mapper_path
# @var $2 partition_path
create_luks_key_and_update_cryptab(){
LUKS_KEY_DIRECTORY="/etc/luks-keys/" &&
info "Creating luks-key-directory..." &&
sudo mkdir $LUKS_KEY_DIRECTORY || warning "Directory exists: $LUKS_KEY_DIRECTORY" || error
luks_key_name="$1.keyfile" &&
secret_key_path="$LUKS_KEY_DIRECTORY$luks_key_name" &&
info "Generate secret key under: $secret_key_path" || error
if [ -f "$secret_key_path" ]
then
warning "File already exists. Overwriting!"
fi
sudo dd if=/dev/urandom of="$secret_key_path" bs=512 count=8 &&
info "Opening and closing device to verify that everything works fine..." &&
sudo cryptsetup -v luksClose "$1" || info "No need to luksClose $1. Device isn't open." &&
sudo cryptsetup luksAddKey $2 $secret_key_path &&
sudo cryptsetup -v luksOpen "$2" "$1" --key-file="$secret_key_path" &&
sudo cryptsetup -v luksClose "$1" &&
info "Reading UUID..." &&
uuid_line=$(sudo cryptsetup luksDump "$2" | grep "UUID") &&
uuid=$(echo "${uuid_line/UUID:/""}"|sed -e "s/[[:space:]]\+//g") &&
crypttab_path="/etc/crypttab" &&
crypttab_entry="$1 UUID=$uuid $secret_key_path luks" &&
info "Adding crypttab entry..." || error
if sudo grep -q "$crypttab_entry" "$crypttab_path";
then
warning "File $crypttab_path already contains the following entry:" &&
echo "$crypttab_entry" &&
info "Skipped." ||
error
else
sudo sh -c "echo '$crypttab_entry' >> $crypttab_path" ||
error
fi
info "The file $crypttab_path contains now the following:" &&
sudo cat $crypttab_path ||
error
}
# @var $1 mapper_name
# @var $2 mount_path
#
# If mount doesn't work adapt it manually to
# @see https://gist.github.com/MaxXor/ba1665f47d56c24018a943bb114640d7
update_fstab(){
fstab_path="/etc/fstab"
fstab_entry="$1 $2 btrfs defaults 0 2"
info "Adding fstab entry..."
if sudo grep -q "$fstab_entry" "$fstab_path"; then
warning "File $fstab_path contains allready a the following entry:" &&
echo "$fstab_entry" &&
info "Skipped." ||
error
else
sudo sh -c "echo '$fstab_entry' >> $fstab_path" ||
error
fi
info "The file $fstab_path contains now the following:" &&
sudo cat $fstab_path ||
error
}

View File

@@ -1,22 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2034 # Unused variables
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/../base.sh" || (echo "Loading base.sh failed." && exit 1)
set_raid1_devices_mount_partition_and_mapper_paths(){
info "RAID1 partition 1..." &&
set_device_mount_partition_and_mapper_paths &&
partition_path_1=$partition_path &&
mapper_name_1=$mapper_name &&
mapper_path_1=$mapper_path &&
mount_path_1=$mount_path &&
device_path_1=$device_path &&
info "RAID1 partition 2..." &&
set_device_mount_partition_and_mapper_paths &&
partition_path_2=$partition_path &&
mapper_name_2=$mapper_name &&
mapper_path_2=$mapper_path &&
mount_path_2=$mount_path &&
device_path_2=$device_path || error
}

View File

@@ -1,20 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2154 # Referenced but not assigned
# shellcheck disable=SC2015 #Deactivate bool hint
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
info "Activate Automount raid1 encrypted storages..." &&
echo ""
for dev in $(lsblk -dno NAME); do
if sudo cryptsetup isLuks /dev/$dev 2>/dev/null; then
info "/dev/$dev is a LUKS encrypted storage device."
fi
done
set_raid1_devices_mount_partition_and_mapper_paths &&
create_luks_key_and_update_cryptab "$mapper_name_1" "$device_path_1" &&
info "Creating mount folder unter \"$mount_path_1\"..." &&
sudo mkdir -vp "$mount_path_1" &&
create_luks_key_and_update_cryptab "$mapper_name_2" "$device_path_2" &&
update_fstab "$mapper_path_1" "$mount_path_1" &&
success "Installation finished. Please restart :)" ||
error

View File

@@ -1,23 +0,0 @@
#!/bin/bash
# @author Kevin Veen-Birkenbach [kevin@veen.world]
# @see https://balaskas.gr/btrfs/raid1.html
# @see https://mutschler.eu/linux/install-guides/ubuntu-btrfs-raid1/
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
set_raid1_devices_mount_partition_and_mapper_paths
info "Encrypting $device_path_1..." &&
cryptsetup luksFormat "$device_path_1" &&
info "Encrypting $device_path_2..." &&
cryptsetup luksFormat "$device_path_2" &&
blkid | tail -2 &&
cryptsetup luksOpen "$device_path_1" "$mapper_name_1" &&
cryptsetup luksOpen "$device_path_2" "$mapper_name_2" &&
cryptsetup status "$mapper_path_1" &&
cryptsetup status "$mapper_path_2" &&
mkfs.btrfs -m raid1 -d raid1 "$mapper_path_1" "$mapper_path_2" &&
success "Encryption successfull :)" ||
error

View File

@@ -1,3 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
source "$(dirname "$(readlink -f "${0}")")/../base.sh" || (echo "Loading base.sh failed." && exit 1)

View File

@@ -1,18 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
echo "Mounts encrypted storages"
set_device_mount_partition_and_mapper_paths
info "Unlock partition..." &&
sudo cryptsetup luksOpen "$partition_path" "$mapper_name" ||
error
info "Mount partition..." &&
sudo mount "$mapper_path" "$mount_path" ||
error
success "Mounting successfull :)"

View File

@@ -1,13 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
echo "Automount encrypted storages"
echo
set_device_mount_partition_and_mapper_paths
create_luks_key_and_update_cryptab "$mapper_name" "$partition_path"
update_fstab "$mapper_path" "$mount_path"
success "Installation finished. Please restart :)"

View File

@@ -1,50 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
echo "Setups disk encryption"
set_device_mount_partition_and_mapper_paths
overwrite_device
info "Creating new GPT partition table..."
( echo "g" # create a new empty GPT partition table
echo "w" # Write partition table
)| sudo fdisk --wipe always "$device_path" ||
error
info "Creating partition table..."
( echo "n" # Create new partition
echo "" # Accept default
echo "" # Accept default
echo "" # Accept default
echo "p" # Create GPT partition table
echo "w" # Write partition table
)| sudo fdisk --wipe always "$device_path" ||
error
info "Encrypt $device_path..." &&
sudo cryptsetup -v -y luksFormat "$partition_path" ||
error
info "Unlock partition..." &&
sudo cryptsetup luksOpen "$partition_path" "$mapper_name" ||
error
info "Create btrfs file system..." &&
sudo mkfs.btrfs "$mapper_path" || error
info "Creating mount folder unter \"$mount_path\"..." &&
sudo mkdir -p "$mount_path" || error
info "Mount partition..." &&
sudo mount "$mapper_path" "$mount_path" ||
error
info "Own partition by user..." &&
sudo chown -R "$USER":"$USER" "$mount_path" ||
error
success "Encryption successfull :)"

View File

@@ -1,15 +0,0 @@
#!/bin/bash
# shellcheck disable=SC1090 # Can't follow non-constant source. Use a directive to specify location.
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2154 # Referenced but not assigned
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
echo "Unmount encrypted storages"
set_device_mount_partition_and_mapper_paths
info "Unmount $mapper_path..."
sudo umount "$mapper_path" &&
sudo cryptsetup luksClose "$mapper_path" ||
error
success "Successfull :)"

View File

@@ -1,30 +0,0 @@
#!/bin/bash
# shellcheck disable=SC2010 # ls | grep allowed
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2154 # Deactivate not referenced link
# shellcheck disable=SC2015 # Deactivate bools hints
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
info "Backupscript for memory devices started..."
echo
set_device_path
while [ "$path" == "" ]
do
echo "Bitte Backupimagepfad+Namen zu $PWD eingeben:"
read -r path
if [ "${path:0:1}" == "/" ]
then
ofi="$path.img"
else
ofi="$PWD/$path.img"
fi
done
info "Input file: $device_path"
info "Output file: $ofi"
question "Please confirm by pushing \"Enter\". To cancel use \"Ctrl + Alt + C\""
read -r bestaetigung && echo "$bestaetigung";
info "Imagetransfer starts. This can take a while..." &&
dd if="$device_path" of="$ofi" bs=1M status=progress || error "\"dd\" failed.";
success "Imagetransfer successfull." && exit 0;

View File

@@ -1,114 +0,0 @@
#!/bin/bash
#
# Offers base functions for the image management
#
# shellcheck disable=SC2034 #Deactivate checking of unused variables
# shellcheck disable=SC2010 # ls | grep allowed
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2015 # Deactivate bools hints
# shellcheck disable=SC2154 # Deactivate referenced but not assigned hints
source "$(dirname "$(readlink -f "${0}")")/../base.sh" || (echo "Loading base.sh failed." && exit 1)
# Writes the full partition name
# @parameter $1 is device path
# @parameter $2 is the partition number
echo_partition_name(){
if [ "${device_path:5:1}" != "s" ]
then
echo "$device_path""p""$1"
else
echo "$device_path$1"
fi
}
set_partition_paths(){
info "Setting partition and mapper paths..."
boot_partition_path=$(echo_partition_name "1")
root_partition_path=$(echo_partition_name "2")
root_mapper_path=$root_partition_path
}
make_mount_folders(){
info "Preparing mount paths..." &&
boot_mount_path="$working_folder_path""boot/" &&
root_mount_path="$working_folder_path""root/" &&
mkdir -v "$boot_mount_path" &&
mkdir -v "$root_mount_path" ||
error
}
make_working_folder(){
working_folder_path="/tmp/linux-image-manager-$(date +%s)/" &&
info "Create temporary working folder in $working_folder_path" &&
mkdir -v "$working_folder_path" ||
error
}
decrypt_root(){
if [ "$(blkid "$root_partition_path" -s TYPE -o value)" == "crypto_LUKS" ]
then
root_partition_uuid=$(blkid "$root_partition_path" -s UUID -o value) &&
root_mapper_name="linux-image-manager-$root_partition_uuid" &&
root_mapper_path="/dev/mapper/$root_mapper_name" &&
info "Decrypting of $root_partition_path is neccessary..." &&
sudo cryptsetup -v luksOpen "$root_partition_path" "$root_mapper_name" || error
fi
}
mount_partitions(){
info "Mount boot and root partition..." &&
mount -v "$boot_partition_path" "$boot_mount_path" &&
mount -v "$root_mapper_path" "$root_mount_path" &&
info "Settind uuid variables..." &&
root_partition_uuid=$(blkid "$root_partition_path" -s UUID -o value) &&
boot_partition_uuid=$(blkid "$boot_partition_path" -s UUID -o value) &&
info "The following mounts refering this setup exist:" && mount | grep "$working_folder_path" ||
error
}
destructor(){
info "Cleaning up..."
info "Unmounting everything..."
umount -lv "$chroot_dev_pts_mount_path" || warning "Umounting $chroot_dev_pts_mount_path failed!"
umount -lv "$chroot_dev_mount_path" || warning "Umounting $chroot_dev_mount_path failed!"
umount -v "$chroot_proc_mount_path" || warning "Umounting $chroot_proc_mount_path failed!"
umount -v "$chroot_sys_mount_path" || warning "Umounting $chroot_sys_mount_path failed!"
umount -v "$root_mount_path""boot/" || warning "Umounting $root_mount_path""boot/ failed!"
umount -v "$root_mount_path" || warning "Umounting $root_mount_path failed!"
umount -v "$boot_mount_path" || warning "Umounting $boot_mount_path failed!"
info "Deleting mount folders..."
rmdir -v "$root_mount_path" || warning "Removing $root_mount_path failed!"
rmdir -v "$boot_mount_path" || warning "Removing $boot_mount_path failed!"
rmdir -v "$working_folder_path" || warning "Removing $working_folder_path failed!"
if [ "$(blkid "$root_partition_path" -s TYPE -o value)" == "crypto_LUKS" ]
then
info "Trying to close decrypted $root_mapper_name..." &&
sudo cryptsetup -v luksClose "$root_mapper_name" || warning "Failed."
fi
}
mount_chroot_binds(){
info "Mount chroot environments..." &&
chroot_sys_mount_path="$root_mount_path""sys/" &&
chroot_proc_mount_path="$root_mount_path""proc/" &&
chroot_dev_mount_path="$root_mount_path""dev/" &&
chroot_dev_pts_mount_path="$root_mount_path""dev/pts" &&
mount --bind "$boot_mount_path" "$root_mount_path""boot" &&
mount --bind /dev "$chroot_dev_mount_path" &&
mount --bind /sys "$chroot_sys_mount_path" &&
mount --bind /proc "$chroot_proc_mount_path" &&
mount --bind /dev/pts "$chroot_dev_pts_mount_path" ||
error
}
copy_qemu(){
info "Copy qemu binary..." &&
cp -v /usr/bin/qemu-arm-static "$root_mount_path""usr/bin/" ||
error
}
copy_resolve_conf(){
info "Copy resolve.conf..." &&
cp --remove-destination -v /etc/resolv.conf "$root_mount_path""etc/" ||
warning "Failed. Propably there is no internet connection available."
}

View File

@@ -1,31 +0,0 @@
#!/bin/bash
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2015 # Deactivating bool hint
# shellcheck disable=SC2154 # Deactivate not referenced link
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
info "Starting chroot..."
set_device_path
make_working_folder
make_mount_folders
set_partition_paths
decrypt_root
mount_partitions
mount_chroot_binds
copy_qemu
copy_resolve_conf
info "Bash shell starts..." &&
chroot "$root_mount_path" /bin/bash ||
error
destructor

View File

@@ -1,696 +0,0 @@
#!/bin/bash
# shellcheck disable=SC2010 # ls | grep allowed
# shellcheck source=/dev/null # Deactivate SC1090
# shellcheck disable=SC2015 # Deactivate bools hints
# shellcheck disable=SC2154 # Deactivate not referenced link
# @see https://wiki.polaire.nl/doku.php?id=archlinux-raspberry-encrypted
source "$(dirname "$(readlink -f "${0}")")/base.sh" || (echo "Loading base.sh failed." && exit 1)
install(){
info "Installing $1..."
case "$distribution" in
"arch"|"manjaro")
echo "pacman --noconfirm -S --needed $1" | chroot "$root_mount_path" /bin/bash || error
;;
"moode"|"retropie")
echo "yes | apt install $1" | chroot "$root_mount_path" /bin/bash || error
;;
*)
error "Package manager not supported."
;;
esac
}
replace_in_file() {
# Assign the first function argument to the local variable search_string
local search_string=$1
# Assign the second function argument to the local variable replace_string
local replace_string=$2
# Assign the third function argument to the local variable file_path
local file_path=$3
# Create a temporary file and store its path in temp_file
temp_file=$(mktemp)
# Use sed to replace the search_string with replace_string in the file at file_path
# Write the output to the temporary file
sed "s/$search_string/$replace_string/g" "$file_path" > "$temp_file"
# Compare the original file with the temporary file
if cmp -s "$file_path" "$temp_file"; then
# If files are identical, remove the temporary file and signal an error
rm -f "$temp_file"
error "Error: Search string '$search_string' not found in $file_path."
else
# If files are different, move the temporary file to overwrite the original file
mv "$temp_file" "$file_path"
fi
}
info "Setupscript for images started..."
info "Checking if root..."
if [ "$(id -u)" != "0" ];then
error "This script must be executed as root!"
fi
make_working_folder
info "Configure user..." &&
question "Please type in a valid working username:" && read -r origin_username &&
getent passwd "$origin_username" > /dev/null 2 || error "User $origin_username doesn't exist."
origin_user_home="/home/$origin_username/"
info "Image routine starts..."
image_folder="$origin_user_home""Software/Images/";
info "The images will be stored in \"$image_folder\"."
if [ ! -d "$image_folder" ]; then
info "Folder \"$image_folder\" doesn't exist. It will be created now." &&
mkdir -v "$image_folder" ||
error
fi
set_device_path
if mount | grep -q "$device_path"
then
error "Device $device_path is allready mounted. Umount with \"umount $device_path*\"."
fi
question "Which operation system would you like to use [linux,windows,...]?" && read -r operation_system || error
case "$operation_system" in
"linux")
question "Which distribution should be used [arch,moode,retropie,manjaro,torbox...]?" && read -r distribution || error
case "$distribution" in
"android-x86")
base_download_url="https://www.fosshub.com/Android-x86.html?dwl=android-x86_64-9.0-r2.iso";
image_name="android-x86_64-9.0-r2.iso"
image_checksum="f7eb8fc56f29ad5432335dc054183acf086c539f3990f0b6e9ff58bd6df4604e"
boot_size="+500M"
;;
"torbox")
base_download_url="https://www.torbox.ch/data/";
image_name="torbox-20220102-v050.gz"
image_checksum="0E1BA7FFD14AAAE5F0462C8293D95B62C3BF1D9E726E26977BD04772C55680D3"
boot_size="+200M"
;;
"arch")
question "Which Raspberry Pi will be used (e.g.: 1, 2, 3b, 3b+, 4...):" && read -r raspberry_pi_version
boot_size="+500M"
base_download_url="http://os.archlinuxarm.org/os/";
case "$raspberry_pi_version" in
"1")
image_name="ArchLinuxARM-rpi-armv7-latest.tar.gz"
luks_memory_cost="64000"
;;
"2")
image_name="ArchLinuxARM-rpi-armv7-latest.tar.gz"
luks_memory_cost="128000"
;;
"3b" | "3b+")
image_name="ArchLinuxARM-rpi-aarch64-latest.tar.gz"
luks_memory_cost="128000"
;;
"4" )
image_name="ArchLinuxARM-rpi-aarch64-latest.tar.gz"
luks_memory_cost="256000"
;;
*)
error "Version $raspberry_pi_version isn't supported."
;;
esac
;;
"manjaro")
question "Which version(e.g.:architect,gnome) should be used:" && read -r gnome_version
boot_size="+500M"
case "$gnome_version" in
"architect")
image_checksum="6b1c2fce12f244c1e32212767a9d3af2cf8263b2"
base_download_url="https://osdn.net/frs/redir.php?m=dotsrc&f=%2Fstorage%2Fg%2Fm%2Fma%2Fmanjaro%2Farchitect%2F20.0%2F";
image_name="manjaro-architect-20.0-200426-linux56.iso"
;;
"gnome")
question "Which release(e.g.:20,21,raspberrypi) should be used:" && read -r release
case "$release" in
"20")
image_checksum="2df3697908483550d4a473815b08c1377e6b6892"
base_download_url="https://osdn.net/projects/manjaro-archive/storage/gnome/20.0/"
image_name="manjaro-gnome-20.0-200426-linux56.iso"
;;
"21")
base_download_url="https://download.manjaro.org/gnome/21.3.7/"
image_name="manjaro-gnome-21.3.7-220816-linux515.iso"
;;
"22")
base_download_url="https://download.manjaro.org/gnome/22.1.3/"
image_name="manjaro-gnome-22.1.3-230529-linux61.iso"
;;
"raspberrypi")
# at the moment just optimized for raspberry pi 4
base_download_url="https://github.com/manjaro-arm/rpi4-images/releases/download/23.02/"
image_name="Manjaro-ARM-gnome-rpi4-23.02.img.xz"
luks_memory_cost="256000"
raspberry_pi_version="4"
;;
*)
error "Gnome Version $gnome_version isn't supported."
;;
esac
;;
esac
;;
"moode")
boot_size="+200M"
image_checksum="185cbc9a4994534bb7a4bc2744c78197"
base_download_url="https://github.com/moode-player/moode/releases/download/r651prod/"
image_name="moode-r651-iso.zip";
;;
"retropie")
boot_size="+500M"
question "Which version(e.g.:1,2,3,4) should be used:" && read -r raspberry_pi_version
base_download_url="https://github.com/RetroPie/RetroPie-Setup/releases/download/4.8/";
case "$raspberry_pi_version" in
"1")
image_checksum="95a6f84453df36318830de7e8507170e"
image_name="retropie-buster-4.8-rpi1_zero.img.gz"
;;
"2" | "3")
image_checksum="224e64d8820fc64046ba3850f481c87e"
image_name="retropie-buster-4.8-rpi2_3_zero2w.img.gz"
;;
"4")
image_checksum="b5daa6e7660a99c246966f3f09b4014b"
image_name="retropie-buster-4.8-rpi4_400.img.gz"
;;
esac
;;
esac
question "Should the system be encrypted?(y/N)" && read -r encrypt_system
info "Generating os-image..."
download_url="$base_download_url$image_name"
image_path="$image_folder$image_name"
question "Should the image download be forced?(y/N)" && read -r force_image_download
if [ "$force_image_download" = "y" ]
then
if [ -f "$image_path" ]
then
info "Removing image $image_path." &&
rm "$image_path" || error "Removing image \"$image_path\" failed."
else
info "Forcing download wasn't neccessary. File $image_path doesn't exist."
fi
fi
info "Start Download procedure..."
if [ -f "$image_path" ]
then
info "Image exist local. Download skipped."
else
info "Image \"$image_name\" doesn't exist under local path \"$image_path\"." &&
info "Image \"$image_name\" gets downloaded from \"$download_url\"..." &&
wget "$download_url" -O "$image_path" || error "Download from \"$download_url\" failed."
fi
;;
*)
info "Available images:"
ls -l "$image_folder"
question "Which image would you like to use?" && read -r image_name || error
image_path="$image_folder$image_name"
;;
esac
info "Verifying image..."
info "Verifying checksum..."
if [ -z "$image_checksum" ]; then
for ext in sha1 sha512 md5; do
sha_download_url="$download_url.$ext"
info "Image Checksum is not defined. Try to download image signature from $sha_download_url."
if wget -q --method=HEAD "$sha_download_url"; then
image_checksum="$(wget $sha_download_url -q -O - | cut -d ' ' -f1)"
info "Defined image_checksum as $image_checksum"
break
else
warning "No checksum found under $sha_download_url."
fi
done
fi
if [[ -v image_checksum ]]
then
(info "Checking md5 checksum..." && echo "$image_checksum $image_path"| md5sum -c -) ||
(info "Checking sha1 checksum..." && echo "$image_checksum $image_path"| sha1sum -c -) ||
(info "Checking sha256 checksum..." && echo "$image_checksum $image_path"| sha256sum -c -) ||
error "Verification failed. HINT: Force the download of the image."
else
warning "Verification is not possible. No checksum is defined."
fi
info "Verifying signature..."
signature_download_url="$download_url.sig"
info "Try to download image signature from $signature_download_url."
if wget -q --method=HEAD "$signature_download_url"; then
signature_name="${image_name}.sig"
signature_path="${image_folder}${signature_name}"
info "Download the signature file"
if wget -q -O "$signature_path" "$signature_download_url"; then
info "Extract the key ID from the signature file"
key_id=$(gpg --status-fd 1 --verify "$signature_path" "$image_path" 2>&1 | grep 'NO_PUBKEY' | awk '{print $NF}')
if [ -n "$key_id" ]; then
info "Check if the key is already in the keyring"
if gpg --list-keys "$key_id" > /dev/null 2>&1; then
info "Key $key_id already in keyring."
else
info "Import the public key"
gpg --keyserver keyserver.ubuntu.com --recv-keys "$key_id"
fi
info "Verify the signature again after importing the key"
if gpg --verify "$signature_path" "$image_path"; then
info "Signature verification succeeded."
else
warning "Signature verification failed."
fi
else
warning "No public key found in the signature file."
fi
else
warning "Failed to download the signature file."
fi
else
warning "No signature found under $signature_download_url."
fi
make_mount_folders
set_partition_paths
question "Which filesystem should be used? E.g.:btrfs,ext4... (none):" && read -r root_filesystem
question "Should the image be transfered to $device_path?(y/N)" && read -r transfer_image
if [ "$transfer_image" = "y" ]
then
question "Should the partition table of $device_path be deleted?(y/N)" && read -r delete_partition_table
if [ "$delete_partition_table" = "y" ]
then
info "Deleting..." &&
wipefs -a "$device_path" || error
else
info "Skipping partition table deletion..."
fi
overwrite_device
info "Starting image transfer..."
if [ "$distribution" = "arch" ]
then
# Set default size of the boot partition
boot_size=${boot_size:-"+500M"}
# Use the provided size or the default size
info "The boot partition will be set to $boot_size."
# Partitioning with the specified size
info "Creating partitions..." &&
(
echo "o" # Type o. This will clear out any partitions on the drive.
echo "p" # Type p to list partitions. There should be no partitions left
echo "n" # Type n,
echo "p" # then p for primary,
echo "1" # 1 for the first partition on the drive,
echo "" # Default start sector
echo "$boot_size" # Size of the boot partition
echo "t" # Type t,
echo "c" # then c to set the first partition to type W95 FAT32 (LBA).
echo "n" # Type n,
echo "p" # then p for primary,
echo "2" # 2 for the second partition on the drive,
echo "" # Default start sector
echo "" # Default end sector
echo "w" # Write the partition table and exit by typing w.
) | fdisk "$device_path" || error
info "Format boot partition..." &&
mkfs.vfat "$boot_partition_path" || error
if [ "$encrypt_system" == "y" ]
then
# Check if luks_memory_cost is defined and set the luksAddKey command accordingly
# @see https://chatgpt.com/share/008ea5f1-670c-467c-8320-1ca67f25ac9a
if [ -n "$luks_memory_cost" ]; then
info "Formating $root_partition_path with LUKS with --pbkdf-memory set to $luks_memory_cost" &&
sudo cryptsetup -v luksFormat -c aes-xts-plain64 -s 512 -h sha512 --use-random -i 1000 --pbkdf-memory "$luks_memory_cost" "$root_partition_path" || error
else
info "Formating $root_partition_path with LUKS" &&
sudo cryptsetup -v luksFormat -c aes-xts-plain64 -s 512 -h sha512 --use-random -i 1000 "$root_partition_path" || error
fi
decrypt_root || error
fi
info "Format root partition..." &&
"mkfs.$root_filesystem" -f "$root_mapper_path" || error
mount_partitions;
info "Root files will be transfered to device..." &&
bsdtar -xpf "$image_path" -C "$root_mount_path" &&
sync ||
error
info "Boot files will be transfered to device..." &&
mv -v "$root_mount_path""boot/"* "$boot_mount_path" ||
error
elif [ "${image_path: -4}" = ".zip" ]
then
info "Transfering .zip file..." &&
unzip -p "$image_path" | sudo dd of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync status=progress || error "DD $image_path to $device_path failed." &&
sync ||
error
elif [ "${image_path: -3}" = ".gz" ]
then
info "Transfering .gz file..." &&
gunzip -c "$image_path" | sudo dd of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync status=progress &&
sync ||
error
elif [ "${image_path: -4}" = ".iso" ]
then
info "Transfering .iso file..." &&
sudo dd if="$image_path" of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync status=progress &&
sync ||
error
elif [ "${image_path: -3}" = ".xz" ]
then
info "Transferring .xz file..." &&
unxz -c "$image_path" | sudo dd of="$device_path" bs="$OPTIMAL_BLOCKSIZE" conv=fsync status=progress &&
sync ||
error
else
error "Image transfer for operation system \"$distribution\" is not supported yet!";
fi
else
info "Skipping image transfer..."
fi
# Execute Raspberry Pi specific procedures
if [ -n "$raspberry_pi_version" ]
then
info "Start regular mounting procedure..."
if mount | grep -q "$boot_partition_path"
then
info "$boot_partition_path is allready mounted..."
else
if mount | grep -q "$root_mapper_path"
then
info "$root_mapper_path is allready mounted..."
else
decrypt_root
mount_partitions
fi
fi
fstab_path="$root_mount_path""etc/fstab" &&
fstab_search_string=$(echo "/dev/mmcblk0p1"| sed -e 's/[\/&]/\\&/g') &&
fstab_replace_string=$(echo "UUID=$boot_partition_uuid"| sed -e 's/[\/&]/\\&/g') &&
info "Seeding UUID to $fstab_path to avoid path conflicts..." &&
sed -i "s/$fstab_search_string/$fstab_replace_string/g" "$fstab_path" &&
info "Content of $fstab_path:$(cat "$fstab_path")" || error
info "Define target paths..." &&
administrator_username="administrator"
target_home_path="$root_mount_path""home/" &&
default_username=$(ls "$target_home_path") &&
question "Should the $default_username be renamed to $administrator_username? (y/N):" && read -r rename_decision
if [ "$rename_decision" == "y" ];
then
variable_old_username="$default_username" &&
target_username="$administrator_username" &&
info "Rename home directory from $target_home_path$variable_old_username to $target_home_path$target_username..." &&
mv -v "$target_home_path$variable_old_username" "$target_home_path$target_username" || error "Failed to rename home directory"
else
target_username="$default_username"
fi
target_user_home_folder_path="$target_home_path$target_username/" &&
target_user_ssh_folder_path="$target_user_home_folder_path"".ssh/" &&
target_authorized_keys="$target_user_ssh_folder_path""authorized_keys" &&
# Activate later. Here was a bug
question "Should the $target_username have sudo rights? (y/N):" && read -r sudo_decision
if [ "$sudo_decision" == "y" ]; then
sudo_config_dir="$root_mount_path""etc/sudoers.d/"
sudo_config_file="$sudo_config_dir$target_username"
mkdir -vp $sudo_config_dir
echo "$target_username ALL=(ALL:ALL) ALL" > "$sudo_config_file" || error "Failed to create sudoers file for $target_username"
chmod 440 "$sudo_config_file" || error "Failed to set permissions on sudoers file for $target_username"
fi
question "Enter the path to the SSH key to be added to the image (default: none):" && read -r origin_user_rsa_pub || error
if [ -z "$origin_user_rsa_pub" ]
then
info "Skipped SSH-key copying.."
else
if [ -f "$origin_user_rsa_pub" ]
then
info "Copy ssh key to target..."
mkdir -v "$target_user_ssh_folder_path" || warning "Folder \"$target_user_ssh_folder_path\" exists. Can't be created."
cat "$origin_user_rsa_pub" > "$target_authorized_keys" &&
target_authorized_keys_content=$(cat "$target_authorized_keys") &&
info "$target_authorized_keys contains the following: $target_authorized_keys_content" &&
info "Set permissions with chmod..." &&
chmod -v 700 "$target_user_ssh_folder_path" &&
chmod -v 600 "$target_authorized_keys" || error "Failed to set ownership and permissions on ssh folder"
else
error "The ssh key \"$origin_user_rsa_pub\" can't be copied to \"$target_authorized_keys\" because it doesn't exist."
fi
fi
info "Start chroot procedures..."
mount_chroot_binds
copy_qemu
copy_resolve_conf
chroot_user_home_path="/home/$target_username/"
chroot_user_ssh_folder_path="$chroot_user_home_path.ssh"
if [ "$rename_decision" == "y" ]; then
info "Delete old user and create new user" &&
(
echo "userdel -r $variable_old_username"
echo "useradd -m -d $chroot_user_home_path -s /bin/bash $target_username"
echo "chown -R $target_username:$target_username $chroot_user_home_path"
) | chroot "$root_mount_path" /bin/bash || error "Failed to delete old user and create new user"
fi
if [ -n "$origin_user_rsa_pub" ]
then
info "Chroot to set ownership..." &&
( echo "chown -vR $target_username:$target_username $chroot_user_ssh_folder_path" ) | chroot "$root_mount_path" /bin/bash || error
fi
question "Type in new password for user root and $target_username (leave empty to skip): " && read -r password_1
if [ -n "$password_1" ]; then
question "Repeat new password for \"$target_username\": " && read -r password_2
if [ "$password_1" = "$password_2" ]; then
info "Changing passwords on target system..."
(
echo "(
echo '$password_1'
echo '$password_1'
) | passwd $target_username"
echo "(
echo '$password_1'
echo '$password_1'
) | passwd"
) | chroot "$root_mount_path" /bin/bash || error "Failed to change password."
else
error "Passwords didn't match."
fi
else
info "No password change requested, skipped password change..."
fi
hostname_path="$root_mount_path/etc/hostname"
question "Type in the hostname (leave empty to skip): " && read -r target_hostname
if [ -n "$target_hostname" ]; then
echo "$target_hostname" > "$hostname_path" || error "Failed to set hostname."
else
target_hostname=$(cat "$hostname_path")
info "No hostname change requested, skipped hostname change..."
fi
info "Used hostname is: $target_hostname"
case "$distribution" in
"arch"|"manjaro")
info "Populating keys..." &&
(
echo "yes | pacman-key --init"
echo "yes | pacman-key --populate archlinuxarm"
) | chroot "$root_mount_path" /bin/bash || error
;;
esac
question "Should the system be updated?(y/N)" && read -r update_system
if [ "$update_system" == "y" ]
then
info "Updating system..."
case "$distribution" in
"arch"|"manjaro")
echo "pacman --noconfirm -Syyu" | chroot "$root_mount_path" /bin/bash || error
;;
"moode"|"retropie")
(
echo "yes | apt update"
echo "yes | apt upgrade"
) | chroot "$root_mount_path" /bin/bash || error
;;
*)
warning "System update for operation system \"$distribution\" is not supported yet. Skipped."
;;
esac
fi
info "Installing software for filesystem $root_filesystem..."
if [ "$root_filesystem" == "btrfs" ]
then
install "btrfs-progs"
else
info "Skipped."
fi
if [ "$encrypt_system" == "y" ]
then
# Adapted this instruction for setting up encrypted systems
# @see https://gist.github.com/gea0/4fc2be0cb7a74d0e7cc4322aed710d38
# @see https://gist.github.com/EnigmaCurry/2f9bed46073da8e38057fe78a61e7994
info "Setup encryption..." &&
info "Installing neccessary software..." &&
install "$(get_packages "server/luks")" &&
dropbear_root_key_path="$root_mount_path""etc/dropbear/root_key" &&
info "Adding $target_authorized_keys to dropbear..." &&
cp -v "$target_authorized_keys" "$dropbear_root_key_path" &&
# Concerning mkinitcpio warning
# @see https://gist.github.com/imrvelj/c65cd5ca7f5505a65e59204f5a3f7a6d
mkinitcpio_path="$root_mount_path""etc/mkinitcpio.conf" &&
info "Configuring $mkinitcpio_path..." &&
mkinitcpio_search_modules="MODULES=()" || error
# Concerning which moduls to load
# @see https://raspberrypi.stackexchange.com/questions/67051/raspberry-pi-3-with-archarm-and-encrypted-disk-will-not-boot-how-can-be-identif
case "$raspberry_pi_version" in
"1" | "2")
mkinitcpio_additional_modules=""
;;
"3b")
mkinitcpio_additional_modules="smsc95xx"
;;
"3b+" | "4")
mkinitcpio_additional_modules="lan78xx"
;;
*)
warning "Version $raspberry_pi_version isn't supported."
;;
esac
mkinitcpio_replace_modules="MODULES=(g_cdc usb_f_acm usb_f_ecm $mkinitcpio_additional_modules g_ether)" || error
mkinitcpio_search_binaries="BINARIES=()" &&
mkinitcpio_replace_binaries=$(echo "BINARIES=(/usr/lib/libgcc_s.so.1)"| sed -e 's/[\/&]/\\&/g') &&
mkinitcpio_encrypt_hooks="sleep netconf dropbear encryptssh" &&
mkinitcpio_hooks_prefix="base udev autodetect microcode modconf kms keyboard keymap consolefont block"
mkinitcpio_hooks_suffix="filesystems fsck"
mkinitcpio_search_hooks="HOOKS=($mkinitcpio_hooks_prefix $mkinitcpio_hooks_suffix)" &&
mkinitcpio_replace_hooks="HOOKS=($mkinitcpio_hooks_prefix $mkinitcpio_encrypt_hooks $mkinitcpio_hooks_suffix)" &&
replace_in_file "$mkinitcpio_search_modules" "$mkinitcpio_replace_modules" "$mkinitcpio_path" &&
replace_in_file "$mkinitcpio_search_binaries" "$mkinitcpio_replace_binaries" "$mkinitcpio_path" &&
replace_in_file "$mkinitcpio_search_hooks" "$mkinitcpio_replace_hooks" "$mkinitcpio_path" &&
info "Content of $mkinitcpio_path:$(cat "$mkinitcpio_path")" &&
info "Generating mkinitcpio..." &&
echo "mkinitcpio -vP" | chroot "$root_mount_path" /bin/bash &&
fstab_insert_line="UUID=$root_partition_uuid / $root_filesystem defaults,noatime 0 1" &&
info "Configuring $fstab_path..." || error
if grep -q "$fstab_insert_line" "$fstab_path"
then
warning "$fstab_path contains allready $fstab_insert_line - Skipped."
else
echo "$fstab_insert_line" >> "$fstab_path" || error
fi
info "Content of $fstab_path:$(cat "$fstab_path")" &&
crypttab_path="$root_mount_path""etc/crypttab" &&
crypttab_insert_line="$root_mapper_name UUID=$root_partition_uuid none luks" &&
info "Configuring $crypttab_path..." || error
if grep -q "$crypttab_insert_line" "$crypttab_path"
then
warning "$crypttab_path contains allready $crypttab_insert_line - Skipped."
else
echo "$crypttab_insert_line" >> "$crypttab_path" || error
fi
info "Content of $crypttab_path:$(cat "$crypttab_path")" &&
boot_txt_path="$boot_mount_path""boot.txt" &&
cryptdevice_configuration="cryptdevice=UUID=$root_partition_uuid:$root_mapper_name root=$root_mapper_path" || error
if [ -f "$boot_txt_path" ];
then
info "Configuring $boot_txt_path..." &&
boot_txt_delete_line=$(echo "part uuid \${devtype} \${devnum}:2 uuid" | sed -e 's/[]\/$*.^[]/\\&/g') &&
boot_txt_setenv_origin=$(echo "setenv bootargs console=ttyS1,115200 console=tty0 root=PARTUUID=\${uuid} rw rootwait smsc95xx.macaddr=\"\${usbethaddr}\"" | sed -e 's/[]\/$*.^[]/\\&/g') &&
# Concerning issues with network adapter names;
# @see https://forum.iobroker.net/topic/40542/raspberry-pi4-kein-eth0-mehr/16
boot_txt_setenv_replace=$(echo "setenv bootargs console=ttyS1,115200 console=tty0 ip=::::$target_hostname:eth0:dhcp $cryptdevice_configuration rw rootwait smsc95xx.macaddr=\"\${usbethaddr}\" net.ifnames=0 biosdevname=0"| sed -e 's/[\/&]/\\&/g') &&
replace_in_file "$boot_txt_delete_line" "" "$boot_txt_path" &&
replace_in_file "$boot_txt_setenv_origin" "$boot_txt_setenv_replace" "$boot_txt_path" &&
info "Content of $boot_txt_path:$(cat "$boot_txt_path")" &&
info "Generating..." &&
echo "cd /boot/ && ./mkscr || exit 1" | chroot "$root_mount_path" /bin/bash || error
else
cmdline_txt_path="$boot_mount_path""cmdline.txt" &&
info "Configuring $cmdline_txt_path..." &&
cmdline_search_string=$(echo "root=/dev/mmcblk0p2" | sed -e 's/[\/&]/\\&/g') &&
cmdline_replace_string=$(echo "$cryptdevice_configuration rootfstype=$root_filesystem"| sed -e 's/[\/&]/\\&/g') &&
replace_in_file "$cmdline_search_string" "$cmdline_replace_string" "$cmdline_txt_path" &&
info "Content of $cmdline_txt_path:$(cat "$cmdline_txt_path")" || error
fi
fi
info "Running system specific procedures..."
if [ "$distribution" = "retropie" ]
then
if [ -n "$origin_user_rsa_pub" ]
then
ssh_file="$boot_mount_path""ssh" &&
echo "" > "$ssh_file"
fi
question "Should the RetroFlag specific procedures be executed?(y/N)" && read -r setup_retroflag
if [ "$setup_retroflag" == "y" ]
then
info "Executing RetroFlag specific procedures..." &&
(
echo 'wget -O - "https://raw.githubusercontent.com/RetroFlag/retroflag-picase/master/install_gpi.sh" | bash'
) | chroot "$root_mount_path" /bin/bash || error
fi
fi
fi
destructor
success "Setup successfull :)" && exit 0

0
tests/__init__.py Normal file
View File

108
tests/conftest.py Normal file
View File

@@ -0,0 +1,108 @@
import subprocess
import sys
from pathlib import Path
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from lim import runner, ui
from lim.errors import LimError
class FakeRunner:
"""Records every command instead of executing it.
``outputs``/``success_by_fragment``/``failures`` map a substring of the
joined command to the canned behavior.
"""
def __init__(self):
self.calls: list[tuple[str, list, str | None]] = []
self.outputs: dict[str, str] = {}
self.failures: set[str] = set()
self.success_by_fragment: dict[str, bool] = {}
self.sudo_log: list[tuple[list[str], bool]] = []
def _fails(self, cmd: list[str]) -> bool:
joined = " ".join(cmd)
return any(fragment in joined for fragment in self.failures)
def run(self, cmd, *, sudo=False, input_text=None, check=True, error_msg=None):
cmd = [str(part) for part in cmd]
self.calls.append(("run", cmd, input_text))
self.sudo_log.append((cmd, sudo))
returncode = 1 if self._fails(cmd) else 0
if check and returncode != 0:
raise LimError(error_msg or f"Command failed: {' '.join(cmd)}")
return subprocess.CompletedProcess(cmd, returncode)
def output(self, cmd, *, sudo=False, check=True, error_msg=None):
cmd = [str(part) for part in cmd]
self.calls.append(("output", cmd, None))
joined = " ".join(cmd)
for fragment, value in self.outputs.items():
if fragment in joined:
return value
return ""
def succeeds(self, cmd, *, sudo=False):
cmd = [str(part) for part in cmd]
self.calls.append(("succeeds", cmd, None))
joined = " ".join(cmd)
for fragment, value in self.success_by_fragment.items():
if fragment in joined:
return value
return False
def pipeline(self, *cmds, sudo_last=False, error_msg=None):
cmds = [[str(part) for part in cmd] for cmd in cmds]
self.calls.append(("pipeline", cmds, None))
for cmd in cmds:
if self._fails(cmd):
raise LimError(error_msg or "Pipeline failed")
def sync_disks(self):
self.calls.append(("run", ["sync"], None))
def commands(self) -> list[list[str]]:
return [cmd for _, cmd, _ in self.calls]
def find(self, *fragments: str) -> list[list[str]]:
"""All recorded commands whose joined form contains every fragment."""
result = []
for command in self.commands():
joined = " ".join(
" ".join(part) if isinstance(part, list) else part for part in command
)
if all(fragment in joined for fragment in fragments):
result.append(command)
return result
@pytest.fixture
def fake_runner(monkeypatch):
fake = FakeRunner()
monkeypatch.setattr(runner, "run", fake.run)
monkeypatch.setattr(runner, "output", fake.output)
monkeypatch.setattr(runner, "succeeds", fake.succeeds)
monkeypatch.setattr(runner, "pipeline", fake.pipeline)
monkeypatch.setattr(runner, "sync_disks", fake.sync_disks)
return fake
@pytest.fixture
def answers(monkeypatch):
"""Feed scripted answers to ui.ask (and thereby ui.confirm)."""
queue: list[str] = []
def fake_ask(prompt: str) -> str:
assert queue, f"No scripted answer left for prompt: {prompt}"
return queue.pop(0)
monkeypatch.setattr(ui, "ask", fake_ask)
def feed(*items: str) -> None:
queue.extend(items)
return feed

0
tests/e2e/__init__.py Normal file
View File

109
tests/e2e/qemu/README.md Normal file
View File

@@ -0,0 +1,109 @@
# Virtualized end-to-end Tor unlock harness
This harness models the **entire** encrypted-image process in a virtual
machine: it builds a LUKS image carrying the real `lim` Tor-in-initramfs
artifacts, boots it in QEMU, lets the real `netconf → tor → dropbear →
encryptssh` chain run in early userspace, then unlocks the root over Tor and
proves the real system booted.
It complements the lightweight, always-rootless
[`test_tor_unlock_e2e.py`](../test_tor_unlock_e2e.py): that one round-trips a
passphrase through Tor against a dropbear *stand-in*; this one runs the real
dropbear, real cryptsetup, real mkinitcpio initramfs, and a real tor daemon in
a booted machine.
## Why a virtio machine and not an emulated Raspberry Pi
QEMU's `raspi3b`/`raspi4b` machines do not emulate the Pi's USB-gadget
Ethernet — exactly the network path the Pi images use in the initramfs
(`g_ether`/`smsc95xx`/`lan78xx`). Without early networking there is no Tor and
no unlock, so emulating the literal board is pointless here. Instead we
reproduce the same *software stack* (same HOOKS chain, same tor hook, same
torrc and onion keys) on a QEMU-friendly `virt`/`q35` machine with
`virtio-net`. The only deltas from a real Pi image are the NIC driver and the
CPU architecture.
## Stages
| Stage | File | Privilege |
|---|---|---|
| Build encrypted image (Arch) | `build_image.sh` | **root** (loop, cryptsetup, chroot) |
| Build encrypted image (Debian) | `build_image_debian.sh` | **root** (debootstrap, loop, cryptsetup, chroot) |
| Tor network | `tor_net.py` | rootless |
| Boot + unlock | `boot_unlock.py` | rootless (QEMU user-mode net) |
| Orchestration | `harness.py` | mixed (uses `sudo` for the build only) |
| Pure command builders | `config.py` | none — unit-tested offline |
## Keeping the host rootless
Only `build_image.sh` needs root (there is no rootless dm-crypt). To keep your
host unprivileged, run the whole harness **inside a throwaway VM** that has
`/dev/kvm`, and drive it from there. QEMU itself, the tor client and the unlock
are rootless: user-mode networking (`-netdev user`) needs no tap device, and a
Tor onion service needs no inbound port forward (its rendezvous is outbound
from both ends).
When run directly on the host, `harness.py` invokes the build via `sudo -E`
and then `chown`s the artifacts back so rootless QEMU can read them.
## Requirements
- `qemu-system-x86_64` (or `-aarch64` for `LIM_E2E_ARCH=aarch64`)
- `arch-install-scripts` (`pacstrap`, `arch-chroot`)
- `cryptsetup`, `mkinitcpio`, `tor`, `openssh`, `ncat` (SOCKS5 ProxyCommand)
- Network access to the public Tor network, **or** `chutney` for a private one
- `/dev/kvm` recommended (TCG works but is slow; aarch64-on-x86 is always TCG)
For the Debian build (`LIM_E2E_OS=debian`), additionally: `debootstrap` and
network access to a Debian mirror. The guest uses the initramfs-tools backend
(cryptsetup-initramfs + dropbear-initramfs), so the unlock SSH session runs
`cryptroot-unlock` instead of Arch's login-time encryptssh prompt.
## Unlock transport: direct vs Tor
`LIM_E2E_TRANSPORT` (default `direct`) selects how the passphrase reaches the
guest's dropbear:
- **direct** — QEMU forwards a host port to the guest's dropbear (`:22`) and the
passphrase is delivered over a plain SSH. Deterministic; it verifies the whole
LUKS-unlock stack (cryptsetup-initramfs + dropbear-initramfs + cryptroot-unlock
+ LUKS + boot). This is what CI/`make` runs.
- **tor** — the full onion path: the guest publishes its onion and the host
reaches it through Tor. Representative of production but **flaky over public
Tor inside QEMU** (fresh guest Tor + user-mode net make the onion rendezvous
unreliable), so it is opt-in. The onion transport itself is covered
deterministically by the rootless `test_tor_unlock_e2e.py`.
## Running
```bash
# Arch (mkinitcpio), direct transport (deterministic, default):
LIM_E2E_QEMU=1 pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
# Debian (initramfs-tools) via debootstrap, direct transport:
LIM_E2E_QEMU=1 LIM_E2E_OS=debian pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
# Full onion transport (opt-in, flaky over public Tor):
LIM_E2E_QEMU=1 LIM_E2E_TRANSPORT=tor pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
```
`make test-qemu` / `make test-qemu-debian` wrap the direct-transport runs.
Environment knobs: `LIM_E2E_ARCH` (`x86_64` default, or `aarch64`),
`CHUTNEY_PATH` (enables the private network), `CHUTNEY_PATH` unset → public.
## Determinism note (chutney)
For a fully private loop the **guest image** must trust the same authorities as
the host client. `ChutneyNetwork.test_network_conf()` distils the
`TestingTorNetwork`/`DirAuthority` lines (rewriting `127.0.0.1` to the QEMU
user-net host alias `10.0.2.2`) into a file, which `build_image.sh` appends to
the baked-in torrc via `TOR_TEST_NETWORK_CONF`. The offline onion key
generation is unaffected — it never touches the network either way.
## What runs in CI without any of this
The pure logic (`config.py`, the env parser, the chutney torrc parsing) and the
drift guards that keep `build_image.sh` aligned with what the harness expects
are covered by [`test_qemu_harness_unit.py`](../test_qemu_harness_unit.py),
which runs in the normal `pytest` suite — no QEMU, root, or network.

View File

View File

@@ -0,0 +1,212 @@
"""Boot a built image in QEMU and unlock its LUKS root over Tor.
This is the live half of the harness. QEMU runs rootless (user-mode net);
the passphrase is delivered exactly as an operator would: SSH through Tor to
the initramfs dropbear, piping the passphrase into the encryptssh prompt.
Success is observed on the serial console, not inferred.
"""
import contextlib
import subprocess
import threading
import time
from pathlib import Path
from tests.e2e.qemu import config
from tests.e2e.qemu.config import QemuSpec
# One generous budget covers guest boot + tor bootstrap + onion publish +
# unlock. TCG (no KVM) is slow, so keep it roomy.
UNLOCK_TIMEOUT = 480
DELIVERY_INTERVAL = 8
# Hold off the first delivery until the guest onion has had time to publish.
# A connect to a not-yet-published onion makes the client Tor cache the failed
# descriptor fetch and back off, so early hammering can lock us out for
# minutes; waiting until the onion is up means the first fetch succeeds.
INITIAL_DELIVERY_DELAY = 100
_SSH_ATTEMPT_TIMEOUT = 45
_POLL_INTERVAL = 2
_PROMPT_WAIT = 4 # let the encryptssh prompt appear before sending
def _serial_contains(spec: QemuSpec, needle: str) -> bool:
log = spec.serial_log
return log.is_file() and needle in log.read_text(errors="replace")
def _stream_new_serial(spec: QemuSpec, offset: int) -> int:
"""Print serial output written since ``offset``; return the new offset.
Streams the guest's boot messages (netconf, the tor hook, dropbear,
encryptssh) live so the operator sees the VM working, not a black box.
"""
log = spec.serial_log
if not log.is_file():
return offset
data = log.read_bytes()
if len(data) > offset:
print(data[offset:].decode(errors="replace"), end="", flush=True)
return len(data)
def _serial_tail(spec: QemuSpec, lines: int = 15) -> str:
if not spec.serial_log.is_file():
return "(no serial output captured)"
tail = spec.serial_log.read_text(errors="replace").splitlines()[-lines:]
return "serial tail:\n " + "\n ".join(tail)
def _qemu_stderr_path(spec: QemuSpec) -> Path:
return spec.work_dir / "qemu.stderr.log"
def _qemu_stderr_tail(spec: QemuSpec, lines: int = 10) -> str:
path = _qemu_stderr_path(spec)
if not path.is_file() or not path.read_text(errors="replace").strip():
return ""
tail = path.read_text(errors="replace").splitlines()[-lines:]
return "\nqemu stderr:\n " + "\n ".join(tail)
def _ssh_log_path(spec: QemuSpec) -> Path:
return spec.work_dir / "ssh-delivery.log"
def _ssh_log_tail(spec: QemuSpec, lines: int = 15) -> str:
path = _ssh_log_path(spec)
if not path.is_file() or not path.read_text(errors="replace").strip():
return ""
tail = path.read_text(errors="replace").splitlines()[-lines:]
return "\nssh delivery log:\n " + "\n ".join(tail)
def _client_tor_tail(spec: QemuSpec, lines: int = 20) -> str:
"""Return the host Tor client's HS log — onion descriptor fetch/rendezvous."""
path = spec.work_dir / "tor-client-hs.log"
if not path.is_file() or not path.read_text(errors="replace").strip():
return ""
lines_out = [
line
for line in path.read_text(errors="replace").splitlines()
if any(k in line.lower() for k in ("desc", "rend", "intro", "hs_", "onion"))
][-lines:]
return "\nclient tor HS log:\n " + "\n ".join(lines_out)
def _deliver_worker(spec: QemuSpec) -> None:
"""Feed the passphrase over an SSH-through-Tor session, held open.
Runs in a background thread so the main loop keeps streaming serial. The
channel is deliberately kept OPEN after sending: closing stdin immediately
(as a plain pipe does) tears the session down before cryptsetup finishes
its argon2 KDF, killing the unlock mid-flight. Session output is appended to
ssh-delivery.log so a failed unlock (onion unreachable, cryptroot-unlock
error) is diagnosable.
"""
log_file = _ssh_log_path(spec).open("a")
log_file.write("--- delivery attempt ---\n")
log_file.flush()
proc = subprocess.Popen(
config.ssh_argv(spec),
stdin=subprocess.PIPE,
stdout=log_file,
stderr=log_file,
text=True,
)
try:
time.sleep(_PROMPT_WAIT)
if proc.poll() is None and proc.stdin is not None:
try:
proc.stdin.write(spec.passphrase + "\n")
proc.stdin.flush()
except OSError:
pass
proc.wait(timeout=_SSH_ATTEMPT_TIMEOUT)
except subprocess.TimeoutExpired:
proc.kill()
finally:
if proc.stdin is not None:
with contextlib.suppress(OSError):
proc.stdin.close()
log_file.close()
def boot_and_unlock(spec: QemuSpec) -> None:
"""Boot the image, unlock it over Tor, and confirm the real system booted.
Returns normally only when the boot-ok marker appears on the serial
console; the passphrase is (re)delivered on each poll until then. Raises
with the serial tail on QEMU death or timeout.
"""
if spec.serial_log.exists():
spec.serial_log.unlink()
if spec.direct_ssh_port:
target = f"127.0.0.1:{spec.direct_ssh_port} -> guest dropbear (direct)"
# dropbear comes up within seconds; no need to wait for an onion.
initial_delay = 15
else:
target = f"{spec.onion_address}:22 (Tor onion)"
initial_delay = INITIAL_DELIVERY_DELAY
print(f"\n========== booting in QEMU ({spec.arch}, accel={spec.accel}) ==========")
print(f"Unlock target: {target} (budget {UNLOCK_TIMEOUT}s)")
print("---- guest serial console ----", flush=True)
# QEMU's own diagnostics (bad -machine/-cpu, /dev/kvm denied, image busy)
# go to stderr; capture them so an early exit is debuggable, not opaque.
qemu_stderr = _qemu_stderr_path(spec).open("wb")
qemu = subprocess.Popen(
config.qemu_argv(spec),
stdout=subprocess.DEVNULL,
stderr=qemu_stderr,
)
try:
deadline = time.monotonic() + UNLOCK_TIMEOUT
offset = 0
# Hold off the first delivery until the unlock endpoint is up (see above).
next_delivery = time.monotonic() + initial_delay
print(
f"[harness] waiting {initial_delay}s for the unlock endpoint before delivering...",
flush=True,
)
delivery: threading.Thread | None = None
while time.monotonic() < deadline:
offset = _stream_new_serial(spec, offset)
if _serial_contains(spec, config.BOOT_OK_MARKER):
print("\n[harness] boot-ok marker seen — LUKS root unlocked and booted.")
return
if qemu.poll() is not None:
raise RuntimeError(
f"QEMU exited early (code {qemu.returncode}).\n"
f"{_serial_tail(spec)}{_qemu_stderr_tail(spec)}"
)
# One delivery at a time; each holds its SSH session open in the
# background while we keep streaming serial and watching the marker.
idle = delivery is None or not delivery.is_alive()
if idle and time.monotonic() >= next_delivery:
print("\n[harness] delivering passphrase (holding session open) ...", flush=True)
delivery = threading.Thread(target=_deliver_worker, args=(spec,), daemon=True)
delivery.start()
next_delivery = time.monotonic() + DELIVERY_INTERVAL
time.sleep(_POLL_INTERVAL)
raise RuntimeError(
f"Boot-ok marker never appeared within {UNLOCK_TIMEOUT}s.\n"
f"{_serial_tail(spec, lines=45)}{_qemu_stderr_tail(spec)}"
f"{_ssh_log_tail(spec)}{_client_tor_tail(spec)}"
)
finally:
qemu.terminate()
try:
qemu.wait(timeout=15)
except subprocess.TimeoutExpired:
qemu.kill()
qemu.wait(timeout=10)
qemu_stderr.close()
def parse_env_file(path: Path) -> dict[str, str]:
"""Read the KEY=VALUE image.env written by build_image.sh into a dict."""
result: dict[str, str] = {}
for line in path.read_text().splitlines():
if "=" in line and not line.startswith("#"):
key, value = line.split("=", 1)
result[key.strip()] = value.strip()
return result

182
tests/e2e/qemu/build_image.sh Executable file
View File

@@ -0,0 +1,182 @@
#!/bin/bash
# Build a QEMU-bootable, LUKS-encrypted image that reproduces lim's Tor
# unlock stack: the REAL initcpio files from lim/configuration/initcpio/,
# the tor hook wired between netconf and dropbear, offline onion keys, and
# a boot-ok marker service. Boots on a virtio machine (we model the stack,
# not the Raspberry Pi board, whose USB-gadget net QEMU cannot emulate).
#
# Needs root (loop device, cryptsetup, chroot). To keep a host rootless,
# run this inside a throwaway VM — see tests/e2e/qemu/README.md.
#
# Required environment:
# WORK_DIR writable scratch directory (outputs land here)
# REPO_ROOT linux-image-manager checkout (for the real initcpio files)
# ARCH x86_64 (default) or aarch64
# PASSPHRASE LUKS passphrase the harness will send over Tor
# SSH_PUBKEY path to the public key authorized for dropbear
#
# Writes $WORK_DIR/image.env with LUKS_UUID / MAPPER_NAME / KERNEL / INITRAMFS
# / IMAGE for the Python harness to consume.
set -euo pipefail
: "${WORK_DIR:?set WORK_DIR}"
: "${REPO_ROOT:?set REPO_ROOT}"
: "${PASSPHRASE:?set PASSPHRASE}"
: "${SSH_PUBKEY:?set SSH_PUBKEY}"
ARCH="${ARCH:-x86_64}"
MAPPER_NAME="cryptroot"
ROOTFS="$WORK_DIR/rootfs"
IMAGE="$WORK_DIR/image.raw"
INITCPIO_SRC="$REPO_ROOT/lim/configuration/initcpio"
case "$ARCH" in
x86_64) KERNEL_PKG=linux; CONSOLE=ttyS0 ;;
aarch64) KERNEL_PKG=linux; CONSOLE=ttyAMA0 ;;
*) echo "Unsupported ARCH: $ARCH" >&2; exit 2 ;;
esac
banner() { printf "\n========== %s ==========\n" "$1"; }
cleanup() {
set +e
mountpoint -q "$WORK_DIR/mnt" && umount -R "$WORK_DIR/mnt"
[ -e "/dev/mapper/$MAPPER_NAME" ] && cryptsetup close "$MAPPER_NAME"
[ -n "${LOOP:-}" ] && losetup -d "$LOOP"
}
trap cleanup EXIT
banner "bootstrapping rootfs with pacstrap"
mkdir -p "$ROOTFS"
# -c reuses the host package cache (fast). If a cached package is corrupt,
# clear it first: sudo rm -f /var/cache/pacman/pkg/<pkg>-*.pkg.tar.zst*
# A file of blank lines on stdin takes the default answer for every provider
# and confirmation prompt (e.g. the `linux` virtual package), keeping the
# bootstrap non-interactive without the SIGPIPE/pipefail pitfalls of `yes`.
printf '\n%.0s' $(seq 100) > "$WORK_DIR/pacman-answers"
pacstrap -c "$ROOTFS" \
base "$KERNEL_PKG" mkinitcpio cryptsetup \
busybox tor openssh \
mkinitcpio-utils mkinitcpio-netconf mkinitcpio-dropbear \
< "$WORK_DIR/pacman-answers"
banner "configuring mkinitcpio (virtio modules, tor between netconf/dropbear)"
# No `autodetect`: it narrows the initramfs to the *build host's* modules and
# would strip the virtio drivers the guest VM needs for early networking.
cat > "$ROOTFS/etc/mkinitcpio.conf" <<EOF
MODULES=(virtio_pci virtio_blk virtio_net)
BINARIES=(/usr/lib/libgcc_s.so.1)
FILES=()
HOOKS=(base udev modconf keyboard block ptsmount netconf tor dropbear encryptssh filesystems fsck)
EOF
banner "adding a devpts-mount hook (dropbear needs a PTY for encryptssh)"
# Without a mounted devpts, dropbear cannot allocate /dev/pts/0, so the
# interactive encryptssh passphrase prompt never runs and the SSH session
# just exits ("TIOCSCTTY: Input/output error", "/dev/pts/0: No such file").
cat > "$ROOTFS/etc/initcpio/install/ptsmount" <<'EOF'
#!/bin/bash
build() { add_runscript; }
help() { echo "Mount devpts so dropbear can allocate PTYs for encryptssh."; }
EOF
cat > "$ROOTFS/etc/initcpio/hooks/ptsmount" <<'EOF'
#!/usr/bin/ash
run_hook() {
mkdir -p /dev/pts
mount -t devpts devpts /dev/pts -o mode=620,gid=5,ptmxmode=666 2>/dev/null
[ -e /dev/ptmx ] || ln -s /dev/pts/ptmx /dev/ptmx
}
EOF
banner "installing the real lim initcpio files"
install -D -m0644 "$INITCPIO_SRC/tor_install" "$ROOTFS/etc/initcpio/install/tor"
install -D -m0644 "$INITCPIO_SRC/tor_hook" "$ROOTFS/etc/initcpio/hooks/tor"
install -D -m0644 "$INITCPIO_SRC/torrc" "$ROOTFS/etc/tor/initramfs-torrc"
if [ -n "${TOR_TEST_NETWORK_CONF:-}" ]; then
banner "wiring the image to a private (chutney) Tor network"
cat "$TOR_TEST_NETWORK_CONF" >> "$ROOTFS/etc/tor/initramfs-torrc"
fi
banner "generating onion keys offline"
install -d -m0700 "$ROOTFS/etc/tor/initramfs-onion"
: > "$WORK_DIR/keygen-torrc"
tor -f "$WORK_DIR/keygen-torrc" --DisableNetwork 1 \
--DataDirectory "$WORK_DIR/keygen-data" \
--HiddenServiceDir "$ROOTFS/etc/tor/initramfs-onion" \
--HiddenServicePort "22 127.0.0.1:22" \
--SocksPort 0 --RunAsDaemon 0 --Log "notice stderr" &
keygen_pid=$!
for _ in $(seq 1 30); do
[ -s "$ROOTFS/etc/tor/initramfs-onion/hostname" ] && break
sleep 1
done
kill "$keygen_pid" 2>/dev/null || true
ONION_ADDRESS="$(cat "$ROOTFS/etc/tor/initramfs-onion/hostname")"
banner "authorizing the ssh key for dropbear"
install -D -m0600 "$SSH_PUBKEY" "$ROOTFS/etc/dropbear/root_key"
banner "baking the boot-ok marker service"
cat > "$ROOTFS/etc/systemd/system/lim-boot-ok.service" <<'EOF'
[Unit]
Description=Signal a successful LUKS-unlock boot to the serial console
After=multi-user.target
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo LIM_UNLOCK_BOOT_OK > /dev/console'
[Install]
WantedBy=multi-user.target
EOF
ln -sf /etc/systemd/system/lim-boot-ok.service \
"$ROOTFS/etc/systemd/system/multi-user.target.wants/lim-boot-ok.service"
ln -sf "/usr/lib/systemd/system/getty@.service" \
"$ROOTFS/etc/systemd/system/getty.target.wants/getty@$CONSOLE.service"
banner "generating the initramfs"
# arch-chroot binds /proc, /sys, /dev itself; no pre-mount needed.
arch-chroot "$ROOTFS" mkinitcpio -P
banner "creating the LUKS image"
truncate -s 4G "$IMAGE"
LOOP="$(losetup -f --show "$IMAGE")"
# Cap the argon2 memory so the unlock fits in the guest's RAM (see QemuSpec).
printf '%s' "$PASSPHRASE" | cryptsetup luksFormat --type luks2 --batch-mode \
--pbkdf-memory 262144 "$LOOP" -
LUKS_UUID="$(cryptsetup luksUUID "$LOOP")"
printf '%s' "$PASSPHRASE" | cryptsetup open "$LOOP" "$MAPPER_NAME" -
mkfs.ext4 -q "/dev/mapper/$MAPPER_NAME"
banner "copying the rootfs into the encrypted volume"
mkdir -p "$WORK_DIR/mnt"
mount "/dev/mapper/$MAPPER_NAME" "$WORK_DIR/mnt"
cp -a "$ROOTFS/." "$WORK_DIR/mnt/"
echo "/dev/mapper/$MAPPER_NAME / ext4 defaults,noatime 0 1" > "$WORK_DIR/mnt/etc/fstab"
# Kernel/initramfs filenames differ per distro (Arch: vmlinuz-linux /
# initramfs-linux.img; Manjaro: vmlinuz-6.1-x86_64 / initramfs-6.1-x86_64.img).
# Discover them instead of hard-coding, and skip the larger fallback image.
KERNEL_IMG="$(find "$ROOTFS/boot" -maxdepth 1 -name 'vmlinuz-*' | sort | head -1)"
INITRAMFS_IMG="$(find "$ROOTFS/boot" -maxdepth 1 -name 'initramfs-*.img' \
! -name '*fallback*' | sort | head -1)"
[ -n "$KERNEL_IMG" ] || { echo "No kernel image in $ROOTFS/boot" >&2; exit 1; }
[ -n "$INITRAMFS_IMG" ] || { echo "No initramfs in $ROOTFS/boot" >&2; exit 1; }
echo "Using kernel $KERNEL_IMG and initramfs $INITRAMFS_IMG"
cp "$KERNEL_IMG" "$WORK_DIR/kernel"
cp "$INITRAMFS_IMG" "$WORK_DIR/initramfs"
sync
umount -R "$WORK_DIR/mnt"
cryptsetup close "$MAPPER_NAME"
losetup -d "$LOOP"
LOOP=""
banner "writing image.env"
cat > "$WORK_DIR/image.env" <<EOF
LUKS_UUID=$LUKS_UUID
MAPPER_NAME=$MAPPER_NAME
ONION_ADDRESS=$ONION_ADDRESS
IMAGE=$IMAGE
KERNEL=$WORK_DIR/kernel
INITRAMFS=$WORK_DIR/initramfs
EOF
echo "Build complete. Onion address: $ONION_ADDRESS"

View File

@@ -0,0 +1,152 @@
#!/bin/bash
# Build a QEMU-bootable, LUKS-encrypted Debian image that reproduces lim's Tor
# unlock stack for the initramfs-tools backend: the REAL hooks from
# lim/configuration/initramfs-tools/, cryptsetup-initramfs + dropbear-initramfs,
# offline onion keys, and a boot-ok marker service. Boots on a virtio machine
# (models the software stack, not the Raspberry Pi board).
#
# Debian amd64 on an x86_64 host — no qemu-user needed. Needs root (debootstrap,
# loop device, cryptsetup, chroot). See tests/e2e/qemu/README.md.
#
# Same env contract and image.env output as build_image.sh.
set -euo pipefail
: "${WORK_DIR:?set WORK_DIR}"
: "${REPO_ROOT:?set REPO_ROOT}"
: "${PASSPHRASE:?set PASSPHRASE}"
: "${SSH_PUBKEY:?set SSH_PUBKEY}"
SUITE="${DEBIAN_SUITE:-bookworm}"
MIRROR="${DEBIAN_MIRROR:-http://deb.debian.org/debian}"
MAPPER_NAME="cryptroot"
ROOTFS="$WORK_DIR/rootfs"
IMAGE="$WORK_DIR/image.raw"
HOOK_SRC="$REPO_ROOT/lim/configuration/initramfs-tools"
banner() { printf "\n========== %s ==========\n" "$1"; }
# Debian keeps tools in /usr/sbin, which an Arch host's PATH (merged /usr/bin)
# omits, so chroot can't find update-initramfs. Force a Debian-style PATH.
in_chroot() {
chroot "$ROOTFS" /usr/bin/env \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin "$@"
}
cleanup() {
set +e
for m in dev/pts dev proc sys; do
mountpoint -q "$ROOTFS/$m" && umount -l "$ROOTFS/$m"
done
mountpoint -q "$WORK_DIR/mnt" && umount -R "$WORK_DIR/mnt"
[ -e "/dev/mapper/$MAPPER_NAME" ] && cryptsetup close "$MAPPER_NAME"
[ -n "${LOOP:-}" ] && losetup -d "$LOOP"
}
trap cleanup EXIT
banner "bootstrapping Debian rootfs with debootstrap ($SUITE)"
mkdir -p "$ROOTFS"
debootstrap --arch=amd64 --variant=minbase \
--include=linux-image-amd64,cryptsetup,cryptsetup-initramfs,dropbear-initramfs,tor,busybox,systemd-sysv,udev,ifupdown,isc-dhcp-client \
"$SUITE" "$ROOTFS" "$MIRROR"
banner "binding chroot filesystems"
cp /etc/resolv.conf "$ROOTFS/etc/resolv.conf"
for m in proc sys dev dev/pts; do
mkdir -p "$ROOTFS/$m"
mount --bind "/$m" "$ROOTFS/$m"
done
banner "creating the LUKS image (UUID must exist before update-initramfs)"
truncate -s 6G "$IMAGE"
LOOP="$(losetup -f --show "$IMAGE")"
printf '%s' "$PASSPHRASE" | cryptsetup luksFormat --type luks2 --batch-mode \
--pbkdf-memory 262144 "$LOOP" -
LUKS_UUID="$(cryptsetup luksUUID "$LOOP")"
banner "configuring crypttab / fstab / networking"
# `initramfs` bakes the mapping in so cryptsetup-initramfs unlocks before pivot.
echo "$MAPPER_NAME UUID=$LUKS_UUID none luks,initramfs" > "$ROOTFS/etc/crypttab"
echo "/dev/mapper/$MAPPER_NAME / ext4 defaults,noatime 0 1" > "$ROOTFS/etc/fstab"
echo "lim-e2e" > "$ROOTFS/etc/hostname"
# initramfs-tools includes virtio via MODULES=most (default); be explicit.
sed -i 's/^MODULES=.*/MODULES=most/' "$ROOTFS/etc/initramfs-tools/initramfs.conf"
banner "authorizing the ssh key for dropbear-initramfs"
install -D -m0600 "$SSH_PUBKEY" "$ROOTFS/etc/dropbear/initramfs/authorized_keys"
banner "installing the real lim initramfs-tools tor hooks"
install -D -m0755 "$HOOK_SRC/tor_hook" "$ROOTFS/etc/initramfs-tools/hooks/tor"
install -D -m0755 "$HOOK_SRC/tor_premount" "$ROOTFS/etc/initramfs-tools/scripts/init-premount/tor"
install -D -m0755 "$HOOK_SRC/tor_bottom" "$ROOTFS/etc/initramfs-tools/scripts/init-bottom/tor"
install -D -m0644 "$HOOK_SRC/torrc" "$ROOTFS/etc/tor/initramfs-torrc"
if [ -n "${TOR_TEST_NETWORK_CONF:-}" ]; then
cat "$TOR_TEST_NETWORK_CONF" >> "$ROOTFS/etc/tor/initramfs-torrc"
fi
banner "generating onion keys offline"
install -d -m0700 "$ROOTFS/etc/tor/initramfs-onion"
: > "$WORK_DIR/keygen-torrc"
tor -f "$WORK_DIR/keygen-torrc" --DisableNetwork 1 \
--DataDirectory "$WORK_DIR/keygen-data" \
--HiddenServiceDir "$ROOTFS/etc/tor/initramfs-onion" \
--HiddenServicePort "22 127.0.0.1:22" \
--SocksPort 0 --RunAsDaemon 0 --Log "notice stderr" &
keygen_pid=$!
for _ in $(seq 1 30); do
[ -s "$ROOTFS/etc/tor/initramfs-onion/hostname" ] && break
sleep 1
done
kill "$keygen_pid" 2>/dev/null || true
ONION_ADDRESS="$(cat "$ROOTFS/etc/tor/initramfs-onion/hostname")"
banner "baking the boot-ok marker service + serial getty"
cat > "$ROOTFS/etc/systemd/system/lim-boot-ok.service" <<'EOF'
[Unit]
Description=Signal a successful LUKS-unlock boot to the serial console
After=multi-user.target
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo LIM_UNLOCK_BOOT_OK > /dev/console'
[Install]
WantedBy=multi-user.target
EOF
in_chroot systemctl enable lim-boot-ok.service serial-getty@ttyS0.service
banner "generating the initramfs (update-initramfs)"
in_chroot update-initramfs -c -k all || in_chroot update-initramfs -u -k all
banner "exporting kernel + initramfs"
KERNEL_IMG="$(find "$ROOTFS/boot" -maxdepth 1 -name 'vmlinuz-*' | sort | tail -1)"
INITRAMFS_IMG="$(find "$ROOTFS/boot" -maxdepth 1 -name 'initrd.img-*' | sort | tail -1)"
[ -n "$KERNEL_IMG" ] || { echo "No kernel in $ROOTFS/boot" >&2; exit 1; }
[ -n "$INITRAMFS_IMG" ] || { echo "No initramfs in $ROOTFS/boot" >&2; exit 1; }
echo "Using kernel $KERNEL_IMG and initramfs $INITRAMFS_IMG"
cp "$KERNEL_IMG" "$WORK_DIR/kernel"
cp "$INITRAMFS_IMG" "$WORK_DIR/initramfs"
banner "unbinding chroot filesystems"
for m in dev/pts dev proc sys; do umount -l "$ROOTFS/$m"; done
banner "copying the rootfs into the encrypted volume"
printf '%s' "$PASSPHRASE" | cryptsetup open "$LOOP" "$MAPPER_NAME" -
mkfs.ext4 -q "/dev/mapper/$MAPPER_NAME"
mkdir -p "$WORK_DIR/mnt"
mount "/dev/mapper/$MAPPER_NAME" "$WORK_DIR/mnt"
cp -a "$ROOTFS/." "$WORK_DIR/mnt/"
sync
umount -R "$WORK_DIR/mnt"
cryptsetup close "$MAPPER_NAME"
losetup -d "$LOOP"
LOOP=""
banner "writing image.env"
cat > "$WORK_DIR/image.env" <<EOF
LUKS_UUID=$LUKS_UUID
MAPPER_NAME=$MAPPER_NAME
ONION_ADDRESS=$ONION_ADDRESS
IMAGE=$IMAGE
KERNEL=$WORK_DIR/kernel
INITRAMFS=$WORK_DIR/initramfs
EOF
echo "Build complete. Onion address: $ONION_ADDRESS"

174
tests/e2e/qemu/config.py Normal file
View File

@@ -0,0 +1,174 @@
"""Pure, side-effect-free builders for the QEMU unlock harness.
Everything here is deterministic string/list construction so it can be
unit-tested without QEMU, root, or a network (see test_qemu_harness_unit.py).
The stages that actually touch the system live in build_image.sh,
chutney.py and boot_unlock.py.
"""
from dataclasses import dataclass
from pathlib import Path
# Printed to /dev/console by a oneshot service baked into the image; its
# appearance on the serial log is the proof that the LUKS root actually
# unlocked and the real system finished booting.
BOOT_OK_MARKER = "LIM_UNLOCK_BOOT_OK"
# The HOOKS the built image must carry — the tor hook sits between netconf
# and dropbear, exactly as lim.image.encryption wires it for real images.
EXPECTED_HOOKS_ORDER = ("netconf", "tor", "dropbear", "encryptssh")
# Serial console device per architecture (kernel console= and getty).
_SERIAL_CONSOLE = {"x86_64": "ttyS0", "aarch64": "ttyAMA0"}
_QEMU_BINARY = {"x86_64": "qemu-system-x86_64", "aarch64": "qemu-system-aarch64"}
@dataclass
class QemuSpec:
"""Everything needed to boot one built image and unlock it."""
arch: str
work_dir: Path
image_path: Path
kernel_path: Path
initramfs_path: Path
luks_uuid: str
mapper_name: str
passphrase: str
ssh_key_path: Path
onion_address: str = ""
socks_port: int = 9050
# 2 GiB so the LUKS argon2 KDF has enough memory to unlock in the guest
# (build_image.sh also caps --pbkdf-memory to keep this headroom).
memory_mb: int = 2048
accel: str = "tcg" # "kvm" when the host arch matches and /dev/kvm exists
# "arch" (mkinitcpio/encryptssh) or "debian" (initramfs-tools/cryptroot-unlock).
os_family: str = "arch"
# Remote command the unlock SSH session runs; empty means the session
# presents the passphrase prompt itself (Arch encryptssh).
unlock_command: str = ""
# >0 selects the deterministic direct transport: QEMU forwards this host
# port to the guest's dropbear (:22) and the passphrase is delivered over a
# plain SSH (no Tor). 0 uses the Tor onion transport (onion_address).
direct_ssh_port: int = 0
@property
def serial_log(self) -> Path:
return self.work_dir / "serial.log"
def qemu_binary(arch: str) -> str:
"""Return the qemu-system-* binary for an arch (raises on unsupported)."""
if arch not in _QEMU_BINARY:
raise ValueError(f"Unsupported arch: {arch}")
return _QEMU_BINARY[arch]
def kernel_cmdline(spec: QemuSpec) -> str:
"""Kernel command line unlocking the LUKS root over the mkinitcpio hooks.
``ip=dhcp`` is what the netconf hook consumes to bring up virtio-net in
early userspace; the Pi images use USB-gadget networking instead, which
QEMU cannot emulate — hence virtio here (we model the stack, not the board).
"""
console = _SERIAL_CONSOLE[spec.arch]
# Arch's mkinitcpio encrypt/encryptssh finds the LUKS device from the
# cryptdevice= param; Debian's cryptsetup-initramfs reads /etc/crypttab
# baked into the initramfs, so it needs only root=/dev/mapper/<name>.
if spec.os_family == "debian":
crypt = f"root=/dev/mapper/{spec.mapper_name} rw"
else:
crypt = (
f"cryptdevice=UUID={spec.luks_uuid}:{spec.mapper_name} "
f"root=/dev/mapper/{spec.mapper_name} rw"
)
# The netconf/dropbear-initramfs hooks need the explicit ip= form
# <client>:<server>:<gw>:<netmask>:<host>:<device>:<proto> — the device
# name is mandatory; a bare "ip=dhcp" leaves netconf looping on
# "SIOCGIFFLAGS: No such device". net.ifnames=0 keeps the NIC named eth0.
# The LAST console= owns /dev/console, which the boot-ok marker echoes to
# and QEMU captures via -serial file, so the serial console must come last.
return f"{crypt} ip=::::lim-e2e:eth0:dhcp net.ifnames=0 console=tty0 console={console}"
def qemu_argv(spec: QemuSpec) -> list[str]:
"""Full QEMU command: direct kernel boot, virtio disk/net, serial to log.
User-mode networking (``-netdev user``) needs no root and no tap device;
it is enough for a Tor onion service, whose rendezvous is outbound from
both ends, so no inbound host port forward is required.
"""
argv = [qemu_binary(spec.arch)] # validates the arch
net_device = "virtio-net-pci" if spec.arch == "x86_64" else "virtio-net-device"
if spec.arch == "x86_64":
argv += ["-machine", "q35"]
else:
argv += ["-machine", "virt", "-cpu", "cortex-a72"]
if spec.accel == "kvm":
argv += ["-cpu", "host", "-accel", "kvm"]
elif spec.arch == "x86_64":
argv += ["-cpu", "max", "-accel", "tcg"]
netdev = "user,id=net0"
if spec.direct_ssh_port:
# Forward a host port to the guest's dropbear for the direct transport.
netdev += f",hostfwd=tcp:127.0.0.1:{spec.direct_ssh_port}-:22"
argv += [
"-m",
str(spec.memory_mb),
"-kernel",
str(spec.kernel_path),
"-initrd",
str(spec.initramfs_path),
"-append",
kernel_cmdline(spec),
"-drive",
f"file={spec.image_path},if=virtio,format=raw",
"-netdev",
netdev,
"-device",
f"{net_device},netdev=net0",
"-nographic",
"-serial",
f"file:{spec.serial_log}",
"-no-reboot",
]
return argv
def ssh_proxy_command(socks_port: int) -> str:
"""ncat-based SOCKS5 ProxyCommand routing the SSH stream through Tor."""
return f"ncat --proxy 127.0.0.1:{socks_port} --proxy-type socks5 %h %p"
def ssh_argv(spec: QemuSpec) -> list[str]:
"""Non-interactive SSH into the initramfs dropbear to feed the passphrase.
``-tt`` forces a pty so the askpass inside the session reads the passphrase
we pipe on stdin. Host-key checking is off (throwaway host key). On Debian
the session runs ``cryptroot-unlock``; on Arch encryptssh presents the
prompt on login. The direct transport connects to a forwarded host port;
the Tor transport routes through the onion via a SOCKS ProxyCommand.
"""
argv = [
"ssh",
"-tt",
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
"BatchMode=yes",
"-i",
str(spec.ssh_key_path),
]
if spec.direct_ssh_port:
argv += ["-p", str(spec.direct_ssh_port), "root@127.0.0.1"]
else:
argv += [
"-o",
f"ProxyCommand={ssh_proxy_command(spec.socks_port)}",
f"root@{spec.onion_address}",
]
if spec.unlock_command:
argv.append(spec.unlock_command)
return argv

218
tests/e2e/qemu/harness.py Normal file
View File

@@ -0,0 +1,218 @@
"""Orchestrate the full virtualized unlock: build -> tor net -> boot -> unlock.
Wires the stages together and owns their teardown. Kept thin so each stage
stays independently testable; the pure command construction lives in
config.py and is unit-tested without any of this running.
"""
import os
import platform
import socket
import subprocess
import threading
from dataclasses import dataclass
from pathlib import Path
from tests.e2e.qemu import boot_unlock, tor_net
from tests.e2e.qemu.config import QemuSpec
IMAGE_ENV_NAME = "image.env"
_SUDO_REFRESH_INTERVAL = 60
DEFAULT_PASSPHRASE = "lim-e2e-luks-passphrase" # noqa: S105 (throwaway test secret)
DEFAULT_ARCH = "x86_64"
DEFAULT_SOCKS_PORT = 9052
@dataclass
class HarnessConfig:
repo_root: Path
work_dir: Path
arch: str = DEFAULT_ARCH
passphrase: str = DEFAULT_PASSPHRASE
chutney_path: Path | None = None
chutney_network: str = "networks/basic"
os_family: str = "arch" # "arch" (pacstrap) or "debian" (debootstrap)
# "direct" (deterministic; QEMU port-forward to dropbear) or "tor" (full
# onion transport, opt-in — flaky over public Tor inside QEMU).
unlock_transport: str = "direct"
class _NullNet:
"""Stand-in tor network for the direct transport: no Tor needed."""
socks_port = 0
test_network_conf: Path | None = None
def start(self) -> None:
pass
def stop(self) -> None:
pass
def _free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
# Per-OS build script and the remote command that delivers the passphrase.
_BUILD_SCRIPT = {"arch": "build_image.sh", "debian": "build_image_debian.sh"}
_UNLOCK_COMMAND = {"arch": "", "debian": "cryptroot-unlock"}
def choose_accel(arch: str) -> str:
"""Use KVM only when the guest arch matches the host and /dev/kvm exists."""
host = "x86_64" if platform.machine() in ("x86_64", "AMD64") else platform.machine()
if arch == host and Path("/dev/kvm").exists():
return "kvm"
return "tcg"
def _generate_ssh_key(work_dir: Path) -> Path:
key_path = work_dir / "unlock_key"
subprocess.run(
["ssh-keygen", "-t", "ed25519", "-N", "", "-f", str(key_path), "-q"],
check=True,
)
return key_path
class _SudoKeepalive:
"""Refresh the cached sudo credential so the long build never re-prompts.
The password is entered once, up front (before the slow tor bootstrap),
then a background thread keeps the timestamp warm until stop().
"""
def __init__(self) -> None:
self._stop = threading.Event()
self._thread = threading.Thread(target=self._loop, daemon=True)
def _loop(self) -> None:
while not self._stop.wait(_SUDO_REFRESH_INTERVAL):
subprocess.run(["sudo", "-n", "-v"], check=False)
def start(self) -> None:
self._thread.start()
def stop(self) -> None:
self._stop.set()
self._thread.join(timeout=5)
def _acquire_sudo() -> _SudoKeepalive | None:
"""Prompt for sudo once now; return a keepalive (None when already root)."""
if os.geteuid() == 0:
return None
print("\n[harness] The build stage needs root — enter your sudo password now.")
subprocess.run(["sudo", "-v"], check=True)
keepalive = _SudoKeepalive()
keepalive.start()
return keepalive
def _build_image(cfg: HarnessConfig, ssh_key: Path, test_network_conf: Path | None) -> None:
script = cfg.repo_root / "tests/e2e/qemu" / _BUILD_SCRIPT[cfg.os_family]
env = {
**os.environ,
"WORK_DIR": str(cfg.work_dir),
"REPO_ROOT": str(cfg.repo_root),
"ARCH": cfg.arch,
"PASSPHRASE": cfg.passphrase,
"SSH_PUBKEY": f"{ssh_key}.pub",
}
if test_network_conf is not None:
env["TOR_TEST_NETWORK_CONF"] = str(test_network_conf)
# -n: never prompt here; the credential was primed by _acquire_sudo().
prefix = [] if os.geteuid() == 0 else ["sudo", "-n", "-E"]
subprocess.run([*prefix, "bash", str(script)], env=env, check=True)
# Hand the root-built artifacts back so rootless QEMU can read them.
_reclaim_work_dir(cfg, required=True)
def _reclaim_work_dir(cfg: HarnessConfig, *, required: bool = False) -> None:
"""Reclaim the root-built work dir for the invoking user via chown.
Needed before boot so rootless QEMU can read the image, and again in the
finally so a failed build never leaves root-owned files that the pytest
tmp cleanup then cannot remove.
"""
if os.geteuid() == 0 or not cfg.work_dir.exists():
return
subprocess.run(
["sudo", "-n", "chown", "-R", str(os.getuid()), str(cfg.work_dir)],
check=required,
)
def _make_tor_net(cfg: HarnessConfig):
if cfg.unlock_transport == "direct":
return _NullNet()
if cfg.chutney_path is not None:
return tor_net.ChutneyNetwork(cfg.chutney_path, cfg.work_dir, cfg.chutney_network)
return tor_net.PublicTorClient(cfg.work_dir, DEFAULT_SOCKS_PORT)
def _load_spec(cfg: HarnessConfig, ssh_key: Path, socks_port: int) -> QemuSpec:
env = boot_unlock.parse_env_file(cfg.work_dir / IMAGE_ENV_NAME)
direct_port = _free_port() if cfg.unlock_transport == "direct" else 0
return QemuSpec(
arch=cfg.arch,
work_dir=cfg.work_dir,
image_path=Path(env["IMAGE"]),
kernel_path=Path(env["KERNEL"]),
initramfs_path=Path(env["INITRAMFS"]),
luks_uuid=env["LUKS_UUID"],
mapper_name=env["MAPPER_NAME"],
onion_address=env["ONION_ADDRESS"],
passphrase=cfg.passphrase,
ssh_key_path=ssh_key,
socks_port=socks_port,
accel=choose_accel(cfg.arch),
os_family=cfg.os_family,
unlock_command=_UNLOCK_COMMAND[cfg.os_family],
direct_ssh_port=direct_port,
)
def run(cfg: HarnessConfig) -> QemuSpec:
"""Execute every stage; return the QemuSpec on a fully verified unlock.
A chutney network must exist before the build so the image can be wired
to its authorities; a public client only needs to be up before boot.
"""
cfg.work_dir.mkdir(parents=True, exist_ok=True)
ssh_key = _generate_ssh_key(cfg.work_dir)
# Prime sudo before the slow tor bootstrap so the prompt is the first thing
# the operator sees, not a surprise five minutes in.
sudo = _acquire_sudo()
net = _make_tor_net(cfg)
try:
if cfg.unlock_transport != "direct":
print("\n[harness] bootstrapping Tor client (~30-60s)...", flush=True)
net.start()
print(
f"[harness] transport={cfg.unlock_transport}, building encrypted "
"image (several minutes)...",
flush=True,
)
_build_image(cfg, ssh_key, net.test_network_conf)
spec = _load_spec(cfg, ssh_key, net.socks_port)
boot_unlock.boot_and_unlock(spec)
return spec
finally:
# Reclaim ownership FIRST so a slow/failing teardown can never leave
# root-owned files behind, and isolate each step so none masks the
# real error propagating out of the try or skips the others.
_reclaim_work_dir(cfg)
try:
net.stop()
except Exception as exc: # noqa: BLE001 - teardown must not mask the real failure
print(f"[harness] tor network teardown failed: {exc}", flush=True)
if sudo is not None:
try:
sudo.stop()
except Exception as exc: # noqa: BLE001 - teardown must not mask the real failure
print(f"[harness] sudo keepalive teardown failed: {exc}", flush=True)

156
tests/e2e/qemu/tor_net.py Normal file
View File

@@ -0,0 +1,156 @@
"""Tor network providers for the QEMU unlock harness.
Two interchangeable ways to give the harness a working SOCKS port that can
reach the guest's onion service:
* PublicTorClient - a local tor client on the public network (default, both
the guest's baked-in torrc and this client use the real Tor authorities).
* ChutneyNetwork - a private, deterministic, offline Tor network via the
official `chutney` tool. Fully private operation also needs the image
built against the same authorities; test_network_conf() emits the file
build_image.sh consumes for that (see README.md).
Both expose ``socks_port``, ``start()`` and ``stop()``.
"""
import subprocess
import time
from pathlib import Path
BOOTSTRAP_TIMEOUT = 300 # public Tor bootstrap can stall on slow guards
_GUEST_HOST_ALIAS = "10.0.2.2" # QEMU user-net alias for the host's 127.0.0.1
class PublicTorClient:
"""A throwaway tor client bootstrapped against the public network."""
def __init__(self, work_dir: Path, socks_port: int) -> None:
self.socks_port = socks_port
self.test_network_conf: Path | None = None
self._work_dir = work_dir
self._log = work_dir / "tor-client.log"
self._data = work_dir / "tor-client-data"
self._process: subprocess.Popen | None = None
def start(self) -> None:
self._data.mkdir(parents=True, exist_ok=True)
self._data.chmod(0o700)
# An empty -f keeps the system /etc/tor/torrc (User tor, ...) out of
# this rootless client, which would otherwise refuse to start.
empty_torrc = self._work_dir / "tor-client-torrc"
empty_torrc.write_text("")
self._process = subprocess.Popen(
[
"tor",
"-f",
str(empty_torrc),
"--SocksPort",
str(self.socks_port),
"--DataDirectory",
str(self._data),
"--Log",
f"notice file {self._log}",
# HS-level log so onion descriptor fetch / rendezvous is visible.
"--Log",
f"[rend]info file {self._work_dir / 'tor-client-hs.log'}",
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
_wait_for_bootstrap(self._log, self._process)
def stop(self) -> None:
if self._process is None:
return
self._process.terminate()
try:
self._process.wait(timeout=15)
except subprocess.TimeoutExpired:
# Never leak a tor holding the fixed SOCKS port into the next run.
self._process.kill()
self._process.wait(timeout=5)
class ChutneyNetwork:
"""A private Tor network managed by the chutney tool."""
def __init__(self, chutney_path: Path, work_dir: Path, network: str) -> None:
self._chutney = chutney_path
self._network = network
self._work_dir = work_dir
self.socks_port = 0
self.test_network_conf: Path | None = None
def _run(self, *args: str) -> None:
subprocess.run(
[str(self._chutney / "chutney"), *args, self._network],
cwd=self._chutney,
check=True,
)
def start(self) -> None:
self._run("configure")
self._run("start")
self._run("wait_for_bootstrap")
client_torrc = _find_client_torrc(self._chutney / "net" / "nodes")
self.socks_port = _parse_socks_port(client_torrc)
self.test_network_conf = self._emit_network_conf(client_torrc)
def _emit_network_conf(self, client_torrc: Path) -> Path:
"""Distil the authority/test-network lines the guest image needs.
127.0.0.1 authority addresses are rewritten to the QEMU user-net host
alias so the guest's initramfs tor can reach them through user-mode
networking.
"""
wanted = ("TestingTorNetwork", "DirAuthority", "DirServer", "AlternateDirAuthority")
lines = [
line.replace("127.0.0.1", _GUEST_HOST_ALIAS)
for line in client_torrc.read_text().splitlines()
if line.strip().startswith(wanted)
]
conf = self._work_dir / "tor-test-network.conf"
conf.write_text("\n".join(lines) + "\n")
return conf
def stop(self) -> None:
self._run("stop")
def _wait_for_bootstrap(log_path: Path, process: subprocess.Popen) -> None:
deadline = time.monotonic() + BOOTSTRAP_TIMEOUT
while time.monotonic() < deadline:
if process.poll() is not None:
raise RuntimeError(
f"tor exited early (code {process.returncode}).\n{_log_tail(log_path)}"
)
if log_path.is_file() and "Bootstrapped 100%" in log_path.read_text():
return
time.sleep(1)
raise RuntimeError(f"tor client did not bootstrap in time.\n{_log_tail(log_path)}")
def _log_tail(log_path: Path, lines: int = 8) -> str:
if not log_path.is_file():
return "(no tor log written)"
tail = log_path.read_text().splitlines()[-lines:]
return "tor log:\n " + "\n ".join(tail)
def _find_client_torrc(nodes_dir: Path) -> Path:
for torrc in sorted(nodes_dir.glob("*/torrc")):
if _parse_socks_port(torrc):
return torrc
raise RuntimeError(f"No chutney client with a SocksPort under {nodes_dir}.")
def _parse_socks_port(torrc: Path) -> int:
if not torrc.is_file():
return 0
for line in torrc.read_text().splitlines():
parts = line.split()
if len(parts) >= 2 and parts[0] == "SocksPort" and parts[1].isdigit():
port = int(parts[1])
if port:
return port
return 0

View File

@@ -0,0 +1,240 @@
"""Always-on unit tests for the QEMU harness logic.
No QEMU, root, or network: they exercise the pure command builders and guard
the root build script against drifting away from what the harness expects.
Run as part of the normal suite.
"""
from pathlib import Path
import pytest
from lim import config as lim_config
from tests.e2e.qemu import boot_unlock, config, harness, tor_net
from tests.e2e.qemu.config import EXPECTED_HOOKS_ORDER, QemuSpec
BUILD_SCRIPT = Path(__file__).resolve().parents[2] / "tests/e2e/qemu/build_image.sh"
def _spec(**overrides) -> QemuSpec:
base = {
"arch": "x86_64",
"work_dir": Path("/work"),
"image_path": Path("/work/image.raw"),
"kernel_path": Path("/work/kernel"),
"initramfs_path": Path("/work/initramfs"),
"luks_uuid": "1111-UUID",
"mapper_name": "cryptroot",
"passphrase": "secret",
"ssh_key_path": Path("/work/unlock_key"),
"onion_address": "abcd.onion",
"socks_port": 9052,
}
base.update(overrides)
return QemuSpec(**base)
class TestKernelCmdline:
def test_unlocks_the_luks_root_over_dhcp(self):
line = config.kernel_cmdline(_spec())
assert "cryptdevice=UUID=1111-UUID:cryptroot" in line
assert "root=/dev/mapper/cryptroot" in line
# Explicit device in the ip= form, else netconf loops on "No such device".
assert ":eth0:dhcp" in line
assert "console=ttyS0" in line
def test_serial_console_is_last_so_marker_reaches_serial_log(self):
line = config.kernel_cmdline(_spec())
assert line.index("console=tty0") < line.index("console=ttyS0")
def test_aarch64_uses_amba_serial(self):
assert "console=ttyAMA0" in config.kernel_cmdline(_spec(arch="aarch64"))
def test_debian_uses_crypttab_not_cryptdevice(self):
# Debian's cryptsetup-initramfs reads /etc/crypttab, so the cmdline
# carries only root=/dev/mapper/<name>, no cryptdevice= param.
line = config.kernel_cmdline(_spec(os_family="debian"))
assert "cryptdevice=" not in line
assert "root=/dev/mapper/cryptroot" in line
assert ":eth0:dhcp" in line
assert line.index("console=tty0") < line.index("console=ttyS0")
class TestQemuArgv:
def test_x86_uses_virtio_net_and_direct_kernel_boot(self):
argv = config.qemu_argv(_spec())
assert argv[0] == "qemu-system-x86_64"
assert "virtio-net-pci,netdev=net0" in argv
assert "-kernel" in argv
assert "/work/kernel" in argv
assert "-initrd" in argv
assert "/work/initramfs" in argv
assert "file=/work/image.raw,if=virtio,format=raw" in argv
assert f"file:{_spec().serial_log}" in argv
assert "user,id=net0" in argv
def test_aarch64_uses_virt_machine_and_net_device(self):
argv = config.qemu_argv(_spec(arch="aarch64"))
assert argv[0] == "qemu-system-aarch64"
assert "virt" in argv
assert "virtio-net-device,netdev=net0" in argv
def test_kvm_accel_is_emitted_when_requested(self):
assert "kvm" in config.qemu_argv(_spec(accel="kvm"))
assert "kvm" not in config.qemu_argv(_spec(accel="tcg"))
def test_unknown_arch_raises(self):
with pytest.raises(ValueError, match="Unsupported arch"):
config.qemu_argv(_spec(arch="riscv64"))
class TestQemuBinary:
def test_derives_binary_per_arch(self):
assert config.qemu_binary("x86_64") == "qemu-system-x86_64"
assert config.qemu_binary("aarch64") == "qemu-system-aarch64"
def test_unknown_arch_raises(self):
with pytest.raises(ValueError, match="Unsupported arch"):
config.qemu_binary("riscv64")
class TestSshInvocation:
def test_proxy_command_routes_through_socks5(self):
proxy = config.ssh_proxy_command(9052)
assert "--proxy 127.0.0.1:9052" in proxy
assert "--proxy-type socks5" in proxy
def test_ssh_argv_targets_onion_via_key_and_proxy(self):
argv = config.ssh_argv(_spec())
assert "root@abcd.onion" in argv
assert "-i" in argv
assert "/work/unlock_key" in argv
assert any("ProxyCommand=" in part for part in argv)
assert "StrictHostKeyChecking=no" in argv
def test_arch_session_runs_no_remote_command(self):
assert config.ssh_argv(_spec())[-1] == "root@abcd.onion"
def test_debian_session_runs_cryptroot_unlock(self):
argv = config.ssh_argv(_spec(unlock_command="cryptroot-unlock"))
assert argv[-1] == "cryptroot-unlock"
class TestDirectTransport:
def test_qemu_forwards_a_host_port_to_dropbear(self):
argv = config.qemu_argv(_spec(direct_ssh_port=2222))
assert "user,id=net0,hostfwd=tcp:127.0.0.1:2222-:22" in argv
def test_tor_transport_adds_no_hostfwd(self):
argv = config.qemu_argv(_spec())
assert "user,id=net0" in argv
assert not any("hostfwd" in part for part in argv)
def test_ssh_argv_direct_connects_to_forwarded_port(self):
argv = config.ssh_argv(_spec(direct_ssh_port=2222))
assert "root@127.0.0.1" in argv
assert "-p" in argv
assert "2222" in argv
assert not any("ProxyCommand" in part for part in argv)
class TestBuildScriptStaysAligned:
"""Guards: the root build script must match what the harness assumes."""
def _script(self) -> str:
return BUILD_SCRIPT.read_text()
def test_hooks_place_tor_between_netconf_and_dropbear(self):
hooks_line = next(line for line in self._script().splitlines() if line.startswith("HOOKS="))
positions = [hooks_line.index(hook) for hook in EXPECTED_HOOKS_ORDER]
assert positions == sorted(positions), hooks_line
def test_installs_the_real_lim_initcpio_files(self):
script = self._script()
for name in ("tor_install", "tor_hook", "torrc"):
assert f"$INITCPIO_SRC/{name}" in script
assert "etc/dropbear/root_key" in script
def test_uses_virtio_net_and_emits_boot_marker(self):
script = self._script()
assert "virtio_net" in script
assert config.BOOT_OK_MARKER in script
def test_mounts_devpts_before_dropbear_for_pty(self):
hooks_line = next(line for line in self._script().splitlines() if line.startswith("HOOKS="))
assert "ptsmount" in hooks_line
assert hooks_line.index("ptsmount") < hooks_line.index("dropbear")
assert "mount -t devpts" in self._script()
def test_real_initcpio_source_directory_exists(self):
assert (lim_config.CONFIGURATION_PATH / "initcpio" / "tor_hook").is_file()
class TestDebianBuildScript:
"""Guards for the debootstrap build script and the OS dispatch."""
def _script(self) -> str:
path = Path(__file__).resolve().parents[2] / "tests/e2e/qemu/build_image_debian.sh"
return path.read_text()
def test_installs_the_real_initramfs_tools_hooks(self):
script = self._script()
for name in ("tor_hook", "tor_premount", "tor_bottom", "torrc"):
assert f"$HOOK_SRC/{name}" in script
assert "etc/dropbear/initramfs/authorized_keys" in script
def test_crypttab_uses_initramfs_option_and_emits_marker(self):
script = self._script()
assert "none luks,initramfs" in script
assert config.BOOT_OK_MARKER in script
def test_harness_maps_os_family_to_script_and_unlock_command(self):
assert harness._BUILD_SCRIPT["debian"] == "build_image_debian.sh"
assert harness._BUILD_SCRIPT["arch"] == "build_image.sh"
assert harness._UNLOCK_COMMAND["debian"] == "cryptroot-unlock"
assert harness._UNLOCK_COMMAND["arch"] == ""
def test_real_initramfs_tools_source_directory_exists(self):
hook = lim_config.CONFIGURATION_PATH / "initramfs-tools" / "tor_premount"
assert hook.is_file()
class TestOrchestratorGlue:
def test_load_spec_reads_image_env(self, tmp_path):
(tmp_path / "image.env").write_text(
"LUKS_UUID=abc-123\n"
"MAPPER_NAME=cryptroot\n"
"ONION_ADDRESS=xyz.onion\n"
"IMAGE=/w/image.raw\n"
"KERNEL=/w/kernel\n"
"INITRAMFS=/w/initramfs\n"
)
cfg = harness.HarnessConfig(repo_root=Path("/repo"), work_dir=tmp_path)
spec = harness._load_spec(cfg, ssh_key=tmp_path / "k", socks_port=9052)
assert spec.luks_uuid == "abc-123"
assert spec.onion_address == "xyz.onion"
assert spec.image_path == Path("/w/image.raw")
assert spec.socks_port == 9052
def test_env_parser_ignores_comments_and_blanks(self, tmp_path):
env = tmp_path / "image.env"
env.write_text("# a comment\n\nKEY=value\n")
assert boot_unlock.parse_env_file(env) == {"KEY": "value"}
def test_choose_accel_falls_back_to_tcg_for_foreign_arch(self):
assert harness.choose_accel("riscv64") == "tcg"
class TestChutneyParsing:
def test_parse_socks_port_picks_first_nonzero(self, tmp_path):
torrc = tmp_path / "torrc"
torrc.write_text("SocksPort 0\nSocksPort 9008\n")
assert tor_net._parse_socks_port(torrc) == 9008
def test_find_client_torrc_skips_relays(self, tmp_path):
(tmp_path / "000a").mkdir()
(tmp_path / "000a" / "torrc").write_text("SocksPort 0\n")
(tmp_path / "001c").mkdir()
(tmp_path / "001c" / "torrc").write_text("SocksPort 9010\n")
found = tor_net._find_client_torrc(tmp_path)
assert found == tmp_path / "001c" / "torrc"

View File

@@ -0,0 +1,81 @@
"""Full virtualized end-to-end test: build -> boot in QEMU -> unlock over Tor.
Models the whole process the way lim runs it, on a QEMU-bootable virtio
machine (the Raspberry Pi's USB-gadget net can't be emulated, so we model
the software stack, not the board):
1. build a LUKS image carrying the REAL lim initcpio tor artifacts,
2. boot it in QEMU (rootless, user-mode net),
3. its initramfs runs the real netconf/tor/dropbear/encryptssh chain and
publishes the onion service,
4. deliver the LUKS passphrase by SSHing to that onion through Tor,
5. assert the boot-ok marker appears on the serial console — i.e. the real
root actually unlocked and booted.
Heavy and privileged (QEMU + a tor network; the build needs root, ideally in
a throwaway VM). Opt-in and skipped unless LIM_E2E_QEMU=1:
LIM_E2E_QEMU=1 pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
Environment knobs: LIM_E2E_ARCH (x86_64|aarch64), CHUTNEY_PATH (use a private
chutney network instead of the public one). See tests/e2e/qemu/README.md.
"""
import os
import shutil
from pathlib import Path
import pytest
from tests.e2e.qemu import config as qemu_config
from tests.e2e.qemu.harness import HarnessConfig, run
REPO_ROOT = Path(__file__).resolve().parents[2]
# The QEMU binary is arch-specific, so derive it from LIM_E2E_ARCH instead of
# hardcoding x86_64 — otherwise the documented aarch64 run is never reachable.
_ARCH = os.environ.get("LIM_E2E_ARCH", "x86_64")
# The OS family selects the build tool: pacstrap (Arch) or debootstrap (Debian).
_OS = os.environ.get("LIM_E2E_OS", "arch")
# "direct" (deterministic port-forward) is the default; "tor" runs the full,
# opt-in onion transport (flaky over public Tor inside QEMU).
_TRANSPORT = os.environ.get("LIM_E2E_TRANSPORT", "direct")
_BUILD_TOOL = {"arch": "pacstrap", "debian": "debootstrap"}
_REQUIRED = [
qemu_config.qemu_binary(_ARCH),
_BUILD_TOOL.get(_OS, "pacstrap"),
"cryptsetup",
"ssh",
]
if _TRANSPORT == "tor": # the onion transport also needs a Tor client + ncat
_REQUIRED += ["tor", "ncat"]
def _missing_tools() -> list[str]:
return [tool for tool in _REQUIRED if shutil.which(tool) is None]
pytestmark = pytest.mark.skipif(
os.environ.get("LIM_E2E_QEMU") != "1" or bool(_missing_tools()),
reason=(
"needs LIM_E2E_QEMU=1 and " + ", ".join(_REQUIRED) + " (build stage needs root; slow)."
),
)
def test_full_build_boot_and_unlock(tmp_path):
chutney_path = os.environ.get("CHUTNEY_PATH")
cfg = HarnessConfig(
repo_root=REPO_ROOT,
work_dir=tmp_path / "qemu-e2e",
arch=_ARCH,
os_family=_OS,
unlock_transport=_TRANSPORT,
chutney_path=Path(chutney_path) if chutney_path else None,
)
spec = run(cfg)
# run() only returns after the boot-ok marker was observed on the console.
serial = spec.serial_log.read_text(errors="replace")
assert qemu_config.BOOT_OK_MARKER in serial

View File

@@ -0,0 +1,110 @@
"""End-to-end test of the Tor onion LUKS-unlock path (rootless).
It reproduces the operator-visible half of the decryption process that the
"tor" mkinitcpio hook drives at boot:
1. generate the v3 onion keys offline (as lim.image.initramfs.keygen does in the chroot),
2. start a real Tor onion service from a torrc mirroring the baked-in one,
forwarding the virtual port 22 to a local dropbear stand-in,
3. connect to the .onion address through Tor and deliver the passphrase,
4. assert the passphrase reached the unlock endpoint and it "unlocked".
The physical flashing half (loop device, cryptsetup, mount, chroot,
mkinitcpio) needs root and is out of scope here by design.
Requires the real `tor` binary and live Tor network access, so it is
opt-in and skipped unless LIM_E2E_TOR=1:
LIM_E2E_TOR=1 pytest tests/e2e/test_tor_unlock_e2e.py -v
"""
import os
import shutil
import time
import pytest
from lim.image.initramfs import keygen
from tests.e2e import tor_harness
# The offline checks only need the tor binary (no network) and are
# deterministic, so they run in CI whenever tor is installed. Only the live
# onion round-trip needs the public Tor network, so it stays opt-in behind
# LIM_E2E_TOR=1 to keep an external, occasionally-flaky dependency out of the
# blocking gate.
_needs_tor = pytest.mark.skipif(shutil.which("tor") is None, reason="needs the tor binary")
_needs_tor_network = pytest.mark.skipif(
shutil.which("tor") is None or os.environ.get("LIM_E2E_TOR") != "1",
reason="needs the tor binary and LIM_E2E_TOR=1 (live Tor network, slow)",
)
PASSPHRASE = b"correct horse battery staple"
@pytest.fixture
def workdir(tmp_path):
(tmp_path / "onion").mkdir()
(tmp_path / "data").mkdir()
return tmp_path
def test_offline_keygen_matches_production_flags():
"""Guard: the harness keygen mirrors the flags production actually runs."""
script = keygen._KEYGEN_SCRIPT
assert "--DisableNetwork 1" in script
assert "HiddenServicePort" in script
assert "22 127.0.0.1:22" in script
assert "--SocksPort 0" in script
@_needs_tor
def test_onion_keygen_is_deterministic_and_offline(workdir):
"""Keys generate without network and the .onion address is stable."""
address = tor_harness.generate_onion_keys(workdir / "onion", workdir / "data")
assert address.endswith(".onion")
assert len(address) == len("v" * 56) + len(".onion") # v3 = 56 base32 chars
for name in ("hs_ed25519_secret_key", "hs_ed25519_public_key", "hostname"):
assert (workdir / "onion" / name).is_file()
@_needs_tor_network
def test_unlock_passphrase_travels_over_onion(workdir):
"""Full rootless round-trip: client -> Tor -> onion -> dropbear stand-in."""
onion_dir = workdir / "onion"
data_dir = workdir / "data"
onion_address = tor_harness.generate_onion_keys(onion_dir, data_dir)
backend_port = tor_harness.free_port()
socks_port = tor_harness.free_port()
torrc = workdir / "torrc"
log_path = workdir / "tor.log"
tor_harness.write_test_torrc(torrc, data_dir, onion_dir, backend_port)
dropbear = tor_harness.FakeDropbear(backend_port)
dropbear.start()
service = tor_harness.start_onion_service(torrc, socks_port, log_path)
try:
tor_harness.wait_bootstrapped(log_path, service)
# Onion descriptors need a moment to publish after bootstrap; retry.
last_error = None
for _ in range(5):
try:
stream = tor_harness.socks5_connect(socks_port, onion_address, 22)
break
except (OSError, RuntimeError) as exc:
last_error = exc
time.sleep(2)
else:
pytest.fail(f"Could not reach {onion_address} via Tor: {last_error}")
with stream:
stream.sendall(PASSPHRASE + b"\n")
reply = stream.recv(64)
finally:
service.terminate()
service.wait(timeout=15)
dropbear.stop()
assert reply.strip() == b"UNLOCKED"
assert dropbear.received == PASSPHRASE

188
tests/e2e/tor_harness.py Normal file
View File

@@ -0,0 +1,188 @@
"""Rootless helpers to stand up a real Tor onion service and reach it.
These reproduce, without root/block-devices/chroot, what the "tor"
mkinitcpio hook does at LUKS-decryption time: generate v3 onion keys
offline, run a real Tor onion service that forwards the virtual port 22
to a local "dropbear", and let a client reach that service through Tor.
"""
import socket
import struct
import subprocess
import threading
import time
from pathlib import Path
# Matches lim.image.initramfs.keygen._KEYGEN_SCRIPT: an empty -f torrc keeps the host's
# /etc/tor/torrc out of the run, --DisableNetwork 1 makes the keys appear
# without ever touching the network.
KEYGEN_TIMEOUT = 30
BOOTSTRAP_TIMEOUT = 180
ONION_CONNECT_TIMEOUT = 90
def free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
return sock.getsockname()[1]
def generate_onion_keys(onion_dir: Path, data_dir: Path) -> str:
"""Create v3 onion keys offline; return the .onion hostname.
Mirrors lim.image.initramfs.keygen._KEYGEN_SCRIPT, run directly on the host instead
of inside the image chroot (the tor invocation is identical).
"""
onion_dir.mkdir(parents=True, exist_ok=True)
onion_dir.chmod(0o700)
empty_torrc = data_dir.parent / "keygen-torrc"
empty_torrc.write_text("")
process = subprocess.Popen(
[
"tor",
"-f",
str(empty_torrc),
"--DisableNetwork",
"1",
"--DataDirectory",
str(data_dir),
"--HiddenServiceDir",
str(onion_dir),
"--HiddenServicePort",
"22 127.0.0.1:22",
"--SocksPort",
"0",
"--RunAsDaemon",
"0",
"--Log",
"notice stderr",
],
stderr=subprocess.DEVNULL,
)
hostname = onion_dir / "hostname"
try:
deadline = time.monotonic() + KEYGEN_TIMEOUT
while time.monotonic() < deadline:
if hostname.is_file() and hostname.read_text().strip():
break
time.sleep(0.5)
finally:
process.terminate()
process.wait(timeout=10)
if not (hostname.is_file() and hostname.read_text().strip()):
raise RuntimeError("Offline onion key generation produced no hostname.")
return hostname.read_text().strip()
def start_onion_service(torrc_path: Path, socks_port: int, log_path: Path) -> subprocess.Popen:
"""Start Tor with the given torrc (service side) plus a SOCKS port.
The torrc carries the HiddenServiceDir/HiddenServicePort just like the
baked-in image torrc; --SocksPort lets the same instance also act as the
client's entry point so the test needs only one bootstrap.
"""
return subprocess.Popen(
[
"tor",
"-f",
str(torrc_path),
"--SocksPort",
str(socks_port),
"--Log",
f"notice file {log_path}",
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
)
def wait_bootstrapped(log_path: Path, process: subprocess.Popen) -> None:
deadline = time.monotonic() + BOOTSTRAP_TIMEOUT
while time.monotonic() < deadline:
if process.poll() is not None:
raise RuntimeError(f"Tor exited early (code {process.returncode}).")
if log_path.is_file() and "Bootstrapped 100%" in log_path.read_text():
return
time.sleep(1)
raise RuntimeError("Tor did not bootstrap in time.")
def _socks5_handshake(sock: socket.socket, onion_host: str, onion_port: int) -> None:
sock.sendall(b"\x05\x01\x00") # VER, 1 method, no-auth
if sock.recv(2) != b"\x05\x00":
raise RuntimeError("SOCKS5 handshake rejected.")
host = onion_host.encode()
request = b"\x05\x01\x00\x03" + bytes([len(host)]) + host + struct.pack(">H", onion_port)
sock.settimeout(ONION_CONNECT_TIMEOUT)
sock.sendall(request)
header = sock.recv(4)
if len(header) < 2 or header[1] != 0x00:
code = header[1] if len(header) >= 2 else -1
raise RuntimeError(f"SOCKS5 CONNECT failed (reply code {code}).")
# Drain the bound-address tail so the stream starts at real payload.
atyp = header[3]
tail = {0x01: 4, 0x04: 16}.get(atyp)
if tail is None: # domain name
tail = sock.recv(1)[0]
sock.recv(tail + 2)
def socks5_connect(socks_port: int, onion_host: str, onion_port: int) -> socket.socket:
"""Open a stream to onion_host:onion_port through Tor's SOCKS5 proxy."""
sock = socket.create_connection(("127.0.0.1", socks_port), timeout=30)
try:
_socks5_handshake(sock, onion_host, onion_port)
except (OSError, RuntimeError):
sock.close()
raise
return sock
class FakeDropbear:
r"""Stand-in for the initramfs dropbear+cryptsetup unlock endpoint.
Accepts one connection on 127.0.0.1:<port>, reads a passphrase line, and
replies "UNLOCKED\n" — the observable proof that a passphrase delivered
over the onion connection reached the unlock shell.
"""
def __init__(self, port: int) -> None:
self.port = port
self.received: bytes | None = None
self._server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self._server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._server.bind(("127.0.0.1", port))
self._server.listen(1)
self._thread = threading.Thread(target=self._serve, daemon=True)
def start(self) -> None:
self._thread.start()
def _serve(self) -> None:
try:
self._server.settimeout(BOOTSTRAP_TIMEOUT + ONION_CONNECT_TIMEOUT)
conn, _ = self._server.accept()
with conn:
conn.settimeout(ONION_CONNECT_TIMEOUT)
self.received = conn.recv(4096).strip()
conn.sendall(b"UNLOCKED\n")
except OSError:
pass
def stop(self) -> None:
self._server.close()
self._thread.join(timeout=5)
def write_test_torrc(path: Path, data_dir: Path, onion_dir: Path, backend_port: int) -> None:
"""Write a torrc mirroring the baked-in one, with test-writable paths.
The virtual onion port stays 22 (as in production); only the loopback
backend and the data/key directories move to rootless temp locations.
"""
path.write_text(
f"DataDirectory {data_dir}\n"
f"HiddenServiceDir {onion_dir}\n"
f"HiddenServicePort 22 127.0.0.1:{backend_port}\n"
"SocksPort 0\n"
)

0
tests/lint/__init__.py Normal file
View File

Some files were not shown because too many files have changed in this diff Show More