ci: run the Debian QEMU e2e on push; add make target and docs
- test.yml: add a qemu-e2e-debian job (build -> boot -> LUKS-unlock in QEMU via the deterministic direct transport) running on push/PR, plus a workflow_dispatch trigger for manual runs. - Makefile: test-qemu-debian target (LIM_E2E_OS=debian); document the direct vs tor transport on test-qemu. - README: document the two build scripts and the direct transport. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
16
.github/workflows/test.yml
vendored
16
.github/workflows/test.yml
vendored
@@ -3,6 +3,7 @@ name: tests
|
|||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
pull_request:
|
pull_request:
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
lint:
|
lint:
|
||||||
@@ -37,3 +38,18 @@ jobs:
|
|||||||
- run: sudo apt-get update && sudo apt-get install -y tor
|
- run: sudo apt-get update && sudo apt-get install -y tor
|
||||||
- run: pip install pytest pyyaml
|
- run: pip install pytest pyyaml
|
||||||
- run: LIM_E2E_TOR=1 make test-tor PYTHON=python
|
- run: LIM_E2E_TOR=1 make test-tor PYTHON=python
|
||||||
|
|
||||||
|
qemu-e2e-debian:
|
||||||
|
# Full build -> boot -> LUKS-unlock in QEMU, deterministic via the direct
|
||||||
|
# transport (debootstrap works on Ubuntu; no Tor). Heavy (root, loop,
|
||||||
|
# cryptsetup, and TCG when the runner has no /dev/kvm).
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 45
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-python@v5
|
||||||
|
with:
|
||||||
|
python-version: "3.12"
|
||||||
|
- run: sudo apt-get update && sudo apt-get install -y qemu-system-x86 debootstrap cryptsetup
|
||||||
|
- run: pip install pytest pyyaml
|
||||||
|
- run: make test-qemu-debian PYTHON=python
|
||||||
|
|||||||
13
Makefile
13
Makefile
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: install test test-tor test-qemu test-all
|
.PHONY: install test test-tor test-qemu test-qemu-debian test-all
|
||||||
|
|
||||||
PREFIX ?= $(HOME)/.local
|
PREFIX ?= $(HOME)/.local
|
||||||
# Interpreter used for the tests. Override if your active venv lacks pytest,
|
# Interpreter used for the tests. Override if your active venv lacks pytest,
|
||||||
@@ -20,11 +20,18 @@ test-tor:
|
|||||||
LIM_E2E_TOR=1 $(PYTHON) -m pytest tests/e2e/test_tor_unlock_e2e.py -v
|
LIM_E2E_TOR=1 $(PYTHON) -m pytest tests/e2e/test_tor_unlock_e2e.py -v
|
||||||
|
|
||||||
# Full virtualized build+boot+unlock e2e. Needs qemu, arch-install-scripts,
|
# Full virtualized build+boot+unlock e2e. Needs qemu, arch-install-scripts,
|
||||||
# cryptsetup, tor and ncat; the build stage runs under sudo. Set CHUTNEY_PATH
|
# cryptsetup and ssh; the build stage runs under sudo. The default "direct"
|
||||||
# to use a private, offline Tor network instead of the public one.
|
# transport delivers the passphrase deterministically via a QEMU port-forward;
|
||||||
|
# set LIM_E2E_TRANSPORT=tor for the full onion path (also needs tor + ncat, and
|
||||||
|
# is flaky over public Tor inside QEMU).
|
||||||
test-qemu:
|
test-qemu:
|
||||||
LIM_E2E_QEMU=1 $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
LIM_E2E_QEMU=1 $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
|
|
||||||
|
# Same, but builds a Debian image with debootstrap (initramfs-tools backend)
|
||||||
|
# instead of Arch. Needs debootstrap + cryptsetup + ssh; runs under sudo.
|
||||||
|
test-qemu-debian:
|
||||||
|
LIM_E2E_QEMU=1 LIM_E2E_OS=debian $(PYTHON) -m pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
|
|
||||||
# Everything: mocked suite plus both opt-in e2e stages, in one run.
|
# Everything: mocked suite plus both opt-in e2e stages, in one run.
|
||||||
test-all:
|
test-all:
|
||||||
LIM_E2E_TOR=1 LIM_E2E_QEMU=1 $(PYTHON) -m pytest -v -s
|
LIM_E2E_TOR=1 LIM_E2E_QEMU=1 $(PYTHON) -m pytest -v -s
|
||||||
|
|||||||
@@ -27,7 +27,8 @@ CPU architecture.
|
|||||||
|
|
||||||
| Stage | File | Privilege |
|
| Stage | File | Privilege |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Build encrypted image | `build_image.sh` | **root** (loop, cryptsetup, chroot) |
|
| Build encrypted image (Arch) | `build_image.sh` | **root** (loop, cryptsetup, chroot) |
|
||||||
|
| Build encrypted image (Debian) | `build_image_debian.sh` | **root** (debootstrap, loop, cryptsetup, chroot) |
|
||||||
| Tor network | `tor_net.py` | rootless |
|
| Tor network | `tor_net.py` | rootless |
|
||||||
| Boot + unlock | `boot_unlock.py` | rootless (QEMU user-mode net) |
|
| Boot + unlock | `boot_unlock.py` | rootless (QEMU user-mode net) |
|
||||||
| Orchestration | `harness.py` | mixed (uses `sudo` for the build only) |
|
| Orchestration | `harness.py` | mixed (uses `sudo` for the build only) |
|
||||||
@@ -53,18 +54,41 @@ and then `chown`s the artifacts back so rootless QEMU can read them.
|
|||||||
- Network access to the public Tor network, **or** `chutney` for a private one
|
- Network access to the public Tor network, **or** `chutney` for a private one
|
||||||
- `/dev/kvm` recommended (TCG works but is slow; aarch64-on-x86 is always TCG)
|
- `/dev/kvm` recommended (TCG works but is slow; aarch64-on-x86 is always TCG)
|
||||||
|
|
||||||
|
For the Debian build (`LIM_E2E_OS=debian`), additionally: `debootstrap` and
|
||||||
|
network access to a Debian mirror. The guest uses the initramfs-tools backend
|
||||||
|
(cryptsetup-initramfs + dropbear-initramfs), so the unlock SSH session runs
|
||||||
|
`cryptroot-unlock` instead of Arch's login-time encryptssh prompt.
|
||||||
|
|
||||||
|
## Unlock transport: direct vs Tor
|
||||||
|
|
||||||
|
`LIM_E2E_TRANSPORT` (default `direct`) selects how the passphrase reaches the
|
||||||
|
guest's dropbear:
|
||||||
|
|
||||||
|
- **direct** — QEMU forwards a host port to the guest's dropbear (`:22`) and the
|
||||||
|
passphrase is delivered over a plain SSH. Deterministic; it verifies the whole
|
||||||
|
LUKS-unlock stack (cryptsetup-initramfs + dropbear-initramfs + cryptroot-unlock
|
||||||
|
+ LUKS + boot). This is what CI/`make` runs.
|
||||||
|
- **tor** — the full onion path: the guest publishes its onion and the host
|
||||||
|
reaches it through Tor. Representative of production but **flaky over public
|
||||||
|
Tor inside QEMU** (fresh guest Tor + user-mode net make the onion rendezvous
|
||||||
|
unreliable), so it is opt-in. The onion transport itself is covered
|
||||||
|
deterministically by the rootless `test_tor_unlock_e2e.py`.
|
||||||
|
|
||||||
## Running
|
## Running
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Public Tor network (simplest; non-deterministic, needs internet):
|
# Arch (mkinitcpio), direct transport (deterministic, default):
|
||||||
LIM_E2E_QEMU=1 pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
LIM_E2E_QEMU=1 pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
|
|
||||||
# Private, offline, deterministic Tor network via chutney:
|
# Debian (initramfs-tools) via debootstrap, direct transport:
|
||||||
git clone https://gitlab.torproject.org/tpo/core/chutney
|
LIM_E2E_QEMU=1 LIM_E2E_OS=debian pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
CHUTNEY_PATH=$PWD/chutney LIM_E2E_QEMU=1 \
|
|
||||||
pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
# Full onion transport (opt-in, flaky over public Tor):
|
||||||
|
LIM_E2E_QEMU=1 LIM_E2E_TRANSPORT=tor pytest tests/e2e/test_qemu_unlock_e2e.py -v -s
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`make test-qemu` / `make test-qemu-debian` wrap the direct-transport runs.
|
||||||
|
|
||||||
Environment knobs: `LIM_E2E_ARCH` (`x86_64` default, or `aarch64`),
|
Environment knobs: `LIM_E2E_ARCH` (`x86_64` default, or `aarch64`),
|
||||||
`CHUTNEY_PATH` (enables the private network), `CHUTNEY_PATH` unset → public.
|
`CHUTNEY_PATH` (enables the private network), `CHUTNEY_PATH` unset → public.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user