Reorganized hal CLI into subcommand groups + MIT licensed

CLI structure now:
  hal {status,diagnose,unlock,forget} HOST
  hal connect {rescue,chroot,server} HOST [CMD]
  hal setup   {image,dropbear,grub,encrypt-root} HOST
  hal fix     {boot,network,grub,kernel,static-ip,upgrade,expand-fs} HOST

Added subcommands cover the previously-manual sections of the README:
  setup image       — upload autosetup + run installimage
  setup dropbear    — install dropbear + mkinitcpio plugins + patch HOOKS
  setup grub        — initial grub install for LUKS boot
  setup encrypt-root — full LUKS conversion of installed root
  connect server    — SSH to booted Arch (vs rescue/chroot)
  unlock            — cryptroot-unlock via dropbear with passphrase from keyring
  fix expand-fs     — lvresize + btrfs resize

Renames (breaking):
  upgrade-system    -> fix upgrade
  expand-fs         -> fix expand-fs
  forget-passphrase -> forget
  reinstall-grub    -> fix grub
  downgrade-kernel  -> fix kernel
  use-static-ip     -> fix static-ip
  fix-{boot,network} -> fix {boot,network}
  install-{image,grub} -> setup {image,grub}
  setup-dropbear    -> setup dropbear
  encrypt-root      -> setup encrypt-root

Removed downgrade-initramfs (never verified, narrow use case).

README rewritten to reference only hal commands; raw bash blocks for
pacman/cryptsetup/grub-install/mount/chroot are gone. Added autosetup.example
as a template for `hal setup image --autosetup PATH`.

Licensed under MIT (LICENSE file added). Author and homepage shown in
hal --version, hal --help, pyproject.toml, and README.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Kevin Veen-Birkenbach
2026-05-12 18:10:06 +02:00
parent 181240eae7
commit 3cf66640b5
10 changed files with 755 additions and 603 deletions

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Kevin Veen-Birkenbach <kevin@veen.world>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

435
README.md
View File

@@ -1,375 +1,158 @@
# Arch Linux with LUKS and btrfs on a Hetzner server
## Software
This guide shows how to set up the following software composition:
* [Arch Linux](https://www.archlinux.de/)
* [btrfs](https://en.wikipedia.org/wiki/Btrfs)
* [LUKS](https://wiki.archlinux.org/index.php/Dm-crypt)
A small Python CLI (`hal`) that wraps every step of installing, encrypting, and
maintaining an [Arch Linux](https://www.archlinux.de/) server on
[Hetzner](https://www.hetzner.com/) Dedicated hardware with software RAID,
[LUKS](https://wiki.archlinux.org/index.php/Dm-crypt) full-disk encryption,
[btrfs](https://en.wikipedia.org/wiki/Btrfs) on top of LVM, and remote unlock
via [dropbear](https://wiki.archlinux.org/title/Dm-crypt/Specialties#busybox-based_initramfs_(built_with_mkinitcpio))
in the initramfs.
## Requirements
Written for a [Dedicated](https://de.wikipedia.org/wiki/Server#Dedizierte_Server) [Hetzner](https://www.hetzner.com/) server with the following hardware specifications:
```
CPU1: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (Cores 8)
Memory: 15973 MB
Disk /dev/sda: 3000 GB (=> 2794 GiB)
Disk /dev/sdb: 3000 GB (=> 2794 GiB)
Total capacity 5589 GiB with 2 Disks
```
**Author:** Kevin Veen-Birkenbach &lt;[kevin@veen.world](mailto:kevin@veen.world)&gt; — [veen.world](https://veen.world)
**License:** MIT — see [LICENSE](./LICENSE)
## Legend
The following symbols show in which environment the code is executed:
* :computer: Client
* :ambulance: [Hetzner Rescue System](https://wiki.hetzner.de/index.php/Hetzner_Rescue-System/en)
* :ghost: Chroot from Rescue System into Arch
* :minidisc: Arch OS
## CLI helper (`hal`)
This repo ships a small Python CLI (`hal`) that wraps the recurring SSH / LUKS / chroot dances. Install it once on your client:
## Install the CLI
```bash
pip install --user -e .
make install # → pip install --user -e .
hal --help
```
After that, `hal` is on your `$PATH`. Subcommands used throughout the guide:
After install, every step below is a single `hal` subcommand.
## Subcommand reference
Run `hal --help`, `hal <group> --help`, or `hal <group> <target> --help` for the live reference.
### Top-level
| Command | What it does |
|---|---|
| `hal status <host>` | Probe reachability (ping, ports 22/222, SSH banner). No login. |
| `hal connect rescue <host>` | Wait for rescue, drop known_hosts entry, SSH in as root. |
| `hal connect chroot <host>` | Prompt LUKS passphrase **first** (hidden), then via rescue: assemble RAID → unlock LUKS → mount → drop into `chroot /mnt /bin/bash`. |
| `hal diagnose <host>` | Same setup as `connect chroot`, then runs a fixed diagnostic script inside the chroot and prints the report to stdout. |
| `hal status <host>` | Ping + port scan + SSH banner. No login. |
| `hal diagnose <host>` | Rescue → chroot, runs a fixed inspection script. Pipe with `tee` to save. |
| `hal unlock <host>` | Send the LUKS passphrase from the keyring to dropbear (`cryptroot-unlock`). |
| `hal forget <host>` | Clear the cached LUKS passphrase from libsecret. |
The passphrase prompt happens *before* the SSH connection is established, so you can type it once, walk away, and the rest runs unattended.
### `hal connect <target> <host> [cmd]`
## Guide
### 1. Configure and Install Image
#### 1.1 Login to Hetzner Rescue System
:computer: :
```bash
hal connect rescue your_server_ip
```
#### 1.2 Create the /autosetup
Open a shell, or run `cmd` non-interactively.
:ambulance: :
| Target | Where it goes |
|---|---|
| `rescue` | Hetzner Rescue OS |
| `server` | Booted Arch system |
| `chroot` | Rescue → chroot of installed Arch (LUKS-unlocks + mounts first) |
### `hal setup <target> <host>` — one-time install operations
| Target | What it does |
|---|---|
| `image --autosetup PATH` | In rescue: upload autosetup, run `installimage`. **Destructive.** |
| `dropbear` | Booted Arch: install dropbear + mkinitcpio plugins, copy authorized_keys, patch HOOKS. |
| `grub` | Rescue → chroot: install grub package, write LUKS-aware `/etc/default/grub`, grub-install on every boot disk. |
| `encrypt-root` | Rescue: LUKS-encrypt `/dev/md1`, preserve data via `/oldroot` copy. **Destructive on `/dev/md1`. Confirms before format.** |
### `hal fix <target> <host>` — recovery + maintenance operations
| Target | What it does |
|---|---|
| `boot` | Patch `PermitRootLogin`, enable persistent journald. |
| `network` | Rewrite `.network` files to match by MACAddress= instead of interface name. |
| `grub` | Refresh Stage1 + core.img in MBR (Arch doesn't do this automatically after grub upgrades). |
| `kernel` | Roll the `linux` package back to the previous version (cache or archive.archlinux.org). |
| `static-ip` | Replace `ip=dhcp` in `/etc/default/grub` with a static cmdline IP derived from `/etc/systemd/network/*.network`. |
| `upgrade` | Full `pacman -Syyu` + initramfs rebuild + grub-install on every boot disk. |
| `expand-fs` | On booted Arch: `lvresize -l +100%FREE /dev/vg0/root && btrfs filesystem resize max /`. |
The LUKS passphrase is prompted (hidden) on first use and cached in the libsecret keyring per host — subsequent runs against the same host don't prompt.
## Setup flow
Each section is a small handful of `hal` commands. Click into the corresponding
table row above for what each one actually does.
### 1. Install Arch via installimage
```bash
nano /autosetup
hal connect rescue YOUR_SERVER_IP # verify rescue is up
hal setup image YOUR_SERVER_IP --autosetup autosetup # see autosetup.example
hal connect rescue YOUR_SERVER_IP reboot
```
Save the following content into this file:
Tip: copy `autosetup.example` to `autosetup`, edit `DRIVE1`/`DRIVE2`/`HOSTNAME`,
then run `setup image`.
```
## Hetzner Online GmbH - installimage - config
DRIVE1 /dev/sda
DRIVE2 /dev/sdb
## SOFTWARE RAID:
## activate software RAID? < 0 | 1 >
SWRAID 1
## Choose the level for the software RAID < 0 | 1 | 10 >
SWRAIDLEVEL 1
## BOOTLOADER:
BOOTLOADER grub
## HOSTNAME:
HOSTNAME hetzner-arch-luks
#Adapt the hostname to your needs
## PARTITIONS / FILESYSTEMS:
PART /boot btrfs 512M
PART lvm vg0 all
LV vg0 swap swap swap 8G
LV vg0 root / btrfs 10G
## OPERATING SYSTEM IMAGE:
IMAGE /root/.oldroot/nfs/install/../images/archlinux-latest-64-minimal.tar.gz
```
#### 1.3 Install Image
:ambulance: :
```bash
installimage
```
#### 1.4 Restart
:ambulance: :
```bash
reboot
```
### 2. Setup System
#### 2.1 Login to server
:computer: :
```bash
ssh-keygen -f "$HOME/.ssh/known_hosts" -R your_server_ip
ssh root@your_server_ip
```
#### 2.2 Update the system
:minidisc: :
```bash
pacman -Syyu
```
#### 2.3 Install administration tools:
:minidisc: :
```bash
pacman -S nano
```
### 3. Prepare System for Unlocking via SSH
#### 3.1 Install software
:minidisc: :
```bash
pacman -S busybox mkinitcpio-dropbear mkinitcpio-utils mkinitcpio-netconf
```
#### 3.2 Copy authorized keys to dropbear
:minidisc: :
```bash
cp -v ~/.ssh/authorized_keys /etc/dropbear/root_key
```
### 2. Boot Arch, install the dropbear stack
```bash
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
systemctl enable sshd
hal connect server YOUR_SERVER_IP # verify SSH works
hal connect server YOUR_SERVER_IP pacman -Syyu # bring system current
hal setup dropbear YOUR_SERVER_IP # dropbear + mkinitcpio plugins + HOOKS
```
#### 3.3 Regenerate OpenSSH keys
:minidisc: :
```bash
rm /etc/ssh/ssh_host_*
ssh-keygen -A -m PEM
```
#### 3.4 Import SSH-keys to dropbear
:minidisc: :
```bash
dropbearconvert openssh dropbear /etc/ssh/ssh_host_rsa_key /etc/dropbear/dropbear_rsa_host_key
```
### 3. Convert root to LUKS
#### 3.5 Modify /etc/mkinitcpio.conf
:minidisc: :
```bash
nano /etc/mkinitcpio.conf
```
##### Replace
**Old:**
```
HOOKS=(base udev autodetect modconf block mdadm_udev lvm2 filesystems keyboard fsck)
```
**New:**
```
HOOKS=(base udev autodetect modconf block mdadm_udev lvm2 netconf dropbear encryptssh filesystems keyboard fsck)
```
### 4. Activate Encryption
#### 4.1 Activate Rescue System
Activate the rescue system https://robot.your-server.de/server
#### 4.2 Reboot
:minidisc: :
```bash
reboot
```
#### 4.3 Login to the rescue system
:computer: :
```bash
hal connect rescue your_server_ip
```
#### 4.4 Mount the "system"
:ambulance: :
```bash
vgscan -v
vgchange -a y
mount /dev/mapper/vg0-root /mnt
```
#### 4.5 Copy "system"
:ambulance: :
```bash
echo 0 >/proc/sys/dev/raid/speed_limit_max
mkdir /oldroot
cp -va /mnt/. /oldroot/.
echo 200000 >/proc/sys/dev/raid/speed_limit_max
```
#### 4.6 Unmount the "system"
:ambulance: :
```bash
umount /mnt
```
#### 4.7 Delete decrypted LVM-Volume-Group
:ambulance: :
```bash
vgremove vg0
```
#### 4.8 Check drive state
:ambulance: :
```bash
cat /proc/mdstat
```
#### 4.9 Encrypt MD1 by executing
:ambulance: :
```bash
cryptsetup --cipher aes-xts-plain64 --key-size 256 --hash sha256 --iter-time=10000 luksFormat /dev/md1
cryptsetup luksOpen /dev/md1 cryptroot
pvcreate /dev/mapper/cryptroot
vgcreate vg0 /dev/mapper/cryptroot
lvcreate -n swap -L8G vg0
lvcreate -n root -L10G vg0
mkfs.btrfs /dev/vg0/root
mkswap /dev/vg0/swap
```
#### 4.10 Mount encrypted
:ambulance: :
```bash
mount /dev/vg0/root /mnt
```
#### 4.12 Copy "system"
:ambulance: :
```bash
echo 0 >/proc/sys/dev/raid/speed_limit_max
cp -av /oldroot/. /mnt/.
echo 200000 >/proc/sys/dev/raid/speed_limit_max
```
#### 4.13 Integrate Finale Installation
:ambulance: :
```bash
mount /dev/md0 /mnt/boot
mount --bind /dev /mnt/dev
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
chroot /mnt
```
#### 4.14
:ghost: :
```bash
echo "cryptroot /dev/md1 none luks" >> /etc/crypttab
```
#### 4.15 Create an initial ramdisk
:ghost: :
```bash
mkinitcpio -p linux
```
### 5 Grub
#### 5.1 Install Grub
:ghost: :
```bash
pacman -S grub
```
#### 5.2 Configure /etc/default/grub
:ghost: :
Activate Rescue in the Hetzner Robot UI, then:
```bash
nano /etc/default/grub
hal connect server YOUR_SERVER_IP reboot # boots back into rescue
hal connect rescue YOUR_SERVER_IP # verify rescue is up
hal setup encrypt-root YOUR_SERVER_IP # LUKS conversion — DESTRUCTIVE
hal setup grub YOUR_SERVER_IP # initial GRUB for LUKS boot
hal fix static-ip YOUR_SERVER_IP # (recommended) harden initramfs network
```
Change the following parameters:
Deactivate Rescue in the Hetzner Robot UI, then:
```bash
GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=dhcp"
GRUB_ENABLE_CRYPTODISK=y
```
:information_source: Further [information](https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system#Configuring_GRUB).
#### 5.3 Make and Install on Hard-drives
:ghost: :
```bash
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda
grub-install /dev/sdb
hal connect rescue YOUR_SERVER_IP reboot # final reboot into encrypted system
```
#### 5.4 Restart System
:ghost: :ambulance: :
### 4. Day-to-day use
After every reboot the system blocks at dropbear in initramfs waiting for the
LUKS passphrase. From your client:
```bash
exit
umount /mnt/boot /mnt/proc /mnt/sys /mnt/dev
umount /mnt
sync
reboot
```
### 6. Encryption Procedure
#### 6.1 Decrypt server
:computer: :
```bash
ssh -o UserKnownHostsFile=/dev/null root@your_server_ip
cryptroot-unlock
exit
```
#### 6.2 Login to server
:computer: :
```bash
ssh-keygen -f "$HOME/.ssh/known_hosts" -R your_server_ip
ssh root@your_server_ip
hal status YOUR_SERVER_IP # wait for dropbear / sshd
hal unlock YOUR_SERVER_IP # send passphrase to dropbear
hal connect server YOUR_SERVER_IP # normal SSH after unlock
```
#### 7. Expand filesystem
:computer: :
### 5. Expand the root filesystem later
If the autosetup gave you a small root LV and the rest is free LVM space:
```bash
lvresize -l +100%FREE /dev/vg0/root
btrfs filesystem resize max /
hal fix expand-fs YOUR_SERVER_IP
```
## 8. Debugging
### 8.1 Login to System from Rescue System
With the rescue system already activated and running, drop straight into the chroot from your client:
## Debugging an unresponsive server
The server isn't booting / SSH never comes up:
:computer: :
```bash
hal connect chroot your_server_ip
```
You'll be prompted for the LUKS passphrase first (hidden input). The CLI then waits for rescue, assembles the RAID, opens LUKS, activates LVM, mounts `/mnt` + `/mnt/boot` + the pseudo-filesystems, and drops you into `chroot /mnt /bin/bash`. Idempotent — re-running while already mounted just re-enters the chroot.
# 1. Reach the server's chroot
hal connect rescue YOUR_SERVER_IP # via Hetzner Robot → Rescue first
hal diagnose YOUR_SERVER_IP | tee "diag-$(date +%F-%H%M).log"
### 8.2 Collect diagnostics in one shot
If you want a non-interactive snapshot of the installed system's state (package versions, last-boot journal errors, sshd status, `/boot` contents, etc.):
# 2. Apply best-guess fixes in roughly this order
hal fix boot YOUR_SERVER_IP # sshd config + journald
hal fix network YOUR_SERVER_IP # interface naming drift
hal fix grub YOUR_SERVER_IP # stale MBR after grub upgrades
hal fix static-ip YOUR_SERVER_IP # DHCP-in-initramfs fragility
:computer: :
```bash
hal diagnose your_server_ip | tee "diagnose-$(date +%F-%H%M).log"
```
The CLI runs the same setup as `connect chroot` and then a fixed inspection script inside the chroot. Output goes to stdout (and the log file via `tee`).
# 3. Last-resort kernel rollback (if a kernel bump is the suspect)
hal fix kernel YOUR_SERVER_IP
<details>
<summary>Manual equivalent of the unlock + mount sequence</summary>
:ambulance: :
```bash
cryptsetup luksOpen /dev/md1 cryptroot
mount /dev/vg0/root /mnt
mount /dev/md0 /mnt/boot
mount --bind /dev /mnt/dev
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
chroot /mnt
```
</details>
### 8.3 Logout from chroot environment
:ghost: :ambulance: :
```bash
exit
umount /mnt/boot /mnt/proc /mnt/sys /mnt/dev
umount /mnt
sync
reboot
# 4. Or, after fixing whatever was broken, upgrade everything cleanly
hal fix upgrade YOUR_SERVER_IP
```
### 8.4 Regenerate GRUB and Arch
:ghost: :
```bash
mkinitcpio -p linux
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda
grub-install /dev/sdb
```
Every `hal` chroot command makes its own backups (`<file>.hal-backup`)
before mutating anything, so individual fixes can be reverted by hand.
## Sources
The code is adapted from the following guides:
* http://daemons-point.com/blog/2019/10/20/hetzner-verschluesselt/
* https://www.howtoforge.com/using-the-btrfs-filesystem-with-raid1-with-ubuntu-12.10-on-a-hetzner-server

31
autosetup.example Normal file
View File

@@ -0,0 +1,31 @@
## Hetzner Online GmbH - installimage - config
## Copy to a working file, adjust DRIVE / HOSTNAME / sizes to your box,
## then upload via: hal install-image <host> --autosetup <path>
## Adjust DRIVE1 / DRIVE2 to your actual disks. Typical values:
## - /dev/sda /dev/sdb (SATA/SAS auction boxes)
## - /dev/nvme0n1 /dev/nvme1n1 (NVMe-based servers)
DRIVE1 /dev/sda
DRIVE2 /dev/sdb
## SOFTWARE RAID:
## activate software RAID? < 0 | 1 >
SWRAID 1
## Choose the level for the software RAID < 0 | 1 | 10 >
SWRAIDLEVEL 1
## BOOTLOADER:
BOOTLOADER grub
## HOSTNAME: adapt to your needs
HOSTNAME hetzner-arch-luks
## PARTITIONS / FILESYSTEMS:
## /boot must be its own partition (btrfs/ext4); root and swap on LVM.
PART /boot btrfs 512M
PART lvm vg0 all
LV vg0 swap swap swap 8G
LV vg0 root / btrfs 10G
## OPERATING SYSTEM IMAGE:
IMAGE /root/.oldroot/nfs/install/../images/archlinux-latest-64-minimal.tar.gz

View File

@@ -1,15 +1,19 @@
[build-system]
requires = ["setuptools>=64"]
# 77+ for PEP 639 SPDX `license = "MIT"` + `license-files`.
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"
[project]
name = "hetzner-arch-luks"
version = "0.1.0"
description = "CLI helpers for the hetzner-arch-luks setup: connect to rescue, drop into the encrypted chroot, probe reachability, collect diagnostics."
description = "End-to-end CLI (`hal`) for installing, encrypting, debugging and maintaining an Arch Linux server on Hetzner Dedicated hardware with software RAID, LUKS full-disk encryption, btrfs on LVM, and remote unlock via dropbear in the initramfs."
readme = "README.md"
requires-python = ">=3.9"
authors = [{ name = "Kevin Veen-Birkenbach" }]
license = { text = "Proprietary" }
authors = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
maintainers = [{ name = "Kevin Veen-Birkenbach", email = "kevin@veen.world" }]
license = "MIT"
license-files = ["LICENSE"]
urls = { Homepage = "https://veen.world", Repository = "https://github.com/kevinveenbirkenbach/hetzner-arch-luks" }
classifiers = [
"Environment :: Console",
"Operating System :: POSIX :: Linux",

View File

@@ -1,209 +1,288 @@
"""Command-line interface for the hetzner-arch-luks helpers.
Entry point: hal <subcommand> <host>
Top-level structure:
Subcommands:
status client-side reachability probe (no login)
connect rescue <host> SSH into the rescue system
connect chroot <host> LUKS unlock + mount + interactive chroot shell
diagnose <host> LUKS unlock + mount + collect diagnostics
hal status HOST
hal diagnose HOST
hal unlock HOST
hal forget HOST
For commands that need the LUKS passphrase, the prompt happens *first*, before
any network IO — so you can type the passphrase, walk away, and the rest runs
unattended.
hal connect {rescue,chroot,server} HOST [CMD...]
hal setup {image,dropbear,grub,encrypt-root} HOST [...]
hal fix {boot,network,grub,kernel,static-ip,upgrade,expand-fs} HOST
For commands that need the LUKS passphrase, the prompt happens *first*,
before any network IO. The passphrase is cached per-host in the libsecret
keyring so subsequent runs against the same host don't prompt.
"""
from __future__ import annotations
import argparse
import sys
from . import probe, remote
from . import __version__, probe, remote
_AUTHOR = "Kevin Veen-Birkenbach <kevin@veen.world>"
_HOMEPAGE = "https://veen.world"
def _add_passphrase_flag(p: argparse.ArgumentParser) -> None:
p.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
)
def _add_host(p: argparse.ArgumentParser) -> None:
p.add_argument("host")
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="hal",
description="Helper CLI for the hetzner-arch-luks workflow.",
description=(
"End-to-end CLI for installing, encrypting, debugging and maintaining "
"an Arch Linux server on Hetzner Dedicated hardware with software RAID, "
"LUKS full-disk encryption, btrfs on LVM, and remote unlock via dropbear "
"in the initramfs."
),
epilog=f"Author: {_AUTHOR}{_HOMEPAGE} License: MIT",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--version",
action="version",
version=(
f"hal {__version__}\n"
f"Author: {_AUTHOR}\n"
f"Homepage: {_HOMEPAGE}\n"
f"License: MIT"
),
)
sub = parser.add_subparsers(dest="cmd", required=True)
p_status = sub.add_parser(
sub = parser.add_subparsers(dest="cmd", required=True, metavar="COMMAND")
# -------------------- Top-level commands --------------------
p = sub.add_parser(
"status",
help="Probe reachability of a host (ping + ports + SSH banner). No login.",
)
p_status.add_argument("host")
_add_host(p)
p = sub.add_parser(
"diagnose",
help="Collect a fixed inspection report from inside the installed system via rescue.",
)
_add_host(p)
_add_passphrase_flag(p)
p = sub.add_parser(
"unlock",
help="Send the LUKS passphrase from the keyring to dropbear (cryptroot-unlock). Use after a reboot.",
)
_add_host(p)
_add_passphrase_flag(p)
p = sub.add_parser(
"forget",
help="Drop the cached LUKS passphrase for a host from the libsecret keyring.",
)
_add_host(p)
# -------------------- `connect` group --------------------
p_connect = sub.add_parser(
"connect",
help="Open an interactive remote shell.",
help="Open a remote shell on rescue / chroot / server, or run a one-off command there.",
)
p_connect_sub = p_connect.add_subparsers(
dest="target", required=True, metavar="TARGET"
)
p_connect_sub = p_connect.add_subparsers(dest="target", required=True)
p_rescue = p_connect_sub.add_parser(
p = p_connect_sub.add_parser(
"rescue",
help="SSH into the Hetzner rescue system (waits for port 22 to come up). "
"Pass extra args after the host to run them non-interactively.",
help="SSH into the Hetzner rescue system. Append a command for non-interactive use.",
)
p_rescue.add_argument("host")
p_rescue.add_argument(
"command",
nargs=argparse.REMAINDER,
help="Optional command + args to run on the rescue instead of opening "
"an interactive shell. Example: hal connect rescue HOST reboot",
_add_host(p)
p.add_argument(
"command", nargs=argparse.REMAINDER,
help="Optional command + args to run on the rescue instead of an interactive shell.",
)
p_chroot = p_connect_sub.add_parser(
p = p_connect_sub.add_parser(
"chroot",
help="Unlock LUKS via rescue, mount, and drop into chroot /mnt /bin/bash. "
"Pass extra args after the host to run them inside the chroot.",
help="Unlock LUKS via rescue, mount, and drop into `chroot /mnt /bin/bash`. Append a command for non-interactive use.",
)
p_chroot.add_argument("host")
p_chroot.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
)
p_chroot.add_argument(
"command",
nargs=argparse.REMAINDER,
help="Optional command + args to run inside the chroot instead of "
"opening an interactive shell. Example: hal connect chroot HOST pacman -Q linux",
_add_host(p)
_add_passphrase_flag(p)
p.add_argument(
"command", nargs=argparse.REMAINDER,
help="Optional command + args to run inside the chroot instead of an interactive shell.",
)
p_diag = sub.add_parser(
"diagnose",
help="Collect diagnostics from inside the installed system via rescue.",
p = p_connect_sub.add_parser(
"server",
help="SSH into the booted Arch system. Append a command for non-interactive use.",
)
p_diag.add_argument("host")
p_diag.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
_add_host(p)
p.add_argument(
"command", nargs=argparse.REMAINDER,
help="Optional command + args to run on the server instead of an interactive shell.",
)
# -------------------- `setup` group (one-time install) --------------------
p_setup = sub.add_parser(
"setup",
help="One-time install operations: image / dropbear / grub / encrypt-root.",
)
p_setup_sub = p_setup.add_subparsers(
dest="target", required=True, metavar="TARGET"
)
p = p_setup_sub.add_parser(
"image",
help="In rescue: upload an autosetup file and run `installimage`. DESTRUCTIVE.",
)
_add_host(p)
p.add_argument(
"--autosetup", required=True,
help="Path to a local autosetup config file (uploaded to /autosetup on rescue).",
)
p = p_setup_sub.add_parser(
"dropbear",
help="On the booted system: install dropbear + mkinitcpio plugins, copy authorized_keys, patch HOOKS. MUTATES.",
)
_add_host(p)
p = p_setup_sub.add_parser(
"grub",
help="In chroot (initial install): install grub package, write LUKS-aware /etc/default/grub, grub-install on every boot disk. MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p = p_setup_sub.add_parser(
"encrypt-root",
help="In rescue: full LUKS conversion of an installed Arch (sections 4.44.15). DESTRUCTIVE — confirms before format.",
)
_add_host(p)
# -------------------- `fix` group (recovery operations) --------------------
p_fix = sub.add_parser(
"fix-boot",
help="Apply boot/SSH fixes inside the chroot. MUTATES the installed system.",
"fix",
help="Recovery + maintenance operations: boot / network / grub / kernel / static-ip / upgrade / expand-fs.",
)
p_fix.add_argument("host")
p_fix.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p_fix_sub = p_fix.add_subparsers(
dest="target", required=True, metavar="TARGET"
)
p_fixnet = sub.add_parser(
"fix-network",
help="Rewrite systemd-networkd .network files to use MACAddress= match. MUTATES.",
)
p_fixnet.add_argument("host")
p_fixnet.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"boot",
help="In chroot: patch PermitRootLogin to prohibit-password, enable persistent journald. MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p_dk = sub.add_parser(
"downgrade-kernel",
help="Roll the linux package back to the previous cached version. MUTATES. "
"Use after a kernel-bump pacman -Syu made the system unbootable.",
)
p_dk.add_argument("host")
p_dk.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"network",
help="In chroot: rewrite /etc/systemd/network/*.network to match by MACAddress= instead of interface name. MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p_fp = sub.add_parser(
"forget-passphrase",
help="Drop the cached LUKS passphrase for a host from the libsecret keyring.",
p = p_fix_sub.add_parser(
"grub",
help="In chroot: re-run grub-install on every disk backing /boot. MUTATES the MBR.",
)
p_fp.add_argument("host")
_add_host(p)
_add_passphrase_flag(p)
p_rg = sub.add_parser(
"reinstall-grub",
help="Re-run grub-install on every disk backing /boot. MUTATES the MBR. "
"Use after a grub-package upgrade that didn't refresh the bootloader.",
)
p_rg.add_argument("host")
p_rg.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"kernel",
help="In chroot: roll the `linux` package back to the previous version (cache or archive.archlinux.org). MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p_di = sub.add_parser(
"downgrade-initramfs",
help="Downgrade mkinitcpio + dropbear + cryptsetup + mdadm + lvm2 to the "
"version before the last pacman -Syu, then rebuild initramfs. MUTATES.",
)
p_di.add_argument("host")
p_di.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"static-ip",
help="In chroot: replace `ip=dhcp` in /etc/default/grub with a static kernel-cmdline IP derived from the .network file. MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p_si = sub.add_parser(
"use-static-ip",
help="Replace ip=dhcp in /etc/default/grub with a static kernel-cmdline "
"network spec (derived from /etc/systemd/network/*.network). MUTATES.",
)
p_si.add_argument("host")
p_si.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"upgrade",
help="In chroot: full `pacman -Syyu` + rebuild initramfs + grub-install on every boot disk. MUTATES.",
)
_add_host(p)
_add_passphrase_flag(p)
p_us = sub.add_parser(
"upgrade-system",
help="Full pacman -Syyu + initramfs rebuild + grub-install on every boot disk "
"+ grub.cfg regen, all in one chroot session. Uses --disable-sandbox "
"to work around the Hetzner Rescue kernel's missing Landlock. MUTATES.",
)
p_us.add_argument("host")
p_us.add_argument(
"--no-passphrase-prompt",
action="store_true",
help="Skip the early LUKS prompt (use when LUKS is already open from a prior run).",
p = p_fix_sub.add_parser(
"expand-fs",
help="On the booted system: `lvresize -l +100%%FREE /dev/vg0/root && btrfs filesystem resize max /`. MUTATES.",
)
_add_host(p)
return parser
def main(argv: list[str] | None = None) -> int:
args = _build_parser().parse_args(argv)
pp = not getattr(args, "no_passphrase_prompt", False)
# Top-level
if args.cmd == "status":
return probe.status(args.host)
if args.cmd == "connect" and args.target == "rescue":
return remote.connect_rescue(args.host, command=args.command or None)
if args.cmd == "connect" and args.target == "chroot":
return remote.connect_chroot(
args.host,
ask_passphrase=not args.no_passphrase_prompt,
command=args.command or None,
)
if args.cmd == "diagnose":
return remote.diagnose(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "fix-boot":
return remote.fix_boot(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "fix-network":
return remote.fix_network(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "downgrade-kernel":
return remote.downgrade_kernel(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "forget-passphrase":
return remote.diagnose(args.host, ask_passphrase=pp)
if args.cmd == "unlock":
return remote.unlock(args.host, ask_passphrase=pp)
if args.cmd == "forget":
return remote.forget_passphrase(args.host)
if args.cmd == "reinstall-grub":
return remote.reinstall_grub(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "downgrade-initramfs":
return remote.downgrade_initramfs(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "use-static-ip":
return remote.use_static_ip(args.host, ask_passphrase=not args.no_passphrase_prompt)
if args.cmd == "upgrade-system":
return remote.upgrade_system(args.host, ask_passphrase=not args.no_passphrase_prompt)
# connect group
if args.cmd == "connect":
cmd_list = getattr(args, "command", None) or None
if args.target == "rescue":
return remote.connect_rescue(args.host, command=cmd_list)
if args.target == "chroot":
return remote.connect_chroot(args.host, ask_passphrase=pp, command=cmd_list)
if args.target == "server":
return remote.connect_server(args.host, command=cmd_list)
# setup group
if args.cmd == "setup":
if args.target == "image":
return remote.install_image(args.host, args.autosetup)
if args.target == "dropbear":
return remote.setup_dropbear(args.host)
if args.target == "grub":
return remote.install_grub(args.host, ask_passphrase=pp)
if args.target == "encrypt-root":
return remote.encrypt_root(args.host)
# fix group
if args.cmd == "fix":
if args.target == "boot":
return remote.fix_boot(args.host, ask_passphrase=pp)
if args.target == "network":
return remote.fix_network(args.host, ask_passphrase=pp)
if args.target == "grub":
return remote.reinstall_grub(args.host, ask_passphrase=pp)
if args.target == "kernel":
return remote.downgrade_kernel(args.host, ask_passphrase=pp)
if args.target == "static-ip":
return remote.use_static_ip(args.host, ask_passphrase=pp)
if args.target == "upgrade":
return remote.upgrade_system(args.host, ask_passphrase=pp)
if args.target == "expand-fs":
return remote.expand_fs(args.host)
return 2

View File

@@ -181,24 +181,34 @@ def _setup(ssh: SshSession, host: str, passphrase: str | None) -> None:
# ---- public entry points (called by cli.py) --------------------------------
def connect_rescue(host: str, *, command: list[str] | None = None) -> int:
"""Wait for rescue to come up, then either open an interactive SSH shell
or run `command` non-interactively and print its output.
No passphrase prompt — rescue itself isn't encrypted.
def _connect_simple(host: str, label: str, command: list[str] | None) -> int:
"""Shared body of `connect_rescue` and `connect_server` — wait for SSH,
then either drop into an interactive shell or run `command` and print.
"""
_wait_rescue(host)
with SshSession(host) as ssh:
if command:
cmd_str = " ".join(shlex.quote(c) for c in command)
print(f"==> Running on rescue: {cmd_str}")
print(f"==> Running on {label}: {cmd_str}")
ssh.run(cmd_str, check=False)
else:
print("==> Connected to rescue. Type 'exit' to leave.")
print(f"==> Connected to {label}. Type 'exit' to leave.")
ssh.run("exec bash -l", tty=True, check=False)
return 0
def connect_rescue(host: str, *, command: list[str] | None = None) -> int:
"""Wait for rescue to come up, then open a shell or run `command`."""
return _connect_simple(host, "rescue", command)
def connect_server(host: str, *, command: list[str] | None = None) -> int:
"""Wait for the booted Arch system to come up, then open a shell or
run `command`. Same SSH plumbing as `connect_rescue`; named differently
for clarity in the docs."""
return _connect_simple(host, "server", command)
def connect_chroot(
host: str,
*,
@@ -250,11 +260,6 @@ def reinstall_grub(host: str, *, ask_passphrase: bool = True) -> int:
return _run_chroot_script(host, "fix/grub.sh", "reinstall-grub", ask_passphrase)
def downgrade_initramfs(host: str, *, ask_passphrase: bool = True) -> int:
"""Downgrade mkinitcpio+dropbear+cryptsetup+mdadm+lvm2, rebuild initramfs. MUTATES."""
return _run_chroot_script(host, "fix/initramfs.sh", "downgrade-initramfs", ask_passphrase)
def use_static_ip(host: str, *, ask_passphrase: bool = True) -> int:
"""Replace ip=dhcp in /etc/default/grub with a static spec parsed from
the existing systemd-networkd .network file. Regenerates grub.cfg. MUTATES."""
@@ -268,6 +273,111 @@ def upgrade_system(host: str, *, ask_passphrase: bool = True) -> int:
return _run_chroot_script(host, "maintain/upgrade.sh", "upgrade-system", ask_passphrase)
def unlock(host: str, *, ask_passphrase: bool = True) -> int:
"""Pipe the LUKS passphrase to `cryptroot-unlock` on the dropbear that
is listening from initramfs. Use after a reboot, before the main sshd
is reachable. Uses a throwaway known_hosts to avoid host-key conflicts
between the dropbear and the real sshd (different host keys, same port).
"""
passphrase = _prompt_passphrase(host) if ask_passphrase else None
if passphrase is None:
print("Need a passphrase to send to cryptroot-unlock.", file=sys.stderr)
return 1
_wait_rescue(host) # really just "wait for port 22"
print(f"==> Sending passphrase to dropbear on {host} ...")
cmd = [
"ssh",
"-o", "UserKnownHostsFile=/dev/null",
"-o", "StrictHostKeyChecking=accept-new",
"-o", "GlobalKnownHostsFile=/dev/null",
"-o", "ConnectTimeout=10",
f"root@{host}",
"cryptroot-unlock",
]
r = subprocess.run(cmd, input=(passphrase + "\n").encode(), check=False)
if r.returncode == 0:
print("==> Passphrase accepted; system continues boot.")
else:
print(f"==> ssh/cryptroot-unlock exited with code {r.returncode}",
file=sys.stderr)
return r.returncode
def expand_fs(host: str) -> int:
"""Run `lvresize -l +100%FREE /dev/vg0/root && btrfs filesystem resize max /`
on the booted system. No LUKS passphrase needed — server is already up."""
_wait_rescue(host)
with SshSession(host) as ssh:
print("==> Expanding LVM root + btrfs filesystem ...")
ssh.run("lvresize -l +100%FREE /dev/vg0/root && btrfs filesystem resize max /")
return 0
def setup_dropbear(host: str) -> int:
"""Install dropbear + supporting packages, configure SSH keys, patch
/etc/mkinitcpio.conf HOOKS. Runs on the booted system. MUTATES."""
inside = (
importlib.resources
.files("hetzner_arch_luks")
.joinpath("resources/setup/dropbear.sh")
.read_bytes()
)
_wait_rescue(host)
with SshSession(host) as ssh:
print("==> Running setup-dropbear on the booted system ...")
ssh.run("bash -s", input_=inside)
return 0
def install_grub(host: str, *, ask_passphrase: bool = True) -> int:
"""Inside chroot: install grub package, write /etc/default/grub for
LUKS-encrypted root, grub-install on every boot disk, grub-mkconfig.
Used during the initial encryption setup. MUTATES."""
return _run_chroot_script(host, "setup/grub.sh", "install-grub", ask_passphrase)
def install_image(host: str, autosetup_path: str) -> int:
"""Upload an autosetup config to the rescue and run `installimage`.
DESTRUCTIVE — formats the disks per the autosetup contents."""
import pathlib
p = pathlib.Path(autosetup_path)
if not p.exists():
print(f"autosetup file not found: {autosetup_path}", file=sys.stderr)
return 1
content = p.read_bytes()
_wait_rescue(host)
with SshSession(host) as ssh:
print(f"==> Uploading {autosetup_path} → /autosetup on rescue ...")
ssh.run("cat > /autosetup", input_=content)
print("==> Running installimage (DESTRUCTIVE — this formats the disks!)")
ssh.run("installimage", tty=True)
return 0
def encrypt_root(host: str) -> int:
"""In rescue (NOT chroot): re-format /dev/md1 with LUKS, preserve the
installed root by copying through /oldroot, then mkinitcpio inside chroot.
Interactive: cryptsetup prompts for the new LUKS passphrase via the rescue
TTY. We upload the script to /root/_encrypt_root.sh and execute it with
a TTY allocated so cryptsetup's prompts work. DESTRUCTIVE on /dev/md1."""
content = (
importlib.resources
.files("hetzner_arch_luks")
.joinpath("resources/setup/encrypt_root.sh")
.read_bytes()
)
_wait_rescue(host)
with SshSession(host) as ssh:
print("==> Uploading encrypt-root script to rescue:/root/_encrypt_root.sh")
ssh.run("cat > /root/_encrypt_root.sh && chmod +x /root/_encrypt_root.sh",
input_=content)
print("==> Running encrypt-root (interactive — answer cryptsetup prompts)")
ssh.run("/root/_encrypt_root.sh", tty=True, check=False)
ssh.run("rm -f /root/_encrypt_root.sh", check=False)
return 0
def forget_passphrase(host: str) -> int:
"""Drop the stored LUKS passphrase for `host` from the libsecret keyring."""
if not _have_secret_tool():

View File

@@ -1,119 +0,0 @@
#!/bin/bash
# Runs INSIDE the chroot. Downgrades the 5 packages that determine how the
# initramfs is built AND what binaries end up inside it, to the version
# they had before the most recent `pacman -Syu`.
#
# The 5 packages:
# mkinitcpio — build tool. mkinitcpio 41 changed hook handling and may
# silently break setups using older third-party hooks
# (mkinitcpio-utils / -dropbear / -netconf).
# dropbear — SSH daemon in initramfs for remote LUKS unlock. The
# 2025.89 → 2026.90 jump may have changed key/config format.
# cryptsetup — LUKS open in initramfs.
# mdadm — RAID assemble in initramfs.
# lvm2 — LVM activate in initramfs.
#
# Source: /var/log/pacman.log tells us the exact previous versions.
# Files: prefer /var/cache/pacman/pkg/, fall back to archive.archlinux.org.
# After: rebuilds initramfs and regenerates grub.cfg.
set -e
banner() { printf "\n========== %s ==========\n" "$1"; }
PKGS=(mkinitcpio dropbear cryptsetup mdadm lvm2)
# Arch convention for package-file naming.
pkg_arch() {
case "$1" in
mkinitcpio|mkinitcpio-utils|mkinitcpio-dropbear|mkinitcpio-netconf) echo "any" ;;
*) echo "x86_64" ;;
esac
}
# Extract previous version from the most recent
# "[ALPM] upgraded <pkg> (OLD -> NEW)" line in pacman.log.
prev_version() {
local pkg="$1"
grep -E "\[ALPM\] upgraded $pkg \(" /var/log/pacman.log 2>/dev/null \
| tail -1 \
| sed -E "s/.*upgraded $pkg \(([^ ]+) -> [^)]+\).*/\1/"
}
banner "discovering previous versions from pacman.log"
declare -A FNAMES
TARGETS=()
for pkg in "${PKGS[@]}"; do
prev=$(prev_version "$pkg")
curr=$(pacman -Q "$pkg" 2>/dev/null | awk '{print $2}')
if [ -z "$prev" ]; then
echo " $pkg: no 'upgraded' entry in pacman.log — SKIP"
continue
fi
if [ "$prev" = "$curr" ]; then
echo " $pkg: already at previous version $curr — skip"
continue
fi
arch=$(pkg_arch "$pkg")
fname="${pkg}-${prev}-${arch}.pkg.tar.zst"
echo " $pkg: $curr$prev ($fname)"
FNAMES[$pkg]="$fname"
TARGETS+=("$pkg")
done
if [ "${#TARGETS[@]}" -eq 0 ]; then
echo "Nothing to downgrade."
exit 0
fi
banner "fetching packages"
FILES=()
for pkg in "${TARGETS[@]}"; do
fname="${FNAMES[$pkg]}"
cache="/var/cache/pacman/pkg/$fname"
if [ -e "$cache" ]; then
echo " $pkg: cached → $cache"
FILES+=("$cache")
continue
fi
first_letter="${pkg:0:1}"
url="https://archive.archlinux.org/packages/${first_letter}/${pkg}/${fname}"
out="/tmp/$fname"
echo " $pkg: fetching"
echo " URL: $url"
if curl -fsSL --connect-timeout 15 -o "$out" "$url"; then
size=$(du -h "$out" | cut -f1)
echo " OK ($size)"
FILES+=("$out")
else
echo " FAILED — cannot continue without all packages"
exit 1
fi
done
banner "downgrading (single transaction)"
pacman -U --noconfirm "${FILES[@]}"
banner "rebuilding initramfs (with downgraded mkinitcpio + tools)"
mkinitcpio -P
banner "regenerating GRUB config"
grub-mkconfig -o /boot/grub/grub.cfg 2>&1 | tail -10
banner "result"
for pkg in "${PKGS[@]}"; do
pacman -Q "$pkg" 2>/dev/null || true
done
banner "next steps"
cat <<EOF
1. Exit chroot, umount -R /mnt, reboot.
2. If the system boots and SSH works:
→ root cause is in one of {mkinitcpio, dropbear, cryptsetup, mdadm, lvm2}.
Pin them so the next pacman -Syu does not re-upgrade:
IgnorePkg = ${PKGS[*]}
in /etc/pacman.conf. Bisect later to find the exact culprit.
3. If still unbootable:
→ not the initramfs stack. Remaining suspects: glibc, systemd, iproute2.
Next attempt would be a full rollback of all May-11 package upgrades.
EOF

View File

@@ -0,0 +1,59 @@
#!/bin/bash
# Runs on the BOOTED Arch system (post-installimage, pre-encryption).
# Wires up dropbear + encryptssh + netconf for later remote-LUKS-unlock.
#
# Performs sections 3.13.5 of the README:
# - install busybox / mkinitcpio-{dropbear,utils,netconf}
# - copy authorized_keys to /etc/dropbear/root_key
# - regenerate OpenSSH host keys in PEM format
# - convert RSA host key to dropbear format
# - replace the HOOKS line in /etc/mkinitcpio.conf
#
# Idempotent: re-running is safe. A backup of /etc/mkinitcpio.conf is taken
# at first patch as /etc/mkinitcpio.conf.hal-backup.
set -e
banner() { printf "\n========== %s ==========\n" "$1"; }
banner "installing dropbear + mkinitcpio plugins"
pacman -S --noconfirm --needed \
busybox mkinitcpio-dropbear mkinitcpio-utils mkinitcpio-netconf
banner "copying authorized_keys to /etc/dropbear/root_key"
install -d -m 0755 /etc/dropbear
install -m 0600 /root/.ssh/authorized_keys /etc/dropbear/root_key
chmod 700 /root/.ssh
chmod 600 /root/.ssh/authorized_keys
banner "enabling sshd"
systemctl enable sshd
banner "regenerating OpenSSH host keys (PEM format)"
rm -f /etc/ssh/ssh_host_*
ssh-keygen -A -m PEM
banner "importing RSA host key into dropbear"
dropbearconvert openssh dropbear \
/etc/ssh/ssh_host_rsa_key /etc/dropbear/dropbear_rsa_host_key
banner "patching HOOKS in /etc/mkinitcpio.conf"
[ -f /etc/mkinitcpio.conf.hal-backup ] \
|| cp -a /etc/mkinitcpio.conf /etc/mkinitcpio.conf.hal-backup
# Replace any existing HOOKS=(...) line with the encryptssh-enabled set.
sed -i -E \
's|^HOOKS=.*|HOOKS=(base udev autodetect modconf block mdadm_udev lvm2 netconf dropbear encryptssh filesystems keyboard fsck)|' \
/etc/mkinitcpio.conf
echo "HOOKS line is now:"
grep '^HOOKS=' /etc/mkinitcpio.conf
banner "done"
cat <<EOF
Next steps:
1. Activate Hetzner Rescue in the Robot, then reboot the server.
2. From your client: hal connect rescue <host>
3. Inside rescue: hal encrypt-root <host>
4. After that: hal install-grub <host>
EOF

View File

@@ -0,0 +1,106 @@
#!/bin/bash
# Runs IN HETZNER RESCUE (NOT in chroot). Re-creates the root LV stack on
# top of LUKS, preserving the installed Arch by copying it through /oldroot.
#
# Performs sections 4.44.15 of the README in one go:
# 4.4 mount the unencrypted /dev/mapper/vg0-root
# 4.5 cp -va /mnt → /oldroot at full RAID resync speed
# 4.6 umount /mnt
# 4.7 vgremove vg0
# 4.8 cat /proc/mdstat (display)
# 4.9 luksFormat /dev/md1 (prompts for NEW passphrase!)
# luksOpen + recreate LVM (vg0 with swap + root)
# mkfs.btrfs / mkswap
# 4.10 mount the encrypted root at /mnt
# 4.12 cp -va /oldroot back into /mnt at full RAID resync speed
# 4.13 bind /dev /sys /proc, mount /boot
# 4.14 echo cryptroot line into /mnt/etc/crypttab
# 4.15 chroot + mkinitcpio -P
#
# DESTRUCTIVE: /dev/md1 will be re-formatted with LUKS. Any data not under
# /mnt (vg0-root) is lost. Confirmation prompted before the format step.
set -e
banner() { printf "\n========== %s ==========\n" "$1"; }
banner "4.4 mount existing unencrypted root"
vgscan -v
vgchange -a y
mount /dev/mapper/vg0-root /mnt
banner "4.5 copy current system to /oldroot (full RAID resync speed)"
mkdir -p /oldroot
echo 0 > /proc/sys/dev/raid/speed_limit_max
cp -va /mnt/. /oldroot/.
echo 200000 > /proc/sys/dev/raid/speed_limit_max
banner "4.6 unmount original root"
umount /mnt
banner "4.7 remove unencrypted VG (frees /dev/md1)"
vgremove -f vg0
banner "4.8 RAID state"
cat /proc/mdstat
banner "CONFIRMATION REQUIRED"
echo "About to luksFormat /dev/md1. This is DESTRUCTIVE for /dev/md1."
echo "Type 'YES' to continue (anything else aborts):"
read -r confirm
if [ "$confirm" != "YES" ]; then
echo "Aborted by user before luksFormat. /oldroot still has your data;"
echo "you can re-create the original LVM by hand from there if needed."
exit 1
fi
banner "4.9 LUKS format /dev/md1 (you will be prompted for the NEW passphrase)"
cryptsetup --cipher aes-xts-plain64 --key-size 256 --hash sha256 \
--iter-time 10000 luksFormat /dev/md1
banner "4.9b open the LUKS volume (re-enter the same passphrase)"
cryptsetup luksOpen /dev/md1 cryptroot
banner "4.9c recreate LVM on top of /dev/mapper/cryptroot"
pvcreate /dev/mapper/cryptroot
vgcreate vg0 /dev/mapper/cryptroot
lvcreate -n swap -L 8G vg0
lvcreate -n root -l 100%FREE vg0
mkfs.btrfs /dev/vg0/root
mkswap /dev/vg0/swap
banner "4.10 mount the encrypted root"
mount /dev/vg0/root /mnt
banner "4.12 copy system back into the encrypted root"
echo 0 > /proc/sys/dev/raid/speed_limit_max
cp -va /oldroot/. /mnt/.
echo 200000 > /proc/sys/dev/raid/speed_limit_max
banner "4.13 bind-mount /dev /sys /proc, mount /boot"
mount /dev/md0 /mnt/boot
mount --bind /dev /mnt/dev
mount --bind /sys /mnt/sys
mount --bind /proc /mnt/proc
banner "4.14 append cryptroot line to /etc/crypttab"
if ! grep -qE '^cryptroot[[:space:]]' /mnt/etc/crypttab 2>/dev/null; then
echo "cryptroot /dev/md1 none luks" >> /mnt/etc/crypttab
fi
grep cryptroot /mnt/etc/crypttab
banner "4.15 regenerate initramfs inside chroot"
chroot /mnt /bin/bash -c "mkinitcpio -P"
banner "done"
cat <<EOF
Encryption setup complete. /oldroot can be deleted manually after you've
confirmed the encrypted boot works.
Recommended next steps:
hal install-grub <host> # configures GRUB for LUKS-encrypted root
hal connect rescue <host> reboot
# Disable rescue in Hetzner Robot
hal status <host> # poll for dropbear / sshd
hal unlock <host> # send LUKS passphrase to dropbear
EOF

View File

@@ -0,0 +1,78 @@
#!/bin/bash
# Runs INSIDE the chroot. Initial GRUB install for the LUKS-encrypted root.
# Performs sections 5.15.3 of the README:
# - install the grub package
# - write /etc/default/grub with the LUKS cmdline + GRUB_ENABLE_CRYPTODISK=y
# - grub-mkconfig
# - grub-install on every disk backing /boot's RAID
set -e
banner() { printf "\n========== %s ==========\n" "$1"; }
# Convert a partition path to its parent disk. (Same helper as fix/grub.sh.)
parent_disk() {
local part="$1"
case "$part" in
/dev/nvme[0-9]*n[0-9]*p[0-9]*) echo "${part%p[0-9]*}" ;;
/dev/mmcblk[0-9]*p[0-9]*) echo "${part%p[0-9]*}" ;;
/dev/sd[a-z]*[0-9]*) echo "$part" | sed -E 's/[0-9]+$//' ;;
/dev/vd[a-z]*[0-9]*) echo "$part" | sed -E 's/[0-9]+$//' ;;
*)
local d
d=$(lsblk -no PKNAME "$part" 2>/dev/null | head -1)
[ -n "$d" ] && echo "/dev/$d"
;;
esac
}
banner "installing grub package"
pacman -S --noconfirm --needed grub
banner "writing /etc/default/grub for LUKS boot"
[ -f /etc/default/grub.hal-backup ] || cp -a /etc/default/grub /etc/default/grub.hal-backup
cat > /etc/default/grub <<'GRUBEOF'
# hetzner-arch-luks default grub config
GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="consoleblank=0"
GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=dhcp"
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
GRUB_ENABLE_CRYPTODISK=y
GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep
GRUB_DISABLE_RECOVERY=true
GRUBEOF
echo "Wrote /etc/default/grub. Showing relevant lines:"
grep -E '^GRUB_(CMDLINE_LINUX|ENABLE_CRYPTODISK|PRELOAD_MODULES)=' /etc/default/grub
banner "identifying boot disks (members of md0)"
BOOT_DISKS=()
for part in $(mdadm --detail /dev/md0 2>/dev/null | awk '/active sync/ {print $NF}'); do
disk=$(parent_disk "$part")
[ -z "$disk" ] && continue
already=0
for d in "${BOOT_DISKS[@]}"; do [ "$d" = "$disk" ] && already=1; done
[ "$already" -eq 0 ] && BOOT_DISKS+=("$disk")
done
echo "Boot disks: ${BOOT_DISKS[*]}"
banner "grub-mkconfig"
grub-mkconfig -o /boot/grub/grub.cfg 2>&1 | tail -10
banner "grub-install on each boot disk"
for disk in "${BOOT_DISKS[@]}"; do
echo "-- grub-install --target=i386-pc --recheck $disk"
grub-install --target=i386-pc --recheck "$disk"
done
banner "done"
cat <<EOF
GRUB installed for LUKS-encrypted boot.
Recommended next step: hal use-static-ip <host> (replaces ip=dhcp with a
static kernel-cmdline IP, making the initramfs network independent of DHCP).
EOF