Compare commits

..
6 Commits
Author SHA1 Message Date
Kevin Veen-Birkenbach d672388f63 Release version 7.2.0
CI (make tests, stable, publish) / make test (push) Canceled after 0s
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Canceled after 0s
2026-11-03 21:40:44 +01:00
Kevin Veen-BirkenbachandClaude Opus 5.5 5817078643 feat(seed): read the database password from stdin with --password-stdin
baudolo-seed took the database password only as a positional argument, so it showed in the process list of the host that seeds the backup catalog (infinito-nexus/core #1213). --password-stdin reads it from stdin without the trailing newline. The positional argument still works for existing callers but warns on stderr that it is visible in the process list; giving the password both ways, or neither, is rejected with a usage error and writes nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:40:43 +01:00
dependabot[bot] 4214d3709a build(deps-dev): bump ruff from 0.17.1 to 0.17.3 (#13)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.17.1 to 0.17.3.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.17.1...0.17.3)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.17.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-31 03:03:46 +00:00
dependabot[bot] 4c51da009c build(deps-dev): bump ruff from 0.16.7 to 0.17.1 (#12)
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.7 to 0.17.1.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.7...0.17.1)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.17.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-10-24 00:45:16 +00:00
Kevin Veen-Birkenbach b43177f0e7 Release version 7.1.2
CI (make tests, stable, publish) / make test (push) Canceled after 0s
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Canceled after 0s
2026-10-18 00:43:44 +02:00
Kevin Veen-BirkenbachandClaude Opus 5.5 0735e5fa7b fix(backup): mount the zfs snapshot itself so a container can read it
The zfs snapshot was read through <subject>/.zfs/snapshot/<name>. The
kernel automounts that directory through a usermode helper that runs in
the host's mount namespace, so a containerised baudolo never sees the
mount and finds no snapshot. The snapshot is now mounted explicitly at
<subject>/.baudolo-<tag>, beside where btrfs carves its own, and removed
in order: umount, rmdir, zfs destroy. A failed mount destroys the
snapshot at once instead of leaking one per run.

The e2e suite could not reach this code at all: zpool create on the /img
file failed with "no such pool or dataset", because the kernel opens
vdevs from a taskq thread that resolves paths in the host's root, where
the container's /img does not exist. The pool now sits on a loop device,
whose /dev/loopN node the kernel creates on the host as well, and it is
destroyed on exit once this script created it, since a pool is
kernel-wide.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-18 00:22:53 +02:00
8 changed files with 225 additions and 19 deletions

No files matched your search

+27
View File
@@ -1,5 +1,32 @@
# Changelog
## [7.2.0] - 2026-11-03
Added:
- Seed: *--password-stdin* reads the database password from stdin
- Seed: a password given both ways or not at all is rejected
Changed:
- Seed: the password argument still works but warns it is visible in *ps*
Upgrade notes:
- Seed: pass the password via *--password-stdin*; the argument is deprecated
## [7.1.2] - 2026-10-18
Fixed:
- Backup: *--snapshot zfs* found no snapshot inside a container; now mounts it
- Tests: e2e zfs pool now sits on a loop device; a file vdev never worked
Upgrade notes:
- Backup: zfs snapshots mount at *.baudolo-TAG* in the subject, not *.zfs*
- Image: first ghcr image since 3.1.2; the 4.0.0 to 7.1.1 notes apply
## [7.1.1] - 2026-10-17
Fixed:
+2 -2
View File
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "backup-docker-to-local"
version = "7.1.1"
version = "7.2.0"
description = "Backup Docker volumes to local with rsync and optional DB dumps."
readme = "README.md"
requires-python = ">=3.9"
@@ -19,7 +19,7 @@ dependencies = [
[project.optional-dependencies]
# Pinned: a ruff minor bump changes which rules fire, and `make test` gates on
# a clean run, so an unpinned lint would fail the suite on an unrelated day.
lint = ["ruff==0.16.7"]
lint = ["ruff==0.17.3"]
[project.scripts]
baudolo = "baudolo.backup.__main__:main"
+30 -11
View File
@@ -58,27 +58,50 @@ def _resolver(subject: str, root: str) -> Callable[[str], str]:
return resolve
def _remove(
root: str, commands: list[list[str]], run: Callable[[list[str]], list[str]]
) -> None:
try:
for command in commands:
run(command)
except BackupError as error:
# Raising here would also mask whatever the body raised.
print(f"WARNING: {root} could not be removed: {error}", flush=True)
def _btrfs(
subject: str, name: str, run: Callable[[list[str]], list[str]]
) -> tuple[str, list[str]]:
) -> tuple[str, list[list[str]]]:
# The snapshot goes inside the subject, never beside it: the kernel rejects
# a snapshot whose destination is on another filesystem, which is exactly
# what the parent directory is when the subject is a mountpoint of its own.
target = str(Path(os.path.abspath(subject)) / f".{name}") # noqa: PTH100 - see _resolver
run(["btrfs", "subvolume", "snapshot", "-r", subject, target])
return target, ["btrfs", "subvolume", "delete", target]
return target, [["btrfs", "subvolume", "delete", target]]
def _zfs(
subject: str, name: str, run: Callable[[list[str]], list[str]]
) -> tuple[str, list[str]]:
) -> tuple[str, list[list[str]]]:
output = run(["zfs", "list", "-H", "-o", "name", subject])
dataset = (output[0] if output else "").strip()
if not dataset:
raise SnapshotError(f"no zfs dataset is mounted at {subject}")
run(["zfs", "snapshot", f"{dataset}@{name}"])
root = str(Path(subject) / ".zfs" / "snapshot" / name)
return root, ["zfs", "destroy", f"{dataset}@{name}"]
snapshot = f"{dataset}@{name}"
# Exception: mounted here rather than read through .zfs/snapshot, which the
# kernel automounts via a usermode helper in the host's mount namespace,
# so a containerised run never sees it.
root = str(Path(os.path.abspath(subject)) / f".{name}") # noqa: PTH100 - see _resolver
run(["zfs", "snapshot", snapshot])
remove = [["zfs", "destroy", snapshot]]
try:
run(["mkdir", root])
remove.insert(0, ["rmdir", root])
run(["mount", "-t", "zfs", snapshot, root])
except BaseException:
_remove(root, remove, run)
raise
return root, [["umount", root], *remove]
_CREATE = {"btrfs": _btrfs, "zfs": _zfs}
@@ -169,8 +192,4 @@ def volume_snapshot(
try:
yield _resolver(subject, root)
finally:
try:
run(remove)
except BackupError as error:
# Raising here would also mask whatever the body raised.
print(f"WARNING: {root} could not be removed: {error}", flush=True)
_remove(root, remove, run)
+23 -2
View File
@@ -73,9 +73,30 @@ def main() -> None:
help="Database name or '*' to dump all databases",
)
parser.add_argument("username", help="Database username")
parser.add_argument("password", help="Database password")
parser.add_argument(
"password",
nargs="?",
help="Deprecated: database password as an argument, visible in the "
"process list; use --password-stdin",
)
parser.add_argument(
"--password-stdin",
action="store_true",
help="Read the database password from stdin",
)
args = parser.parse_args()
if args.password_stdin == (args.password is not None):
parser.error("give the password either via --password-stdin or as an argument")
if args.password_stdin:
password = sys.stdin.read().removesuffix("\n")
else:
print(
"WARNING: a password argument is visible in the process list; "
"use --password-stdin",
file=sys.stderr,
)
password = args.password
try:
check_and_add_entry(
@@ -83,7 +104,7 @@ def main() -> None:
instance=args.instance,
database=args.database,
username=args.username,
password=args.password,
password=password,
)
except Exception as exc: # noqa: BLE001 - CLI boundary: any failure becomes exit 1
print(f"ERROR: {exc}", file=sys.stderr)
+7 -2
View File
@@ -36,9 +36,14 @@ ZFS_READY = "{ [ -c /dev/zfs ] || modprobe zfs 2>/dev/null; }; [ -c /dev/zfs ]"
def mount_script(fstype: str) -> str:
"""Build a filesystem on a loop device and carve out the snapshot subject."""
if fstype == "zfs":
# Exception: a loop device, not the /img file, because the kernel opens
# vdevs from a taskq thread that resolves paths in the host's root.
# The pool is kernel-wide, so it is destroyed on exit, but only once
# this script created it.
return (
f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img "
"&& zpool create -m none baudolo /img "
f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img && {ATTACH} "
'&& zpool create -m none baudolo "$LOOP" '
"&& trap 'zpool destroy -f baudolo; losetup -d \"$LOOP\"' EXIT "
"&& zfs create -o mountpoint=/subject/docker baudolo/docker"
)
return (
@@ -65,6 +65,32 @@ def run_seed_expect_fail(
)
def run_seed_stdin(
csv_path: Path, instance: str, database: str, username: str, stdin: str, *extra: str
) -> subprocess.CompletedProcess:
"""
Run the real CLI with --password-stdin, feeding the password on stdin.
Any extra arguments are appended after the username.
"""
return subprocess.run(
[
sys.executable,
"-m",
"baudolo.seed",
"--password-stdin",
str(csv_path),
instance,
database,
username,
*extra,
],
input=stdin,
text=True,
capture_output=True,
check=False,
)
def read_csv_semicolon(path: Path) -> list[dict]:
with path.open("r", encoding="utf-8", newline="") as f:
reader = csv.DictReader(f, delimiter=";")
@@ -195,6 +221,61 @@ class TestSeedIntegration(unittest.TestCase):
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["database"], "my_db-1")
def test_password_stdin_stores_the_password_without_its_trailing_newline(
self,
) -> None:
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "databases.csv"
cp = run_seed_stdin(p, "docker.test", "appdb", "alice", "s3cr;et\n")
self.assertEqual(cp.returncode, 0, cp.stderr)
self.assertNotIn("WARNING", cp.stderr)
rows = read_csv_semicolon(p)
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["username"], "alice")
self.assertEqual(rows[0]["password"], "s3cr;et")
def test_password_argument_still_works_with_a_deprecation_warning(self) -> None:
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "databases.csv"
cp = run_seed(p, "docker.test", "appdb", "alice", "secret")
self.assertIn("--password-stdin", cp.stderr)
self.assertEqual(read_csv_semicolon(p)[0]["password"], "secret")
def test_password_given_both_ways_is_rejected(self) -> None:
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "databases.csv"
cp = run_seed_stdin(p, "docker.test", "appdb", "alice", "pw\n", "pw")
self.assertNotEqual(cp.returncode, 0)
self.assertFalse(p.exists(), "Should not create file on invalid input")
def test_password_given_no_way_is_rejected(self) -> None:
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "databases.csv"
cp = subprocess.run(
[
sys.executable,
"-m",
"baudolo.seed",
str(p),
"docker.test",
"appdb",
"alice",
],
text=True,
capture_output=True,
check=False,
)
self.assertNotEqual(cp.returncode, 0)
self.assertFalse(p.exists(), "Should not create file on invalid input")
def test_file_is_semicolon_delimited_and_has_header(self) -> None:
with tempfile.TemporaryDirectory() as td:
p = Path(td) / "databases.csv"
+53 -2
View File
@@ -21,6 +21,14 @@ class Runner:
return []
class Unmountable(Runner):
def __call__(self, command: list[str]) -> list[str]:
result = super().__call__(command)
if command[0] == "mount":
raise BackupError("permission denied")
return result
class TestBtrfs(unittest.TestCase):
def test_it_creates_a_read_only_snapshot_inside_the_subject(self) -> None:
run = Runner()
@@ -95,14 +103,57 @@ class TestZfs(unittest.TestCase):
run.calls[-1], ["zfs", "destroy", "tank/docker@baudolo-20260731"]
)
def test_it_maps_a_volume_path_through_the_dot_zfs_directory(self) -> None:
def test_it_mounts_the_snapshot_inside_the_subject(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
pass
self.assertIn(
[
"mount",
"-t",
"zfs",
"tank/docker@baudolo-20260731",
"/var/lib/docker/.baudolo-20260731",
],
run.calls,
)
def test_it_maps_a_volume_path_into_the_mounted_snapshot(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run) as resolve:
self.assertEqual(
resolve("/var/lib/docker/volumes/postgres_data/_data"),
"/var/lib/docker/.zfs/snapshot/baudolo-20260731/volumes/postgres_data/_data",
"/var/lib/docker/.baudolo-20260731/volumes/postgres_data/_data",
)
def test_it_unmounts_before_it_destroys(self) -> None:
run = self._run()
with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
pass
self.assertEqual(
run.calls[-3:],
[
["umount", "/var/lib/docker/.baudolo-20260731"],
["rmdir", "/var/lib/docker/.baudolo-20260731"],
["zfs", "destroy", "tank/docker@baudolo-20260731"],
],
)
def test_a_failed_mount_leaves_no_snapshot_behind(self) -> None:
run = Unmountable({"zfs list": ["tank/docker"]})
with (
self.assertRaises(BackupError),
volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run),
):
pass
self.assertEqual(
run.calls[-2:],
[
["rmdir", "/var/lib/docker/.baudolo-20260731"],
["zfs", "destroy", "tank/docker@baudolo-20260731"],
],
)
def test_an_unmounted_dataset_is_an_error(self) -> None:
run = Runner({"zfs list": [""]})
with (
+2
View File
@@ -164,6 +164,7 @@ class TestSeedMain(unittest.TestCase):
ns.database = "db"
ns.username = "user"
ns.password = "pass"
ns.password_stdin = False
parse_args.return_value = ns
seed_main.main()
@@ -195,6 +196,7 @@ class TestSeedMain(unittest.TestCase):
ns.database = "db"
ns.username = "user"
ns.password = "pass"
ns.password_stdin = False
parse_args.return_value = ns
seed_main.main()