Release version 8.0.0

Dumps stream through gzip into <db>.backup.sql.gz; no uncompressed copy touches the disk. Restore reads gzip and plain dumps by their magic bytes, verifies a gzip dump to its end before anything is dropped, and still finds the uncompressed names of older generations. The manifest names the compression.

🎯 Fix: 92 %
💥 Break: 20 %
💸 Cost: –
⏱️ Time: –
🛡️ Security: ~100 %
🏆 Score: 74
🚦 Gate: ⛔ break 20 % > 5

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -1,5 +1,188 @@
 # Changelog
 
+## [8.0.0] - 2026-11-05
+
+Breaking:
+
+- Backup: dumps are gzip-compressed and named *name.backup.sql.gz*
+
+Added:
+
+- Restore: *--db-password-stdin* reads the database password from stdin
+
+Changed:
+
+- Restore: *--db-password* still works but warns it is visible in *ps*
+- Restore: reads gzip and plain dumps, so older generations still restore
+
+Fixed:
+
+- Security: MariaDB passwords no longer appear in any process argv
+- Restore: a truncated gzip dump stops the restore before anything is dropped
+
+Upgrade notes:
+
+- Scripts that find dumps by name must also accept the *.gz* suffix
+- Scripts that read dumps as plain SQL must decompress them first
+
+## [7.2.0] - 2026-11-03
+
+Added:
+
+- Seed: *--password-stdin* reads the database password from stdin
+- Seed: a password given both ways or not at all is rejected
+
+Changed:
+
+- Seed: the password argument still works but warns it is visible in *ps*
+
+Upgrade notes:
+
+- Seed: pass the password via *--password-stdin*; the argument is deprecated
+
+## [7.1.2] - 2026-10-18
+
+Fixed:
+
+- Backup: *--snapshot zfs* found no snapshot inside a container; now mounts it
+- Tests: e2e zfs pool now sits on a loop device; a file vdev never worked
+
+Upgrade notes:
+
+- Backup: zfs snapshots mount at *.baudolo-TAG* in the subject, not *.zfs*
+- Image: first ghcr image since 3.1.2; the 4.0.0 to 7.1.1 notes apply
+
+## [7.1.1] - 2026-10-17
+
+Fixed:
+
+- Tests: e2e MariaDB probe ran root passwordless and timed out at random
+- Tests: probe now logs in with the root password, skips the temp server
+- Tests: anonymous-preemption bootstrap now uses the root password too
+
+Upgrade notes:
+
+- 7.1.0 never reached stable: its CI failed, no image was published
+- 7.1.1 is the first published -aHAX build; 7.1.0 upgrade notes apply
+
+## [7.1.0] - 2026-10-17
+
+Fixed:
+
+- Backup: rsync -a stored no xattrs, ACLs or hardlinks; now -aHAX
+- Restore: restore_volume_files had the same gap; now -aHAX too
+- An OpenCloud volume lost its storage metadata through a DR drill
+
+Upgrade notes:
+
+- First backup re-copies every attribute-carrying volume once in full
+- A destination without xattr support now fails the backup run outright
+- exFAT, xattr-less NFS and object-storage FUSE mounts must move first
+
+## [7.0.2] - 2026-09-23
+
+* Backup: failed live pre-copy no longer aborts when a stopped copy follows
+* Backup: stopped --checksum copy replaces pre-copy hit by live writer (rc 23)
+* Backup: failing sole live copy or failing stopped copy still fails the run
+* Tests: e2e live writer shrinking files mid-read reproduces the rsync failure
+
+## [7.0.1] - 2026-08-18
+
+- Restore: *--empty* no longer aborts on a database that carries an extension.
+  The pre-clean picked its candidates by owner, on the stated assumption that
+  extension members are superuser-owned and would therefore never be selected.
+  That holds only when a superuser installed the extension: a role that
+  installs one itself owns its functions, so they were listed for a one-by-one
+  *DROP* that postgres refuses — *cannot drop function
+  vector_in(cstring,oid,integer) because extension vector requires it*. Under
+  *ON_ERROR_STOP* that ends the whole restore, which is how a generation of an
+  application declaring the *vector* extension became unreplayable. Membership
+  now comes from *pg_depend* rather than from ownership; each branch carries
+  its oid and classid so one *NOT EXISTS* covers all seven instead of seven
+  separate predicates, and the schema branch is guarded too because an
+  extension can own a schema. Skipping the members suffices — the dump's
+  *CREATE EXTENSION IF NOT EXISTS* finds the surviving extension either way.
+
+## [7.0.0] - 2026-08-18
+
+Breaking:
+- Backup: *mariadb* and *mysql* join the suffix tokens, so a container named
+  *<app>-mariadb* or *<app>-mysql* now resolves to the instance *<app>* instead
+  of to its own name. A *databases.csv* keyed on the full container name has to
+  move to the application name, or name the container in
+  *--database-containers*. This narrows what 6.0.0 broke rather than widening
+  it: a container named exactly *postgres*, *mariadb*, *mysql*, *db* or
+  *database* resolves again without any declaration, which is the shape a
+  compose file writes as *container_name: postgres* and the most common
+  configuration there is.
+
+Fixed:
+- Backup: a container named exactly after its engine is dumped again. The
+  suffix match needs a hyphen or underscore in front of the token, which a bare
+  name does not carry, so 6.0.0 resolved *container_name: postgres* to nothing
+  and stopped dumping it without saying so. *ENGINE_NAMES* now states the set
+  once and serves both readings — carried as a suffix it makes the rest the
+  instance, being one outright makes the container its own instance.
+- Backup: a central MariaDB under swarm is dumped for the first time. Swarm
+  names its task *mariadb_mariadb.1.<id>*, which matches neither the static
+  *mariadb* passed through *--database-containers* nor any token the suffix
+  match knew, since *_mariadb* is not *_db*. The database was silently absent
+  from every swarm backup this tool has ever written, before 6.0.0 as well.
+- Backup: an application container is no longer recorded as a database.
+  *container_engine* recognises an engine by its client tools, which an
+  application image frequently ships, so refusing its dump alone would have
+  written the volume to the manifest as *database: true, dumped: false* — the
+  exact shape a restore drill reads as a database that was missed. Without an
+  instance there is no database to record, and the volume is a file backup like
+  any other.
+
+## [6.0.0] - 2026-08-18
+
+Breaking:
+- Backup: a database container whose name carries no *database*, *db* or
+  *postgres* token — preceded by a hyphen or underscore — must now be named in
+  *--database-containers*. Without that declaration its *databases.csv* rows no
+  longer match and no dump is written, silently, because nothing fails. The
+  shape this hits hardest is a container named exactly *postgres* or *mariadb*:
+  the token needs a separator in front of it, which a bare name does not have.
+  *app-database*, *app_database.1.<task>* from swarm and *app-postgres-1* are
+  unaffected, as is any container already declared.
+
+Fixed:
+- Backup: *docker exec* now forwards *PGPASSWORD* into the container.
+  *execute_to_file* set the variable on baudolo's own process, but nothing
+  carried it across the container boundary, so an engine whose *pg_hba* demands
+  a password on TCP loopback refused every dump — which is every dedicated
+  Postgres instance on a real host. The name travels as a bare *-e NAME* so
+  docker copies the value out of this process's environment; spelling
+  *-e NAME=value* instead would publish the secret in the host's process list.
+- Backup: *get_instance* no longer claims an instance it never derived. It
+  returned the container name unchanged when that name carried no database
+  token, so an application container answered the same *databases.csv* row as
+  its own dedicated engine. Application images frequently ship the engine's
+  client tools, so the dump command started and wrote a file that looked like a
+  backup and held none of the data: measured against Discourse, 1,680 bytes
+  from the application where the engine produced 10,469,439. The regex stays a
+  normaliser — *<app>-database* from compose and *<app>_database.1.<task>* from
+  swarm still resolve to one instance. Only the fallthrough changed.
+
+New:
+- Tests: *get_instance* has unit coverage for the first time. Eleven cases pin
+  the container names that compose, swarm and explicitly-named engines produce,
+  so a future change to the regex has to state which shape it gives up.
+- Tests: two e2e modules cover shapes the suite structurally could not see.
+  Every fixture passed its container in *--database-containers*, which left the
+  regex branch — the only one a dedicated database ever takes — dead code under
+  test, and no scenario made a password mandatory, because stock
+  *postgres:alpine* grants trust on loopback.
+  *test_e2e_postgres_password_required* starts an engine with
+  *--auth-host=scram-sha-256* and carries a negative control asserting the
+  server refuses an unauthenticated dump; without it the module would pass
+  whether or not the password is forwarded at all.
+  *test_e2e_app_container_ships_client_tools* places an application container
+  beside its engine with neither declared, and requires the engine dumped, the
+  application volume copied as files, and no dump written from the application.
+
 ## [5.0.0] - 2026-08-18
 
 **[5.0.0] - 2026-08-18**

README.md

@@ -47,9 +47,13 @@ Backups are stored in a deterministic, fully nested structure:
             └── <volume-name>/
                 ├── files/
                 └── sql/
-                    └── <database>.backup.sql
+                    └── <database>.backup.sql.gz
 ```
 
+Dumps are gzip-compressed while they are written; no uncompressed copy
+touches the disk. Generations written before compression hold
+`<database>.backup.sql`; `baudolo-restore` reads both.
+
 ### Meaning of each level
 
 * `<machine-hash>`
@@ -174,10 +178,14 @@ baudolo-restore postgres \
   <version> \
   --container postgres \
   --db-name appdb \
-  --db-password secret \
-  --empty
+  --db-password-stdin \
+  --empty < password-file
 ```
 
+`--db-password-stdin` reads the password from stdin. The older
+`--db-password <value>` still works but warns: it is visible in the process
+list.
+
 ### Restore MariaDB / MySQL
 
 ```bash
@@ -187,8 +195,8 @@ baudolo-restore mariadb \
   <version> \
   --container mariadb \
   --db-name shopdb \
-  --db-password secret \
-  --empty
+  --db-password-stdin \
+  --empty < password-file
 ```
 
 > `baudolo` automatically detects whether `mariadb` or `mysql`

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "backup-docker-to-local"
-version = "5.0.0"
+version = "8.0.0"
 description = "Backup Docker volumes to local with rsync and optional DB dumps."
 readme = "README.md"
 requires-python = ">=3.9"
@@ -19,7 +19,7 @@ dependencies = [
 [project.optional-dependencies]
 # Pinned: a ruff minor bump changes which rules fire, and `make test` gates on
 # a clean run, so an unpinned lint would fail the suite on an unrelated day.
-lint = ["ruff==0.16.1"]
+lint = ["ruff==0.17.3"]
 
 [project.scripts]
 baudolo = "baudolo.backup.__main__:main"

src/baudolo/backup/app.py

@@ -23,6 +23,7 @@ from .layout import (
     write_manifest,
 )
 from .policy import requires_stop, volume_is_fully_ignored
+from .shell import BackupError
 from .snapshot import snapshot_source, volume_snapshot
 from .volume import backup_volume, inspect_backing
 
@@ -125,8 +126,18 @@ def main() -> int:
                     copy(authoritative=False)
                 continue
 
+            if not requires_stop(containers, args.images_no_stop_required):
                 copy(authoritative=False)
-            if requires_stop(containers, args.images_no_stop_required):
+                continue
+
+            try:
+                copy(authoritative=False)
+            except BackupError as error:
+                print(
+                    f"WARNING: live pre-copy of volume '{volume_name}' failed; "
+                    f"the copy with its containers stopped replaces it.\n{error}",
+                    flush=True,
+                )
             stoppable = filter_stoppable(containers)
             change_containers_status(stoppable, "stop")
             copy(authoritative=True)

src/baudolo/backup/db.py

@@ -9,21 +9,42 @@ from baudolo.databases import CLUSTER_ROW, validate_database
 from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX, SQL_DIR
 
 from .docker import docker_exec_argv
-from .shell import BackupError, execute_to_file
+from .shell import BackupError, execute_to_gzip
 
 if TYPE_CHECKING:
     import pandas as pd
 
 log = logging.getLogger(__name__)
 
+ENGINE_NAMES = ("database", "postgres", "mariadb", "mysql", "db")
+_SUFFIX_RE = re.compile(rf"(_|-)({'|'.join(ENGINE_NAMES)})")
 
-def get_instance(container: str, database_containers: list[str]) -> str:
-    """
-    Derive a stable instance name from the container name.
+
+def get_instance(container: str, database_containers: list[str]) -> str | None:
+    """The databases.csv instance a container serves, or None for no database.
+
+    A declared container is its own instance. Every other name is read against
+    ENGINE_NAMES: carrying one as a suffix makes the rest the instance, which
+    maps `<app>-database` from compose and `<app>_database.1.<task>` from swarm
+    onto the same one; being one outright makes the container its own instance,
+    the shape a compose file writes as `container_name: postgres`.
+
+    Args:
+        container: the running container's name.
+        database_containers: names passed via --database-containers, taken as
+            declared engines whatever they are called.
+
+    Returns:
+        The instance name, or None when the name neither carries nor is an
+        engine name: an application container is not an engine, even when it
+        ships the client tools that would let a dump command start.
     """
     if container in database_containers:
         return container
-    return re.split(r"(_|-)(database|db|postgres)", container)[0]
+    parts = _SUFFIX_RE.split(container)
+    if len(parts) > 1:
+        return parts[0]
+    return container if container in ENGINE_NAMES else None
 
 
 def fallback_pg_dumpall(
@@ -32,11 +53,12 @@ def fallback_pg_dumpall(
     """
     Perform a full Postgres cluster dump using pg_dumpall.
     """
-    execute_to_file(
+    execute_to_gzip(
         docker_exec_argv(
             container,
             ["pg_dumpall", "-U", username, "-h", "localhost"],
             interactive=True,
+            forward_env=["PGPASSWORD"],
         ),
         out_file,
         env={"PGPASSWORD": password},
@@ -62,6 +84,9 @@ def backup_database(
     Returns True if at least one dump was produced.
     """
     instance_name = get_instance(container, database_containers)
+    if instance_name is None:
+        log.debug("Container '%s' carries no database token", container)
+        return False
 
     entries = databases_df[databases_df["instance"] == instance_name]
     if entries.empty:
@@ -97,7 +122,7 @@ def backup_database(
 
         if db_type == "mariadb":
             # Force TCP so auth matches '<user>'@'%' instead of socket -> 'localhost'.
-            execute_to_file(
+            execute_to_gzip(
                 docker_exec_argv(
                     container,
                     [
@@ -107,18 +132,19 @@ def backup_database(
                         "--protocol=tcp",
                         "-u",
                         user,
-                        f"-p{password}",
                         db_name,
                     ],
+                    forward_env=["MYSQL_PWD"],
                 ),
                 dump_file,
+                env={"MYSQL_PWD": password},
             )
             produced = True
             continue
 
         if db_type == "postgres":
             try:
-                execute_to_file(
+                execute_to_gzip(
                     docker_exec_argv(
                         container,
                         [
@@ -133,6 +159,7 @@ def backup_database(
                             "--no-privileges",
                         ],
                         interactive=True,
+                        forward_env=["PGPASSWORD"],
                     ),
                     dump_file,
                     env={"PGPASSWORD": password},

src/baudolo/backup/docker.py

@@ -9,10 +9,35 @@ if TYPE_CHECKING:
 
 
 def docker_exec_argv(
-    container: str, argv: Sequence[str], *, interactive: bool = False
+    container: str,
+    argv: Sequence[str],
+    *,
+    interactive: bool = False,
+    forward_env: Sequence[str] = (),
 ) -> list[str]:
-    """The argv that runs *argv* inside *container*."""
-    return ["docker", "exec", *(["-i"] if interactive else []), container, *argv]
+    """The argv that runs *argv* inside *container*.
+
+    Args:
+        container: the container to run in.
+        argv: the command, already split.
+        interactive: keep stdin open, for a command that is fed a dump.
+        forward_env: names of environment variables to hand to the container.
+            Passed as bare ``-e NAME``, so docker copies the value out of this
+            process's own environment; spelling ``-e NAME=value`` instead would
+            publish a secret in the host's process list.
+
+    Returns:
+        The argv list.
+    """
+    forwarded = [arg for name in forward_env for arg in ("-e", name)]
+    return [
+        "docker",
+        "exec",
+        *(["-i"] if interactive else []),
+        *forwarded,
+        container,
+        *argv,
+    ]
 
 
 def get_image_info(container: str) -> str:

src/baudolo/backup/dumps.py

@@ -10,7 +10,7 @@ from pandas.errors import EmptyDataError
 
 from baudolo.databases import COLUMNS, DELIMITER
 
-from .db import backup_database
+from .db import backup_database, get_instance
 from .docker import has_tool, image_id
 
 DUMP_TOOLS: tuple[tuple[str, str], ...] = (
@@ -76,6 +76,8 @@ def backup_mariadb_or_postgres(
     engine = container_engine(container)
     if engine is None:
         return VolumeOutcome(database=False, dumped=False)
+    if get_instance(container, database_containers) is None:
+        return VolumeOutcome(database=False, dumped=False)
     db_type, dump_tool = engine
     dumped = backup_database(
         container=container,

src/baudolo/backup/shell.py

@@ -7,14 +7,22 @@ string handed to ``shell=True`` allowed.
 
 from __future__ import annotations
 
+import gzip
 import os
 import subprocess
+import tempfile
+import zlib
+from functools import partial
 from pathlib import Path
 from typing import TYPE_CHECKING
 
 if TYPE_CHECKING:
     from collections.abc import Mapping, Sequence
 
+GZIP_LEVEL = 6
+READ_BYTES = 1 << 20
+SYNC_BYTES = 1 << 16
+
 
 class BackupError(Exception):
     """Generic exception for backup errors."""
@@ -53,23 +61,71 @@ def execute_shell_command(
     return [line.decode("utf-8") for line in out.splitlines()]
 
 
-def execute_to_file(
+def is_sync_point(piece: bytes) -> bool:
+    """
+    Args:
+        piece: a line of the dump, or a READ_BYTES slice of a longer one.
+
+    Returns:
+        True when a full flush follows *piece*, decided by its bytes alone, on
+        average once per SYNC_BYTES of input.
+    """
+    return zlib.crc32(piece) % max(1, SYNC_BYTES // len(piece)) == 0
+
+
+def _abort(process: subprocess.Popen, tmp: Path) -> None:
+    process.kill()
+    process.wait()
+    tmp.unlink(missing_ok=True)
+
+
+def execute_to_gzip(
     command: Sequence[str], out_file: str, *, env: Mapping[str, str] | None = None
 ) -> None:
-    """Run *command*, writing its stdout to *out_file* only once it succeeded.
+    """
+    Args:
+        command: argv, the program first; its stdout is the dump.
+        out_file: the final gzip path. The stream is written to
+            ``<out_file>.tmp`` and renamed only after the command exited 0.
+        env: variables added to the child's environment.
 
-    The output goes to a sibling temporary file first, so a partial or empty
-    stream from a failing dump never takes the place of a valid backup.
+    Raises:
+        BackupError: the command failed, or the file could not be written.
     """
     command = list(command)
     print(" ".join(command), flush=True)
     tmp = Path(f"{out_file}.tmp")
-    with tmp.open("wb") as handle:
+    with tempfile.TemporaryFile() as err_sink:
         process = subprocess.Popen(
-            command, stdout=handle, stderr=subprocess.PIPE, env=_child_env(env)
+            command, stdout=subprocess.PIPE, stderr=err_sink, env=_child_env(env)
         )
-        _, err = process.communicate()
-    if process.returncode != 0:
+        try:
+            with (
+                tmp.open("wb") as raw,
+                gzip.GzipFile(
+                    filename="",
+                    mode="wb",
+                    fileobj=raw,
+                    compresslevel=GZIP_LEVEL,
+                    mtime=0,
+                ) as packed,
+            ):
+                for piece in iter(partial(process.stdout.readline, READ_BYTES), b""):
+                    packed.write(piece)
+                    if is_sync_point(piece):
+                        packed.flush(zlib.Z_FULL_FLUSH)
+        except OSError as error:
+            _abort(process, tmp)
+            raise BackupError(f"Error writing {out_file}: {error}") from error
+        except BaseException:
+            _abort(process, tmp)
+            raise
+        finally:
+            process.stdout.close()
+        returncode = process.wait()
+        err_sink.seek(0)
+        err = err_sink.read()
+    if returncode != 0:
         tmp.unlink()
-        _fail(command, process.returncode, b"", err)
+        _fail(command, returncode, b"", err)
     tmp.replace(out_file)

src/baudolo/backup/snapshot.py

@@ -58,27 +58,50 @@ def _resolver(subject: str, root: str) -> Callable[[str], str]:
     return resolve
 
 
+def _remove(
+    root: str, commands: list[list[str]], run: Callable[[list[str]], list[str]]
+) -> None:
+    try:
+        for command in commands:
+            run(command)
+    except BackupError as error:
+        # Raising here would also mask whatever the body raised.
+        print(f"WARNING: {root} could not be removed: {error}", flush=True)
+
+
 def _btrfs(
     subject: str, name: str, run: Callable[[list[str]], list[str]]
-) -> tuple[str, list[str]]:
+) -> tuple[str, list[list[str]]]:
     # The snapshot goes inside the subject, never beside it: the kernel rejects
     # a snapshot whose destination is on another filesystem, which is exactly
     # what the parent directory is when the subject is a mountpoint of its own.
     target = str(Path(os.path.abspath(subject)) / f".{name}")  # noqa: PTH100 - see _resolver
     run(["btrfs", "subvolume", "snapshot", "-r", subject, target])
-    return target, ["btrfs", "subvolume", "delete", target]
+    return target, [["btrfs", "subvolume", "delete", target]]
 
 
 def _zfs(
     subject: str, name: str, run: Callable[[list[str]], list[str]]
-) -> tuple[str, list[str]]:
+) -> tuple[str, list[list[str]]]:
     output = run(["zfs", "list", "-H", "-o", "name", subject])
     dataset = (output[0] if output else "").strip()
     if not dataset:
         raise SnapshotError(f"no zfs dataset is mounted at {subject}")
-    run(["zfs", "snapshot", f"{dataset}@{name}"])
-    root = str(Path(subject) / ".zfs" / "snapshot" / name)
-    return root, ["zfs", "destroy", f"{dataset}@{name}"]
+    snapshot = f"{dataset}@{name}"
+    # Exception: mounted here rather than read through .zfs/snapshot, which the
+    # kernel automounts via a usermode helper in the host's mount namespace,
+    # so a containerised run never sees it.
+    root = str(Path(os.path.abspath(subject)) / f".{name}")  # noqa: PTH100 - see _resolver
+    run(["zfs", "snapshot", snapshot])
+    remove = [["zfs", "destroy", snapshot]]
+    try:
+        run(["mkdir", root])
+        remove.insert(0, ["rmdir", root])
+        run(["mount", "-t", "zfs", snapshot, root])
+    except BaseException:
+        _remove(root, remove, run)
+        raise
+    return root, [["umount", root], *remove]
 
 
 _CREATE = {"btrfs": _btrfs, "zfs": _zfs}
@@ -169,8 +192,4 @@ def volume_snapshot(
     try:
         yield _resolver(subject, root)
     finally:
-        try:
-            run(remove)
-        except BackupError as error:
-            # Raising here would also mask whatever the body raised.
-            print(f"WARNING: {root} could not be removed: {error}", flush=True)
+        _remove(root, remove, run)

src/baudolo/backup/volume.py

@@ -75,7 +75,7 @@ def backup_volume(
     pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
 
     last = get_last_backup_dir(versions_dir, volume_name, dest)
-    cmd = ["rsync", "-aP", "--no-D", "--delete", "--delete-excluded"]
+    cmd = ["rsync", "-aHAX", "-P", "--no-D", "--delete", "--delete-excluded"]
     if authoritative:
         cmd.append("--checksum")
     if last:

src/baudolo/generation.py

@@ -18,8 +18,12 @@ from __future__ import annotations
 
 FILES_DIR = "files"
 SQL_DIR = "sql"
-DUMP_SUFFIX = ".backup.sql"
-CLUSTER_SUFFIX = ".cluster.backup.sql"
+LEGACY_DUMP_SUFFIX = ".backup.sql"
+LEGACY_CLUSTER_SUFFIX = ".cluster.backup.sql"
+COMPRESSED_SUFFIX = ".gz"
+DUMP_COMPRESSION = "gzip"
+DUMP_SUFFIX = LEGACY_DUMP_SUFFIX + COMPRESSED_SUFFIX
+CLUSTER_SUFFIX = LEGACY_CLUSTER_SUFFIX + COMPRESSED_SUFFIX
 
 MANIFEST_FILE = "manifest.json"
 MANIFEST_SCHEMA = 1
@@ -42,6 +46,7 @@ def manifest_document(volumes: dict[str, object]) -> dict[str, object]:
             "sql_dir": SQL_DIR,
             "dump_suffix": DUMP_SUFFIX,
             "cluster_suffix": CLUSTER_SUFFIX,
+            "dump_compression": DUMP_COMPRESSION,
         },
         "volumes": {
             name: {

src/baudolo/restore/__main__.py

@@ -6,10 +6,34 @@ import sys
 from .db.cluster import restore_cluster_sql
 from .db.mariadb import restore_mariadb_sql
 from .db.postgres import restore_postgres_sql
+from .dumpfile import locate
 from .files import restore_volume_files
 from .paths import BackupPaths
 
 
+def _password(parser: argparse.ArgumentParser, args: argparse.Namespace) -> str:
+    """
+    Args:
+        parser: reports a password given both ways or not at all.
+        args: the parsed engine subcommand.
+
+    Returns:
+        The database password, from stdin or from the deprecated argument.
+    """
+    if args.db_password_stdin == (args.db_password is not None):
+        parser.error(
+            "give the password either via --db-password-stdin or --db-password"
+        )
+    if args.db_password_stdin:
+        return sys.stdin.read().removesuffix("\n")
+    print(
+        "WARNING: --db-password is visible in the process list; "
+        "use --db-password-stdin",
+        file=sys.stderr,
+    )
+    return args.db_password
+
+
 def _add_common_backup_args(p: argparse.ArgumentParser) -> None:
     p.add_argument("volume_name", help="Docker volume name (target volume)")
     p.add_argument("backup_hash", help="Hashed machine id")
@@ -29,7 +53,16 @@ def _add_common_backup_args(p: argparse.ArgumentParser) -> None:
 
 def _add_common_engine_args(p: argparse.ArgumentParser) -> None:
     p.add_argument("--container", required=True)
-    p.add_argument("--db-password", required=True)
+    p.add_argument(
+        "--db-password",
+        default=None,
+        help="Deprecated: visible in the process list; use --db-password-stdin",
+    )
+    p.add_argument(
+        "--db-password-stdin",
+        action="store_true",
+        help="Read the database password from stdin",
+    )
     p.add_argument("--empty", action="store_true")
     p.add_argument(
         "--no-version-check",
@@ -74,7 +107,7 @@ def main(argv: list[str] | None = None) -> int:
     p_cluster.add_argument(
         "--instance",
         required=True,
-        help="Instance the dump was taken from; names <instance>.cluster.backup.sql",
+        help="Instance the dump was taken from; names <instance>.cluster.backup.sql.gz",
     )
     p_cluster.add_argument(
         "--db-user",
@@ -91,6 +124,7 @@ def main(argv: list[str] | None = None) -> int:
     p_mdb.add_argument("--db-user", default=None, help="Defaults to db-name if omitted")
 
     args = parser.parse_args(argv)
+    password = None if args.cmd == "files" else _password(parser, args)
 
     try:
         if args.cmd == "files":
@@ -115,14 +149,16 @@ def main(argv: list[str] | None = None) -> int:
                 container=args.container,
                 db_name=args.db_name,
                 user=user,
-                password=args.db_password,
-                sql_path=BackupPaths(
+                password=password,
+                sql_path=locate(
+                    BackupPaths(
                         args.volume_name,
                         args.backup_hash,
                         args.version,
                         repo_name=args.repo_name,
                         backups_dir=args.backups_dir,
-                ).sql_file(args.db_name),
+                    ).sql_file(args.db_name)
+                ),
                 empty=args.empty,
                 check_version=not args.no_version_check,
             )
@@ -132,14 +168,16 @@ def main(argv: list[str] | None = None) -> int:
             restore_cluster_sql(
                 container=args.container,
                 user=args.db_user,
-                password=args.db_password,
-                sql_path=BackupPaths(
+                password=password,
+                sql_path=locate(
+                    BackupPaths(
                         args.volume_name,
                         args.backup_hash,
                         args.version,
                         repo_name=args.repo_name,
                         backups_dir=args.backups_dir,
-                ).cluster_file(args.instance),
+                    ).cluster_file(args.instance)
+                ),
                 empty=args.empty,
                 check_version=not args.no_version_check,
             )
@@ -151,14 +189,16 @@ def main(argv: list[str] | None = None) -> int:
                 container=args.container,
                 db_name=args.db_name,
                 user=user,
-                password=args.db_password,
-                sql_path=BackupPaths(
+                password=password,
+                sql_path=locate(
+                    BackupPaths(
                         args.volume_name,
                         args.backup_hash,
                         args.version,
                         repo_name=args.repo_name,
                         backups_dir=args.backups_dir,
-                ).sql_file(args.db_name),
+                    ).sql_file(args.db_name)
+                ),
                 empty=args.empty,
                 check_version=not args.no_version_check,
             )

src/baudolo/restore/db/cluster.py

@@ -24,6 +24,7 @@ import tempfile
 from pathlib import Path
 from typing import TYPE_CHECKING
 
+from baudolo.restore.dumpfile import open_dump
 from baudolo.restore.run import docker_exec
 
 from .version import guard
@@ -96,7 +97,7 @@ def dump_inventory(sql_path: str) -> tuple[list[str], list[str]]:
     """
     databases: list[str] = []
     roles: list[str] = []
-    with Path(sql_path).open("rb") as handle:
+    with open_dump(sql_path) as handle:
         for raw in handle:
             line = raw.decode("utf-8", "replace")
             for pattern, sink, read in (
@@ -212,7 +213,8 @@ def restore_cluster_sql(
         user: a superuser of that instance; the dump creates roles and
             databases, which an application role may not do.
         password: its password, handed to psql through the container's env.
-        sql_path: the ``<instance>.cluster.backup.sql`` of a generation.
+        sql_path: the ``<instance>.cluster.backup.sql.gz`` of a generation, or
+            the uncompressed ``<instance>.cluster.backup.sql`` of an older one.
         empty: drop the cluster's databases and roles first. Without it the
             replay stops at the first object that already exists, which is the
             honest outcome: recreating a cluster over a populated one is a
@@ -243,7 +245,7 @@ def restore_cluster_sql(
             docker_env=docker_env,
         )
 
-    with Path(sql_path).open("rb") as src, tempfile.TemporaryFile() as filtered:
+    with open_dump(sql_path) as src, tempfile.TemporaryFile() as filtered:
         for line in filter_own_role_creation(src, user):
             filtered.write(line)
         filtered.seek(0)

src/baudolo/restore/db/empty_preclean.sql

@@ -3,8 +3,12 @@
 -- would run every DROP in one transaction and exhaust max_locks_per_transaction on
 -- large schemas (e.g. gitlab). Also drops user-owned non-public schemas so a dump
 -- that CREATE SCHEMAs (e.g. discourse's discourse_functions) does not fail on an
--- already-existing schema. Extension members (pg_trgm's set_limit) are
--- superuser-owned; IF EXISTS absorbs the CASCADE fallout.
+-- already-existing schema. Objects belonging to an extension are skipped: postgres
+-- refuses to drop them one by one ("cannot drop function vector_in(...) because
+-- extension vector requires it"), and the dump's CREATE EXTENSION IF NOT EXISTS
+-- finds the surviving extension either way. Owning them is not enough to make them
+-- droppable - an extension a role installed itself is owned by that role, so the
+-- owner filter alone lets pgvector's members through.
 SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
   FROM (
     SELECT format('%I', c.relname) AS name,
@@ -13,7 +17,8 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
              WHEN 'm' THEN 'MATERIALIZED VIEW'
              WHEN 'f' THEN 'FOREIGN TABLE'
              ELSE 'TABLE'
-           END AS type
+           END AS type,
+           c.oid AS objid, 'pg_class'::regclass AS classid
       FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
      WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p', 'v', 'm', 'f')
        AND pg_get_userbyid(c.relowner) = current_user
@@ -21,17 +26,20 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
     -- Overloaded functions share a proname; DROP needs the identity
     -- signature or psql aborts with "function name is not unique".
     SELECT format('%I(%s)', p.proname, pg_get_function_identity_arguments(p.oid)) AS name,
-           CASE p.prokind WHEN 'p' THEN 'PROCEDURE' ELSE 'FUNCTION' END AS type
+           CASE p.prokind WHEN 'p' THEN 'PROCEDURE' ELSE 'FUNCTION' END AS type,
+           p.oid AS objid, 'pg_proc'::regclass AS classid
       FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
      WHERE n.nspname = 'public' AND p.prokind IN ('f', 'p', 'w')
        AND pg_get_userbyid(p.proowner) = current_user
     UNION ALL
-    SELECT format('%I', c.relname) AS name, 'SEQUENCE' AS type
+    SELECT format('%I', c.relname) AS name, 'SEQUENCE' AS type,
+           c.oid AS objid, 'pg_class'::regclass AS classid
       FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace
      WHERE n.nspname = 'public' AND c.relkind = 'S'
        AND pg_get_userbyid(c.relowner) = current_user
     UNION ALL
-    SELECT format('%I', t.typname) AS name, 'TYPE' AS type
+    SELECT format('%I', t.typname) AS name, 'TYPE' AS type,
+           t.oid AS objid, 'pg_type'::regclass AS classid
       FROM pg_type t JOIN pg_namespace n ON n.oid = t.typnamespace
      WHERE n.nspname = 'public'
        AND pg_get_userbyid(t.typowner) = current_user
@@ -40,25 +48,36 @@ SELECT format('DROP %s IF EXISTS public.%s CASCADE', obj.type, obj.name)
                   SELECT 1 FROM pg_class c2
                    WHERE c2.oid = t.typrelid AND c2.relkind = 'c')))
     UNION ALL
-    SELECT format('%I', col.collname) AS name, 'COLLATION' AS type
+    SELECT format('%I', col.collname) AS name, 'COLLATION' AS type,
+           col.oid AS objid, 'pg_collation'::regclass AS classid
       FROM pg_collation col JOIN pg_namespace n ON n.oid = col.collnamespace
      WHERE n.nspname = 'public'
        AND pg_get_userbyid(col.collowner) = current_user
     UNION ALL
-    SELECT format('%I', ts.cfgname) AS name, 'TEXT SEARCH CONFIGURATION' AS type
+    SELECT format('%I', ts.cfgname) AS name, 'TEXT SEARCH CONFIGURATION' AS type,
+           ts.oid AS objid, 'pg_ts_config'::regclass AS classid
       FROM pg_ts_config ts JOIN pg_namespace n ON n.oid = ts.cfgnamespace
      WHERE n.nspname = 'public'
        AND pg_get_userbyid(ts.cfgowner) = current_user
     UNION ALL
-    SELECT format('%I', d.dictname) AS name, 'TEXT SEARCH DICTIONARY' AS type
+    SELECT format('%I', d.dictname) AS name, 'TEXT SEARCH DICTIONARY' AS type,
+           d.oid AS objid, 'pg_ts_dict'::regclass AS classid
       FROM pg_ts_dict d JOIN pg_namespace n ON n.oid = d.dictnamespace
      WHERE n.nspname = 'public'
        AND pg_get_userbyid(d.dictowner) = current_user
   ) obj
+ WHERE NOT EXISTS (
+         SELECT 1 FROM pg_depend dep
+          WHERE dep.classid = obj.classid AND dep.objid = obj.objid
+            AND dep.deptype = 'e')
 UNION ALL
 SELECT format('DROP SCHEMA IF EXISTS %I CASCADE', n.nspname)
   FROM pg_namespace n
  WHERE NOT starts_with(n.nspname, 'pg_')
    AND n.nspname NOT IN ('public', 'information_schema')
    AND pg_get_userbyid(n.nspowner) = current_user
+   AND NOT EXISTS (
+         SELECT 1 FROM pg_depend dep
+          WHERE dep.classid = 'pg_namespace'::regclass AND dep.objid = n.oid
+            AND dep.deptype = 'e')
 \gexec

src/baudolo/restore/db/mariadb.py

@@ -3,7 +3,8 @@ from __future__ import annotations
 import sys
 from pathlib import Path
 
-from baudolo.restore.run import docker_exec, docker_exec_sh
+from baudolo.restore.dumpfile import open_dump, verify
+from baudolo.restore.run import docker_exec, docker_exec_sh, docker_exec_stream
 
 from .version import guard
 
@@ -47,6 +48,9 @@ def restore_mariadb_sql(
     if not Path(sql_path).is_file():
         raise FileNotFoundError(sql_path)
 
+    verify(sql_path)
+    docker_env = {"MYSQL_PWD": password}
+
     if check_version:
         guard(
             sql_path=sql_path,
@@ -65,12 +69,12 @@ def restore_mariadb_sql(
                 client,
                 "-u",
                 user,
-                f"--password={password}",
                 "-N",
                 "-e",
                 f"SELECT table_name FROM information_schema.tables WHERE table_schema = '{db_name}';",  # noqa: S608 - validate_database() constrains the name to ^[a-zA-Z0-9_][a-zA-Z0-9_-]*$
             ],
             capture=True,
+            docker_env=docker_env,
         )
         tables = result.stdout.decode().split()
 
@@ -90,15 +94,18 @@ def restore_mariadb_sql(
                     client,
                     "-u",
                     user,
-                    f"--password={password}",
                     "-e",
                     drop_sql,
                 ],
+                docker_env=docker_env,
             )
 
-    with Path(sql_path).open("rb") as f:
-        docker_exec(
-            container, [client, "-u", user, f"--password={password}", db_name], stdin=f
+    with open_dump(sql_path) as source:
+        docker_exec_stream(
+            container,
+            [client, "-u", user, db_name],
+            source=source,
+            docker_env=docker_env,
         )
 
     print(f"MariaDB/MySQL restore complete for db '{db_name}'.")

src/baudolo/restore/db/postgres.py

@@ -4,6 +4,7 @@ import tempfile
 from pathlib import Path
 from typing import TYPE_CHECKING
 
+from baudolo.restore.dumpfile import open_dump
 from baudolo.restore.run import docker_exec
 
 from .version import guard
@@ -67,6 +68,13 @@ def restore_postgres_sql(
 
     docker_env = {"PGPASSWORD": password}
 
+    # Filter into a spooled temp file instead of building the whole dump in
+    # memory: production dumps reach many GB and the previous read/splitlines/
+    # join needed roughly three times the dump size in RSS.
+    with open_dump(sql_path) as src, tempfile.TemporaryFile() as filtered:
+        for line in filter_superuser_only_lines(src):
+            filtered.write(line)
+        filtered.seek(0)
         if empty:
             with _EMPTY_PRECLEAN_SQL.open(encoding="utf-8") as preclean:
                 drop_sql = preclean.read()
@@ -76,14 +84,6 @@ def restore_postgres_sql(
                 stdin=drop_sql.encode(),
                 docker_env=docker_env,
             )
-
-    # Filter into a spooled temp file instead of building the whole dump in
-    # memory: production dumps reach many GB and the previous read/splitlines/
-    # join needed roughly three times the dump size in RSS.
-    with Path(sql_path).open("rb") as src, tempfile.TemporaryFile() as filtered:
-        for line in filter_superuser_only_lines(src):
-            filtered.write(line)
-        filtered.seek(0)
         docker_exec(
             container,
             [

src/baudolo/restore/db/version.py

@@ -23,9 +23,10 @@ with the cluster banner and the roles section, and the first
 
 from __future__ import annotations
 
+import io
 import re
-from pathlib import Path
 
+from baudolo.restore.dumpfile import open_dump
 from baudolo.restore.run import docker_exec, stdout_of
 
 SCAN_LINES = 2000
@@ -69,7 +70,9 @@ def dump_version(sql_path: str, engine: str) -> str:
         VersionMismatchError: no version line within the first ``SCAN_LINES``.
     """
     pattern = DUMP_VERSION[engine]
-    with Path(sql_path).open(encoding="utf-8", errors="replace") as handle:
+    with io.TextIOWrapper(
+        open_dump(sql_path), encoding="utf-8", errors="replace"
+    ) as handle:
         for _ in range(SCAN_LINES):
             line = handle.readline()
             if not line:
@@ -102,13 +105,13 @@ def server_version(
                 client or "mariadb",
                 "-u",
                 user,
-                f"--password={password}",
                 "-N",
                 "-B",
                 "-e",
                 "SELECT VERSION()",
             ],
             capture=True,
+            docker_env={"MYSQL_PWD": password},
         )
     )
 

src/baudolo/restore/dumpfile.py

@@ -0,0 +1,81 @@
+from __future__ import annotations
+
+import gzip
+from pathlib import Path
+from typing import BinaryIO
+
+from baudolo.generation import COMPRESSED_SUFFIX
+
+GZIP_MAGIC = b"\x1f\x8b"
+VERIFY_BYTES = 1 << 20
+
+
+def is_gzip(path: str) -> bool:
+    """
+    Args:
+        path: a dump file.
+
+    Returns:
+        True when its first bytes are the gzip magic, whatever its name says.
+    """
+    with Path(path).open("rb") as handle:
+        return handle.read(len(GZIP_MAGIC)) == GZIP_MAGIC
+
+
+def open_dump(path: str) -> BinaryIO:
+    """
+    Args:
+        path: a dump, gzip-compressed or plain.
+
+    Returns:
+        A binary reader yielding the SQL text either way.
+    """
+    if is_gzip(path):
+        return gzip.open(path, "rb")
+    return Path(path).open("rb")
+
+
+def verify(path: str) -> None:
+    """
+    Args:
+        path: a dump; a gzip one is decompressed to its end, a plain one is
+            left alone.
+
+    Raises:
+        EOFError: the gzip stream is truncated.
+        gzip.BadGzipFile: its header, CRC or length is wrong.
+        zlib.error: its deflate data is corrupt.
+    """
+    if not is_gzip(path):
+        return
+    with gzip.open(path, "rb") as handle:
+        while handle.read(VERIFY_BYTES):
+            pass
+
+
+def locate(path: str) -> str:
+    """
+    Args:
+        path: where this version writes the dump, ending in COMPRESSED_SUFFIX.
+
+    Returns:
+        *path*, or the uncompressed name earlier versions wrote, whichever
+        exists.
+
+    Raises:
+        FileNotFoundError: neither exists.
+        FileExistsError: both exist.
+    """
+    legacy = path.removesuffix(COMPRESSED_SUFFIX)
+    found = [
+        candidate
+        for candidate in dict.fromkeys((path, legacy))
+        if Path(candidate).is_file()
+    ]
+    if len(found) > 1:
+        raise FileExistsError(
+            f"both {path} and {legacy} exist; remove the one that must not be replayed"
+        )
+    if not found:
+        raise FileNotFoundError(f"neither {path} nor {legacy} exists")
+    return found[0]

src/baudolo/restore/files.py

@@ -59,6 +59,6 @@ def restore_volume_files(volume_name: str, backup_files_dir: str) -> int:
     # rsync reads "dir/" as its contents and "dir" as the directory itself.
     src = f"{Path(backup_files_dir)}{os.sep}"
     dest = f"{Path(mountpoint)}{os.sep}"
-    run(["rsync", "-avv", "--delete", src, dest])
+    run(["rsync", "-aHAX", "-vv", "--delete", src, dest])
     print("File restore complete.")
     return 0

src/baudolo/restore/run.py

@@ -1,7 +1,13 @@
 from __future__ import annotations
 
+import contextlib
+import os
+import shutil
 import subprocess
 import sys
+from typing import BinaryIO
+
+STREAM_BYTES = 1 << 20
 
 
 def run(
@@ -46,6 +52,38 @@ def stdout_of(completed: subprocess.CompletedProcess) -> str:
     return (raw.decode() if isinstance(raw, bytes) else raw).strip()
 
 
+def docker_exec_cmd(
+    container: str, argv: list[str], docker_env: dict[str, str] | None
+) -> list[str]:
+    """
+    Args:
+        container: the container to run in.
+        argv: the command, already split.
+        docker_env: variables to hand to the container; only their names
+            reach the returned argv.
+
+    Returns:
+        The ``docker exec -i`` argv.
+    """
+    forwarded = [arg for name in docker_env or {} for arg in ("-e", name)]
+    return ["docker", "exec", "-i", *forwarded, container, *argv]
+
+
+def child_env(env: dict | None, docker_env: dict[str, str] | None) -> dict | None:
+    """
+    Args:
+        env: the docker CLI's environment, or None to inherit this process's.
+        docker_env: variables whose values the docker CLI copies into the
+            container for a bare ``-e NAME``.
+
+    Returns:
+        The environment for the docker CLI, or None to inherit unchanged.
+    """
+    if not docker_env:
+        return env
+    return {**(os.environ if env is None else env), **docker_env}
+
+
 def docker_exec(
     container: str,
     argv: list[str],
@@ -55,12 +93,49 @@ def docker_exec(
     env: dict | None = None,
     docker_env: dict[str, str] | None = None,
 ) -> subprocess.CompletedProcess:
-    cmd: list[str] = ["docker", "exec", "-i"]
-    if docker_env:
-        for k, v in docker_env.items():
-            cmd.extend(["-e", f"{k}={v}"])
-    cmd.extend([container, *argv])
-    return run(cmd, stdin=stdin, capture=capture, env=env)
+    return run(
+        docker_exec_cmd(container, argv, docker_env),
+        stdin=stdin,
+        capture=capture,
+        env=child_env(env, docker_env),
+    )
+
+
+def docker_exec_stream(
+    container: str,
+    argv: list[str],
+    *,
+    source: BinaryIO,
+    docker_env: dict[str, str] | None = None,
+) -> None:
+    """
+    Args:
+        container: the container to run in.
+        argv: the command, fed *source* on stdin.
+        source: a binary reader, copied in STREAM_BYTES chunks, so a reader
+            without a file descriptor of its own, such as a gzip one, works.
+        docker_env: as for :func:`docker_exec`.
+
+    Raises:
+        subprocess.CalledProcessError: the command exited non-zero.
+    """
+    cmd = docker_exec_cmd(container, argv, docker_env)
+    process = subprocess.Popen(
+        cmd, stdin=subprocess.PIPE, env=child_env(None, docker_env)
+    )
+    try:
+        with contextlib.suppress(BrokenPipeError):
+            shutil.copyfileobj(source, process.stdin, STREAM_BYTES)
+        with contextlib.suppress(BrokenPipeError):
+            process.stdin.close()
+    except BaseException:
+        process.kill()
+        process.wait()
+        raise
+    returncode = process.wait()
+    if returncode != 0:
+        print(f"ERROR: command failed ({returncode}): {' '.join(cmd)}", file=sys.stderr)
+        raise subprocess.CalledProcessError(returncode, cmd)
 
 
 def docker_exec_sh(

src/baudolo/seed/__main__.py

@@ -73,9 +73,30 @@ def main() -> None:
         help="Database name or '*' to dump all databases",
     )
     parser.add_argument("username", help="Database username")
-    parser.add_argument("password", help="Database password")
+    parser.add_argument(
+        "password",
+        nargs="?",
+        help="Deprecated: database password as an argument, visible in the "
+        "process list; use --password-stdin",
+    )
+    parser.add_argument(
+        "--password-stdin",
+        action="store_true",
+        help="Read the database password from stdin",
+    )
 
     args = parser.parse_args()
+    if args.password_stdin == (args.password is not None):
+        parser.error("give the password either via --password-stdin or as an argument")
+    if args.password_stdin:
+        password = sys.stdin.read().removesuffix("\n")
+    else:
+        print(
+            "WARNING: a password argument is visible in the process list; "
+            "use --password-stdin",
+            file=sys.stderr,
+        )
+        password = args.password
 
     try:
         check_and_add_entry(
@@ -83,7 +104,7 @@ def main() -> None:
             instance=args.instance,
             database=args.database,
             username=args.username,
-            password=args.password,
+            password=password,
         )
     except Exception as exc:  # noqa: BLE001 - CLI boundary: any failure becomes exit 1
         print(f"ERROR: {exc}", file=sys.stderr)

tests/e2e/helpers/fixtures.py

@@ -2,10 +2,14 @@
 
 from __future__ import annotations
 
+import gzip
 import shutil
 import subprocess
 from pathlib import Path
 
+from baudolo.generation import DUMP_SUFFIX, LEGACY_DUMP_SUFFIX
+from baudolo.restore.dumpfile import open_dump
+
 from .process import machine_hash, run
 
 # postgres 18+ mounts at /var/lib/postgresql, not /var/lib/postgresql/data.
@@ -25,7 +29,7 @@ def backup_run(
     images_no_stop_required: list[str],
     images_no_backup_required: list[str] | None = None,
     only_sql: bool = False,
-) -> None:
+) -> subprocess.CompletedProcess:
     cmd = [
         "baudolo",
         "--compose-dir",
@@ -49,7 +53,7 @@ def backup_run(
         cmd += ["--only-sql"]
 
     try:
-        run(cmd, capture=True, check=True)
+        return run(cmd, capture=True, check=True)
     except subprocess.CalledProcessError as e:
         print(">>> baudolo failed (exit code:", e.returncode, ")")
         if e.stdout:
@@ -112,3 +116,54 @@ def ensure_empty_dir(path: str) -> None:
     if p.exists():
         shutil.rmtree(p)
     p.mkdir(parents=True, exist_ok=True)
+
+
+def dump_text(path: Path | str) -> str:
+    """
+    Args:
+        path: a dump, gzip-compressed or plain.
+
+    Returns:
+        Its SQL text.
+    """
+    with open_dump(str(path)) as handle:
+        return handle.read().decode("utf-8", errors="replace")
+
+
+def write_dump_text(path: Path | str, text: str) -> None:
+    """
+    Args:
+        path: a gzip dump to overwrite.
+        text: the SQL text it carries afterwards.
+    """
+    with gzip.open(str(path), "wb") as handle:
+        handle.write(text.encode("utf-8"))
+
+
+def dumps_in(sql_dir: Path) -> list[Path]:
+    """
+    Args:
+        sql_dir: a volume's sql directory, which may be absent.
+
+    Returns:
+        Every dump in it, compressed or plain, single-database or cluster.
+    """
+    if not sql_dir.is_dir():
+        return []
+    return sorted(
+        {*sql_dir.glob(f"*{DUMP_SUFFIX}"), *sql_dir.glob(f"*{LEGACY_DUMP_SUFFIX}")}
+    )
+
+
+def assert_dumps_in_sees_both_suffixes(case, scratch: str) -> None:
+    """
+    Args:
+        case: the running ``unittest.TestCase``.
+        scratch: an empty directory to place the probe dumps in.
+    """
+    probe = Path(scratch) / "dumps-in-probe"
+    probe.mkdir(parents=True, exist_ok=True)
+    expected = [probe / f"a{DUMP_SUFFIX}", probe / f"b{LEGACY_DUMP_SUFFIX}"]
+    for path in expected:
+        path.write_bytes(b"")
+    case.assertEqual(dumps_in(probe), expected)

tests/e2e/helpers/process.py

@@ -13,7 +13,17 @@ def run(
     capture: bool = True,
     check: bool = True,
     cwd: str | None = None,
+    stdin_text: str | None = None,
 ) -> subprocess.CompletedProcess:
+    """
+    Args:
+        cmd: argv, the program first.
+        capture: keep stdout and stderr on the result.
+        check: raise on a non-zero exit.
+        cwd: working directory.
+        stdin_text: text fed to the command's stdin, e.g. a password for
+            ``--db-password-stdin``.
+    """
     try:
         return subprocess.run(
             cmd,
@@ -21,6 +31,7 @@ def run(
             cwd=cwd,
             text=True,
             capture_output=capture,
+            input=stdin_text,
         )
     except subprocess.CalledProcessError as e:
         # Print captured output so failing E2E tests are "live" / debuggable in CI logs
@@ -96,12 +107,13 @@ def wait_for_mariadb(
     container: str, *, root_password: str, timeout_s: int = 90
 ) -> None:
     """
-    Liveness probe for MariaDB.
+    Readiness probe for MariaDB, as root over the socket.
 
-    IMPORTANT (MariaDB 11):
-    Root TCP auth is often restricted (unix_socket auth), so a TCP ping like
-    `mariadb-admin -uroot -p... -h localhost ping` can fail even though the server is up.
-    We therefore check readiness via a socket-based query.
+    The entrypoint drops root@127.0.0.1, so root is socket-only, and it
+    installs root@localhost without unix_socket auth: a probe without the
+    password only passes in the init window before the password is set.
+    `@@skip_networking` must be 0 so the entrypoint's temporary server,
+    which shuts down right after init, does not count as ready.
     """
     deadline = time.time() + timeout_s
     while time.time() < deadline:
@@ -112,12 +124,15 @@ def wait_for_mariadb(
                 container,
                 "sh",
                 "-lc",
-                'mariadb -uroot --protocol=socket -e "SELECT 1;"',
+                (
+                    f"mariadb -uroot -p{root_password} --protocol=socket "
+                    '-N -e "SELECT @@skip_networking;"'
+                ),
             ],
             capture=True,
             check=False,
         )
-        if p.returncode == 0:
+        if p.returncode == 0 and p.stdout.strip() == "0":
             return
         time.sleep(1)
     raise TimeoutError(

tests/e2e/test_e2e_app_container_ships_client_tools.py

@@ -0,0 +1,185 @@
+"""An application container that ships the engine's client tools.
+
+This is the shape a dedicated database deploys in: the engine runs as
+`<app>-database` while the application itself runs as `<app>`, and neither is
+declared through --database-containers, so both names go through the instance
+regex. `<app>-database` loses its suffix and lands on the instance `<app>` -
+and `<app>` carries no database token at all, so a fallback that returns the
+name unchanged lands on that same instance and offers the application container
+as a second engine for the same row.
+
+Discourse is the live example: its application container is named `discourse`
+by its own launcher and ships pg_dumpall, so a dump command starts there and
+writes a file that looks like a backup and holds none of the data.
+"""
+
+import json
+import unittest
+from pathlib import Path
+
+from baudolo.generation import DUMP_SUFFIX, FILES_DIR, MANIFEST_FILE, SQL_DIR
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    dump_text,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+MARKER = "the-application-volume-holds-files"
+PAYLOAD = "shop-payload"
+
+
+class TestE2EAppContainerShipsClientTools(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        # uuid4 hex may begin with "db", which the instance regex would split
+        # on and turn the application container into a different instance,
+        # hiding exactly the collision this module is about.
+        cls.prefix = unique("baudolo-e2e-app-tools").replace("-db", "-xb")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.engine = f"{cls.prefix}-shop-database"
+        cls.app = f"{cls.prefix}-shop"
+        cls.engine_volume = f"{cls.prefix}-shop-database-vol"
+        cls.app_volume = f"{cls.prefix}-shop-app-vol"
+        cls.containers = [cls.engine, cls.app]
+        cls.volumes = [cls.engine_volume, cls.app_volume]
+
+        run(["docker", "volume", "create", cls.engine_volume])
+        run(["docker", "volume", "create", cls.app_volume])
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.engine,
+                "-e",
+                "POSTGRES_PASSWORD=shoppw",
+                "-e",
+                "POSTGRES_DB=shopdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                "-v",
+                f"{cls.engine_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.app,
+                "--entrypoint",
+                "sh",
+                "-v",
+                f"{cls.app_volume}:/data",
+                POSTGRES_IMAGE,
+                "-c",
+                f"echo '{MARKER}' > /data/marker.txt && sleep 3600",
+            ]
+        )
+
+        wait_for_postgres(cls.engine, user="postgres", timeout_s=90)
+        run(
+            [
+                "docker",
+                "exec",
+                cls.engine,
+                "sh",
+                "-lc",
+                (
+                    'psql -U postgres -d shopdb -c "CREATE TABLE orders (id int, '
+                    f"note text); INSERT INTO orders VALUES (1,'{PAYLOAD}');\""
+                ),
+            ],
+            check=True,
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [(cls.app, "shopdb", "postgres", "shoppw")],
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=["dummy-db"],
+            images_no_stop_required=[POSTGRES_IMAGE],
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def volume_dir(self, volume: str) -> Path:
+        return backup_path(self.backups_dir, self.repo_name, self.version, volume)
+
+    def test_the_engine_volume_was_dumped(self) -> None:
+        dump = self.volume_dir(self.engine_volume) / SQL_DIR / f"shopdb{DUMP_SUFFIX}"
+        self.assertTrue(dump.is_file(), f"expected a dump at {dump}")
+        self.assertIn(PAYLOAD, dump_text(dump))
+
+    def test_the_application_volume_produced_no_dump(self) -> None:
+        """The collision this module exists for: the application container
+        answers the same instance as the engine and starts a dump of its own."""
+        sql_dir = self.volume_dir(self.app_volume) / SQL_DIR
+        self.assertFalse(
+            sql_dir.exists(),
+            f"the application container was dumped: {sorted(sql_dir.iterdir())}"
+            if sql_dir.exists()
+            else "",
+        )
+
+    def test_the_application_volume_was_backed_up_as_files(self) -> None:
+        """Refusing the dump must not cost the volume its backup."""
+        marker = self.volume_dir(self.app_volume) / FILES_DIR / "marker.txt"
+        self.assertTrue(marker.is_file(), f"expected a file backup at {marker}")
+        self.assertIn(MARKER, marker.read_text(encoding="utf-8"))
+
+    def test_the_manifest_does_not_call_the_application_volume_a_database(self) -> None:
+        manifest = json.loads(
+            (self.volume_dir(self.app_volume).parent / MANIFEST_FILE).read_text(
+                encoding="utf-8"
+            )
+        )
+        entry = manifest["volumes"][self.app_volume]
+        self.assertFalse(entry["database"], entry)
+        self.assertFalse(entry["dumped"], entry)
+
+    def test_the_manifest_records_the_engine_volume_as_dumped(self) -> None:
+        manifest = json.loads(
+            (self.volume_dir(self.engine_volume).parent / MANIFEST_FILE).read_text(
+                encoding="utf-8"
+            )
+        )
+        entry = manifest["volumes"][self.engine_volume]
+        self.assertTrue(entry["database"], entry)
+        self.assertTrue(entry["dumped"], entry)
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_engine_detection_by_tool.py

@@ -13,6 +13,8 @@ inside Alpine and takes the whole run with it.
 
 import unittest
 
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
@@ -20,6 +22,8 @@ from .helpers import (
     backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
+    dumps_in,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -142,16 +146,20 @@ class TestE2EEngineDetectionByTool(unittest.TestCase):
         return backup_path(self.backups_dir, self.repo_name, self.version, volume)
 
     def test_an_engine_without_an_engine_name_is_still_dumped(self) -> None:
-        dump = self._volume_dir(self.engine_volume) / "sql" / "appdb.backup.sql"
+        dump = self._volume_dir(self.engine_volume) / SQL_DIR / f"appdb{DUMP_SUFFIX}"
         self.assertTrue(
             dump.is_file(),
             f"a Postgres tagged '{self.engine_image}' produced no dump at {dump}",
         )
-        self.assertIn("Dumped by pg_dump", dump.read_text(encoding="utf-8"))
+        self.assertIn("Dumped by pg_dump", dump_text(dump))
 
     def test_an_engine_name_without_an_engine_is_not_dumped(self) -> None:
-        sql_dir = self._volume_dir(self.impostor_volume) / "sql"
-        dumps = list(sql_dir.glob("*.sql")) if sql_dir.exists() else []
+        self.assertEqual(
+            dumps_in(self._volume_dir(self.engine_volume) / SQL_DIR),
+            [self._volume_dir(self.engine_volume) / SQL_DIR / f"appdb{DUMP_SUFFIX}"],
+            "the glob must see the real engine's dump, or the check below is vacuous",
+        )
+        dumps = dumps_in(self._volume_dir(self.impostor_volume) / SQL_DIR)
         self.assertEqual(
             dumps,
             [],

tests/e2e/test_e2e_engine_detection_registry_prefix.py

@@ -1,5 +1,7 @@
 import unittest
 
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
@@ -7,6 +9,7 @@ from .helpers import (
     backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -94,11 +97,11 @@ class TestE2EEngineDetectionRegistryPrefix(unittest.TestCase):
     def test_the_registry_host_does_not_pick_the_engine(self) -> None:
         p = (
             backup_path(self.backups_dir, self.repo_name, self.version, self.pg_volume)
-            / "sql"
-            / "appdb.backup.sql"
+            / SQL_DIR
+            / f"appdb{DUMP_SUFFIX}"
         )
         self.assertTrue(p.is_file(), f"Expected a pg_dump at: {p}")
-        self.assertIn("Dumped by pg_dump", p.read_text(encoding="utf-8"))
+        self.assertIn("Dumped by pg_dump", dump_text(p))
 
 
 if __name__ == "__main__":

tests/e2e/test_e2e_files_live_writer_precopy.py

@@ -0,0 +1,110 @@
+"""A writer that shrinks files mid-read fails the live pre-copy, not the run."""
+
+import unittest
+
+from .helpers import (
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_log,
+    write_databases_csv,
+)
+
+CHURN_FILES = 8
+
+WRITER = f"""
+trap 'exit 0' TERM
+echo hello > /data/hello.txt
+for i in $(seq 1 {CHURN_FILES}); do
+  (while :; do truncate -s 0 /data/churn$i; truncate -s 16M /data/churn$i; done) &
+done
+echo ready
+wait
+"""
+
+
+class TestE2EFilesLiveWriterPreCopy(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-live-writer")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.volume = f"{cls.prefix}-vol"
+        cls.writer = f"{cls.prefix}-writer"
+        cls.containers = [cls.writer]
+        cls.volumes = [cls.volume]
+
+        run(["docker", "volume", "create", cls.volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.writer,
+                "-v",
+                f"{cls.volume}:/data",
+                "alpine:3.20",
+                "sh",
+                "-c",
+                WRITER,
+            ]
+        )
+        wait_for_log(cls.writer, "ready")
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(cls.databases_csv, [])
+
+        cls.result = backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=["dummy-db"],
+            images_no_stop_required=["dummy-image"],
+        )
+
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def test_the_live_pre_copy_hit_the_writer(self) -> None:
+        self.assertIn(
+            f"WARNING: live pre-copy of volume '{self.volume}' failed",
+            self.result.stdout,
+            "the writer never shrank a file under rsync, so this run proves nothing",
+        )
+
+    def test_the_stopped_copy_captured_the_volume(self) -> None:
+        files = (
+            backup_path(self.backups_dir, self.repo_name, self.version, self.volume)
+            / "files"
+        )
+        self.assertEqual((files / "hello.txt").read_text().strip(), "hello")
+        self.assertEqual(
+            sorted(p.name for p in files.glob("churn*")),
+            sorted(f"churn{i}" for i in range(1, CHURN_FILES + 1)),
+        )
+
+    def test_the_writer_runs_again_after_the_backup(self) -> None:
+        state = run(
+            ["docker", "inspect", "-f", "{{.State.Running}}", self.writer]
+        ).stdout.strip()
+        self.assertEqual(state, "true")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_mariadb_anonymous_preemption.py

@@ -34,6 +34,7 @@ from .helpers import (
     MARIADB_DATA_DIR,
     MARIADB_IMAGE,
     cleanup_docker,
+    dump_text,
     require_docker,
     run,
     unique,
@@ -100,7 +101,10 @@ class TestE2EMariaDBAnonymousPreemption(unittest.TestCase):
                 cls.db_container,
                 "sh",
                 "-lc",
-                f'mariadb -uroot --protocol=socket -e "{bootstrap_sql}"',
+                (
+                    f"mariadb -uroot -p{cls.root_password} --protocol=socket "
+                    f'-e "{bootstrap_sql}"'
+                ),
             ]
         )
 
@@ -156,8 +160,7 @@ class TestE2EMariaDBAnonymousPreemption(unittest.TestCase):
             self.assertTrue(produced, "backup_database did not produce a dump")
             dump_path = Path(volume_dir) / SQL_DIR / f"{self.db_name}{DUMP_SUFFIX}"
             self.assertTrue(dump_path.is_file(), f"expected dump at {dump_path}")
-            with dump_path.open(encoding="utf-8", errors="replace") as f:
-                content = f.read()
+            content = dump_text(dump_path)
             self.assertIn("INSERT INTO", content)
             self.assertIn("'ok'", content)
 

tests/e2e/test_e2e_mariadb_full.py

@@ -1,5 +1,7 @@
 import unittest
 
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
 from .helpers import (
     MARIADB_DATA_DIR,
     MARIADB_IMAGE,
@@ -7,6 +9,7 @@ from .helpers import (
     backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -137,10 +140,10 @@ class TestE2EMariaDBFull(unittest.TestCase):
                 cls.db_name,
                 "--db-user",
                 cls.db_user,
-                "--db-password",
-                cls.db_password,
+                "--db-password-stdin",
                 "--empty",
-            ]
+            ],
+            stdin_text=cls.db_password,
         )
 
     @classmethod
@@ -150,10 +153,11 @@ class TestE2EMariaDBFull(unittest.TestCase):
     def test_dump_file_exists(self) -> None:
         p = (
             backup_path(self.backups_dir, self.repo_name, self.version, self.db_volume)
-            / "sql"
-            / f"{self.db_name}.backup.sql"
+            / SQL_DIR
+            / f"{self.db_name}{DUMP_SUFFIX}"
         )
         self.assertTrue(p.is_file(), f"Expected dump file at: {p}")
+        self.assertIn("INSERT INTO", dump_text(p))
 
     def test_data_restored(self) -> None:
         p = run(

tests/e2e/test_e2e_only_files.py

@@ -7,12 +7,16 @@ make the flag useless there.
 
 import unittest
 
+from baudolo.generation import SQL_DIR
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
+    assert_dumps_in_sees_both_suffixes,
     backup_path,
     cleanup_docker,
     create_minimal_compose_dir,
+    dumps_in,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -104,8 +108,8 @@ class TestE2EOnlyFiles(unittest.TestCase):
         self.assertEqual(marker.read_text(encoding="utf-8").strip(), MARKER)
 
     def test_no_dump_is_written(self) -> None:
-        sql_dir = self._volume_dir() / "sql"
-        dumps = list(sql_dir.glob("*.sql")) if sql_dir.exists() else []
+        assert_dumps_in_sees_both_suffixes(self, f"/tmp/{self.prefix}")
+        dumps = dumps_in(self._volume_dir() / SQL_DIR)
         self.assertEqual(dumps, [], f"did not expect any dump, found: {dumps}")
 
     def test_the_missing_databases_csv_is_not_reported(self) -> None:

tests/e2e/test_e2e_only_sql_fallback_to_files.py

@@ -1,14 +1,24 @@
 import json
 import unittest
 
-from baudolo.generation import FILES_DIR, MANIFEST_FILE, MANIFEST_SCHEMA, SQL_DIR
+from baudolo.generation import (
+    CLUSTER_SUFFIX,
+    DUMP_COMPRESSION,
+    DUMP_SUFFIX,
+    FILES_DIR,
+    MANIFEST_FILE,
+    MANIFEST_SCHEMA,
+    SQL_DIR,
+)
 
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
+    assert_dumps_in_sees_both_suffixes,
     backup_path,
     cleanup_docker,
     create_minimal_compose_dir,
+    dumps_in,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -143,7 +153,7 @@ class TestE2EOnlySqlFallbackToFiles(unittest.TestCase):
         self.assertTrue(p.is_dir(), f"Expected files backup dir at: {p}")
 
     def test_sql_dump_not_present(self) -> None:
-        # There should be no sql dumps because databases.csv had no matching entry.
+        assert_dumps_in_sees_both_suffixes(self, f"/tmp/{self.prefix}")
         sql_dir = (
             backup_path(
                 self.backups_dir,
@@ -151,16 +161,10 @@ class TestE2EOnlySqlFallbackToFiles(unittest.TestCase):
                 self.version,
                 self.pg_volume,
             )
-            / "sql"
-        )
-        # Could exist (dir created) in some edge cases, but should contain no *.sql dumps.
-        if sql_dir.exists():
-            dumps = list(sql_dir.glob("*.sql"))
-            self.assertEqual(
-                len(dumps),
-                0,
-                f"Did not expect SQL dump files, found: {dumps}",
+            / SQL_DIR
         )
+        dumps = dumps_in(sql_dir)
+        self.assertEqual(dumps, [], f"Did not expect SQL dump files, found: {dumps}")
 
     def manifest(self) -> dict:
         generation = backup_path(
@@ -183,6 +187,9 @@ class TestE2EOnlySqlFallbackToFiles(unittest.TestCase):
         self.assertTrue((volume_dir / layout["files_dir"]).is_dir())
         self.assertEqual(layout["files_dir"], FILES_DIR)
         self.assertEqual(layout["sql_dir"], SQL_DIR)
+        self.assertEqual(layout["dump_suffix"], DUMP_SUFFIX)
+        self.assertEqual(layout["cluster_suffix"], CLUSTER_SUFFIX)
+        self.assertEqual(layout["dump_compression"], DUMP_COMPRESSION)
 
     def test_the_manifest_states_a_schema_a_reader_can_check(self) -> None:
         self.assertEqual(self.manifest()["schema"], MANIFEST_SCHEMA)

tests/e2e/test_e2e_postgres_cluster_restore.py

@@ -1,5 +1,7 @@
 import unittest
 
+from baudolo.generation import CLUSTER_SUFFIX, SQL_DIR
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
@@ -85,37 +87,16 @@ class TestE2EPostgresClusterRestore(unittest.TestCase):
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
         cls.dump = (
             backup_path(cls.backups_dir, cls.repo_name, cls.version, cls.pg_volume)
-            / "sql"
-            / f"{cls.pg_container}.cluster.backup.sql"
+            / SQL_DIR
+            / f"{cls.pg_container}{CLUSTER_SUFFIX}"
         )
 
-        run(
-            [
-                "baudolo-restore",
-                "cluster",
-                cls.pg_volume,
-                cls.hash,
-                cls.version,
-                "--backups-dir",
-                cls.backups_dir,
-                "--repo-name",
-                cls.repo_name,
-                "--container",
-                cls.pg_container,
-                "--instance",
-                cls.pg_container,
-                "--db-user",
-                "postgres",
-                "--db-password",
-                "pgpw",
-                "--empty",
-            ]
-        )
+        run(cls._restore_argv(), stdin_text="pgpw")
 
         for statement in SIBLING_SQL:
             cls._psql("postgres", statement)
         cls._psql("sibling", SIBLING_PAYLOAD)
-        cls.refused = run(cls._restore_argv(), check=False)
+        cls.refused = run(cls._restore_argv(), check=False, stdin_text="pgpw")
 
     @classmethod
     def _restore_argv(cls) -> list:
@@ -135,8 +116,7 @@ class TestE2EPostgresClusterRestore(unittest.TestCase):
             cls.pg_container,
             "--db-user",
             "postgres",
-            "--db-password",
-            "pgpw",
+            "--db-password-stdin",
             "--empty",
         ]
 

tests/e2e/test_e2e_postgres_empty_drop_hard.py

@@ -24,6 +24,7 @@ from .helpers import (
 # is not unique") and english_stem_nostop reproduces taiga's text search
 # dictionary abort (duplicate pg_ts_dict_dictname_index).
 SCENARIO_SQL = (
+    "CREATE EXTENSION pg_trgm;"
     "CREATE SCHEMA discourse_functions;"
     "CREATE TABLE discourse_functions.helper (id int);"
     "INSERT INTO discourse_functions.helper VALUES (1);"
@@ -166,6 +167,13 @@ class TestE2EPostgresEmptyDropHard(unittest.TestCase):
         )
         self.assertEqual(self._scalar("SELECT public.f(41) + public.f();"), "42")
 
+    def test_the_extension_survived_the_preclean(self) -> None:
+        self.assertEqual(
+            self._scalar("SELECT count(*) FROM pg_extension WHERE extname='pg_trgm';"),
+            "1",
+        )
+        self.assertEqual(self._scalar("SELECT similarity('abc','abc')::int;"), "1")
+
     def test_text_search_dictionary_restored_once(self) -> None:
         self.assertEqual(
             self._scalar(

tests/e2e/test_e2e_postgres_full.py

@@ -1,5 +1,7 @@
 import unittest
 
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
@@ -7,6 +9,7 @@ from .helpers import (
     backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -94,7 +97,7 @@ class TestE2EPostgresFull(unittest.TestCase):
         )
 
         # Restore
-        run(
+        cls.restored = run(
             [
                 "baudolo-restore",
                 "postgres",
@@ -124,10 +127,14 @@ class TestE2EPostgresFull(unittest.TestCase):
     def test_dump_file_exists(self) -> None:
         p = (
             backup_path(self.backups_dir, self.repo_name, self.version, self.pg_volume)
-            / "sql"
-            / "appdb.backup.sql"
+            / SQL_DIR
+            / f"appdb{DUMP_SUFFIX}"
         )
         self.assertTrue(p.is_file(), f"Expected dump file at: {p}")
+        self.assertIn("Dumped by pg_dump", dump_text(p))
+
+    def test_the_deprecated_password_argument_still_restores_but_warns(self) -> None:
+        self.assertIn("visible in the process list", self.restored.stderr)
 
     def test_data_restored(self) -> None:
         p = run(

tests/e2e/test_e2e_postgres_password_required.py

@@ -0,0 +1,160 @@
+"""An engine whose loopback auth really demands a password.
+
+Every other Postgres scenario runs stock postgres:alpine, whose generated
+pg_hba grants trust on 127.0.0.1 and ::1 - so `pg_dump -h localhost` never
+needs the password and a dump succeeds whether or not baudolo hands one to the
+container. This module makes the password mandatory, which is what a dedicated
+engine on a real host does.
+"""
+
+import unittest
+from pathlib import Path
+
+from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX, FILES_DIR, SQL_DIR
+
+from .helpers import (
+    POSTGRES_DATA_DIR,
+    POSTGRES_IMAGE,
+    backup_path,
+    backup_run,
+    cleanup_docker,
+    create_minimal_compose_dir,
+    dump_text,
+    ensure_empty_dir,
+    latest_version_dir,
+    require_docker,
+    run,
+    unique,
+    wait_for_postgres,
+    write_databases_csv,
+)
+
+
+class TestE2EPostgresPasswordRequired(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls) -> None:
+        require_docker()
+        cls.prefix = unique("baudolo-e2e-pg-password-required")
+        cls.backups_dir = f"/tmp/{cls.prefix}/Backups"
+        ensure_empty_dir(cls.backups_dir)
+        cls.compose_dir = create_minimal_compose_dir(f"/tmp/{cls.prefix}")
+        cls.repo_name = cls.prefix
+
+        cls.pg_container = f"{cls.prefix}-pg"
+        cls.pg_volume = f"{cls.prefix}-pg-vol"
+        cls.containers = [cls.pg_container]
+        cls.volumes = [cls.pg_volume]
+
+        run(["docker", "volume", "create", cls.pg_volume])
+        run(
+            [
+                "docker",
+                "run",
+                "-d",
+                "--name",
+                cls.pg_container,
+                "-e",
+                "POSTGRES_PASSWORD=pgpw",
+                "-e",
+                "POSTGRES_DB=appdb",
+                "-e",
+                "POSTGRES_USER=postgres",
+                # The entrypoint evals this into its initdb call, so the host
+                # lines of pg_hba demand scram while the local socket stays
+                # trust - the entrypoint's own init and the seeding below keep
+                # working, and only a TCP connection needs the password.
+                "-e",
+                "POSTGRES_INITDB_ARGS=--auth-host=scram-sha-256",
+                "-v",
+                f"{cls.pg_volume}:{POSTGRES_DATA_DIR}",
+                POSTGRES_IMAGE,
+            ]
+        )
+        wait_for_postgres(cls.pg_container, user="postgres", timeout_s=90)
+
+        run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                (
+                    'psql -U postgres -d appdb -c "CREATE TABLE t (id int primary '
+                    "key, v text); INSERT INTO t VALUES (1,'ok');\""
+                ),
+            ],
+            check=True,
+        )
+
+        cls.unauthenticated = run(
+            [
+                "docker",
+                "exec",
+                cls.pg_container,
+                "sh",
+                "-lc",
+                "pg_dump -U postgres -d appdb -h localhost",
+            ],
+            capture=True,
+            check=False,
+        )
+
+        cls.databases_csv = f"/tmp/{cls.prefix}/databases.csv"
+        write_databases_csv(
+            cls.databases_csv,
+            [
+                (cls.pg_container, "appdb", "postgres", "pgpw"),
+                (cls.pg_container, "*", "postgres", "pgpw"),
+            ],
+        )
+
+        backup_run(
+            backups_dir=cls.backups_dir,
+            repo_name=cls.repo_name,
+            compose_dir=cls.compose_dir,
+            databases_csv=cls.databases_csv,
+            database_containers=[cls.pg_container],
+            images_no_stop_required=[POSTGRES_IMAGE],
+            only_sql=True,
+        )
+        cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
+
+    @classmethod
+    def tearDownClass(cls) -> None:
+        cleanup_docker(containers=cls.containers, volumes=cls.volumes)
+
+    def volume_dir(self) -> Path:
+        return backup_path(
+            self.backups_dir, self.repo_name, self.version, self.pg_volume
+        )
+
+    def test_a_dump_without_the_password_is_refused_by_the_server(self) -> None:
+        """Without this the module is vacuous: a pg_hba still saying trust would
+        let a baudolo that forwards nothing pass just as well."""
+        self.assertNotEqual(self.unauthenticated.returncode, 0)
+        self.assertIn("no password supplied", self.unauthenticated.stderr or "")
+
+    def test_the_configured_database_was_dumped(self) -> None:
+        dump = self.volume_dir() / SQL_DIR / f"appdb{DUMP_SUFFIX}"
+        self.assertTrue(dump.is_file(), f"expected a dump at {dump}")
+        self.assertIn("Dumped by pg_dump", dump_text(dump))
+
+    def test_the_dump_carries_the_payload(self) -> None:
+        """pg_dump emits table data as COPY ... FROM stdin, so the row reads as
+        tab-separated values rather than as an INSERT literal."""
+        text = dump_text(self.volume_dir() / SQL_DIR / f"appdb{DUMP_SUFFIX}")
+        self.assertIn("COPY public.t (id, v) FROM stdin;", text)
+        self.assertIn("1\tok", text)
+
+    def test_the_cluster_row_was_dumped_too(self) -> None:
+        cluster = self.volume_dir() / SQL_DIR / f"{self.pg_container}{CLUSTER_SUFFIX}"
+        self.assertTrue(cluster.is_file(), f"expected a cluster dump at {cluster}")
+        self.assertIn("CREATE DATABASE", dump_text(cluster))
+
+    def test_only_sql_left_no_file_copy_behind(self) -> None:
+        self.assertFalse((self.volume_dir() / FILES_DIR).exists())
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/e2e/test_e2e_seed_star_and_db_entries_backup_postgres.py

@@ -1,11 +1,14 @@
 import unittest
 
+from baudolo.generation import CLUSTER_SUFFIX, DUMP_SUFFIX
+
 from .helpers import (
     POSTGRES_DATA_DIR,
     POSTGRES_IMAGE,
     backup_path,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -197,15 +200,15 @@ class TestE2ESeedStarAndDbEntriesBackupPostgres(unittest.TestCase):
         )
 
         # Cluster dump file produced by '*' entry
-        cluster = sql_dir / f"{self.pg_container}.cluster.backup.sql"
+        cluster = sql_dir / f"{self.pg_container}{CLUSTER_SUFFIX}"
         self.assertTrue(cluster.is_file(), f"Expected cluster dump file at: {cluster}")
 
         # Concrete DB dump produced by normal entry
-        db1 = sql_dir / f"{self.pg_db1}.backup.sql"
+        db1 = sql_dir / f"{self.pg_db1}{DUMP_SUFFIX}"
         self.assertTrue(db1.is_file(), f"Expected db dump file at: {db1}")
 
         # Basic sanity: cluster dump usually contains CREATE DATABASE statements
-        txt = cluster.read_text(encoding="utf-8", errors="ignore")
+        txt = dump_text(cluster)
         self.assertIn(
             "CREATE DATABASE",
             txt,

tests/e2e/test_e2e_snapshot.py

@@ -36,9 +36,14 @@ ZFS_READY = "{ [ -c /dev/zfs ] || modprobe zfs 2>/dev/null; }; [ -c /dev/zfs ]"
 def mount_script(fstype: str) -> str:
     """Build a filesystem on a loop device and carve out the snapshot subject."""
     if fstype == "zfs":
+        # Exception: a loop device, not the /img file, because the kernel opens
+        # vdevs from a taskq thread that resolves paths in the host's root.
+        # The pool is kernel-wide, so it is destroyed on exit, but only once
+        # this script created it.
         return (
-            f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img "
-            "&& zpool create -m none baudolo /img "
+            f"{PACKAGES} && {ZFS_READY} && truncate -s 400M /img && {ATTACH} "
+            '&& zpool create -m none baudolo "$LOOP" '
+            "&& trap 'zpool destroy -f baudolo; losetup -d \"$LOOP\"' EXIT "
             "&& zfs create -o mountpoint=/subject/docker baudolo/docker"
         )
     return (

tests/e2e/test_e2e_version_gate.py

@@ -10,6 +10,8 @@ import re
 import unittest
 from pathlib import Path
 
+from baudolo.generation import DUMP_SUFFIX, SQL_DIR
+
 from .helpers import (
     MARIADB_DATA_DIR,
     MARIADB_IMAGE,
@@ -19,6 +21,7 @@ from .helpers import (
     backup_run,
     cleanup_docker,
     create_minimal_compose_dir,
+    dump_text,
     ensure_empty_dir,
     latest_version_dir,
     require_docker,
@@ -28,6 +31,7 @@ from .helpers import (
     wait_for_mariadb_sql,
     wait_for_postgres,
     write_databases_csv,
+    write_dump_text,
 )
 
 PAYLOAD = "gate-payload"
@@ -36,14 +40,14 @@ FUTURE = "99.0"
 
 def rewrite_version(dump: Path, pattern: str, version: str) -> str:
     """Make the dump claim ``version``; return what it claimed before."""
-    text = dump.read_text(encoding="utf-8", errors="replace")
+    text = dump_text(dump)
     found = re.search(pattern, text)
     if not found:
         raise AssertionError(f"{dump} carries no version header matching {pattern}")
     claimed = found.group(1)
-    dump.write_text(
+    write_dump_text(
+        dump,
         text.replace(found.group(0), found.group(0).replace(claimed, version), 1),
-        encoding="utf-8",
     )
     return claimed
 
@@ -73,12 +77,12 @@ class GateCase:
                 cls.db_name,
                 "--db-user",
                 cls.db_user,
-                "--db-password",
-                cls.db_password,
+                "--db-password-stdin",
                 "--empty",
                 *extra,
             ],
             check=False,
+            stdin_text=cls.db_password,
         )
 
     @classmethod
@@ -103,8 +107,8 @@ class GateCase:
         cls.hash, cls.version = latest_version_dir(cls.backups_dir, cls.repo_name)
         cls.dump = (
             backup_path(cls.backups_dir, cls.repo_name, cls.version, cls.volume)
-            / "sql"
-            / f"{cls.db_name}.backup.sql"
+            / SQL_DIR
+            / f"{cls.db_name}{DUMP_SUFFIX}"
         )
 
         cls.truthful_version = rewrite_version(cls.dump, cls.pattern, FUTURE)

tests/integration/test_seed_integration.py

@@ -65,6 +65,32 @@ def run_seed_expect_fail(
     )
 
 
+def run_seed_stdin(
+    csv_path: Path, instance: str, database: str, username: str, stdin: str, *extra: str
+) -> subprocess.CompletedProcess:
+    """
+    Run the real CLI with --password-stdin, feeding the password on stdin.
+    Any extra arguments are appended after the username.
+    """
+    return subprocess.run(
+        [
+            sys.executable,
+            "-m",
+            "baudolo.seed",
+            "--password-stdin",
+            str(csv_path),
+            instance,
+            database,
+            username,
+            *extra,
+        ],
+        input=stdin,
+        text=True,
+        capture_output=True,
+        check=False,
+    )
+
+
 def read_csv_semicolon(path: Path) -> list[dict]:
     with path.open("r", encoding="utf-8", newline="") as f:
         reader = csv.DictReader(f, delimiter=";")
@@ -195,6 +221,61 @@ class TestSeedIntegration(unittest.TestCase):
             self.assertEqual(len(rows), 1)
             self.assertEqual(rows[0]["database"], "my_db-1")
 
+    def test_password_stdin_stores_the_password_without_its_trailing_newline(
+        self,
+    ) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed_stdin(p, "docker.test", "appdb", "alice", "s3cr;et\n")
+
+            self.assertEqual(cp.returncode, 0, cp.stderr)
+            self.assertNotIn("WARNING", cp.stderr)
+            rows = read_csv_semicolon(p)
+            self.assertEqual(len(rows), 1)
+            self.assertEqual(rows[0]["username"], "alice")
+            self.assertEqual(rows[0]["password"], "s3cr;et")
+
+    def test_password_argument_still_works_with_a_deprecation_warning(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed(p, "docker.test", "appdb", "alice", "secret")
+
+            self.assertIn("--password-stdin", cp.stderr)
+            self.assertEqual(read_csv_semicolon(p)[0]["password"], "secret")
+
+    def test_password_given_both_ways_is_rejected(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = run_seed_stdin(p, "docker.test", "appdb", "alice", "pw\n", "pw")
+
+            self.assertNotEqual(cp.returncode, 0)
+            self.assertFalse(p.exists(), "Should not create file on invalid input")
+
+    def test_password_given_no_way_is_rejected(self) -> None:
+        with tempfile.TemporaryDirectory() as td:
+            p = Path(td) / "databases.csv"
+
+            cp = subprocess.run(
+                [
+                    sys.executable,
+                    "-m",
+                    "baudolo.seed",
+                    str(p),
+                    "docker.test",
+                    "appdb",
+                    "alice",
+                ],
+                text=True,
+                capture_output=True,
+                check=False,
+            )
+
+            self.assertNotEqual(cp.returncode, 0)
+            self.assertFalse(p.exists(), "Should not create file on invalid input")
+
     def test_file_is_semicolon_delimited_and_has_header(self) -> None:
         with tempfile.TemporaryDirectory() as td:
             p = Path(td) / "databases.csv"

tests/unit/backup/test_app_live_precopy.py

@@ -0,0 +1,70 @@
+"""Contract of app.main's live copy when no snapshot is taken."""
+
+from __future__ import annotations
+
+import unittest
+from unittest import mock
+
+from baudolo.backup import app
+from baudolo.backup.shell import BackupError
+from baudolo.backup.volume import Backing
+
+from . import BASE_ARGV
+
+
+def drive(*, stop: bool, fail_live: bool) -> list[str]:
+    events: list[str] = []
+
+    def record(versions_dir, volume_name, volume_dir, *, authoritative, source):
+        events.append("authoritative" if authoritative else "live")
+        if fail_live and not authoritative:
+            raise BackupError("rsync exit code 23")
+
+    with (
+        mock.patch("sys.argv", BASE_ARGV),
+        mock.patch.object(app, "get_machine_id", return_value="machine"),
+        mock.patch.object(app, "create_version_directory", return_value="/gen"),
+        mock.patch.object(app, "create_volume_directory", return_value="/gen/vol"),
+        mock.patch.object(app, "load_databases_df", return_value=None),
+        mock.patch.object(app, "docker_volume_names", return_value=["vol"]),
+        mock.patch.object(app, "containers_using_volume", return_value=["c"]),
+        mock.patch.object(app, "volume_is_fully_ignored", return_value=False),
+        mock.patch.object(app, "backup_dumps_for_volume", return_value=(False, False)),
+        mock.patch.object(
+            app,
+            "inspect_backing",
+            return_value=Backing("/var/lib/docker/volumes/vol/_data"),
+        ),
+        mock.patch.object(app, "requires_stop", return_value=stop),
+        mock.patch.object(app, "filter_stoppable", return_value=["c"]),
+        mock.patch.object(
+            app,
+            "change_containers_status",
+            side_effect=lambda containers, status: events.append(status),
+        ),
+        mock.patch.object(app, "write_manifest"),
+        mock.patch.object(app, "stamp_directory"),
+        mock.patch.object(app, "handle_docker_compose_services"),
+        mock.patch.object(app, "backup_volume", side_effect=record),
+    ):
+        app.main()
+    return events
+
+
+class TestLivePreCopy(unittest.TestCase):
+    def test_a_failed_pre_copy_is_replaced_by_the_stopped_copy(self) -> None:
+        self.assertEqual(
+            drive(stop=True, fail_live=True),
+            ["live", "stop", "authoritative", "start"],
+        )
+
+    def test_a_failed_live_copy_without_a_stop_aborts_the_run(self) -> None:
+        with self.assertRaises(BackupError):
+            drive(stop=False, fail_live=True)
+
+    def test_a_volume_without_a_stop_is_copied_live_once(self) -> None:
+        self.assertEqual(drive(stop=False, fail_live=False), ["live"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_db_mariadb_dump.py

@@ -16,11 +16,11 @@ def _capture_dumps(*, db_type, rows, container, dump_tool="mariadb-dump"):
     captured = []
 
     def _capture(command, out_file, *, env=None):
-        captured.append((list(command), env))
+        captured.append((list(command), env, out_file))
 
     with (
         tempfile.TemporaryDirectory() as td,
-        patch.object(db_mod, "execute_to_file", side_effect=_capture),
+        patch.object(db_mod, "execute_to_gzip", side_effect=_capture),
     ):
         db_mod.backup_database(
             container=container,
@@ -46,14 +46,13 @@ class TestMariaDBDumpUsesTCP(unittest.TestCase):
         )
         self.assertEqual(len(captured), 1, f"expected one dump, got: {captured}")
 
-        argv, env = captured[0]
-        self.assertEqual(argv[:3], ["docker", "exec", "mariadb"])
+        argv, _env, _out = captured[0]
+        self.assertEqual(argv[:2], ["docker", "exec"])
+        self.assertEqual(argv[argv.index(dump_tool_of(argv)) - 1], "mariadb")
         self.assertIn("--protocol=tcp", argv)
         self.assertEqual(argv[argv.index("-h") + 1], "127.0.0.1")
         self.assertEqual(argv[argv.index("-u") + 1], "appuser")
-        self.assertIn("-ps3cret", argv)
         self.assertEqual(argv[-1], "appdb")
-        self.assertIsNone(env)
 
     def test_the_probed_client_is_the_one_invoked(self):
         captured = _capture_dumps(
@@ -62,7 +61,7 @@ class TestMariaDBDumpUsesTCP(unittest.TestCase):
             container="mariadb",
             dump_tool="mysqldump",
         )
-        argv, _env = captured[0]
+        argv, _env, _out = captured[0]
         self.assertIn("mysqldump", argv)
         self.assertNotIn("mariadb-dump", argv)
 
@@ -72,20 +71,46 @@ class TestMariaDBDumpUsesTCP(unittest.TestCase):
             rows=[("pg", "appdb", "appuser", "s3cret")],
             container="pg",
         )
-        argv, _env = captured[0]
+        argv, _env, _out = captured[0]
         self.assertIn("pg_dump", argv)
         self.assertNotIn("--protocol=tcp", argv)
 
     def test_the_password_travels_in_the_environment_not_the_argv(self):
-        """A process listing shows argv; PGPASSWORD must not be in it."""
-        captured = _capture_dumps(
-            db_type="postgres",
-            rows=[("pg", "appdb", "appuser", "s3cret")],
-            container="pg",
+        for db_type, container, variable in (
+            ("postgres", "pg", "PGPASSWORD"),
+            ("mariadb", "mariadb", "MYSQL_PWD"),
+        ):
+            with self.subTest(engine=db_type):
+                argv, env, _out = _capture_dumps(
+                    db_type=db_type,
+                    rows=[(container, "appdb", "appuser", "s3cret")],
+                    container=container,
+                )[0]
+                self.assertEqual(env, {variable: "s3cret"})
+                self.assertEqual(argv[argv.index("-e") + 1], variable)
+                self.assertLess(argv.index("-e"), argv.index(container))
+                self.assertFalse(
+                    [arg for arg in argv if "s3cret" in arg],
+                    f"password reached the argv: {argv}",
                 )
-        argv, env = captured[0]
-        self.assertEqual(env, {"PGPASSWORD": "s3cret"})
-        self.assertNotIn("s3cret", argv)
+
+    def test_every_dump_is_written_gzip_named(self):
+        for db_type, container, database, suffix in (
+            ("postgres", "pg", "appdb", ".backup.sql.gz"),
+            ("mariadb", "mariadb", "appdb", ".backup.sql.gz"),
+            ("postgres", "pg", "*", ".cluster.backup.sql.gz"),
+        ):
+            with self.subTest(engine=db_type, database=database):
+                _argv, _env, out = _capture_dumps(
+                    db_type=db_type,
+                    rows=[(container, database, "appuser", "s3cret")],
+                    container=container,
+                )[0]
+                self.assertTrue(out.endswith(suffix), out)
+
+
+def dump_tool_of(argv):
+    return next(arg for arg in argv if arg in ("mariadb-dump", "mysqldump"))
 
 
 class TestNoShellReachesTheDump(unittest.TestCase):

tests/unit/backup/test_docker_exec_env.py

@@ -0,0 +1,61 @@
+"""How a secret reaches the command running inside the container."""
+
+from __future__ import annotations
+
+import unittest
+
+from baudolo.backup.db import fallback_pg_dumpall
+from baudolo.backup.docker import docker_exec_argv
+
+
+class TestForwardEnv(unittest.TestCase):
+    def test_nothing_is_added_when_no_variable_is_named(self) -> None:
+        self.assertEqual(
+            docker_exec_argv("c1", ["true"]),
+            ["docker", "exec", "c1", "true"],
+        )
+
+    def test_a_named_variable_is_forwarded_without_its_value(self) -> None:
+        """-e NAME=value would publish the secret in the host's process list."""
+        argv = docker_exec_argv("c1", ["true"], forward_env=["PGPASSWORD"])
+        self.assertEqual(argv, ["docker", "exec", "-e", "PGPASSWORD", "c1", "true"])
+
+    def test_the_flag_precedes_the_container(self) -> None:
+        """docker reads options before the container name, arguments after it."""
+        argv = docker_exec_argv(
+            "c1", ["pg_dump", "-U", "u"], interactive=True, forward_env=["PGPASSWORD"]
+        )
+        self.assertLess(argv.index("-e"), argv.index("c1"))
+        self.assertLess(argv.index("-i"), argv.index("c1"))
+        self.assertGreater(argv.index("pg_dump"), argv.index("c1"))
+
+    def test_several_variables_each_get_their_own_flag(self) -> None:
+        argv = docker_exec_argv("c1", ["true"], forward_env=["A", "B"])
+        self.assertEqual(argv[:6], ["docker", "exec", "-e", "A", "-e", "B"])
+
+
+class TestPostgresDumpCarriesThePassword(unittest.TestCase):
+    def test_the_cluster_dump_forwards_pgpassword(self) -> None:
+        seen: dict = {}
+
+        def fake(command, out_file, *, env=None):
+            seen["command"] = command
+            seen["env"] = env
+
+        import baudolo.backup.db as db
+
+        original = db.execute_to_gzip
+        db.execute_to_gzip = fake
+        try:
+            fallback_pg_dumpall("pg", "user", "secret", "/tmp/out.sql")
+        finally:
+            db.execute_to_gzip = original
+
+        self.assertIn("-e", seen["command"])
+        self.assertEqual(seen["command"][seen["command"].index("-e") + 1], "PGPASSWORD")
+        self.assertEqual(seen["env"], {"PGPASSWORD": "secret"})
+        self.assertFalse([arg for arg in seen["command"] if "secret" in arg])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_get_instance.py

@@ -0,0 +1,80 @@
+"""Which databases.csv instance a container name resolves to.
+
+The cases are the container names real deployments produce, in both compose
+and swarm, so a change to the regex has to state which shape it gives up.
+"""
+
+from __future__ import annotations
+
+import unittest
+
+from baudolo.backup.db import get_instance
+
+
+class TestDeclaredContainers(unittest.TestCase):
+    def test_a_declared_container_is_its_own_instance(self) -> None:
+        self.assertEqual(
+            get_instance("postgres-central", ["postgres-central"]), "postgres-central"
+        )
+
+    def test_a_declaration_beats_the_regex(self) -> None:
+        """A declared name is taken whole even when it carries a token the
+        fallback would otherwise strip."""
+        self.assertEqual(
+            get_instance("shop-database", ["shop-database"]), "shop-database"
+        )
+
+    def test_a_qualified_central_name_still_has_to_be_declared(self) -> None:
+        self.assertIsNone(get_instance("postgres-central", []))
+
+
+class TestContainersNamedAfterTheirEngine(unittest.TestCase):
+    def test_a_bare_engine_name_is_its_own_instance(self) -> None:
+        for name in ("postgres", "mariadb", "mysql", "db", "database"):
+            with self.subTest(container=name):
+                self.assertEqual(get_instance(name, []), name)
+
+    def test_a_swarm_task_of_such_a_container_keeps_the_instance(self) -> None:
+        self.assertEqual(get_instance("postgres_postgres.1.k3f9x2", []), "postgres")
+        self.assertEqual(get_instance("mariadb_mariadb.1.k3f9x2", []), "mariadb")
+
+
+class TestDedicatedEngines(unittest.TestCase):
+    def test_compose_names_the_container_with_a_hyphen(self) -> None:
+        self.assertEqual(get_instance("discourse-database", []), "discourse")
+
+    def test_swarm_names_the_task_with_an_underscore_and_a_slot(self) -> None:
+        """Swarm suppresses container_name and names the task
+        <stack>_<service>.<slot>.<id>, which must land on the same instance as
+        the compose name so one databases.csv serves both modes."""
+        self.assertEqual(get_instance("discourse_database.1.k3f9x2", []), "discourse")
+
+    def test_an_explicitly_named_engine_keeps_its_entity(self) -> None:
+        self.assertEqual(get_instance("bigbluebutton-postgres-1", []), "bigbluebutton")
+
+    def test_the_short_token_is_stripped_too(self) -> None:
+        self.assertEqual(get_instance("matomo-db", []), "matomo")
+
+    def test_mariadb_uses_the_same_suffix(self) -> None:
+        self.assertEqual(get_instance("matomo-database", []), "matomo")
+
+    def test_an_engine_named_suffix_is_stripped_too(self) -> None:
+        self.assertEqual(get_instance("shop-mariadb", []), "shop")
+        self.assertEqual(get_instance("shop-mysql", []), "shop")
+
+
+class TestApplicationContainers(unittest.TestCase):
+    def test_a_bare_application_name_is_not_a_database(self) -> None:
+        """Returning the name unchanged here would offer the application as a
+        second engine for its own dedicated database's instance."""
+        self.assertIsNone(get_instance("discourse", []))
+
+    def test_a_swarm_application_task_is_not_a_database(self) -> None:
+        self.assertIsNone(get_instance("discourse_discourse.1.k3f9x2", []))
+
+    def test_an_application_that_merely_starts_with_a_token_is_not_split(self) -> None:
+        self.assertIsNone(get_instance("dbeaver", []))
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_shell_gzip.py

@@ -0,0 +1,126 @@
+import contextlib
+import gzip
+import io
+import sys
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+from baudolo.backup import shell
+from baudolo.backup.shell import BackupError, execute_to_gzip
+
+CAT = (
+    "import shutil, sys; shutil.copyfileobj(open(sys.argv[1], 'rb'), sys.stdout.buffer)"
+)
+
+
+def sql_lines(count: int, changed: int = -1) -> bytes:
+    return b"".join(
+        (
+            b"INSERT INTO t VALUES (%d, 'changed');\n" % i
+            if i == changed
+            else b"INSERT INTO t VALUES (%d, 'name-%d');\n" % (i, i * 7919)
+        )
+        for i in range(count)
+    )
+
+
+def common_suffix(a: bytes, b: bytes) -> int:
+    n = 0
+    while n < min(len(a), len(b)) and a[-1 - n] == b[-1 - n]:
+        n += 1
+    return n
+
+
+class GzipDumpCase(unittest.TestCase):
+    def setUp(self):
+        self.dir = Path(tempfile.mkdtemp())
+
+    def dump(self, payload: bytes, name: str = "db.backup.sql.gz") -> bytes:
+        src = self.dir / f"{name}.src"
+        src.write_bytes(payload)
+        out = self.dir / name
+        with contextlib.redirect_stdout(io.StringIO()):
+            execute_to_gzip([sys.executable, "-c", CAT, str(src)], str(out))
+        return out.read_bytes()
+
+
+class TestRoundTrip(GzipDumpCase):
+    def test_the_dump_round_trips_through_gzip(self):
+        payload = sql_lines(5000) + b"-- " + b"x" * (3 * shell.READ_BYTES) + b"\n"
+        packed = self.dump(payload)
+        self.assertEqual(packed[:2], b"\x1f\x8b")
+        self.assertEqual(gzip.decompress(packed), payload)
+
+    def test_an_empty_dump_is_a_valid_empty_gzip(self):
+        self.assertEqual(gzip.decompress(self.dump(b"")), b"")
+
+    def test_no_temporary_file_is_left_behind(self):
+        self.dump(sql_lines(10))
+        self.assertEqual(sorted(p.suffix for p in self.dir.iterdir()), [".gz", ".src"])
+
+
+class TestDeterminism(GzipDumpCase):
+    def test_identical_dumps_give_identical_bytes(self):
+        payload = sql_lines(20000)
+        self.assertEqual(self.dump(payload, "a.gz"), self.dump(payload, "b.gz"))
+
+    def test_the_header_carries_neither_a_name_nor_a_time(self):
+        packed = self.dump(sql_lines(10))
+        self.assertEqual(packed[3] & 0x08, 0, "FNAME flag set")
+        self.assertEqual(packed[4:8], b"\x00\x00\x00\x00", "MTIME set")
+
+
+class TestResync(GzipDumpCase):
+    def test_a_change_near_the_start_leaves_the_tail_identical(self):
+        before = sql_lines(100000)
+        after = sql_lines(100000, changed=5)
+        ours = common_suffix(
+            self.dump(before, "a.gz")[:-8], self.dump(after, "b.gz")[:-8]
+        )
+        plain = common_suffix(gzip.compress(before)[:-8], gzip.compress(after)[:-8])
+        self.assertLess(plain, 1024, "control: plain gzip must not resync")
+        self.assertGreater(ours, len(gzip.compress(before)) // 2)
+
+
+class TestFailure(GzipDumpCase):
+    def test_a_failing_command_leaves_no_file(self):
+        out = self.dir / "db.backup.sql.gz"
+        script = "import sys; sys.stdout.write('partial'); sys.exit(3)"
+        with (
+            contextlib.redirect_stdout(io.StringIO()),
+            self.assertRaises(BackupError) as caught,
+        ):
+            execute_to_gzip([sys.executable, "-c", script], str(out))
+        self.assertIn("Exit code: 3", str(caught.exception))
+        self.assertEqual(list(self.dir.iterdir()), [])
+
+    def test_a_write_error_kills_the_dump_and_leaves_no_file(self):
+        src = self.dir / "src"
+        src.write_bytes(sql_lines(200000))
+        out = self.dir / "db.backup.sql.gz"
+        with (
+            patch.object(
+                gzip.GzipFile, "write", side_effect=OSError(28, "No space left")
+            ),
+            contextlib.redirect_stdout(io.StringIO()),
+            self.assertRaises(BackupError),
+        ):
+            execute_to_gzip([sys.executable, "-c", CAT, str(src)], str(out))
+        self.assertEqual(sorted(p.name for p in self.dir.iterdir()), ["src"])
+
+    def test_env_reaches_the_child_without_the_argv(self):
+        out = self.dir / "env.gz"
+        script = "import os, sys; sys.stdout.write(os.environ['MYSQL_PWD'])"
+        printed = io.StringIO()
+        with contextlib.redirect_stdout(printed):
+            execute_to_gzip(
+                [sys.executable, "-c", script], str(out), env={"MYSQL_PWD": "s3cret"}
+            )
+        self.assertEqual(gzip.decompress(out.read_bytes()), b"s3cret")
+        self.assertNotIn("s3cret", printed.getvalue())
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/backup/test_snapshot.py

@@ -21,6 +21,14 @@ class Runner:
         return []
 
 
+class Unmountable(Runner):
+    def __call__(self, command: list[str]) -> list[str]:
+        result = super().__call__(command)
+        if command[0] == "mount":
+            raise BackupError("permission denied")
+        return result
+
+
 class TestBtrfs(unittest.TestCase):
     def test_it_creates_a_read_only_snapshot_inside_the_subject(self) -> None:
         run = Runner()
@@ -95,12 +103,55 @@ class TestZfs(unittest.TestCase):
             run.calls[-1], ["zfs", "destroy", "tank/docker@baudolo-20260731"]
         )
 
-    def test_it_maps_a_volume_path_through_the_dot_zfs_directory(self) -> None:
+    def test_it_mounts_the_snapshot_inside_the_subject(self) -> None:
+        run = self._run()
+        with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertIn(
+            [
+                "mount",
+                "-t",
+                "zfs",
+                "tank/docker@baudolo-20260731",
+                "/var/lib/docker/.baudolo-20260731",
+            ],
+            run.calls,
+        )
+
+    def test_it_maps_a_volume_path_into_the_mounted_snapshot(self) -> None:
         run = self._run()
         with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run) as resolve:
             self.assertEqual(
                 resolve("/var/lib/docker/volumes/postgres_data/_data"),
-                "/var/lib/docker/.zfs/snapshot/baudolo-20260731/volumes/postgres_data/_data",
+                "/var/lib/docker/.baudolo-20260731/volumes/postgres_data/_data",
+            )
+
+    def test_it_unmounts_before_it_destroys(self) -> None:
+        run = self._run()
+        with volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run):
+            pass
+        self.assertEqual(
+            run.calls[-3:],
+            [
+                ["umount", "/var/lib/docker/.baudolo-20260731"],
+                ["rmdir", "/var/lib/docker/.baudolo-20260731"],
+                ["zfs", "destroy", "tank/docker@baudolo-20260731"],
+            ],
+        )
+
+    def test_a_failed_mount_leaves_no_snapshot_behind(self) -> None:
+        run = Unmountable({"zfs list": ["tank/docker"]})
+        with (
+            self.assertRaises(BackupError),
+            volume_snapshot("zfs", "/var/lib/docker", "20260731", run=run),
+        ):
+            pass
+        self.assertEqual(
+            run.calls[-2:],
+            [
+                ["rmdir", "/var/lib/docker/.baudolo-20260731"],
+                ["zfs", "destroy", "tank/docker@baudolo-20260731"],
+            ],
         )
 
     def test_an_unmounted_dataset_is_an_error(self) -> None:

tests/unit/backup/test_volume.py

@@ -46,9 +46,12 @@ class TestBackupVolume(unittest.TestCase):
     def test_it_carries_no_kernel_objects_into_a_generation(self) -> None:
         self.assertIn("--no-D", self.copy())
 
+    def test_it_stores_the_metadata_a_restore_has_to_give_back(self) -> None:
+        self.assertIn("-aHAX", self.copy())
+
     def test_it_keeps_no_twin_of_what_the_second_pass_replaces(self) -> None:
         command = self.copy(authoritative=True)
-        self.assertEqual(command[:2], ["rsync", "-aP"])
+        self.assertEqual(command[:3], ["rsync", "-aHAX", "-P"])
         self.assertNotIn("--backup", command)
 
     def test_it_creates_the_destination(self) -> None:

tests/unit/restore/test_cli_password_stdin.py

@@ -0,0 +1,66 @@
+import contextlib
+import io
+import unittest
+from unittest.mock import patch
+
+from baudolo.restore import __main__ as cli
+
+BASE = [
+    "postgres",
+    "app_vol",
+    "hash",
+    "20260817000000",
+    "--repo-name",
+    "repo",
+    "--container",
+    "db",
+    "--db-name",
+    "app",
+]
+
+
+class TestRestorePasswordChannel(unittest.TestCase):
+    def call(self, extra, stdin=""):
+        stderr = io.StringIO()
+        with (
+            patch.object(cli, "restore_postgres_sql") as restore,
+            patch.object(cli, "locate", side_effect=lambda path: path),
+            patch("sys.stdin", io.StringIO(stdin)),
+            contextlib.redirect_stderr(stderr),
+        ):
+            code = cli.main([*BASE, *extra])
+        return code, restore, stderr.getvalue()
+
+    def test_stdin_carries_the_password_without_its_newline(self):
+        code, restore, stderr = self.call(["--db-password-stdin"], stdin="s3cret\n")
+        self.assertEqual(code, 0)
+        self.assertEqual(restore.call_args.kwargs["password"], "s3cret")
+        self.assertNotIn("WARNING", stderr)
+
+    def test_the_argument_still_works_but_warns(self):
+        code, restore, stderr = self.call(["--db-password", "s3cret"])
+        self.assertEqual(code, 0)
+        self.assertEqual(restore.call_args.kwargs["password"], "s3cret")
+        self.assertIn("visible in the process list", stderr)
+
+    def test_both_ways_is_a_usage_error(self):
+        with self.assertRaises(SystemExit) as caught:
+            self.call(["--db-password", "x", "--db-password-stdin"], stdin="y")
+        self.assertEqual(caught.exception.code, 2)
+
+    def test_neither_way_is_a_usage_error(self):
+        with self.assertRaises(SystemExit) as caught:
+            self.call([])
+        self.assertEqual(caught.exception.code, 2)
+
+    def test_files_needs_no_password(self):
+        with patch.object(cli, "restore_volume_files", return_value=0) as restore:
+            code = cli.main(
+                ["files", "app_vol", "hash", "20260817000000", "--repo-name", "repo"]
+            )
+        self.assertEqual(code, 0)
+        restore.assert_called_once()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_cli_version_flag.py

@@ -27,7 +27,10 @@ class TestVersionFlagReachesEveryEngine(unittest.TestCase):
             *required,
             *extra,
         ]
-        with patch.object(cli, target) as restore:
+        with (
+            patch.object(cli, target) as restore,
+            patch.object(cli, "locate", side_effect=lambda path: path),
+        ):
             self.assertEqual(cli.main(argv), 0)
         return restore.call_args.kwargs
 

tests/unit/restore/test_cluster_replay.py

@@ -117,7 +117,7 @@ class TestClusterReplay(unittest.TestCase):
         paths = BackupPaths("vol", "hash", "v1", repo_name="repo", backups_dir="/B")
         self.assertEqual(
             paths.cluster_file("bigbluebutton"),
-            "/B/hash/repo/v1/vol/sql/bigbluebutton.cluster.backup.sql",
+            "/B/hash/repo/v1/vol/sql/bigbluebutton.cluster.backup.sql.gz",
         )
 
 

tests/unit/restore/test_compressed_restore.py

@@ -0,0 +1,242 @@
+import gzip
+import sys
+import tempfile
+import unittest
+import zlib
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from baudolo.restore import run as run_mod
+from baudolo.restore.db import cluster as cluster_mod
+from baudolo.restore.db import mariadb as mdb_mod
+from baudolo.restore.db import postgres as pg_mod
+from baudolo.restore.db import version as ver
+from baudolo.restore.dumpfile import is_gzip, locate, open_dump, verify
+from baudolo.restore.paths import BackupPaths
+
+from .test_version_gate import MARIADB_HEADER, POSTGRES_HEADER, cluster_header
+
+SQL = (
+    POSTGRES_HEADER + "CREATE TABLE t (id int);\nINSERT INTO t VALUES (1);\n"
+).encode()
+
+
+def write(directory: Path, name: str, data: bytes) -> str:
+    path = directory / name
+    path.write_bytes(data)
+    return str(path)
+
+
+class DumpCase(unittest.TestCase):
+    def setUp(self):
+        self.dir = Path(tempfile.mkdtemp())
+
+
+class TestOpenDump(DumpCase):
+    def test_gzip_and_plain_read_the_same_sql(self):
+        packed = write(self.dir, "a.backup.sql.gz", gzip.compress(SQL))
+        plain = write(self.dir, "b.backup.sql", SQL)
+        for path in (packed, plain):
+            with self.subTest(path=path), open_dump(path) as handle:
+                self.assertEqual(handle.read(), SQL)
+
+    def test_the_magic_decides_not_the_name(self):
+        misnamed = write(self.dir, "c.backup.sql", gzip.compress(SQL))
+        self.assertTrue(is_gzip(misnamed))
+        with open_dump(misnamed) as handle:
+            self.assertEqual(handle.read(), SQL)
+
+
+class TestLocate(DumpCase):
+    def test_the_compressed_name_wins_when_alone(self):
+        path = write(self.dir, "db.backup.sql.gz", gzip.compress(SQL))
+        self.assertEqual(locate(path), path)
+
+    def test_an_old_uncompressed_dump_is_found(self):
+        legacy = write(self.dir, "db.backup.sql", SQL)
+        self.assertEqual(locate(str(self.dir / "db.backup.sql.gz")), legacy)
+
+    def test_both_present_is_refused(self):
+        write(self.dir, "db.backup.sql", SQL)
+        path = write(self.dir, "db.backup.sql.gz", gzip.compress(SQL))
+        with self.assertRaises(FileExistsError):
+            locate(path)
+
+    def test_neither_present_names_both(self):
+        with self.assertRaises(FileNotFoundError) as caught:
+            locate(str(self.dir / "db.backup.sql.gz"))
+        self.assertIn("db.backup.sql.gz", str(caught.exception))
+        self.assertIn("db.backup.sql ", str(caught.exception) + " ")
+
+    def test_the_path_helper_names_what_this_version_writes(self):
+        paths = BackupPaths("vol", "h", "v", repo_name="r", backups_dir="/B")
+        self.assertEqual(paths.sql_file("app"), "/B/h/r/v/vol/sql/app.backup.sql.gz")
+
+
+class TestVerify(DumpCase):
+    def test_a_sound_gzip_and_a_plain_file_pass(self):
+        verify(write(self.dir, "a.gz", gzip.compress(SQL)))
+        verify(write(self.dir, "b.sql", SQL[:10]))
+
+    def test_a_truncated_gzip_is_refused(self):
+        packed = gzip.compress(SQL * 50)
+        with self.assertRaises(EOFError):
+            verify(write(self.dir, "t.gz", packed[: len(packed) // 2]))
+
+    def test_a_corrupt_gzip_is_refused(self):
+        packed = bytearray(gzip.compress(SQL * 50))
+        packed[-6] ^= 0xFF
+        with self.assertRaises((gzip.BadGzipFile, zlib.error, EOFError)):
+            verify(write(self.dir, "c.gz", bytes(packed)))
+
+
+class TestReadersAcceptGzip(DumpCase):
+    def test_the_version_header_is_read_through_gzip(self):
+        for header, engine, expected in (
+            (POSTGRES_HEADER, "postgres", "17.11"),
+            (MARIADB_HEADER, "mariadb", "11.8.8-MariaDB-ubu2404"),
+        ):
+            with self.subTest(engine=engine):
+                path = write(self.dir, f"{engine}.gz", gzip.compress(header.encode()))
+                self.assertEqual(ver.dump_version(path, engine), expected)
+
+    def test_the_cluster_inventory_is_read_through_gzip(self):
+        data = (cluster_header(roles=2) + "CREATE DATABASE app2;\n").encode()
+        path = write(self.dir, "c.cluster.backup.sql.gz", gzip.compress(data))
+        databases, roles = cluster_mod.dump_inventory(path)
+        self.assertIn("app2", databases)
+        self.assertEqual(roles, ["app0", "app1"])
+
+
+class TestReplayFeedsSql(DumpCase):
+    def test_postgres_replays_decompressed_sql(self):
+        fed = []
+
+        def capture(container, argv, **kwargs):
+            stdin = kwargs.get("stdin")
+            fed.append(stdin if isinstance(stdin, bytes) else stdin.read())
+            return MagicMock()
+
+        for name, data in (
+            ("new.backup.sql.gz", gzip.compress(SQL)),
+            ("old.backup.sql", SQL),
+        ):
+            fed.clear()
+            path = write(self.dir, name, data)
+            with (
+                self.subTest(name=name),
+                patch.object(pg_mod, "docker_exec", side_effect=capture),
+            ):
+                pg_mod.restore_postgres_sql(
+                    container="db",
+                    db_name="app",
+                    user="app",
+                    password="pw",
+                    sql_path=path,
+                    empty=False,
+                    check_version=False,
+                )
+                self.assertEqual(fed, [SQL])
+
+    def test_mariadb_streams_decompressed_sql(self):
+        fed = []
+
+        def capture(container, argv, *, source, docker_env):
+            fed.append(source.read())
+
+        for name, data in (
+            ("new.backup.sql.gz", gzip.compress(SQL)),
+            ("old.backup.sql", SQL),
+        ):
+            fed.clear()
+            path = write(self.dir, name, data)
+            with (
+                self.subTest(name=name),
+                patch.object(mdb_mod, "_pick_client", return_value="mariadb"),
+                patch.object(mdb_mod, "docker_exec_stream", side_effect=capture),
+            ):
+                mdb_mod.restore_mariadb_sql(
+                    container="db",
+                    db_name="app",
+                    user="app",
+                    password="pw",
+                    sql_path=path,
+                    empty=False,
+                    check_version=False,
+                )
+                self.assertEqual(fed, [SQL])
+
+    def test_the_stream_reaches_the_child_decompressed(self):
+        out = self.dir / "received.sql"
+        sink = "import shutil, sys; shutil.copyfileobj(sys.stdin.buffer, open(sys.argv[1], 'wb'))"
+        path = write(self.dir, "s.gz", gzip.compress(SQL * 2000))
+        with (
+            patch.object(
+                run_mod,
+                "docker_exec_cmd",
+                return_value=[sys.executable, "-c", sink, str(out)],
+            ),
+            open_dump(path) as source,
+        ):
+            run_mod.docker_exec_stream("db", ["ignored"], source=source)
+        self.assertEqual(out.read_bytes(), SQL * 2000)
+
+    def test_a_child_that_dies_early_is_reported_by_its_exit_code(self):
+        path = write(self.dir, "s.gz", gzip.compress(SQL * 20000))
+        with (
+            patch.object(
+                run_mod,
+                "docker_exec_cmd",
+                return_value=[sys.executable, "-c", "import sys; sys.exit(3)"],
+            ),
+            open_dump(path) as source,
+            patch("sys.stderr"),
+            self.assertRaises(run_mod.subprocess.CalledProcessError) as caught,
+        ):
+            run_mod.docker_exec_stream("db", ["ignored"], source=source)
+        self.assertEqual(caught.exception.returncode, 3)
+
+
+class TestCorruptionStopsBeforeDestroying(DumpCase):
+    def truncated(self) -> str:
+        packed = gzip.compress(SQL * 500)
+        return write(self.dir, "x.backup.sql.gz", packed[: len(packed) // 2])
+
+    def test_postgres_runs_no_preclean_on_a_truncated_dump(self):
+        with (
+            patch.object(pg_mod, "docker_exec") as exec_,
+            self.assertRaises(EOFError),
+        ):
+            pg_mod.restore_postgres_sql(
+                container="db",
+                db_name="app",
+                user="app",
+                password="pw",
+                sql_path=self.truncated(),
+                empty=True,
+                check_version=False,
+            )
+        exec_.assert_not_called()
+
+    def test_mariadb_drops_nothing_on_a_truncated_dump(self):
+        with (
+            patch.object(mdb_mod, "_pick_client", return_value="mariadb"),
+            patch.object(mdb_mod, "docker_exec") as exec_,
+            patch.object(mdb_mod, "docker_exec_stream") as stream,
+            self.assertRaises(EOFError),
+        ):
+            mdb_mod.restore_mariadb_sql(
+                container="db",
+                db_name="app",
+                user="app",
+                password="pw",
+                sql_path=self.truncated(),
+                empty=True,
+                check_version=False,
+            )
+        exec_.assert_not_called()
+        stream.assert_not_called()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/restore/test_mariadb_empty_drop.py

@@ -20,6 +20,7 @@ class TestMariadbEmptyDrop(unittest.TestCase):
             sql.flush()
             with (
                 patch.object(mariadb_mod, "docker_exec", side_effect=_capture),
+                patch.object(mariadb_mod, "docker_exec_stream"),
                 patch.object(mariadb_mod, "_pick_client", return_value="mariadb"),
             ):
                 mariadb_mod.restore_mariadb_sql(

tests/unit/restore/test_password_off_argv.py

@@ -0,0 +1,135 @@
+import gzip
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from baudolo.restore import run as run_mod
+from baudolo.restore.db import cluster as cluster_mod
+from baudolo.restore.db import mariadb as mdb_mod
+from baudolo.restore.db import postgres as pg_mod
+from baudolo.restore.db import version as ver
+
+from .test_version_gate import MARIADB_HEADER, POSTGRES_HEADER, cluster_header
+
+SECRET = "s3cret-Pw"
+
+
+class TestDockerExecForwardsByName(unittest.TestCase):
+    def test_the_value_stays_out_of_the_argv(self):
+        with patch.object(run_mod, "run") as run:
+            run_mod.docker_exec("db", ["psql"], docker_env={"PGPASSWORD": SECRET})
+        cmd = run.call_args.args[0]
+        self.assertEqual(
+            cmd, ["docker", "exec", "-i", "-e", "PGPASSWORD", "db", "psql"]
+        )
+        self.assertFalse([arg for arg in cmd if SECRET in arg])
+        self.assertEqual(run.call_args.kwargs["env"]["PGPASSWORD"], SECRET)
+
+    def test_the_rest_of_the_environment_is_kept(self):
+        with (
+            patch.dict(run_mod.os.environ, {"PATH": "/x"}),
+            patch.object(run_mod, "run") as run,
+        ):
+            run_mod.docker_exec("db", ["psql"], docker_env={"PGPASSWORD": SECRET})
+        self.assertEqual(run.call_args.kwargs["env"]["PATH"], "/x")
+
+    def test_no_docker_env_inherits_unchanged(self):
+        with patch.object(run_mod, "run") as run:
+            run_mod.docker_exec("db", ["true"])
+        self.assertIsNone(run.call_args.kwargs["env"])
+        self.assertEqual(run.call_args.args[0], ["docker", "exec", "-i", "db", "true"])
+
+    def test_the_stream_forwards_by_name_too(self):
+        with patch.object(run_mod.subprocess, "Popen") as popen:
+            popen.return_value.wait.return_value = 0
+            run_mod.docker_exec_stream(
+                "db",
+                ["mariadb"],
+                source=MagicMock(read=MagicMock(return_value=b"")),
+                docker_env={"MYSQL_PWD": SECRET},
+            )
+        cmd = popen.call_args.args[0]
+        self.assertFalse([arg for arg in cmd if SECRET in arg])
+        self.assertIn("MYSQL_PWD", cmd)
+        self.assertEqual(popen.call_args.kwargs["env"]["MYSQL_PWD"], SECRET)
+
+
+class TestNoRestoreArgvCarriesThePassword(unittest.TestCase):
+    def setUp(self):
+        self.argvs = []
+        self.envs = []
+        self.dir = Path(tempfile.mkdtemp())
+
+        def fake_run(cmd, *, stdin=None, capture=False, env=None):
+            self.argvs.append(cmd)
+            self.envs.append(env or {})
+            stdout = b"17.11\n" if "SHOW server_version" in cmd else b"t1\n"
+            if "SELECT VERSION()" in cmd:
+                stdout = b"11.8.8-MariaDB\n"
+            return MagicMock(stdout=stdout)
+
+        def fake_popen(cmd, *, stdin=None, env=None):
+            self.argvs.append(cmd)
+            self.envs.append(env or {})
+            process = MagicMock()
+            process.wait.return_value = 0
+            return process
+
+        patch.object(run_mod, "run", side_effect=fake_run).start()
+        patch.object(run_mod.subprocess, "Popen", side_effect=fake_popen).start()
+        self.addCleanup(patch.stopall)
+
+    def dump(self, name: str, text: str) -> str:
+        path = self.dir / name
+        path.write_bytes(gzip.compress(text.encode()))
+        return str(path)
+
+    def assert_clean(self, variable: str):
+        self.assertTrue(self.argvs)
+        for cmd in self.argvs:
+            self.assertFalse([arg for arg in cmd if SECRET in arg], cmd)
+        self.assertTrue(all(env.get(variable) == SECRET for env in self.envs))
+
+    def test_postgres(self):
+        pg_mod.restore_postgres_sql(
+            container="db",
+            db_name="app",
+            user="app",
+            password=SECRET,
+            sql_path=self.dump("app.backup.sql.gz", POSTGRES_HEADER),
+            empty=True,
+        )
+        self.assert_clean("PGPASSWORD")
+
+    def test_cluster(self):
+        with patch.object(cluster_mod, "assert_instance_matches_dump"):
+            cluster_mod.restore_cluster_sql(
+                container="db",
+                user="postgres",
+                password=SECRET,
+                sql_path=self.dump("c.cluster.backup.sql.gz", cluster_header(roles=1)),
+                empty=True,
+            )
+        self.assert_clean("PGPASSWORD")
+
+    def test_mariadb(self):
+        with patch.object(mdb_mod, "_pick_client", return_value="mariadb"):
+            mdb_mod.restore_mariadb_sql(
+                container="db",
+                db_name="app",
+                user="app",
+                password=SECRET,
+                sql_path=self.dump("app.backup.sql.gz", MARIADB_HEADER),
+                empty=True,
+            )
+        self.assert_clean("MYSQL_PWD")
+        self.assertEqual(len(self.argvs), 4, "version, list, drop, replay")
+
+    def test_mariadb_version_probe(self):
+        ver.server_version("db", "mariadb", "app", SECRET, client="mariadb")
+        self.assert_clean("MYSQL_PWD")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/seed/test_main.py

@@ -164,6 +164,7 @@ class TestSeedMain(unittest.TestCase):
         ns.database = "db"
         ns.username = "user"
         ns.password = "pass"
+        ns.password_stdin = False
         parse_args.return_value = ns
 
         seed_main.main()
@@ -195,6 +196,7 @@ class TestSeedMain(unittest.TestCase):
         ns.database = "db"
         ns.username = "user"
         ns.password = "pass"
+        ns.password_stdin = False
         parse_args.return_value = ns
 
         seed_main.main()

tests/unit/test_generation.py

@@ -13,6 +13,8 @@ from baudolo.generation import (
     CLUSTER_SUFFIX,
     DUMP_SUFFIX,
     FILES_DIR,
+    LEGACY_CLUSTER_SUFFIX,
+    LEGACY_DUMP_SUFFIX,
     MANIFEST_FILE,
     MANIFEST_SCHEMA,
     SQL_DIR,
@@ -30,9 +32,19 @@ class TestManifestDocument(unittest.TestCase):
                 "sql_dir": SQL_DIR,
                 "dump_suffix": DUMP_SUFFIX,
                 "cluster_suffix": CLUSTER_SUFFIX,
+                "dump_compression": "gzip",
             },
         )
 
+    def test_dumps_are_named_as_gzip_and_the_legacy_names_stay_known(self) -> None:
+        self.assertEqual(DUMP_SUFFIX, ".backup.sql.gz")
+        self.assertEqual(CLUSTER_SUFFIX, ".cluster.backup.sql.gz")
+        self.assertEqual(LEGACY_DUMP_SUFFIX, ".backup.sql")
+        self.assertEqual(LEGACY_CLUSTER_SUFFIX, ".cluster.backup.sql")
+
+    def test_the_schema_stays_one_so_existing_readers_keep_reading(self) -> None:
+        self.assertEqual(MANIFEST_SCHEMA, 1)
+
     def test_it_carries_a_schema_so_a_reader_can_refuse_a_newer_one(self) -> None:
         self.assertEqual(manifest_document({})["schema"], MANIFEST_SCHEMA)