kevinveenbirkenbach/backup-docker-to-local
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
309 Commits 2 Branches 17 Tags
a0204fd3ea778747e2e6694e59fd599253cac39f
Commit Graph
7 Commits
Author SHA1 Message Date
Kevin Veen-BirkenbachandClaude Fable 5 e9030e8443 fix(backup,restore): make restore drills replayable and leave swarm tasks alone
CI (make tests, stable, publish) / make test (push) Has been cancelled
Details
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Has been cancelled
Details 
Restore fixes, both hit by the infinito svc-bkp e2e drill:

- mariadb --empty dropped tables one docker exec at a time with SET
  FOREIGN_KEY_CHECKS=0 issued in a separate client session, so the
  session-scoped toggle never applied and any FK-referenced parent table
  (mailu.users) died with ERROR 1451. Issue the toggle and all DROPs in
  one session.
- postgres --empty now drops only current_user-owned objects (extension
  members like pg_trgm's set_limit are superuser-owned) with IF EXISTS
  absorbing CASCADE fallout, and the replay skips superuser-only dump
  lines (COMMENT ON EXTENSION, ALTER DEFAULT PRIVILEGES) that abort an
  app-user psql run under ON_ERROR_STOP.

Backup fixes:

- pg_dump now runs with --no-owner --no-privileges so future dumps are
  replayable by the owning app user in the first place.
- Swarm task containers are never stopped or started manually: the
  orchestrator replaces a stopped task and a later docker start fails on
  the detached overlay network. filter_stoppable skips them visibly and
  the whitelist stop check ignores them.

Validated end to end against a live infinito compose stack: the full
svc-bkp-volume-2-local drill (verify, restore cycle, sql replay for
mailu, keycloak and one more db) passes with these patches applied.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-11 08:23:11 +02:00
Kevin Veen-BirkenbachandClaude Opus 4.7 ad5d8fcda3 fix(backup): force TCP for mariadb-dump to match '<user>'@'%' grant 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 00:46:47 +02:00
Kevin Veen-Birkenbach d976640312 fix(seed): handle empty databases.csv and add unit tests 
- Gracefully handle empty databases.csv by creating header columns and emitting a warning
- Add _empty_df() helper for consistent DataFrame initialization
- Add unit tests for baudolo-seed including empty-file regression case
- Apply minor formatting fixes across backup and e2e test files

https://chatgpt.com/share/69628f0b-8744-800f-b08d-2633e05167da
 2026-01-10 18:40:22 +01:00
Kevin Veen-Birkenbach f3ef86a444 feat(backup): stricter databases.csv semantics + atomic SQL dumps
CI (make tests, stable, publish) / make test (push) Has been cancelled
Details
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Has been cancelled
Details 
- read databases.csv with stable types (dtype=str, keep_default_na=False)
- validate database field: require '*' or concrete name (no empty/NaN)
- support Postgres cluster dumps via '*' entries (pg_dumpall)
- write SQL dumps atomically to avoid partial/empty files
- early-skip fully ignored volumes before creating backup directories
- update seed CLI to enforce new contract and update by (instance,database)
- adjust tests: sql dir naming + add E2E coverage for early-skip and '*' seeding
 2025-12-29 11:39:57 +01:00
Kevin Veen-Birkenbach df32671cec fix(backup): fallback to file backup in dump-only mode when no DB dump is possible
CI (make tests, stable, publish) / make test (push) Has been cancelled
Details
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Has been cancelled
Details 
- Change DB backup helpers to return whether a dump was actually produced
- Detect DB containers without successful dumps in --dump-only mode
- Fallback to file backups with a warning instead of skipping silently
- Refactor DB dump logic to return boolean status
- Add E2E test covering dump-only fallback when databases.csv entry is missing

https://chatgpt.com/share/6951a659-2b0c-800f-aafa-3e89ae1eb697
 2025-12-28 22:51:12 +01:00
Kevin Veen-Birkenbach 6adafe6b1f fix(backup): log missing db config instead of raising
CI (make tests, stable, publish) / make test (push) Has been cancelled
Details
CI (make tests, stable, publish) / Mark stable + publish image (SemVer tags only) (push) Has been cancelled
Details 
- Use module logger in backup/db.py
- Skip db dump when no databases.csv entry is present
- Apply black/formatting cleanup across backup/restore/tests

https://chatgpt.com/share/69519d45-b0dc-800f-acb6-6fb8504e9b46
 2025-12-28 22:12:31 +01:00
Kevin Veen-Birkenbach c30b4865d4 refactor: migrate to src/ package + add DinD-based E2E runner with debug artifacts 
- Replace legacy standalone scripts with a proper src-layout Python package
  (baudolo backup/restore/configure entrypoints via pyproject.toml)
- Remove old scripts/files (backup-docker-to-local.py, recover-docker-from-local.sh,
  databases.csv.tpl, Todo.md)
- Add Dockerfile to build the project image for local/E2E usage
- Update Makefile: build image and run E2E via external runner script
- Add scripts/test-e2e.sh:
  - start DinD + dedicated network
  - recreate DinD data volume (and shared /tmp volume)
  - pre-pull helper images (alpine-rsync, alpine)
  - load local baudolo:local image into DinD
  - run unittest E2E suite inside DinD and abort on first failure
  - on failure: dump host+DinD diagnostics and archive shared /tmp into artifacts/
- Add artifacts/ debug outputs produced by failing E2E runs (logs, events, tmp archive)

https://chatgpt.com/share/694ec23f-0794-800f-9a59-8365bc80f435
 2025-12-26 18:13:26 +01:00