kevinveenbirkenbach/backup-docker-to-local
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
264 Commits 2 Branches 16 Tags
7a7ec57b546447df3034cf019709e34eb417e147
Commit Graph

40 Commits

Author SHA1 Message Date
Kevin Veen-Birkenbach
96e6b3ea93 fix(backup,restore): harden the branch fixes and prove them with tests 
Backup: a container that vanishes between the docker ps listing and the
swarm-task inspect (--rm one-shots, task-history GC) no longer aborts the
whole backup run; it counts as not stoppable and is skipped.

Restore: the postgres replay streams the dump through a spooled temp file
instead of buffering it three times in memory (multi-GB dumps OOMed the
restore mid-replay), and the superuser-only line filter is COPY-aware: data
rows inside COPY ... FROM stdin blocks pass through untouched, so a row
that happens to start with COMMENT ON EXTENSION or ALTER DEFAULT PRIVILEGES
is no longer silently dropped.

The e2e runner talks to the DinD daemon through docker exec instead of a
host-published tcp://127.0.0.1:2375: port publishing is unreachable from
sandboxed runners and from hosts with broken loopback publishing, and the
unencrypted root API port disappears from the host. The debug tmp dump
shrinks to tar plus docker cp against the DinD container itself.

New coverage: an e2e reproducing the swarm flake end to end (service task
on the volume, nothing whitelisted: the backup must succeed, the very same
task container must keep running, and the service must never replace a
task), unit tests for the COPY-aware filter, the swarm-task probe including
the vanished-container path, filter_stoppable ordering, and the one-session
FOREIGN_KEY_CHECKS drop assembly. Full suite: 35 unit, 9 integration,
30 e2e green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-07-11 09:19:11 +02:00
Kevin Veen-Birkenbach
bfa596ae30 fix(test-e2e): set DinD MTU to 1280 to fix ghcr.io pull timeouts on broken-PMTUD host paths 
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-26 00:45:29 +02:00
Kevin Veen-Birkenbach
a69074c302 test(e2e): replace ls with find to satisfy shellcheck SC2012 2026-02-06 10:37:31 +01:00
Kevin Veen-Birkenbach
fbfdb8615f **Commit message:** 
Fix Docker CLI install, switch test runners to bash, and stabilize unit tests for compose/seed mocks

https://chatgpt.com/share/697e68cd-d22c-800f-9b2e-47ef231b6502
 2026-01-31 21:40:39 +01:00
Kevin Veen-Birkenbach
c30b4865d4 refactor: migrate to src/ package + add DinD-based E2E runner with debug artifacts 
- Replace legacy standalone scripts with a proper src-layout Python package
  (baudolo backup/restore/configure entrypoints via pyproject.toml)
- Remove old scripts/files (backup-docker-to-local.py, recover-docker-from-local.sh,
  databases.csv.tpl, Todo.md)
- Add Dockerfile to build the project image for local/E2E usage
- Update Makefile: build image and run E2E via external runner script
- Add scripts/test-e2e.sh:
  - start DinD + dedicated network
  - recreate DinD data volume (and shared /tmp volume)
  - pre-pull helper images (alpine-rsync, alpine)
  - load local baudolo:local image into DinD
  - run unittest E2E suite inside DinD and abort on first failure
  - on failure: dump host+DinD diagnostics and archive shared /tmp into artifacts/
- Add artifacts/ debug outputs produced by failing E2E runs (logs, events, tmp archive)

https://chatgpt.com/share/694ec23f-0794-800f-9a59-8365bc80f435
 2025-12-26 18:13:26 +01:00
Kevin Veen-Birkenbach [aka. Frantz]
65151194f9 Updated README.md 2020-10-12 14:50:17 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
f64d22fb77 Optimized 2020-10-12 13:40:43 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
cafd49f216 Implemented log 2020-10-12 13:36:03 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
3eef4cf70a Optimized destination path 2020-10-12 13:30:02 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
4105c9b3b8 optimized 2020-10-12 13:01:26 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
a05bf52e54 removed file rsync 2020-10-12 12:58:52 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
0f6acc7834 Added --delete-excluded 2020-10-12 12:42:35 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
7998a7dc03 Removed / 2020-10-12 12:31:08 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
0859dbbedd Optimized constructor 2020-10-12 12:29:01 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
fc09e21519 Implemented cloning 2020-10-12 12:25:57 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
8175353377 activated backup 2020-10-12 12:06:18 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
2bbf5bcf18 solved bug 2020-10-12 11:56:53 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
6b3a161a06 optimized 2020-10-12 11:54:01 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
c7a8fa734b Solved bug 2020-10-12 11:52:26 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
8b5b16be94 Added / 2020-10-12 11:47:28 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
074915709a Changed variable 2020-10-12 11:43:00 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
cebd984697 Optimized shellcheck hints 2020-10-12 11:38:44 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
84dbc469df Refactored and implemented versioning 2020-10-12 11:35:02 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
c9cba04199 Optimized backup 2020-10-12 10:05:39 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
19c46981f6 Changed to just backup volumes 2020-10-11 22:28:07 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
ed1dfca985 Optimized logic 2020-10-11 22:22:47 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
3d77545020 Implemented sh 2020-10-11 22:05:05 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
3692325288 Solved bug 2020-10-11 21:42:29 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
37be4ec3af Removed hardcoded backup paths 2020-10-11 21:33:48 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
23b84fba39 Optimized 2020-10-11 21:20:59 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
a9209f8def Optimized script 2020-10-11 21:12:34 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
6b9ffcb331 Optimized usability 2020-10-11 20:59:26 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
82cf191520 Solved variable bug 2020-10-11 20:50:45 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
f714f9b57c Optimized 2020-10-11 20:48:55 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
ea3fb0158b Optimized 2020-10-11 20:34:56 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
e063a05f91 Changed image 2020-10-11 20:19:14 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
54db0da552 Added some options 2020-10-11 19:11:36 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
e6c348b3bf Optimized 2020-10-11 19:05:38 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
029115f63e Implemented draft 2020-10-11 18:49:03 +02:00
Kevin Veen-Birkenbach [aka. Frantz]
4ab4fbeef3 Implemented draft 2020-10-11 18:39:06 +02:00