Merge branch 'fix/hub-cleanup-pkgmgr-local' into unify/skills-20261114

This commit is contained in:
Kevin Veen-Birkenbach committed 2026-11-14 21:52:08 +00:00
commit 568747651b
19 files changed
+1531 -186

No files matched your search

+48 -10
View File
@@ -125,6 +125,33 @@ from three views down to any agent, branch or artefact.
worktree holds is skipped and left to `kvb-git-gc`, and no worktree, worktree holds is skipped and left to `kvb-git-gc`, and no worktree,
Compose stack or volume is touched. Rejected: per Scope, also removing Compose stack or volume is touched. Rejected: per Scope, also removing
the worktree and running `compose down` without `-v`. the worktree and running `compose down` without `-v`.
- **Amended by the operator 2026-11-14**: only a branch whose local tip
the `pkgmgr branch cleanup --local` dry run deletes is offered: the tree
timer writes that dry run (`--json`, per known repository within a
budget, again only when its refs, HEAD, config, reftable or worktree
heads changed, the file rewritten after each repository) as
`local-plan.json` beside the tree file; a repository whose dry run
failed, timed out or was killed keeps its old plan marked stale, and
the hub offers nothing there and says why. The hub and the host decide
with one predicate on its item for that tip (`merged-clean`, no skip
reason, exactly the action `local-branch`); the hub also keeps the
local upstream of another branch, as pkgmgr keeps it for the host's
dry run of that branch alone. A branch that dry run keeps, such as one
merged by pull request (`pr#N`), squash or patch whose local tip is no
ancestor of the default branch, shows its dry run without a Delete
button, with the skip reason of the dry run and branch gc
(`kvb-git-gc`, `pkgmgr gc`) as the way to remove it, and the host
refuses its signed delete. The host runs one dry run and one apply per
repository and job; an item pkgmgr deleted at another merged tip than
the signed one is done with a warning, one the apply keeps is failed
with pkgmgr's skip reason, and only a branch pkgmgr reports deleted is
done: a signed branch whose ref is gone otherwise is `gone` ("deleted
outside this run" when pkgmgr answered without deleting it, "gone
after an interrupted apply" when the apply ended without output),
rewrites the tree and shows apart from deleted and failed. A dry run
or apply that names gone branches unknown runs once more without
them. Rejected: deciding by
`merged_by` alone, and offering a Delete the host then refuses.
22. **Organisation update**: a new skill `kvb-org-update` tells every agent 22. **Organisation update**: a new skill `kvb-org-update` tells every agent
and client which skills changed and how to use them, as one and client which skills changed and how to use them, as one
`newsletter` per update; phase 1 announces the radio change through it. `newsletter` per update; phase 1 announces the radio change through it.
@@ -641,23 +668,34 @@ flowchart TD
- [x] The Artifacts Tree marks every branch and worktree with icons for - [x] The Artifacts Tree marks every branch and worktree with icons for
merged (with what merged it), clean, uncommitted changes (with counts), merged (with what merged it), clean, uncommitted changes (with counts),
unpushed commits, gone upstream, locked, stale and protected. unpushed commits, gone upstream, locked, stale and protected.
- [x] A `merged-clean` item without a gc skip reason shows a delete button, - [x] A `merged-clean` item without a gc skip reason whose local tip the
and the Tree header shows "Delete all merged"; both are disabled, `pkgmgr branch cleanup --local` dry run in `local-plan.json` deletes
with the reason as tooltip, while `delete_ready` is false. shows a delete button, also when its remote tip was merged by squash or
patch, and the Tree header shows "Delete all merged"; both are
disabled, with the reason as tooltip, while `delete_ready` is false.
- [x] A `merged-clean` item that dry run keeps, or that it has not planned
at this tip yet, shows no delete button in its Tree row, its popup or
the delete sheet; its dry run says `pkgmgr branch cleanup --local` keeps
it with the skip reason of that dry run and names branch gc
(`kvb-git-gc`, `pkgmgr gc`) as the way to remove it, and the unit
refuses its signed delete by the same predicate.
- [x] A click shows the gc dry-run lines of exactly the chosen items; - [x] A click shows the gc dry-run lines of exactly the chosen items;
"Delete" requires the operator signature through `/api/sign`. "Delete" requires the operator signature through `/api/sign`.
- [x] The orchestrator writes the signed job (repository, branch, worktree - [x] The orchestrator writes the signed job (repository, branch, worktree
and tip sha per item, no command) into a host-mounted job directory and and tip sha per item, no command) into a host-mounted job directory and
holds no git, token or key. holds no git, token or key.
- [ ] A systemd user path unit verifies the signature on the host and runs - [x] A systemd user path unit verifies the signature on the host and runs
`pkgmgr branch cleanup --apply --local` (the `--local` release is `pkgmgr branch cleanup --apply --local` (pkgmgr 2.5.0) for exactly the
pending in pkgmgr) for exactly the signed items, matched exactly, never signed items, matched exactly, never as a pattern; it removes the local
as a pattern; it removes the local branch, never a remote branch, a branch, never a remote branch, a worktree, a Compose stack or a volume,
worktree, a Compose stack or a volume, and needs no SSH key; a branch a and needs no SSH key; a branch a worktree holds is skipped and left to
worktree holds is skipped and left to `kvb-git-gc`. `kvb-git-gc`.
- [x] The unit refuses, and changes nothing for, an item whose tip or state - [x] The unit refuses, and changes nothing for, an item whose tip or state
differs from the signed one, an unknown repository, a non-`merged-clean` differs from the signed one, an unknown repository, a non-`merged-clean`
item and a job with a bad or missing signature, and reports why. item and a job with a bad or missing signature, and reports why; an
item pkgmgr deleted at another merged tip than the signed one is done
with a warning, and one it did not delete there is failed, or `gone`,
never done, when its ref is gone.
- [x] The unit writes each item's result back and rewrites the tree file; the - [x] The unit writes each item's result back and rewrites the tree file; the
Tree shows the result. Tree shows the result.
+11 -5
View File
@@ -52,9 +52,13 @@ changed and what it costs, and a gate means something again.
labelled stale and counted apart. Rejected: count everything, only drop labelled stale and counted apart. Rejected: count everything, only drop
operator rows. operator rows.
10. **Clean all merged**: dry run, one passkey, signed job. Local branches 10. **Clean all merged**: dry run, one passkey, signed job. Local branches
merged by ancestry and without a remote branch need no token; remote merged by ancestry and without a remote branch need no token; the local
branches and squash-merge candidates need the GitHub token, stored through branch of one with a remote branch needs the GitHub token, stored through
an operator row. Rejected: wait for the token, ancestry for remotes. an operator row. A branch the `pkgmgr branch cleanup --local` dry run
keeps, such as a squash or patch merge whose local tip is no ancestor of
the default branch, is never deleted here, token or not: its dry run says
why and names branch gc (`kvb-git-gc`, `pkgmgr gc`) (operator
2026-11-14). Rejected: wait for the token, ancestry for remotes.
11. **Ancestry**: Artifacts and Agents get a family tree (generations top down) 11. **Ancestry**: Artifacts and Agents get a family tree (generations top down)
in the slot of today's Tree view. A branch may have several parents; in the slot of today's Tree view. A branch may have several parents;
parents are recorded at creation (post-checkout hook), older branches are parents are recorded at creation (post-checkout hook), older branches are
@@ -227,8 +231,10 @@ flowchart TD
- [x] "Clean all merged" shows the gc dry run of every `merged-clean` item; one - [x] "Clean all merged" shows the gc dry run of every `merged-clean` item; one
passkey runs the signed job. passkey runs the signed job.
- [ ] Without a GitHub token, local branches merged by ancestry into the - [ ] Without a GitHub token, local branches merged by ancestry into the
default branch and without a remote branch can be deleted; the others default branch and without a remote branch can be deleted; the other
show "token needed", and an operator row stores the token. branches the `--local` dry run deletes show "token needed", and an
operator row stores the token. A branch that dry run keeps shows no
delete and no "token needed", with or without the token.
- [ ] Artifacts and Agents show Ancestry in the Tree slot: generations top - [ ] Artifacts and Agents show Ancestry in the Tree slot: generations top
down, several parents per node; a derived parent edge is dashed. down, several parents per node; a derived parent edge is dashed.
- [ ] Dragging a branch onto another stages a parent edge, applied with the - [ ] Dragging a branch onto another stages a parent edge, applied with the
+217 -66
View File
@@ -15,18 +15,28 @@
Returns: Returns:
Runs once over every job in --jobs, as the operator on the host: per item Runs once over every job in --jobs, as the operator on the host: per item
it verifies the operator's passkey signature over repository, branch, it verifies the operator's passkey signature over repository, branch,
worktree and tip, then in the repository runs `pkgmgr branch cleanup worktree and tip; then per repository it runs one `pkgmgr branch cleanup
--json --branch <name>` as a dry run and, only when that plan names --local --json --branch <name>...` dry run over its signed branches and
exactly this branch as merged-clean without skip reason at the signed one `pkgmgr branch cleanup --apply --local --json --branch <name>...`
tip and worktree and deletes no remote branch, `pkgmgr branch cleanup over those the plan names exactly at the signed tip and worktree and
--apply --json --branch <name>`. While the tree file says pkgmgr has no GitHub token deletes exactly the local branch of (signing.local_keeps, the predicate
(delete_ready false), only a branch the dry run shows merged by ancestry the hub offers by); pkgmgr deletes only a local branch whose planned tip
into the default branch and without a remote branch is deleted; the is an ancestor of the default branch, by update-ref with that tip. While
others are refused with "token needed". Every other item is refused and the tree file says pkgmgr has no GitHub token (delete_ready false), only
nothing changes for it. An applied item is done only when pkgmgr's a branch the dry run shows merged by ancestry into the default branch
result for it is done; a skipped one or one without result is failed and without a remote branch is deleted; the others are refused with
with pkgmgr's reason. Writes each item's result, rewrites the tree file "token needed". Every other item is refused and nothing changes for it.
after a deletion and removes the job. The exit code is 0. An applied item is done only when pkgmgr's result for it is done, with a
warning when pkgmgr deleted it at another merged tip than the signed
one; one the apply keeps is failed with pkgmgr's skip reason. A signed
branch pkgmgr did not report deleted whose ref is gone is gone, never
done: "deleted outside this run" when pkgmgr answered without deleting
it (a failed result, no result, or "Unknown --branch"), "gone after an
interrupted apply" when the apply ended without JSON; a present one is
failed. When a dry run or apply answers "Unknown --branch" for gone
branches, it runs once more without them.
Writes each item's result, rewrites the tree file after a deletion or a
gone branch and removes the job. The exit code is 0.
""" """
from __future__ import annotations from __future__ import annotations
@@ -49,7 +59,10 @@ UNSAFE = re.compile(r"[*?\[\]\\\s\x00-\x1f\x7f]|^-|\.\.|@\{")
OUTPUT_LIMIT = 16 << 10 OUTPUT_LIMIT = 16 << 10
TIMEOUT = 600 TIMEOUT = 600
DONE = "done" DONE = "done"
REMOTE_ACTION = "remote-branch:" GONE = "gone"
DELETED_OUTSIDE = "deleted outside this run"
GONE_INTERRUPTED = "gone after an interrupted apply"
UNKNOWN = re.compile(r"^Unknown --branch: (.*)$", re.MULTILINE)
TOKEN_NEEDED = "token needed: without the GitHub token only a local branch merged by ancestry into the default branch and without a remote branch is deleted" TOKEN_NEEDED = "token needed: without the GitHub token only a local branch merged by ancestry into the default branch and without a remote branch is deleted"
# the cwd is a repository agents can write: its config may name hooks, and # the cwd is a repository agents can write: its config may name hooks, and
# pkgmgr releases before 4c8b137 run git branch -D without scan mode # pkgmgr releases before 4c8b137 run git branch -D without scan mode
@@ -84,23 +97,57 @@ def tree_facts(tree: Path) -> tuple[dict[str, str], bool]:
return {}, False return {}, False
def run(pkgmgr: Path, argv: list[str], cwd: str) -> tuple[int, str, str]: def run(pkgmgr: Path, argv: list[str], cwd: str, timeout: int) -> tuple[int, str, str]:
"""Args:
pkgmgr: the pkgmgr executable.
argv: its arguments.
cwd: the repository it runs in, without hooks or fsmonitor.
timeout: seconds before pkgmgr is killed.
Returns:
Exit code, stdout and the tail of stderr; 127 with the error when
pkgmgr could not start or was killed at the timeout.
"""
try: try:
done = subprocess.run([str(pkgmgr), *argv], cwd=cwd, env={**os.environ, **git_overrides()}, stdin=subprocess.DEVNULL, capture_output=True, text=True, errors="replace", timeout=TIMEOUT, check=False) done = subprocess.run([str(pkgmgr), *argv], cwd=cwd, env={**os.environ, **git_overrides()}, stdin=subprocess.DEVNULL, capture_output=True, text=True, errors="replace", timeout=timeout, check=False)
except (OSError, subprocess.TimeoutExpired) as error: except (OSError, subprocess.TimeoutExpired) as error:
return 127, "", str(error) return 127, "", str(error)
return done.returncode, done.stdout, done.stderr[-OUTPUT_LIMIT:] return done.returncode, done.stdout, done.stderr[-OUTPUT_LIMIT:]
def plan_argv(branch: str, apply: bool) -> list[str]: def branch_tip(repository: str, branch: str) -> str | None:
"""Returns:
The local branch's tip as git reads it in the repository and no
parent of it, "" when the repository has no such branch, None when
git cannot tell.
"""
env = {**os.environ, **git_overrides(), "GIT_CEILING_DIRECTORIES": str(Path(repository).parent)}
try:
done = subprocess.run(["git", "rev-parse", "--verify", "--quiet", f"refs/heads/{branch}"], cwd=repository, env=env, stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=60, check=False)
except (OSError, subprocess.TimeoutExpired):
return None
if done.returncode == 0 and TIP.fullmatch(done.stdout.strip()):
return done.stdout.strip()
return "" if done.returncode == 1 and not done.stdout.strip() else None
def unknown(stderr: str) -> set[str]:
"""Returns:
The branch names pkgmgr's "Unknown --branch: a, b" line names.
"""
return {name for found in UNKNOWN.findall(stderr) for name in found.split(", ")}
def plan_argv(branches: list[str], apply: bool) -> list[str]:
"""Args: """Args:
branch: the exact branch name. branches: the exact branch names of one repository.
apply: execute the plan instead of the dry run. apply: execute the plan instead of the dry run.
Returns: Returns:
The `pkgmgr branch cleanup` arguments for this branch's plan in JSON. The `pkgmgr branch cleanup --local` arguments for these branches'
plan in JSON: local branches only, no forge token, no remote delete.
""" """
return ["branch", "cleanup", *(["--apply"] if apply else []), "--json", "--branch", branch] return ["branch", "cleanup", *(["--apply"] if apply else []), "--local", "--json", *[part for branch in branches for part in ("--branch", branch)]]
def plan_items(stdout: str) -> list[dict] | None: def plan_items(stdout: str) -> list[dict] | None:
@@ -126,20 +173,85 @@ def shaped(item: object) -> bool:
return item["repository"].startswith("/") and bool(TIP.fullmatch(item["tip"])) and (item["worktree"] == "" or item["worktree"].startswith("/")) return item["repository"].startswith("/") and bool(TIP.fullmatch(item["tip"])) and (item["worktree"] == "" or item["worktree"].startswith("/"))
def mismatch(entry: dict, item: dict) -> str | None: def tip_changed(entry: dict, item: dict) -> str | None:
if entry.get("state") != "merged-clean":
return f"state is {entry.get('state')}, not merged-clean"
if entry.get("skip"):
return f"gc skip reason: {entry['skip']}"
tip = entry.get("local_sha") or entry.get("head") tip = entry.get("local_sha") or entry.get("head")
if tip != item["tip"]: return None if tip == item["tip"] else f"tip changed: signed {item['tip'][:12]}, now {str(tip)[:12]}"
return f"tip changed: signed {item['tip'][:12]}, now {str(tip)[:12]}"
def mismatch(entry: dict, item: dict) -> str | None:
changed = tip_changed(entry, item)
if changed:
return changed
if (entry.get("worktree") or "") != item["worktree"]: if (entry.get("worktree") or "") != item["worktree"]:
return f"worktree changed: signed {item['worktree'] or 'none'}, now {entry.get('worktree') or 'none'}" return f"worktree changed: signed {item['worktree'] or 'none'}, now {entry.get('worktree') or 'none'}"
remote = [str(action) for action in entry.get("actions") or [] if str(action).startswith(REMOTE_ACTION)] kept = signing.local_keeps(entry)
if remote: return f"pkgmgr branch cleanup --local keeps it: {kept}" if kept else None
return f"the plan also deletes {', '.join(remote)[:200]}; a signed delete is local only"
return None
def refused(item: dict, reason: str) -> dict:
return {**item, "status": "refused", "reason": reason}
def applied_result(item: dict, applied: list[dict] | None, code: int, stderr: str) -> dict:
"""Args:
item: one signed item the apply chose.
applied: every item `--apply` printed for its repository, None when
it printed no JSON.
code: the apply's exit code, 1 when any of its items failed.
stderr: the apply's error output.
Returns:
The item done only when pkgmgr's result for it is done, with a
warning as reason when pkgmgr deleted it at another tip than the
signed one, which update-ref only does for the tip pkgmgr planned as
merged; failed with pkgmgr's skip reason when the apply kept it.
Without a done result its ref decides: gone is gone, "gone after an
interrupted apply" when the apply printed no JSON and named no
unknown branch, else "deleted outside this run"; present is failed
with pkgmgr's reason.
"""
mine = next((entry for entry in applied or [] if entry.get("branch") == item["branch"]), {})
outcome = mine.get("result")
moved = tip_changed(mine, item) if mine else None
if isinstance(outcome, str) and outcome.split(";")[0] == DONE:
warning = f"deleted at another merged tip than the signed dry run: {moved}" if moved else ""
return {**item, "status": DONE, "reason": warning, "code": code, "applied": [mine]}
if mine.get("skip") and not outcome:
reason = f"pkgmgr branch cleanup --local keeps it: {mine['skip']}"
elif moved:
reason = f"the apply planned another tip than the signed dry run: {moved}"
elif not (isinstance(outcome, str) and outcome.startswith("skipped")) and branch_tip(item["repository"], item["branch"]) == "":
why = GONE_INTERRUPTED if applied is None and not unknown(stderr) else DELETED_OUTSIDE
return {**item, "status": GONE, "reason": why, "code": code, "applied": [mine] if mine else []}
else:
reason = outcome if isinstance(outcome, str) and outcome else stderr.strip()[-500:] or f"pkgmgr reported no result for {item['branch']}"
return {**item, "status": "failed", "reason": reason[:500], "code": code, "applied": [mine] if mine else []}
def screened(items: list[dict], plan: list[dict], default: str, ready: bool) -> dict[str, dict]:
"""Args:
items: admitted items of one repository, each branch once.
plan: the items of their dry run.
default: the repository's default branch.
ready: whether pkgmgr holds the GitHub token.
Returns:
The refusal by branch of each item the plan does not delete exactly
at its signed tip and worktree, or without the token not token-free;
of every item when the plan names other branches too.
"""
others = sorted({str(entry.get("branch")) for entry in plan} - {item["branch"] for item in items})
if others:
return {item["branch"]: refused(item, f"the dry run names other branches too, a pattern match: {', '.join(others)[:500]}") for item in items}
refusals = {}
for item in items:
mine = [entry for entry in plan if entry.get("branch") == item["branch"]]
reason = mismatch(mine[0], item) if len(mine) == 1 else f"the dry run names {len(mine)} items for {item['branch']}"
if reason is None and not ready and not signing.token_free(mine[0], default):
reason = TOKEN_NEEDED
if reason is not None:
refusals[item["branch"]] = refused(item, reason)
return refusals
class Cleanup: class Cleanup:
@@ -148,53 +260,78 @@ class Cleanup:
self.tree = tree self.tree = tree
self.pkgmgr = pkgmgr self.pkgmgr = pkgmgr
def item(self, item: dict, proof: object, known: dict[str, str], ready: bool, now: float) -> dict: def admitted(self, item: dict, proof: object, known: dict[str, str], now: float) -> str | None:
"""Args: """Args:
item: one signed item with repository, branch, worktree and tip. item: one signed item with repository, branch, worktree and tip.
proof: the job's {"payload", "assertion"}. proof: the job's {"payload", "assertion"}.
known: the repositories the tree file says pkgmgr knows, each known: the repositories the tree file says pkgmgr knows.
with its default branch.
ready: whether pkgmgr holds the GitHub token; without it only a
branch signing.token_free() accepts in the dry run is deleted.
now: the current time in epoch seconds. now: the current time in epoch seconds.
Returns: Returns:
The item with status done, refused or failed and the reason; Why the item is refused before any dry run; None when the
refused items changed nothing, failed ones are those pkgmgr did operator's signature lists it, pkgmgr knows its repository and
not report done. its branch is an exact name.
""" """
def refused(reason: str) -> dict:
return {**item, "status": "refused", "reason": reason}
try: try:
verifier.check(proof, signing.cleanup_item(item["repository"], item["branch"], item["worktree"], item["tip"]), "", self.trust, now) verifier.check(proof, signing.cleanup_item(item["repository"], item["branch"], item["worktree"], item["tip"]), "", self.trust, now)
except (signing.Refused, OSError) as error: except (signing.Refused, OSError) as error:
return refused(f"no valid operator signature: {error}") return f"no valid operator signature: {error}"
if item["repository"] not in known or not Path(item["repository"]).is_dir(): if item["repository"] not in known or not Path(item["repository"]).is_dir():
return refused("unknown repository: pkgmgr does not list it") return "unknown repository: pkgmgr does not list it"
if UNSAFE.search(item["branch"]) or not item["branch"]: if UNSAFE.search(item["branch"]) or not item["branch"]:
return refused("the branch name holds pattern or option characters; only an exact branch name is deleted") return "the branch name holds pattern or option characters; only an exact branch name is deleted"
code, stdout, stderr = run(self.pkgmgr, plan_argv(item["branch"], False), item["repository"]) return None
def attempt(self, items: list[dict], apply: bool, results: dict[str, dict]) -> tuple[list[dict], int, str, str]:
"""Args:
items: items of one repository to plan or apply.
apply: run the apply instead of the dry run.
results: gets, by branch, each item that run names unknown whose
ref is gone, as gone "deleted outside this run".
Returns:
The items of the last run with its exit code, stdout and stderr;
after an "Unknown --branch" for gone branches the run repeats
once without them.
"""
location = items[0]["repository"]
code, stdout, stderr = run(self.pkgmgr, plan_argv([item["branch"] for item in items], apply), location, TIMEOUT)
named = unknown(stderr) if code != 0 else set()
gone = [item for item in items if item["branch"] in named and branch_tip(location, item["branch"]) == ""]
if gone:
results.update({item["branch"]: {**item, "status": GONE, "reason": DELETED_OUTSIDE, "code": code} for item in gone})
items = [item for item in items if item not in gone]
if items:
code, stdout, stderr = run(self.pkgmgr, plan_argv([item["branch"] for item in items], apply), location, TIMEOUT)
return items, code, stdout, stderr
def repository(self, items: list[dict], default: str, ready: bool) -> list[dict]:
"""Args:
items: admitted items of one repository, each branch once.
default: the repository's default branch.
ready: whether pkgmgr holds the GitHub token; without it only a
branch signing.token_free() accepts in the dry run is deleted.
Returns:
Each item with status done, gone, refused or failed and the
reason, from one dry run over all of them and one apply over
those the dry run plans to delete at the signed tip and worktree,
each repeated once without the branches it names unknown and
gone; refused items changed nothing.
"""
results: dict[str, dict] = {}
planned, code, stdout, stderr = self.attempt(items, False, results)
plan = plan_items(stdout) if code == 0 else None plan = plan_items(stdout) if code == 0 else None
if plan is None: if plan is None:
return refused(f"the dry run failed ({code}): {stderr.strip()[-500:]}") results.update({item["branch"]: refused(item, f"the dry run failed ({code}): {stderr.strip()[-500:]}") for item in planned})
others = sorted({str(entry.get("branch")) for entry in plan} - {item["branch"]}) else:
if others: results.update(screened(planned, plan, default, ready))
return refused(f"the dry run names other branches too, a pattern match: {', '.join(others)[:500]}") chosen = [item for item in planned if item["branch"] not in results]
if len(plan) != 1: if chosen:
return refused(f"the dry run names {len(plan)} items for {item['branch']}") applying, code, stdout, stderr = self.attempt(chosen, True, results)
reason = mismatch(plan[0], item) applied = plan_items(stdout)
if reason: results.update({item["branch"]: applied_result(item, applied, code, stderr) for item in applying})
return refused(reason) return [results[item["branch"]] for item in items]
if not ready and not signing.token_free(plan[0], known[item["repository"]]):
return refused(TOKEN_NEEDED)
code, stdout, stderr = run(self.pkgmgr, plan_argv(item["branch"], True), item["repository"])
applied = plan_items(stdout) or []
outcome = next((entry.get("result") for entry in applied if entry.get("branch") == item["branch"]), None)
if code == 0 and isinstance(outcome, str) and outcome.split(";")[0] == DONE:
return {**item, "status": "done", "reason": "", "code": code, "applied": applied}
reason = outcome if isinstance(outcome, str) and outcome else stderr.strip()[-500:] or f"pkgmgr reported no result for {item['branch']}"
return {**item, "status": "failed", "reason": reason[:500], "code": code, "applied": applied}
def job(self, folder: int, path: Path, now: float) -> dict: def job(self, folder: int, path: Path, now: float) -> dict:
try: try:
@@ -205,10 +342,24 @@ class Cleanup:
if not isinstance(items, list) or not items or not all(map(shaped, items)) or job.get("id") != path.stem: if not isinstance(items, list) or not items or not all(map(shaped, items)) or job.get("id") != path.stem:
return {"id": path.stem, "finished": now, "items": [], "error": "malformed job: refused, nothing deleted"} return {"id": path.stem, "finished": now, "items": [], "error": "malformed job: refused, nothing deleted"}
known, ready = tree_facts(self.tree) known, ready = tree_facts(self.tree)
results = [self.item(item, job.get("proof"), known, ready, now) for item in items] results: list[dict | None] = [None] * len(items)
groups: dict[str, list[int]] = {}
seen: set[tuple[str, str]] = set()
for index, item in enumerate(items):
reason = self.admitted(item, job.get("proof"), known, now)
if reason is None and (item["repository"], item["branch"]) in seen:
reason = "the job lists this branch twice"
if reason is not None:
results[index] = refused(item, reason)
continue
seen.add((item["repository"], item["branch"]))
groups.setdefault(item["repository"], []).append(index)
for location, indexes in groups.items():
for index, outcome in zip(indexes, self.repository([items[index] for index in indexes], known[location], ready)):
results[index] = outcome
result = {"id": path.stem, "finished": time.time(), "items": results} result = {"id": path.stem, "finished": time.time(), "items": results}
if any(entry["status"] == "done" for entry in results): if any(entry["status"] in (DONE, GONE) for entry in results):
code, _, stderr = run(self.pkgmgr, ["branch", "tree", "--format", "json", "--output", str(self.tree)], str(self.tree.parent)) code, _, stderr = run(self.pkgmgr, ["branch", "tree", "--format", "json", "--output", str(self.tree)], str(self.tree.parent), TIMEOUT)
result["tree"] = {"code": code, "error": stderr.strip()[-500:]} result["tree"] = {"code": code, "error": stderr.strip()[-500:]}
return result return result
+159
View File
@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""Args:
--tree: the tree file `pkgmgr branch tree` writes; the plan goes beside
it as local-plan.json, in the directory the orchestrator mounts
read-only.
--pkgmgr: the pkgmgr executable built from a signed tag.
--budget: seconds one repository's dry run may take, well below the
tree unit's TimeoutStartSec.
Returns:
Keeps local-plan.json beside the tree file: the dry run of `pkgmgr
branch cleanup --local --json` of every repository the tree file says
pkgmgr knows, run in it as the cleanup runner runs its dry run
(cleanup.run: no repository hook or fsmonitor), named by its real path.
A repository whose refs, HEAD, config, reftable and worktree heads
(fingerprint) are unchanged since its last attempt is not planned
again. Before each dry run the file is rewritten atomically with that
repository marked stale ("replan started <time> has not finished") at
the attempted fingerprint, and again after it, so a kill keeps every
plan written before; a dry run that fails or outlasts --budget keeps
the repository's old items, marked stale with the reason and time, and
is retried only once its fingerprint changes. The hub offers nothing
from a stale repository. Deleting local-plan.json plans every
repository again. The exit code is 0 when every dry run of this run
printed a plan, else 1.
"""
from __future__ import annotations
import argparse
import datetime
import hashlib
import json
import os
import sys
import tempfile
from pathlib import Path
import cleanup
LOCAL_PLAN = "local-plan.json"
PLAN_ARGV = ["branch", "cleanup", "--local", "--json"]
GIT_FILES = ("HEAD", "config", "packed-refs", "reftable/tables.list")
WORKTREE_FILES = ("HEAD", "gitdir", "locked", "config.worktree")
def git_dir(repo: Path) -> Path:
dot = repo / ".git"
if dot.is_file():
return (repo / dot.read_text(encoding="utf-8").strip().removeprefix("gitdir: ")).resolve()
return dot
def fingerprint(repo: str) -> str | None:
"""Returns:
sha256 over name, mtime and size of every file the `--local` plan
reads from the repository's git dir: HEAD, config, packed-refs, the
reftable table list, every ref, and each worktree's HEAD, gitdir,
lock and config; None when the git dir cannot be read.
"""
try:
root = git_dir(Path(repo))
paths = [root / name for name in GIT_FILES]
for folder, _, files in os.walk(root / "refs"):
paths += [Path(folder, name) for name in files]
worktrees = root / "worktrees"
if worktrees.is_dir():
paths += [entry / name for entry in worktrees.iterdir() for name in WORKTREE_FILES]
digest = hashlib.sha256()
for path in sorted(paths):
try:
info = path.stat()
except FileNotFoundError:
continue
digest.update(f"{path.relative_to(root)}\0{info.st_mtime_ns}\0{info.st_size}\n".encode())
return digest.hexdigest()
except OSError:
return None
def known_repos(tree: Path) -> list[str]:
data = json.loads(tree.read_text(encoding="utf-8"))
return [entry["path"] for entry in data["repos"] if entry.get("known") is True and entry.get("missing") is not True and isinstance(entry.get("path"), str)]
def old_plans(target: Path) -> dict[str, dict]:
try:
return {entry["repo"]: entry for entry in json.loads(target.read_text(encoding="utf-8"))["repos"] if isinstance(entry, dict) and isinstance(entry.get("repo"), str)}
except (OSError, ValueError, KeyError, TypeError, AttributeError):
return {}
def plan(pkgmgr: Path, repo: str, budget: int) -> tuple[dict | None, str]:
"""Returns:
The repository's {"repo", "default", "merged_into", "warnings",
"items"} as pkgmgr prints it, named by its real path, and ""; or
None and "timed out" or "failed" when pkgmgr printed no plan of
exactly one repository within the budget.
"""
code, stdout, stderr = cleanup.run(pkgmgr, PLAN_ARGV, repo, budget)
if cleanup.plan_items(stdout) is None:
return None, "timed out" if code == 127 and "timed out" in stderr else "failed"
repos = json.loads(stdout)["repos"]
if len(repos) != 1 or not isinstance(repos[0].get("repo"), str):
return None, "failed"
return {**repos[0], "repo": os.path.realpath(repos[0]["repo"])}, ""
def write(target: Path, data: dict) -> None:
with tempfile.NamedTemporaryFile("w", dir=target.parent, prefix=f".{target.name}-", delete=False, encoding="utf-8") as handle:
json.dump(data, handle)
os.replace(handle.name, target)
def now() -> str:
return datetime.datetime.now(datetime.UTC).isoformat(timespec="seconds")
def main() -> int:
parser = argparse.ArgumentParser(prog="local_plan")
parser.add_argument("--tree", type=Path, required=True)
parser.add_argument("--pkgmgr", type=Path, required=True)
parser.add_argument("--budget", type=int, required=True)
args = parser.parse_args()
target = args.tree.with_name(LOCAL_PLAN)
try:
known = [os.path.realpath(repo) for repo in known_repos(args.tree)]
except (OSError, ValueError, KeyError, TypeError, AttributeError) as error:
print(f"local_plan: tree file {args.tree} unreadable ({error}), the old plan stays", file=sys.stderr)
return 1
plans = {repo: entry for repo, entry in old_plans(target).items() if repo in known}
def save() -> None:
write(target, {"repos": [plans[repo] for repo in known if repo in plans]})
failed = []
for repo in known:
current = fingerprint(repo)
if current is not None and plans.get(repo, {}).get("fingerprint") == current:
continue
previous = plans.get(repo, {"repo": repo, "default": None, "merged_into": [], "warnings": [], "items": []})
plans[repo] = {**previous, "fingerprint": current, "stale": f"replan started {now()} has not finished"}
save()
planned, reason = plan(args.pkgmgr, repo, args.budget)
if planned is None:
plans[repo] = {**previous, "fingerprint": current, "stale": f"replan {reason} at {now()}"}
failed.append(f"{repo}: {reason}")
else:
plans[repo] = {**planned, "fingerprint": current}
save()
if not known:
save()
for line in failed:
print(f"local_plan: no --local plan for {line}, the hub offers no delete there until its refs change", file=sys.stderr)
return 1 if failed else 0
if __name__ == "__main__":
raise SystemExit(main())
+31 -2
View File
@@ -35,6 +35,7 @@ GATE_ACTIONS = ("commit", "merge", "push", "pr")
GATE_REPO = re.compile(r"/[^\x00-\x1f\x7f]{0,4095}") GATE_REPO = re.compile(r"/[^\x00-\x1f\x7f]{0,4095}")
GATE_BRANCH = re.compile(r"(?![-/])(?!.*\.\.)(?!.*@\{)(?!.*//)[^\x00-\x20\x7f~^:?*\[\\]{1,255}(?<![/.])") GATE_BRANCH = re.compile(r"(?![-/])(?!.*\.\.)(?!.*@\{)(?!.*//)[^\x00-\x20\x7f~^:?*\[\\]{1,255}(?<![/.])")
TOKEN_FREE_MERGE = "ancestor" TOKEN_FREE_MERGE = "ancestor"
LOCAL_DELETE = ["local-branch"]
APPROVAL_KEYS = {"v", "kind", "items", "nonce", "expires"} APPROVAL_KEYS = {"v", "kind", "items", "nonce", "expires"}
RULE_KEYS = {"v", "kind", "id", "kinds", "tool", "pattern", "scope", "expires"} RULE_KEYS = {"v", "kind", "id", "kinds", "tool", "pattern", "scope", "expires"}
TARGET = "agents" TARGET = "agents"
@@ -273,7 +274,8 @@ def names_default(ref: object, default: object) -> bool:
"""Args: """Args:
ref: a ref as pkgmgr names it, bare such as main or behind one ref: a ref as pkgmgr names it, bare such as main or behind one
remote such as origin/main, the remote-tracking ref it compares remote such as origin/main, the remote-tracking ref it compares
against when the remote has a HEAD. against when the remote has a HEAD; `branch cleanup --local`
names it in full, refs/remotes/origin/main.
default: the repository's default branch as the tree file names it, default: the repository's default branch as the tree file names it,
without the remote (pkgmgr tree.py strips it). without the remote (pkgmgr tree.py strips it).
@@ -282,7 +284,7 @@ def names_default(ref: object, default: object) -> bool:
""" """
if not isinstance(ref, str) or not isinstance(default, str) or not default: if not isinstance(ref, str) or not isinstance(default, str) or not default:
return False return False
remote, _, name = ref.partition("/") remote, _, name = ref.removeprefix("refs/remotes/").partition("/")
return ref == default or (bool(remote) and name == default) return ref == default or (bool(remote) and name == default)
@@ -300,6 +302,33 @@ def token_free(entry: dict, default: object) -> bool:
return entry.get("merged_by") == TOKEN_FREE_MERGE and names_default(entry.get("merged_into"), default) and not entry.get("remotes") return entry.get("merged_by") == TOKEN_FREE_MERGE and names_default(entry.get("merged_into"), default) and not entry.get("remotes")
def local_keeps(entry: dict) -> str:
"""Args:
entry: one item of `pkgmgr branch cleanup --local --json`, the dry
run of what `--apply --local` does to this branch's local tip.
Returns:
Why that dry run keeps the branch, in pkgmgr's words where it gives
them: a state other than merged-clean, its skip reason (for a tip
that is no ancestor of the default branch "merged by patch, tip is
no ancestor of refs/remotes/origin/main"), a remote branch delete,
or a plan without exactly the local-branch delete; "" when it
deletes exactly the local branch, the only case the hub offers and
the host runs.
"""
if entry.get("state") != "merged-clean":
return f"state is {entry.get('state')}, not merged-clean"
if entry.get("skip"):
return str(entry["skip"])
actions = [str(action) for action in entry.get("actions") or []]
remote = [action for action in actions if action.startswith("remote-branch")]
if remote:
return f"the plan also deletes {', '.join(remote)[:200]}; a signed delete is local only"
if actions != LOCAL_DELETE:
return f"the dry run plans {', '.join(actions)[:200] or 'no delete'}, not exactly the local branch"
return ""
def approval_payload(hashes: list[str], nonce: str, expires: int) -> dict: def approval_payload(hashes: list[str], nonce: str, expires: int) -> dict:
return {"v": VERSION, "kind": "approve", "items": sorted(set(hashes)), "nonce": nonce, "expires": expires} return {"v": VERSION, "kind": "approve", "items": sorted(set(hashes)), "nonce": nonce, "expires": expires}
+6 -4
View File
@@ -22,7 +22,7 @@ ITEM_LIMIT = 100
RESULTS_SHOWN = 20 RESULTS_SHOWN = 20
TOKEN_ROW = "pkgmgr-github-token" TOKEN_ROW = "pkgmgr-github-token"
TOKEN_COMMAND = "env secret-tool store --label=pkgmgr-github service pkgmgr:github username 'github.com|-' </dev/tty" TOKEN_COMMAND = "env secret-tool store --label=pkgmgr-github service pkgmgr:github username 'github.com|-' </dev/tty"
TOKEN_REASON = "Store the GitHub token in the Secret Service under the key pkgmgr reads; Clean all merged then also deletes remote and squash-merged branches." TOKEN_REASON = "Store the GitHub token in the Secret Service under the key pkgmgr reads; Delete all merged then also deletes the local branch of a merged branch that has a remote branch; never the remote branch."
TOKEN_ADDER = "Operator" TOKEN_ADDER = "Operator"
@@ -71,8 +71,10 @@ class Cleaning:
Returns: Returns:
The signed items, each still a merged-clean branch without a gc The signed items, each still a merged-clean branch without a gc
skip reason at the same tip in the current tree file; without skip reason at the same tip in the current tree file whose
the GitHub token (delete_ready false) each also token-free. `--local` dry run deletes exactly its local branch at that tip
(tree.kept_by); without the GitHub token (delete_ready false)
each also token-free.
""" """
if not isinstance(raw, list) or not 1 <= len(raw) <= ITEM_LIMIT: if not isinstance(raw, list) or not 1 <= len(raw) <= ITEM_LIMIT:
raise ToolError(f"items must be a list of 1 to {ITEM_LIMIT} objects") raise ToolError(f"items must be a list of 1 to {ITEM_LIMIT} objects")
@@ -90,7 +92,7 @@ class Cleaning:
if entry in locked and entry not in known: if entry in locked and entry not in known:
raise ToolError(f"delete blocked: {scan.get('delete_reason') or 'delete_ready is false'}") raise ToolError(f"delete blocked: {scan.get('delete_reason') or 'delete_ready is false'}")
if entry not in known: if entry not in known:
raise ToolError(f"{entry.get('branch')!s:.200} in {entry.get('repository')!s:.200} is no merged-clean branch without skip reason at that tip in the tree file") raise ToolError(f"{entry.get('branch')!s:.200} in {entry.get('repository')!s:.200} is no merged-clean branch without skip reason that the --local dry run deletes at that tip")
if entry in chosen: if entry in chosen:
raise ToolError("items must be distinct") raise ToolError("items must be distinct")
chosen.append(entry) chosen.append(entry)
+108 -14
View File
@@ -20,6 +20,7 @@ INVALID = "invalid"
TREE_HEARTBEAT = "tree-heartbeat" TREE_HEARTBEAT = "tree-heartbeat"
GC_SKILL = "kvb-git-gc" GC_SKILL = "kvb-git-gc"
GC_COMPOSE_DOWN = ("docker", "compose", "-p", "{project}", "down", "-v", "--remove-orphans") GC_COMPOSE_DOWN = ("docker", "compose", "-p", "{project}", "down", "-v", "--remove-orphans")
LOCAL_PLAN = "local-plan.json"
def flow(item: dict) -> dict | None: def flow(item: dict) -> dict | None:
@@ -72,15 +73,64 @@ def held_by(worktree: str) -> str:
return f"held by worktree {worktree}: pkgmgr branch cleanup skips it, use branch gc ({GC_SKILL}, pkgmgr gc)" return f"held by worktree {worktree}: pkgmgr branch cleanup skips it, use branch gc ({GC_SKILL}, pkgmgr gc)"
def dry_run(item: dict) -> list[str]: def kept_by(planned: dict | None, tip: str, upstream_of: str, stale: str) -> str:
"""Returns: """Args:
planned: this branch's item of the `pkgmgr branch cleanup --local`
dry run beside the tree file, None when it has none.
tip: the branch's local tip in the tree file.
upstream_of: a branch of the same dry run whose local upstream this
branch is, "" for none; pkgmgr's _config_keeps keeps it when the
host's `--branch` dry run chooses it alone.
stale: why the repository's dry run is stale, "" when it is fresh.
Returns:
Why the hub offers no delete: a stale dry run, or from that dry run
what keeps the branch (signing.local_keeps, the predicate the host
runner refuses by, then the local upstream), or that it has not
planned this tip yet; "" when it deletes exactly the local branch at
this tip.
"""
if stale:
return f"pkgmgr branch cleanup --local plan stale: {stale}; use branch gc ({GC_SKILL}, pkgmgr gc)"
if planned is None or planned.get("local_sha") != tip:
return f"no pkgmgr branch cleanup --local dry run of {tip[:12]} yet: the tree timer writes {LOCAL_PLAN} within a minute"
keeps = signing.local_keeps(planned) or (f"kept: local upstream of {upstream_of}" if upstream_of else "")
return f"pkgmgr branch cleanup --local keeps it: {keeps}; use branch gc ({GC_SKILL}, pkgmgr gc)" if keeps else ""
def local_upstreams(items: list[dict]) -> dict[str, str]:
"""Args:
items: one repository's items of the `--local` dry run, each with
its `%(upstream:short)` as upstream.
Returns:
Each branch that is the local upstream of another branch there, with
the first such branch by name, as pkgmgr's _config_keeps names it.
"""
names = {item["branch"] for item in items}
found: dict[str, str] = {}
for item in sorted(items, key=lambda entry: entry["branch"]):
upstream = str(item.get("upstream") or "")
parent = upstream if upstream in names else upstream.removeprefix("heads/")
if parent in names and parent != item["branch"]:
found.setdefault(parent, item["branch"])
return found
def dry_run(item: dict, kept: str) -> list[str]:
"""Args:
item: one tree item.
kept: kept_by() for it, "" when the hub offers its delete.
Returns:
What the runner's `pkgmgr branch cleanup` does for one tree item, What the runner's `pkgmgr branch cleanup` does for one tree item,
read from the gc plan the tree file carries: it deletes the local read from the gc plan the tree file carries: it deletes the local
branch and never touches a worktree, Compose stack or volume; a branch and never touches a worktree, Compose stack or volume; a
remote branch stays, the runner refuses a plan that deletes one. A remote branch stays, the runner refuses a plan that deletes one. A
branch held by a worktree it skips; its lines name what branch gc branch held by a worktree it skips, and one the `--local` dry run
would do instead, its `docker compose down -v` deleting the stack's keeps or has not planned it does not delete; their lines name what
volumes. branch gc would do instead, its `docker compose down -v` deleting
the stack's volumes.
""" """
tip = (item["local_sha"] or item["head"] or "")[:12] tip = (item["local_sha"] or item["head"] or "")[:12]
merged = f", merged by {item['merged_by']}" if item["merged_by"] else "" merged = f", merged by {item['merged_by']}" if item["merged_by"] else ""
@@ -91,6 +141,10 @@ def dry_run(item: dict) -> list[str]:
lines += [f"branch gc would run {' '.join(GC_COMPOSE_DOWN).format(project=name)}, which deletes the volumes of {name}" for name in item["compose_projects"]] lines += [f"branch gc would run {' '.join(GC_COMPOSE_DOWN).format(project=name)}, which deletes the volumes of {name}" for name in item["compose_projects"]]
lines += [f"branch gc would remove the image {name}" for name in item["images"]] lines += [f"branch gc would remove the image {name}" for name in item["images"]]
return lines return lines
if kept:
lines.append(kept)
lines += [f"branch gc would delete the local branch {item['branch']}" if action == "local-branch" else f"branch gc would delete {action}" for action in item["actions"]]
return lines
for action in item["actions"]: for action in item["actions"]:
if action == "local-branch": if action == "local-branch":
lines.append(f"deletes the local branch {item['branch']}") lines.append(f"deletes the local branch {item['branch']}")
@@ -101,14 +155,25 @@ def dry_run(item: dict) -> list[str]:
return lines return lines
def branch(item: dict, path: str, default: object) -> dict: def branch(item: dict, path: str, default: object, planned: dict | None, upstream_of: str, stale: str) -> dict:
"""Args:
item: one tree item with a branch.
path: its repository's path.
default: the repository's default branch.
planned: the branch's item of the `--local` dry run, None without.
upstream_of: as kept_by() takes it.
stale: as kept_by() takes it.
"""
remote = next(iter(item["remotes"].values()), "") remote = next(iter(item["remotes"].values()), "")
head = item["local_sha"] or item["head"] or remote head = item["local_sha"] or item["head"] or remote
held = held_by(item["worktree"]) if item["worktree"] and item["state"] == "merged-clean" else "" held = held_by(item["worktree"]) if item["worktree"] and item["state"] == "merged-clean" else ""
open_to_delete = item["state"] == "merged-clean" and not item["skip"] and bool(item["local_sha"]) and not item["worktree"] candidate = item["state"] == "merged-clean" and bool(item["local_sha"]) and not item["worktree"]
kept = kept_by(planned, item["local_sha"], upstream_of, stale) if candidate else ""
open_to_delete = candidate and not item["skip"] and not kept
return { return {
"held": held, "held": held,
"token_free": signing.token_free(item, default), "kept": kept,
"token_free": signing.token_free(planned, default) if candidate and planned is not None else False,
"name": item["branch"], "name": item["branch"],
"head": head, "head": head,
"upstream": item["upstream"] or "", "upstream": item["upstream"] or "",
@@ -120,22 +185,45 @@ def branch(item: dict, path: str, default: object) -> dict:
"merged_into": item["merged_into"], "merged_into": item["merged_into"],
"skip": item["skip"], "skip": item["skip"],
"reason": item["reason"], "reason": item["reason"],
"plan": dry_run(item), "plan": dry_run(item, kept),
"cleanup": signing.cleanup_item(path, item["branch"], item["worktree"] or "", item["local_sha"]) if open_to_delete else None, "cleanup": signing.cleanup_item(path, item["branch"], item["worktree"] or "", item["local_sha"]) if open_to_delete else None,
"worktrees": [worktree(item)] if item["worktree"] else [], "worktrees": [worktree(item)] if item["worktree"] else [],
} }
def repo(entry: dict) -> dict: def repo(entry: dict, local: dict[tuple[str, str], tuple[dict, str]], stale: dict[str, str]) -> dict:
items = entry["items"] items = entry["items"]
return { return {
**{key: value for key, value in entry.items() if key != "items"}, **{key: value for key, value in entry.items() if key != "items"},
"name": "/".join(Path(entry["path"]).parts[-LABEL_PARTS:]), "name": "/".join(Path(entry["path"]).parts[-LABEL_PARTS:]),
"branches": [branch(item, entry["path"], entry.get("default")) for item in items if item["branch"]], "branches": [branch(item, entry["path"], entry.get("default"), *local.get((entry["path"], item["branch"]), (None, "")), stale.get(entry["path"], "")) for item in items if item["branch"]],
"detached": [worktree(item) for item in items if item["detached"]], "detached": [worktree(item) for item in items if item["detached"]],
} }
def local_plan(path: Path) -> tuple[dict[tuple[str, str], tuple[dict, str]], dict[str, str]]:
"""Returns:
Every item of the `pkgmgr branch cleanup --local` dry run that
client/local_plan.py writes at path, keyed by repository path and
branch, with the branch whose local upstream it is (local_upstreams,
"" for none); and each repository whose dry run is stale, with why.
Both are empty when the file is missing or unreadable, so the hub
offers no delete.
"""
try:
data = json.loads(path.read_text(encoding="utf-8"))
found, stale = {}, {}
for repository in data["repos"]:
items = [item for item in repository["items"] if isinstance(item, dict) and isinstance(item.get("branch"), str)]
upstreams = local_upstreams(items)
found.update({(repository["repo"], item["branch"]): (item, upstreams.get(item["branch"], "")) for item in items})
if repository.get("stale"):
stale[repository["repo"]] = str(repository["stale"])[:200]
return found, stale
except (OSError, ValueError, KeyError, TypeError, AttributeError):
return {}, {}
def offered(scan: dict, ready: bool) -> list[dict]: def offered(scan: dict, ready: bool) -> list[dict]:
"""Args: """Args:
scan: the tree. scan: the tree.
@@ -151,8 +239,9 @@ def offered(scan: dict, ready: bool) -> list[dict]:
def read_tree(path: Path) -> dict: def read_tree(path: Path) -> dict:
try: try:
data = json.loads(path.read_text(encoding="utf-8")) data = json.loads(path.read_text(encoding="utf-8"))
local, stale = local_plan(path.with_name(LOCAL_PLAN))
return { return {
"repos": sorted((repo(entry) for entry in data["repos"]), key=lambda entry: entry["name"]), "repos": sorted((repo(entry, local, stale) for entry in data["repos"]), key=lambda entry: entry["name"]),
"scanned": iso_epoch(data["generated_at"]), "scanned": iso_epoch(data["generated_at"]),
"delete_ready": data["delete_ready"], "delete_ready": data["delete_ready"],
"delete_reason": data["delete_reason"], "delete_reason": data["delete_reason"],
@@ -204,7 +293,7 @@ class Tree:
heartbeat: Path heartbeat: Path
lock: threading.Lock lock: threading.Lock
cached: dict | None cached: dict | None
stamp: tuple[int, int] | None stamp: tuple[tuple[int, int], tuple[int, int] | None] | None
def beat(self) -> None: def beat(self) -> None:
self.heartbeat.touch() self.heartbeat.touch()
@@ -218,7 +307,12 @@ class Tree:
f"no pkgmgr tree file at {self.file}: {error.strerror}. The tree timer runs `pkgmgr branch tree`, which the installed pkgmgr " f"no pkgmgr tree file at {self.file}: {error.strerror}. The tree timer runs `pkgmgr branch tree`, which the installed pkgmgr "
"lacks until a release ships it: release pkgmgr and install it, then the timer writes the file within a minute." "lacks until a release ships it: release pkgmgr and install it, then the timer writes the file within a minute."
)} )}
stamp = (info.st_mtime_ns, info.st_size) try:
planned = self.file.with_name(LOCAL_PLAN).stat()
plan_stamp = (planned.st_mtime_ns, planned.st_size)
except OSError:
plan_stamp = None
stamp = ((info.st_mtime_ns, info.st_size), plan_stamp)
if self.cached is None or stamp != self.stamp: if self.cached is None or stamp != self.stamp:
self.cached, self.stamp = read_tree(self.file), stamp self.cached, self.stamp = read_tree(self.file), stamp
return self.cached return self.cached
@@ -1,11 +1,12 @@
[Unit] [Unit]
Description=Agent Orchestrator tree: pkgmgr branch tree into the file the orchestrator reads, while its page polls Description=Agent Orchestrator tree: pkgmgr branch tree and the pkgmgr branch cleanup --local dry run into the files the orchestrator reads, while its page polls
[Service] [Service]
Type=oneshot Type=oneshot
Environment=PATH=/usr/bin:/bin Environment=PATH=/usr/bin:/bin
ExecCondition=/bin/sh -c 'test -n "$(find %h/.local/state/kvb-orchestrator/tree-heartbeat -mmin -5 2>/dev/null)"' ExecCondition=/bin/sh -c 'test -n "$(find %h/.local/state/kvb-orchestrator/tree-heartbeat -mmin -5 2>/dev/null)"'
ExecStart=%h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json ExecStart=%h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json
ExecStart=/usr/bin/env python3 %h/.local/share/kvb-orchestrator/client/local_plan.py --tree %h/.local/state/pkgmgr/tree/branch-tree.json --pkgmgr %h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr --budget 240
Nice=10 Nice=10
IOSchedulingClass=idle IOSchedulingClass=idle
CPUQuota=50% CPUQuota=50%
+9 -7
View File
@@ -1,6 +1,6 @@
const CLEANUP_POLL = 2000; const CLEANUP_POLL = 2000;
const CLEANUP_PATIENCE = 900000; const CLEANUP_PATIENCE = 900000;
const RESULT_LABELS = {done: ["deleted", "ok"], refused: ["refused", "bad"], failed: ["failed", "bad"]}; const RESULT_LABELS = {done: ["deleted", "ok"], gone: ["gone", "warn"], refused: ["refused", "bad"], failed: ["failed", "bad"]};
function planOf(item) { function planOf(item) {
const repo = (tree.data.repos || []).find((entry) => entry.path === item.repository); const repo = (tree.data.repos || []).find((entry) => entry.path === item.repository);
@@ -8,16 +8,18 @@
return {repo, entry}; return {repo, entry};
} }
function deleteDialog(items, locked = [], held = []) { function deleteDialog(items, locked = [], held = [], kept = []) {
if (!items.length) { if (!items.length) {
return; return;
} }
const rows = [...items, ...locked, ...held].map((item) => { const rows = [...items, ...locked, ...held, ...kept].map((item) => {
const {repo, entry} = planOf(item); const {repo, entry} = planOf(item);
const waits = locked.includes(item); const waits = locked.includes(item);
const kept = held.includes(item); const holds = held.includes(item);
return h("li", {class: waits ? "dry-run locked" : kept ? "dry-run held" : "dry-run", "data-token": waits ? "needed" : null, "data-held": kept && entry ? entry.held : null}, const keeps = kept.includes(item);
h("div", {class: "row"}, h("strong", {}, revealed(repo ? repo.name : item.repository)), h("code", {}, revealed(item.branch)), waits ? chip("bi-key", "token needed", "warn") : kept ? chip("bi-folder-symlink", "held by a worktree: not deleted here", "neutral") : null), return h("li", {class: waits ? "dry-run locked" : holds ? "dry-run held" : keeps ? "dry-run kept" : "dry-run", "data-token": waits ? "needed" : null, "data-held": holds && entry ? entry.held : null, "data-kept": keeps && entry ? entry.kept : null},
h("div", {class: "row"}, h("strong", {}, revealed(repo ? repo.name : item.repository)), h("code", {}, revealed(item.branch)),
waits ? chip("bi-key", "token needed", "warn") : holds ? chip("bi-folder-symlink", "held by a worktree: not deleted here", "neutral") : keeps ? chip("bi-sign-merge-left", "kept by the --local dry run: not deleted here", "neutral") : null),
h("pre", {}, (entry ? entry.plan : [`${item.branch} @ ${item.tip.slice(0, 12)}`]).map(revealed).join("\n"))); h("pre", {}, (entry ? entry.plan : [`${item.branch} @ ${item.tip.slice(0, 12)}`]).map(revealed).join("\n")));
}); });
const outcome = h("div", {class: "mini", "aria-live": "polite"}); const outcome = h("div", {class: "mini", "aria-live": "polite"});
@@ -29,7 +31,7 @@
} }
}); });
openSheet(items.length > 1 ? `Delete ${items.length} merged branches` : `Delete ${items[0].branch}`, [ openSheet(items.length > 1 ? `Delete ${items.length} merged branches` : `Delete ${items[0].branch}`, [
h("p", {class: "muted"}, "Dry run of pkgmgr branch cleanup for exactly these entries: it deletes the local branch and never touches a worktree, Compose stack or volume. No remote branch: the host refuses an entry whose plan deletes one. A branch held by a worktree is not deleted here: branch gc (kvb-git-gc, pkgmgr gc) removes it, and its docker compose down -v deletes the stack's volumes. The host checks your signature and refuses every entry whose state has changed since."), h("p", {class: "muted"}, "Dry run of pkgmgr branch cleanup for exactly these entries: it deletes the local branch and never touches a worktree, Compose stack or volume. No remote branch: the host refuses an entry whose plan deletes one. A branch held by a worktree is not deleted here: branch gc (kvb-git-gc, pkgmgr gc) removes it, and its docker compose down -v deletes the stack's volumes. Neither is a branch the pkgmgr branch cleanup --local dry run keeps, such as one merged by pull request, squash or patch whose local tip is no ancestor of the default branch: branch gc removes it. The host checks your signature and refuses every entry whose state has changed since."),
h("ul", {class: "mini", "aria-label": "Dry run"}, rows), h("ul", {class: "mini", "aria-label": "Dry run"}, rows),
outcome, outcome,
h("div", {class: "actions"}, h("span", {class: "spacer"}), btn("Cancel", "btn ghost", closeSheet), confirm), h("div", {class: "actions"}, h("span", {class: "spacer"}), btn("Cancel", "btn ghost", closeSheet), confirm),
+12 -4
View File
@@ -27,12 +27,15 @@
const FORGE_UNAVAILABLE = "forge unavailable"; const FORGE_UNAVAILABLE = "forge unavailable";
const deletableNow = (entry) => Boolean(entry.cleanup) && (tree.data.delete_ready === true || entry.token_free === true); const deletableNow = (entry) => Boolean(entry.cleanup) && (tree.data.delete_ready === true || entry.token_free === true);
const tokenLocked = (entry) => tree.data.delete_ready !== true && entry.state === "merged-clean" && !entry.held && (entry.cleanup ? entry.token_free !== true : entry.skip === FORGE_UNAVAILABLE); const tokenLocked = (entry) => tree.data.delete_ready !== true && entry.state === "merged-clean" && !entry.held && !entry.kept && (entry.cleanup ? entry.token_free !== true : entry.skip === FORGE_UNAVAILABLE);
function deleteButton(entry) { function deleteButton(entry) {
if (entry.held) { if (entry.held) {
return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral")); return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));
} }
if (entry.kept) {
return h("span", {class: "hint kept", title: entry.kept}, chip("bi-sign-merge-left", entry.kept, "neutral"));
}
if (!entry.cleanup) { if (!entry.cleanup) {
return tokenLocked(entry) ? chip("bi-key", "token needed", "warn") : null; return tokenLocked(entry) ? chip("bi-key", "token needed", "warn") : null;
} }
@@ -53,14 +56,17 @@
const items = branches.filter(deletableNow).map((entry) => entry.cleanup); const items = branches.filter(deletableNow).map((entry) => entry.cleanup);
const waiting = branches.filter(tokenLocked); const waiting = branches.filter(tokenLocked);
const locked = waiting.filter((entry) => entry.cleanup).map((entry) => entry.cleanup); const locked = waiting.filter((entry) => entry.cleanup).map((entry) => entry.cleanup);
const held = (tree.data.repos || []).flatMap((repo) => repo.branches.filter((entry) => entry.held).map((entry) => ({repository: repo.path, branch: entry.name, tip: entry.head}))); const preview = (key) => (tree.data.repos || []).flatMap((repo) => repo.branches.filter((entry) => entry[key]).map((entry) => ({repository: repo.path, branch: entry.name, tip: entry.head})));
const held = preview("held");
const kept = preview("kept");
const ready = items.length > 0; const ready = items.length > 0;
const hint = !ready && tree.data.delete_ready !== true ? `Delete all merged blocked: ${tree.data.delete_reason || "delete_ready is false"}` : items.length ? `Delete all merged: ${items.length} branches after dry run and signature` : "Delete all merged: no merged branch without a skip reason"; const hint = !ready && tree.data.delete_ready !== true ? `Delete all merged blocked: ${tree.data.delete_reason || "delete_ready is false"}` : items.length ? `Delete all merged: ${items.length} branches after dry run and signature` : "Delete all merged: no merged branch without a skip reason";
const button = iconBtn("bi-trash3-fill", "Delete all merged", "btn small danger", () => deleteDialog(items, locked, held), hint); const button = iconBtn("bi-trash3-fill", "Delete all merged", "btn small danger", () => deleteDialog(items, locked, held, kept), hint);
button.disabled = !ready; button.disabled = !ready;
return h("span", {class: "hint", title: hint}, button, items.length ? counter(items.length) : null, return h("span", {class: "hint", title: hint}, button, items.length ? counter(items.length) : null,
waiting.length ? chip("bi-key", `${waiting.length} token needed`, "warn", waiting.length) : null, waiting.length ? chip("bi-key", `${waiting.length} token needed`, "warn", waiting.length) : null,
held.length ? chip("bi-folder-symlink", `${held.length} held by a worktree: use branch gc`, "neutral", held.length) : null, held.length ? chip("bi-folder-symlink", `${held.length} held by a worktree: use branch gc`, "neutral", held.length) : null,
kept.length ? chip("bi-sign-merge-left", `${kept.length} kept by the --local dry run: see each dry run`, "neutral", kept.length) : null,
tree.data.delete_ready !== true ? iconBtn("bi-key-fill", "Store token", "btn small soft", storeToken, "Store token: add the operator row that stores the GitHub token in the Secret Service") : null); tree.data.delete_ready !== true ? iconBtn("bi-key-fill", "Store token", "btn small soft", storeToken, "Store token: add the operator row that stores the GitHub token in the Secret Service") : null);
} }
@@ -92,8 +98,10 @@
return found.length ? found[found.length - 1] : null; return found.length ? found[found.length - 1] : null;
} }
const RESULT_CHIPS = {done: ["bi-check2-all", "deleted", "ok"], gone: ["bi-question-octagon", "gone", "warn"], refused: ["bi-x-octagon", "refused", "bad"], failed: ["bi-exclamation-triangle", "failed", "bad"]};
function resultChip(result) { function resultChip(result) {
const [name, label, tone] = result.status === "done" ? ["bi-check2-all", "deleted", "ok"] : result.status === "refused" ? ["bi-x-octagon", "refused", "bad"] : ["bi-exclamation-triangle", "failed", "bad"]; const [name, label, tone] = RESULT_CHIPS[result.status] || RESULT_CHIPS.failed;
return chip(name, `${label}${result.reason ? `: ${result.reason}` : ""}`, tone); return chip(name, `${label}${result.reason ? `: ${result.reason}` : ""}`, tone);
} }
+144
View File
@@ -0,0 +1,144 @@
{
"repos": [
{
"repo": "@REPO@",
"default": "refs/remotes/origin/main",
"merged_into": [],
"warnings": [],
"items": [
{
"branch": "main",
"worktree": "@REPO@",
"head": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f",
"detached": false,
"locked": null,
"worktree_state": "clean",
"staged": 0,
"unstaged": 0,
"untracked": 0,
"upstream": null,
"ahead": null,
"behind": null,
"gone": false,
"local_sha": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f",
"remotes": {
"origin": "f48b13b73a443ae8a887a0f6b0190cfb66a3fc3f"
},
"compose_projects": [],
"images": [],
"state": "protected",
"merged": false,
"dirty": 0,
"merged_by": null,
"merged_into": null,
"unpushed": null,
"last_activity": null,
"size_bytes": null,
"skip": null,
"actions": [],
"reason": "default or protected branch",
"result": null
},
{
"branch": "side",
"worktree": null,
"head": null,
"detached": false,
"locked": null,
"worktree_state": null,
"staged": null,
"unstaged": null,
"untracked": null,
"upstream": null,
"ahead": null,
"behind": null,
"gone": false,
"local_sha": "cf287347ef836eeebeab3c87cdb149d99a3cee91",
"remotes": {},
"compose_projects": [],
"images": [],
"state": "merged-clean",
"merged": true,
"dirty": 0,
"merged_by": "ancestor",
"merged_into": "refs/remotes/origin/main",
"unpushed": null,
"last_activity": null,
"size_bytes": null,
"skip": null,
"actions": [
"local-branch"
],
"reason": "merged by ancestor",
"result": null
},
{
"branch": "squashed",
"worktree": null,
"head": null,
"detached": false,
"locked": null,
"worktree_state": null,
"staged": null,
"unstaged": null,
"untracked": null,
"upstream": null,
"ahead": null,
"behind": null,
"gone": false,
"local_sha": "73e7e81781ea1bf4c3fb0b1a9aa99e7809529e7d",
"remotes": {},
"compose_projects": [],
"images": [],
"state": "merged-clean",
"merged": true,
"dirty": 0,
"merged_by": "patch",
"merged_into": "refs/remotes/origin/main",
"unpushed": null,
"last_activity": null,
"size_bytes": null,
"skip": "merged by patch, tip is no ancestor of refs/remotes/origin/main",
"actions": [],
"reason": "merged by patch",
"result": null
},
{
"branch": "topic",
"worktree": null,
"head": null,
"detached": false,
"locked": null,
"worktree_state": null,
"staged": null,
"unstaged": null,
"untracked": null,
"upstream": null,
"ahead": null,
"behind": null,
"gone": false,
"local_sha": "ab372c4ea8b3ee5d062f1f5ea43c8154bf70b62a",
"remotes": {
"origin": "99dce520c1c6076008c6178cba98b360d41dbd0d"
},
"compose_projects": [],
"images": [],
"state": "merged-clean",
"merged": true,
"dirty": 0,
"merged_by": "patch",
"merged_into": "refs/remotes/origin/main",
"unpushed": null,
"last_activity": null,
"size_bytes": null,
"skip": null,
"actions": [
"local-branch"
],
"reason": "merged by patch",
"result": null
}
]
}
]
}
+2 -2
View File
@@ -78,7 +78,7 @@
"demo:feature-done" "demo:feature-done"
], ],
"state": "merged-clean", "state": "merged-clean",
"merged_by": "#12", "merged_by": "pr#12",
"merged_into": "main", "merged_into": "main",
"size_bytes": 4096, "size_bytes": 4096,
"skip": null, "skip": null,
@@ -87,7 +87,7 @@
"local-branch", "local-branch",
"remote-branch" "remote-branch"
], ],
"reason": "merged into main by #12", "reason": "merged by pr#12",
"result": null, "result": null,
"flow": null "flow": null
}, },
+1 -1
View File
@@ -297,7 +297,7 @@ class TestOrchestratorGaps(HubCase):
self.assertNotIn("error", icons, icons) self.assertNotIn("error", icons, icons)
chips = icons["chips"] chips = icons["chips"]
expected = { expected = {
"merged by #12 into main": "#bi-sign-merge-left", "clean": "#bi-check-circle", "not committed: 0 staged, 0 changed, 1 new": "#bi-pencil-square", "1 commits unpushed": "#bi-cloud-upload", "merged by pr#12 into main": "#bi-sign-merge-left", "clean": "#bi-check-circle", "not committed: 0 staged, 0 changed, 1 new": "#bi-pencil-square", "1 commits unpushed": "#bi-cloud-upload",
"upstream gone": "#bi-cloud-slash", f"locked: {LOCKED}": "#bi-lock-fill", "stale": "#bi-hourglass-bottom", "protected": "#bi-shield-lock", "upstream gone": "#bi-cloud-slash", f"locked: {LOCKED}": "#bi-lock-fill", "stale": "#bi-hourglass-bottom", "protected": "#bi-shield-lock",
} }
for label, href in expected.items(): for label, href in expected.items():
+84 -6
View File
@@ -384,6 +384,45 @@ AFTER = """(async () => {@HELPERS@
const past = await until(() => all("#main details.repo").find((details) => details.textContent.includes("Recently deleted")), 15000); const past = await until(() => all("#main details.repo").find((details) => details.textContent.includes("Recently deleted")), 15000);
return {past: past ? [past.querySelector("[data-result]").dataset.result, name(past.querySelector("span.state"))] : null, branches: all("#main details.repo").filter((details) => details !== past).flatMap((details) => all(".branch-name strong", details)).map(text)}; return {past: past ? [past.querySelector("[data-result]").dataset.result, name(past.querySelector("span.state"))] : null, branches: all("#main details.repo").filter((details) => details !== past).flatMap((details) => all(".branch-name strong", details)).map(text)};
})()""".replace("@HELPERS@", HELPERS) })()""".replace("@HELPERS@", HELPERS)
KEPT_TREE = """(async () => {@HELPERS@
const card = await until(() => all("#main details.repo").find((details) => details.textContent.includes("github.com/me/demo")), 8000);
if (!card) {
return {error: text(document.getElementById("main"))};
}
card.open = true;
await sleep(300);
const row = all(".branch-row", card).find((entry) => text(entry.querySelector(".branch-name strong")) === "feature/squashed");
const kept = row ? row.querySelector(".hint.kept") : null;
const deletes = all('button[aria-label="Delete"]', card).map((button) => text(button.closest(".branch-row").querySelector(".branch-name strong")));
const header = all("#main .delete-all span.state").map(name);
document.querySelector('#main button[aria-label="Delete all merged"]').click();
const sheet = await until(() => document.querySelector("#sheet:not([hidden]) li.dry-run") && document.getElementById("sheet"), 8000);
const kind = (li) => (li.dataset.token ? "token" : li.dataset.held ? "held" : li.dataset.kept ? "kept" : null);
const rows = sheet ? all("li.dry-run", sheet).map((li) => [text(li.querySelector("code")), kind(li), all("button", li).length, text(li.querySelector("pre"))]) : null;
const confirms = sheet ? all("#sheet button").filter((button) => name(button) === "Delete").length : null;
const title = text(document.querySelector(".sheet-head .panel-title"));
document.querySelector(".sheet-head button").click();
return {kept: kept ? [kept.title, name(kept.querySelector("span.state"))] : null, rowDeletes: row ? all('button[aria-label="Delete"]', row).length : null, deletes, header, rows, confirms, title};
})()""".replace("@HELPERS@", HELPERS)
KEPT_POPUP = """(async () => {@HELPERS@
await screenShown(8000);
const picker = await until(() => document.querySelector('#main select[aria-label="Repository"]'), 8000);
if (picker && picker.value !== @REPO@) {
picker.value = @REPO@;
picker.dispatchEvent(new Event("change", {bubbles: true}));
}
const id = `branch:${@REPO@}\\u0000feature/squashed`;
const target = await until(() => all("#main svg [data-pick]").find((node) => node.dataset.pick === id), 8000);
if (!target) {
return {error: text(document.getElementById("main"))};
}
target.dispatchEvent(new MouseEvent("click", {bubbles: true, cancelable: true}));
const dialog = document.getElementById("popup");
const dry = await until(() => dialog.open && dialog.querySelector(".popup-box pre.notes"), 6000);
const kept = dialog.querySelector(".popup-box .actions .hint.kept");
return {title: text(dialog.querySelector(".popup-box .panel-title")), heading: dry ? text(dry.previousElementSibling) : null, dry: dry ? dry.textContent : null,
deletes: all('#popup button[aria-label="Delete"]').length, kept: kept ? kept.title : null};
})()""".replace("@HELPERS@", HELPERS)
OVERLAP = """ OVERLAP = """
const shown = (node) => node.getClientRects().length > 0 && node.getBoundingClientRect().width > 0; const shown = (node) => node.getClientRects().length > 0 && node.getBoundingClientRect().width > 0;
const words = (label) => [...label.childNodes].filter((node) => node.nodeType === 3).map((node) => node.textContent).join("").trim(); const words = (label) => [...label.childNodes].filter((node) => node.nodeType === 3).map((node) => node.textContent).join("").trim();
@@ -1884,7 +1923,7 @@ class TestOrchestratorE2E(HubCase):
self.assertIn(["twig", "false"], by_hash["#agents/tree"]["spawn"]) self.assertIn(["twig", "false"], by_hash["#agents/tree"]["spawn"])
artifacts = by_hash["#artifacts/tree"] artifacts = by_hash["#artifacts/tree"]
self.assertEqual(artifacts["deletes"], [["Delete all merged", True, "Delete all merged blocked: Secret Service locked"]]) self.assertEqual(artifacts["deletes"], [["Delete all merged", True, "Delete all merged blocked: Secret Service locked"]])
for state in ("merged by #12 into main", "upstream gone", "protected", "1 commits unpushed", "clean", "not committed: 0 staged, 0 changed, 1 new", "gc skips: ignored files beyond known caches: notes.txt"): for state in ("merged by pr#12 into main", "upstream gone", "protected", "1 commits unpushed", "clean", "not committed: 0 staged, 0 changed, 1 new", "gc skips: ignored files beyond known caches: notes.txt"):
with self.subTest(state=state): with self.subTest(state=state):
self.assertIn(state, artifacts["states"]) self.assertIn(state, artifacts["states"])
for view in ("orbit", "chord"): for view in ("orbit", "chord"):
@@ -2237,15 +2276,16 @@ class TestOrchestratorE2E(HubCase):
def test_chromium_deletes_a_merged_branch_after_dry_run_and_signature(self): def test_chromium_deletes_a_merged_branch_after_dry_run_and_signature(self):
tree = json.loads(self.tree_file.read_text(encoding="utf-8")) tree = json.loads(self.tree_file.read_text(encoding="utf-8"))
tree["delete_ready"], tree["delete_reason"] = True, "" tree["delete_ready"], tree["delete_reason"] = True, ""
next(item for item in tree["repos"][0]["items"] if item["branch"] == "feature/done").update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[]) next(item for item in tree["repos"][0]["items"] if item["branch"] == "feature/done").update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[], merged_by="ancestor")
self.tree_file.write_text(json.dumps(tree), encoding="utf-8") self.tree_file.write_text(json.dumps(tree), encoding="utf-8")
self.write_local_plan(tree, {})
demo = f"{self.root}/repos/github.com/me/demo" demo = f"{self.root}/repos/github.com/me/demo"
Path(demo).mkdir(parents=True) Path(demo).mkdir(parents=True)
fake = self.outside / "bin" / "pkgmgr" fake = self.outside / "bin" / "pkgmgr"
fake.parent.mkdir() fake.parent.mkdir()
fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8") fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8")
fake.chmod(0o755) fake.chmod(0o755)
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "feature/done", "worktree": None, "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "skip": None, "actions": ["local-branch"]}]), encoding="utf-8") (fake.parent / "plan.json").write_text(json.dumps([{"branch": "feature/done", "worktree": None, "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"]}]), encoding="utf-8")
following = json.loads(json.dumps(tree)) following = json.loads(json.dumps(tree))
following["repos"][0]["items"] = [item for item in following["repos"][0]["items"] if item["branch"] != "feature/done"] following["repos"][0]["items"] = [item for item in following["repos"][0]["items"] if item["branch"] != "feature/done"]
(fake.parent / "next-tree.json").write_text(json.dumps(following), encoding="utf-8") (fake.parent / "next-tree.json").write_text(json.dumps(following), encoding="utf-8")
@@ -2269,7 +2309,7 @@ class TestOrchestratorE2E(HubCase):
self.assertEqual(opened["buttons"], [[False, "Delete: remove feature/done locally, after dry run and signature"]]) self.assertEqual(opened["buttons"], [[False, "Delete: remove feature/done locally, after dry run and signature"]])
self.assertEqual(opened["title"], "Delete feature/done") self.assertEqual(opened["title"], "Delete feature/done")
self.assertEqual(len(opened["dry"]), 1) self.assertEqual(len(opened["dry"]), 1)
self.assertTrue(opened["dry"][0].startswith("feature/done @ 5b1e6fd3a7c0: merged-clean, merged by #12"), opened["dry"]) self.assertTrue(opened["dry"][0].startswith("feature/done @ 5b1e6fd3a7c0: merged-clean, merged by ancestor"), opened["dry"])
self.assertIn("stays: remote-branch (local only; the host refuses a plan that deletes it)", opened["dry"][0]) self.assertIn("stays: remote-branch (local only; the host refuses a plan that deletes it)", opened["dry"][0])
self.assertEqual((confirmed["result"], confirmed["text"]), ("done", "feature/donedeleted"), confirmed) self.assertEqual((confirmed["result"], confirmed["text"]), ("done", "feature/donedeleted"), confirmed)
self.assertEqual(confirmed["toasts"], ["Delete: donefeature/done: deleted"]) self.assertEqual(confirmed["toasts"], ["Delete: donefeature/done: deleted"])
@@ -2279,11 +2319,49 @@ class TestOrchestratorE2E(HubCase):
self.assertEqual([[(item["branch"], item["status"]) for item in result["items"]] for result in written], [[("feature/done", "done")]]) self.assertEqual([[(item["branch"], item["status"]) for item in result["items"]] for result in written], [[("feature/done", "done")]])
calls = [json.loads(line)["argv"] for line in (fake.parent / "calls.jsonl").read_text(encoding="utf-8").splitlines()] calls = [json.loads(line)["argv"] for line in (fake.parent / "calls.jsonl").read_text(encoding="utf-8").splitlines()]
self.assertEqual(calls, [ self.assertEqual(calls, [
["branch", "cleanup", "--json", "--branch", "feature/done"], ["branch", "cleanup", "--local", "--json", "--branch", "feature/done"],
["branch", "cleanup", "--apply", "--json", "--branch", "feature/done"], ["branch", "cleanup", "--apply", "--local", "--json", "--branch", "feature/done"],
["branch", "tree", "--format", "json", "--output", str(self.tree_file)], ["branch", "tree", "--format", "json", "--output", str(self.tree_file)],
]) ])
def test_chromium_shows_a_branch_the_local_dry_run_keeps_without_delete_and_offers_one_whose_local_tip_it_deletes(self):
squashed_tip, cherry_tip, cherry_remote = "9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b", "8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b9c", "7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b9c8d"
tree = json.loads(self.tree_file.read_text(encoding="utf-8"))
tree["delete_ready"], tree["delete_reason"] = True, ""
items = tree["repos"][0]["items"]
kept = next(item for item in items if item["branch"] == "feature/kept")
kept["skip"] = None
items.append({**kept, "branch": "feature/squashed", "local_sha": squashed_tip, "upstream": "origin/feature/squashed", "ahead": 0, "behind": 0, "remotes": {"origin": squashed_tip},
"merged_by": "pr#7", "actions": ["local-branch", "remote-branch:origin"], "reason": "merged by pr#7"})
items.append({**kept, "branch": "feature/cherry", "local_sha": cherry_tip, "upstream": "origin/feature/cherry", "ahead": 0, "behind": 1, "remotes": {"origin": cherry_remote},
"merged_by": "patch", "actions": ["local-branch", "remote-branch:origin"], "reason": "merged by patch"})
self.tree_file.write_text(json.dumps(tree), encoding="utf-8")
self.write_local_plan(tree, {"feature/cherry": {"skip": None, "actions": ["local-branch"]}})
demo = f"{self.root}/repos/github.com/me/demo"
local = f"http://localhost:{self.port}/"
steps = [
{"width": 1366, "height": 768, "url": f"{local}#artifacts/tree", "wait": 1200, "probe": KEPT_TREE, "shot": self.shot("1366-kept-tree")},
{"width": 1366, "height": 768, "url": f"{local}?kept#artifacts/orbit", "wait": 1200, "probe": KEPT_POPUP.replace("@REPO@", json.dumps(demo)), "shot": self.shot("1366-kept-popup")},
]
data = self.drive(steps, None)
(OUT / "chromium-kept.json").write_text(json.dumps(data, indent=1), encoding="utf-8")
self.assertEqual([error for error in data["errors"] if "/favicon.ico" not in error], [])
shown, popup = data["results"]
self.assertNotIn("error", shown, shown)
self.assertNotIn("error", popup, popup)
kept_line = "pkgmgr branch cleanup --local keeps it: merged by pr#7, tip is no ancestor of refs/remotes/origin/main; use branch gc (kvb-git-gc, pkgmgr gc)"
plan = f"feature/squashed @ {squashed_tip[:12]}: merged-clean, merged by pr#7\n{kept_line}\nbranch gc would delete the local branch feature/squashed\nbranch gc would delete remote-branch:origin"
self.assertEqual((shown["kept"], shown["rowDeletes"], shown["deletes"]), ([kept_line, kept_line], 0, ["feature/kept", "feature/cherry"]))
self.assertIn("1 kept by the --local dry run: see each dry run", shown["header"])
self.assertEqual([row[:3] for row in shown["rows"]], [["feature/kept", None, 0], ["feature/cherry", None, 0], ["feature/done", "held", 0], ["feature/squashed", "kept", 0]])
self.assertEqual(shown["rows"][1][3], f"feature/cherry @ {cherry_tip[:12]}: merged-clean, merged by patch\ndeletes the local branch feature/cherry\nstays: remote-branch:origin (local only; the host refuses a plan that deletes it)")
self.assertEqual(shown["rows"][3][3], plan)
self.assertEqual((shown["confirms"], shown["title"]), (1, "Delete 2 merged branches"))
self.assertEqual((popup["title"], popup["heading"], popup["dry"], popup["deletes"], popup["kept"]), ("feature/squashed", "Dry run", plan, 0, kept_line))
status, _, body = self.post_json("/api/sign", {"items": [{"kind": "cleanup", "repository": demo, "branch": "feature/squashed", "worktree": "", "tip": squashed_tip}]}, self.guard(self.host))
self.assertEqual((status, body.decode()), (400, f"feature/squashed in {demo} is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"))
self.assertEqual(list((self.cleanup / "jobs").glob("*.json")), [])
def test_chromium_folds_the_hierarchy_by_keyboard_reorders_agents_and_hands_an_office_over_with_a_signature(self): def test_chromium_folds_the_hierarchy_by_keyboard_reorders_agents_and_hands_an_office_over_with_a_signature(self):
signer = self.pin() signer = self.pin()
trust = verifier.Trust(approvers=self.approvers, replay=self.replay / "offices.json", origin=self.origin) trust = verifier.Trust(approvers=self.approvers, replay=self.replay / "offices.json", origin=self.origin)
+32
View File
@@ -242,8 +242,40 @@ class HubCase(unittest.TestCase):
root = self.outside / "scan" root = self.outside / "scan"
sample = (Path(__file__).parent / fixture).read_text(encoding="utf-8") sample = (Path(__file__).parent / fixture).read_text(encoding="utf-8")
self.tree_file.write_text(sample.replace("@ROOT@", str(root)), encoding="utf-8") self.tree_file.write_text(sample.replace("@ROOT@", str(root)), encoding="utf-8")
self.write_local_plan(json.loads(self.tree_file.read_text(encoding="utf-8")), {})
return root return root
def write_local_plan(self, tree: dict, local: dict[str, dict]) -> None:
"""Args:
tree: a tree file's data; each branch becomes the item `pkgmgr
branch cleanup --local --json` plans for it as gc.py
_local_actions does: a merged-clean local branch without
worktree merged by ancestry gets exactly local-branch, any
other merged-clean one the skip pkgmgr gives a tip that is no
ancestor of the default branch.
local: branch -> fields that replace the modelled ones, for a
local tip that is an ancestor while the tree's merged_by,
which covers every tip, is not.
"""
repos = []
for entry in tree["repos"]:
default = f"refs/remotes/origin/{entry.get('default')}"
items = []
for item in entry["items"]:
if not item["branch"]:
continue
candidate = item["state"] == "merged-clean" and bool(item["local_sha"]) and not item["worktree"]
ancestry = candidate and item["merged_by"] == "ancestor"
planned = {
"branch": item["branch"], "worktree": item["worktree"], "head": item["head"], "local_sha": item["local_sha"], "remotes": item["remotes"], "upstream": item["upstream"], "state": item["state"],
"merged_by": item["merged_by"], "merged_into": default if item["state"] == "merged-clean" else None,
"skip": f"merged by {item['merged_by']}, tip is no ancestor of {default}" if candidate and not ancestry else None,
"actions": ["local-branch"] if ancestry else [], "result": None,
}
items.append({**planned, **local.get(item["branch"], {})})
repos.append({"repo": entry["path"], "default": default, "merged_into": [], "warnings": [], "items": items})
self.tree_file.with_name("local-plan.json").write_text(json.dumps({"repos": repos}), encoding="utf-8")
def jsonl(self, rows: list[dict]) -> str: def jsonl(self, rows: list[dict]) -> str:
return "".join(json.dumps(row) + "\n" for row in rows) return "".join(json.dumps(row) + "\n" for row in rows)
+1 -1
View File
@@ -106,7 +106,7 @@ class TestHubArtifacts(HubCase):
self.assertEqual(branches["feature"]["worktrees"][0]["artefacts"], []) self.assertEqual(branches["feature"]["worktrees"][0]["artefacts"], [])
self.assertEqual([(task["task"], [leaf["ref"] for leaf in task["artefacts"]]) for task in data["tasks"]], [("work/loose", ["log:/tmp/loose.log"])]) self.assertEqual([(task["task"], [leaf["ref"] for leaf in task["artefacts"]]) for task in data["tasks"]], [("work/loose", ["log:/tmp/loose.log"])])
self.assertEqual(repo["overlaps"], [{"a": "feature", "b": "feature/done", "count": 1, "files": ["a.py"]}]) self.assertEqual(repo["overlaps"], [{"a": "feature", "b": "feature/done", "count": 1, "files": ["a.py"]}])
self.assertEqual(repo["merges"], [{"from": "feature/done", "into": "main", "by": "#12"}, {"from": "feature/kept", "into": "main", "by": "ancestor"}]) self.assertEqual(repo["merges"], [{"from": "feature/done", "into": "main", "by": "pr#12"}, {"from": "feature/kept", "into": "main", "by": "ancestor"}])
self.assertEqual((branches["feature"]["files"], branches["feature/done"]["files"]), (2, 2)) self.assertEqual((branches["feature"]["files"], branches["feature/done"]["files"]), (2, 2))
kinds = [(event["kind"], event["ok"]) for event in branches["feature/done"]["events"]] kinds = [(event["kind"], event["ok"]) for event in branches["feature/done"]["events"]]
self.assertEqual(kinds, [("gate", True), ("review", None), ("gate", True), ("commit", True)]) self.assertEqual(kinds, [("gate", True), ("review", None), ("gate", True), ("commit", True)])
+419 -28
View File
@@ -16,6 +16,7 @@ from pathlib import Path
from hub_case import HubCase from hub_case import HubCase
from hub_support import GIT_ENV, HUB, ORIGIN, key, signing, verifier from hub_support import GIT_ENV, HUB, ORIGIN, key, signing, verifier
from kvb_orchestrator import tree
CLEANUP = HUB / "client" / "cleanup.py" CLEANUP = HUB / "client" / "cleanup.py"
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_runner", CLEANUP) CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_runner", CLEANUP)
@@ -24,24 +25,28 @@ CLEANUP_SPEC.loader.exec_module(host)
INSTALLED_PKGMGR = shutil.which("pkgmgr") INSTALLED_PKGMGR = shutil.which("pkgmgr")
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4" DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
OTHER_TIP = "1111111111111111111111111111111111111111" OTHER_TIP = "1111111111111111111111111111111111111111"
# Param: format with python=<interpreter>; parses argv as pkgmgr's branch_cmd.py does (argparse exits 2 on any other flag) and prints the plan as gc.py _render does, {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}; plan.json beside it is the plan's items, next-tree.json the rewritten tree, outcome.txt the result --apply reports per item (done without it), exit 0 also for a skipped item. # Param: format with python=<interpreter>; parses argv as pkgmgr's branch_cmd.py does (argparse exits 2 on any other flag) and prints the plan as gc.py _render does, {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}; plan.json beside it is the plan's items, apply.json the items --apply plans instead when present, next-tree.json the rewritten tree, outcome.txt the result --apply reports per item without skip (done without it; an item with skip gets none, as pkgmgr applies only eligible items), killed makes --apply exit 137 without output, unknown.json {"plan": [names], "apply": [names]} the branches that vanished before that run (pkgmgr's "Unknown --branch: a, b" on stderr and exit 2 when one is asked for, else left out of the plan), slow.txt names a cwd it sleeps 30 s in before planning, exit 0 also for a skipped item.
FAKE_PKGMGR = """#!{python} FAKE_PKGMGR = """#!{python}
import argparse import argparse
import json import json
import os import os
import sys import sys
import time
from pathlib import Path from pathlib import Path
here = Path(__file__).parent here = Path(__file__).parent
with (here / "calls.jsonl").open("a", encoding="utf-8") as log: with (here / "calls.jsonl").open("a", encoding="utf-8") as log:
log.write(json.dumps({{"argv": sys.argv[1:], "cwd": os.getcwd()}}) + "\\n") log.write(json.dumps({{"argv": sys.argv[1:], "cwd": os.getcwd()}}) + "\\n")
slow = here / "slow.txt"
if slow.exists() and slow.read_text(encoding="utf-8") == os.getcwd():
time.sleep(30)
parser = argparse.ArgumentParser(prog="pkgmgr") parser = argparse.ArgumentParser(prog="pkgmgr")
branch = parser.add_subparsers(dest="command", required=True).add_parser("branch").add_subparsers(dest="subcommand", required=True) branch = parser.add_subparsers(dest="command", required=True).add_parser("branch").add_subparsers(dest="subcommand", required=True)
tree = branch.add_parser("tree") tree = branch.add_parser("tree")
tree.add_argument("--format", choices=("json", "yaml"), default="json") tree.add_argument("--format", choices=("json", "yaml"), default="json")
tree.add_argument("--output") tree.add_argument("--output")
cleanup = branch.add_parser("cleanup") cleanup = branch.add_parser("cleanup")
for flag in ("--apply", "--json", "--all", "--sizes"): for flag in ("--apply", "--json", "--local", "--all", "--sizes"):
cleanup.add_argument(flag, action="store_true") cleanup.add_argument(flag, action="store_true")
for option in ("--branch", "--exclude", "--merged-into"): for option in ("--branch", "--exclude", "--merged-into"):
cleanup.add_argument(option, action="append", default=[]) cleanup.add_argument(option, action="append", default=[])
@@ -53,31 +58,112 @@ if args.subcommand == "tree":
Path(args.output).write_text(rewritten, encoding="utf-8") Path(args.output).write_text(rewritten, encoding="utf-8")
raise SystemExit(0) raise SystemExit(0)
items = json.loads((here / "plan.json").read_text(encoding="utf-8")) items = json.loads((here / "plan.json").read_text(encoding="utf-8"))
if args.apply and (here / "apply.json").exists():
items = json.loads((here / "apply.json").read_text(encoding="utf-8"))
unknown = here / "unknown.json"
vanished = json.loads(unknown.read_text(encoding="utf-8")).get("apply" if args.apply else "plan", []) if unknown.exists() else []
missing = [name for name in args.branch if name in vanished]
if missing:
print(f"Unknown --branch: {{', '.join(missing)}}", file=sys.stderr)
raise SystemExit(2)
items = [item for item in items if item["branch"] not in vanished]
if args.apply and (here / "killed").exists():
raise SystemExit(137)
if args.apply: if args.apply:
outcome = here / "outcome.txt" outcome = here / "outcome.txt"
result = outcome.read_text(encoding="utf-8") if outcome.exists() else "done" result = outcome.read_text(encoding="utf-8") if outcome.exists() else "done"
items = [{{**item, "result": result}} for item in items] items = [item if item.get("skip") else {{**item, "result": result}} for item in items]
plan = {{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}}]}} plan = {{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}}]}}
print(json.dumps(plan) if args.json else "\\n".join(str(item["branch"]) for item in items)) print(json.dumps(plan) if args.json else "\\n".join(str(item["branch"]) for item in items))
""" """
RECORDED = Path(__file__).parent / "branch-cleanup-local.json"
RECORDED_REPO = "@REPO@"
CAPTURE_DATE = "2026-11-14T12:00:00+00:00"
KEPT_PATCH = "merged by patch, tip is no ancestor of refs/remotes/origin/main"
def capture_local_plan(base: Path, pkgmgr: Path) -> tuple[Path, dict]:
"""Args:
base: an empty directory for the origin and the repository.
pkgmgr: the pkgmgr to plan with.
Returns:
The repository and the plan pkgmgr prints for `branch cleanup
--local --json` in it, built with fixed dates so every sha repeats:
side merged by ancestry; squashed squash-merged, so its tip is no
ancestor of main; topic, whose local tip is an ancestor of main
while its remote tip was cherry-picked onto main.
"""
env = {**os.environ, **GIT_ENV, "GIT_AUTHOR_DATE": CAPTURE_DATE, "GIT_COMMITTER_DATE": CAPTURE_DATE}
def git(cwd: Path, *args: str) -> str:
return subprocess.run(["git", *args], cwd=cwd, env=env, check=True, capture_output=True, text=True).stdout.strip()
def commit(repo: Path, name: str) -> str:
(repo / f"{name}.txt").write_text(name, encoding="utf-8")
git(repo, "add", f"{name}.txt")
git(repo, "commit", "-q", "-m", name)
return git(repo, "rev-parse", "HEAD")
origin, repo = base / "origin.git", base / "repo"
git(base, "init", "-q", "--bare", "-b", "main", str(origin))
git(base, "init", "-q", "-b", "main", str(repo))
git(repo, "remote", "add", "origin", origin.as_uri())
git(repo, "commit", "-q", "--allow-empty", "-m", "one")
for name in ("side", "squashed", "topic"):
git(repo, "checkout", "-q", "-b", name, "main")
commit(repo, name)
git(repo, "checkout", "-q", "topic")
remote_tip = commit(repo, "topic-remote")
git(repo, "push", "-q", "origin", "topic:topic")
git(repo, "reset", "-q", "--hard", "topic~1")
git(repo, "checkout", "-q", "main")
git(repo, "merge", "-q", "--no-ff", "-m", "merge side and topic", "side", "topic")
git(repo, "merge", "-q", "--squash", "squashed")
git(repo, "commit", "-q", "-m", "squashed as one commit")
git(repo, "cherry-pick", remote_tip)
git(repo, "push", "-q", "origin", "main:main")
git(repo, "fetch", "-q", "origin")
git(repo, "remote", "set-head", "origin", "main")
code, stdout, stderr = host.run(pkgmgr, ["branch", "cleanup", "--local", "--json"], str(repo), host.TIMEOUT)
if code != 0:
raise AssertionError(stderr)
return repo, json.loads(stdout)
def recorded_plan(repo: str) -> dict:
"""Returns:
The plan the installed pkgmgr printed for capture_local_plan(),
recorded in branch-cleanup-local.json, its repository named repo.
"""
return json.loads(RECORDED.read_text(encoding="utf-8").replace(RECORDED_REPO, repo))
class TestHubCleanup(HubCase): class TestHubCleanup(HubCase):
def ready(self, ready: bool, held: bool) -> dict: def ready(self, ready: bool, held: bool, merged_by: str, local: dict[str, dict]) -> dict:
"""Args: """Args:
ready: delete_ready of the tree file. ready: delete_ready of the tree file.
held: keep feature/done in its worktree; otherwise it is a local held: keep feature/done in its worktree; otherwise it is a local
branch without one. branch without one.
merged_by: what merged feature/done over all its tips, as pkgmgr
gc.py _merged_by names it: ancestor, pr#N, line:X or patch.
local: branch -> fields of its `--local` dry run item that
replace the ones HubCase.write_local_plan models.
""" """
root = self.write_tree() root = self.write_tree()
data = json.loads(self.tree_file.read_text(encoding="utf-8")) data = json.loads(self.tree_file.read_text(encoding="utf-8"))
data["delete_ready"], data["delete_reason"] = ready, "" if ready else "Secret Service locked" data["delete_ready"], data["delete_reason"] = ready, "" if ready else "Secret Service locked"
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done") done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
done["merged_by"] = merged_by
if not held: if not held:
done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], compose_projects=[], images=[]) done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch"], remotes={"origin": DONE_TIP}, compose_projects=[], images=[])
self.tree_file.write_text(json.dumps(data), encoding="utf-8") self.tree_file.write_text(json.dumps(data), encoding="utf-8")
self.write_local_plan(data, local)
return signing.cleanup_item(f"{root}/repos/github.com/me/demo", "feature/done", done["worktree"] or "", DONE_TIP) return signing.cleanup_item(f"{root}/repos/github.com/me/demo", "feature/done", done["worktree"] or "", DONE_TIP)
def refusal(self, item: dict) -> str:
return f"{item['branch']} in {item['repository']} is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"
def branches(self) -> dict: def branches(self) -> dict:
status, _, body = self.get("/api/tree") status, _, body = self.get("/api/tree")
self.assertEqual(status, 200, body) self.assertEqual(status, 200, body)
@@ -89,24 +175,109 @@ class TestHubCleanup(HubCase):
return status, body return status, body
def test_the_tree_offers_a_delete_only_for_merged_clean_branches_without_skip_reason(self): def test_the_tree_offers_a_delete_only_for_merged_clean_branches_without_skip_reason(self):
item = self.ready(False, False) item = self.ready(False, False, "ancestor", {})
branches = self.branches() branches = self.branches()
done = branches["feature/done"] done = branches["feature/done"]
self.assertEqual((done["cleanup"], done["held"]), (item, "")) self.assertEqual((done["cleanup"], done["held"], done["kept"]), (item, "", ""))
self.assertEqual(done["plan"], [ self.assertEqual(done["plan"], [
"feature/done @ 5b1e6fd3a7c0: merged-clean, merged by #12", "feature/done @ 5b1e6fd3a7c0: merged-clean, merged by ancestor",
"deletes the local branch feature/done", "deletes the local branch feature/done",
"stays: remote-branch (local only; the host refuses a plan that deletes it)", "stays: remote-branch (local only; the host refuses a plan that deletes it)",
]) ])
self.assertEqual((done["merged_by"], done["merged_into"], done["gone"]), ("#12", "main", True)) self.assertEqual((done["merged_by"], done["merged_into"], done["gone"]), ("ancestor", "main", True))
self.assertEqual({name: entry["cleanup"] for name, entry in branches.items() if name != "feature/done"}, {"feature": None, "feature/kept": None, "main": None}) self.assertEqual({name: entry["cleanup"] for name, entry in branches.items() if name != "feature/done"}, {"feature": None, "feature/kept": None, "main": None})
self.assertEqual(branches["feature/kept"]["skip"], "ignored files beyond known caches: notes.txt") self.assertEqual(branches["feature/kept"]["skip"], "ignored files beyond known caches: notes.txt")
def test_the_local_dry_run_decides_by_its_actions_not_by_what_merged_the_branch(self):
deletes = {"state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"]}
cases = (
("ancestry", deletes, ""),
("local tip an ancestor, remote tip patch-merged", {**deletes, "merged_by": "patch"}, ""),
("tip no ancestor", {**deletes, "merged_by": "patch", "skip": KEPT_PATCH, "actions": []}, KEPT_PATCH),
("not merged", {**deletes, "state": "unmerged", "actions": []}, "state is unmerged, not merged-clean"),
("no state", {"actions": ["local-branch"]}, "state is None, not merged-clean"),
("remote delete", {**deletes, "actions": ["local-branch", "remote-branch:origin"]}, "the plan also deletes remote-branch:origin; a signed delete is local only"),
("nothing planned", {**deletes, "actions": []}, "the dry run plans no delete, not exactly the local branch"),
("no actions", {key: value for key, value in deletes.items() if key != "actions"}, "the dry run plans no delete, not exactly the local branch"),
("worktree too", {**deletes, "actions": ["worktree", "local-branch"]}, "the dry run plans worktree, local-branch, not exactly the local branch"),
)
for name, entry, kept in cases:
with self.subTest(name=name):
self.assertEqual(signing.local_keeps(entry), kept)
def test_a_branch_the_local_dry_run_keeps_offers_no_delete_and_its_dry_run_says_why_from_that_dry_run(self):
for merged_by in ("pr#12", "patch"):
with self.subTest(merged_by=merged_by):
item = self.ready(True, False, merged_by, {})
done = self.branches()["feature/done"]
kept = f"pkgmgr branch cleanup --local keeps it: merged by {merged_by}, tip is no ancestor of refs/remotes/origin/main; use branch gc (kvb-git-gc, pkgmgr gc)"
self.assertEqual((done["cleanup"], done["held"], done["kept"], done["token_free"]), (None, "", kept, False))
self.assertEqual(done["plan"], [
f"feature/done @ 5b1e6fd3a7c0: merged-clean, merged by {merged_by}",
kept,
"branch gc would delete the local branch feature/done",
"branch gc would delete remote-branch",
])
status, body = self.sign([item])
self.assertEqual((status, body.decode()), (400, self.refusal(item)))
page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8")
self.assertIn('if (entry.kept) {\n return h("span", {class: "hint kept", title: entry.kept}, chip("bi-sign-merge-left", entry.kept, "neutral"));\n }\n if (!entry.cleanup) {', page)
self.assertIn("!entry.held && !entry.kept && (entry.cleanup", page)
self.assertIn("deleteDialog(items, locked, held, kept)", page)
self.assertIn('keeps ? chip("bi-sign-merge-left", "kept by the --local dry run: not deleted here", "neutral")', (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8"))
def test_a_branch_whose_local_tip_the_local_dry_run_deletes_is_offered_although_its_remote_tip_was_patch_merged(self):
item = self.ready(True, False, "patch", {"feature/done": {"skip": None, "actions": ["local-branch"]}})
done = self.branches()["feature/done"]
self.assertEqual((done["cleanup"], done["kept"], done["merged_by"]), (item, "", "patch"))
self.assertEqual(done["plan"][1], "deletes the local branch feature/done")
status, body = self.sign([item])
self.assertEqual(status, 200, body)
def test_a_branch_another_branch_tracks_as_local_upstream_is_kept_as_pkgmgr_keeps_it_when_the_host_chooses_it_alone(self):
kept = "pkgmgr branch cleanup --local keeps it: kept: local upstream of feature; use branch gc (kvb-git-gc, pkgmgr gc)"
cases = (
("local upstream", {"feature": {"upstream": "feature/done"}}, kept),
("local upstream git shortens with heads/", {"feature": {"upstream": "heads/feature/done"}}, kept),
("remote upstream of the same name", {"feature": {"upstream": "origin/feature/done"}}, ""),
("its own upstream", {"feature/done": {"upstream": "feature/done"}}, ""),
)
for name, local, expected in cases:
with self.subTest(name=name):
item = self.ready(True, False, "ancestor", local)
done = self.branches()["feature/done"]
self.assertEqual((done["cleanup"], done["kept"]), (None, expected) if expected else (item, ""))
if expected:
self.assertEqual(done["plan"][1], expected)
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
def test_a_repository_whose_replan_did_not_finish_offers_no_delete_and_says_why(self):
item = self.ready(True, False, "ancestor", {})
planned = self.tree_file.with_name("local-plan.json")
data = json.loads(planned.read_text(encoding="utf-8"))
data["repos"][0]["stale"] = "replan timed out at 2026-11-14T18:00:00+00:00"
planned.write_text(json.dumps(data), encoding="utf-8")
done = self.branches()["feature/done"]
stale = "pkgmgr branch cleanup --local plan stale: replan timed out at 2026-11-14T18:00:00+00:00; use branch gc (kvb-git-gc, pkgmgr gc)"
self.assertEqual((done["cleanup"], done["kept"], done["plan"][1]), (None, stale, stale))
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
def test_without_a_local_dry_run_of_the_tip_the_tree_offers_no_delete(self):
pending = "no pkgmgr branch cleanup --local dry run of 5b1e6fd3a7c0 yet: the tree timer writes local-plan.json within a minute"
for name, local in (("dry run of another tip", {"feature/done": {"local_sha": OTHER_TIP}}), ("no dry run file", None)):
with self.subTest(name=name):
item = self.ready(True, False, "ancestor", local or {})
if local is None:
self.tree_file.with_name("local-plan.json").unlink()
done = self.branches()["feature/done"]
self.assertEqual((done["cleanup"], done["kept"], done["plan"][1]), (None, pending, pending))
self.assertEqual(self.sign([item]), (400, self.refusal(item).encode()))
def test_signing_a_delete_needs_delete_ready_and_exactly_an_offered_item(self): def test_signing_a_delete_needs_delete_ready_and_exactly_an_offered_item(self):
item = self.ready(False, False) item = self.ready(False, False, "ancestor", {})
status, body = self.sign([item]) status, body = self.sign([item])
self.assertEqual((status, body.decode()), (400, "delete blocked: Secret Service locked\n")) self.assertEqual((status, body.decode()), (400, "delete blocked: Secret Service locked\n"))
item = self.ready(True, False) item = self.ready(True, False, "ancestor", {})
kept = {**item, "branch": "feature/kept", "worktree": "", "tip": "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"} kept = {**item, "branch": "feature/kept", "worktree": "", "tip": "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"}
for name, items in (("changed tip", [{**item, "tip": OTHER_TIP}]), ("skip reason", [kept]), ("pattern", [{**item, "branch": "feature/*"}]), ("twice", [item, item]), ("extra key", [{**item, "command": "rm -rf /"}])): for name, items in (("changed tip", [{**item, "tip": OTHER_TIP}]), ("skip reason", [kept]), ("pattern", [{**item, "branch": "feature/*"}]), ("twice", [item, item]), ("extra key", [{**item, "command": "rm -rf /"}])):
with self.subTest(name=name): with self.subTest(name=name):
@@ -116,15 +287,15 @@ class TestHubCleanup(HubCase):
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(item)]) self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(item)])
def test_a_branch_held_by_a_worktree_offers_no_delete_and_names_branch_gc(self): def test_a_branch_held_by_a_worktree_offers_no_delete_and_names_branch_gc(self):
item = self.ready(True, True) item = self.ready(True, True, "pr#12", {})
done = self.branches()["feature/done"] done = self.branches()["feature/done"]
held = f"held by worktree {item['worktree']}: pkgmgr branch cleanup skips it, use branch gc (kvb-git-gc, pkgmgr gc)" held = f"held by worktree {item['worktree']}: pkgmgr branch cleanup skips it, use branch gc (kvb-git-gc, pkgmgr gc)"
self.assertEqual((done["cleanup"], done["held"], done["plan"][1]), (None, held, held)) self.assertEqual((done["cleanup"], done["held"], done["kept"], done["plan"][1]), (None, held, "", held))
status, body = self.sign([item]) status, body = self.sign([item])
self.assertEqual((status, body.decode()), (400, f"feature/done in {item['repository']} is no merged-clean branch without skip reason at that tip in the tree file\n")) self.assertEqual((status, body.decode()), (400, self.refusal(item)))
page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8") page = (HUB / "templates" / "page-tree.js").read_text(encoding="utf-8")
self.assertIn('if (entry.held) {\n return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));', page) self.assertIn('if (entry.held) {\n return h("span", {class: "hint held", title: entry.held}, chip("bi-folder-symlink", entry.held, "neutral"));', page)
self.assertIn("deleteDialog(items, locked, held)", page) self.assertIn("deleteDialog(items, locked, held, kept)", page)
self.assertIn("held by a worktree: not deleted here", (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8")) self.assertIn("held by a worktree: not deleted here", (HUB / "templates" / "page-cleanup.js").read_text(encoding="utf-8"))
def test_the_delete_sheet_renders_the_dry_run_note_as_visible_text(self): def test_the_delete_sheet_renders_the_dry_run_note_as_visible_text(self):
@@ -132,7 +303,8 @@ class TestHubCleanup(HubCase):
sheet = dialog[dialog.index("openSheet("):] sheet = dialog[dialog.index("openSheet("):]
note = re.search(r'\[\n\s*h\("p", \{class: "muted"\}, "([^"]+)"\),\n', sheet) note = re.search(r'\[\n\s*h\("p", \{class: "muted"\}, "([^"]+)"\),\n', sheet)
self.assertIsNotNone(note, sheet[:400]) self.assertIsNotNone(note, sheet[:400])
for part in ("it deletes the local branch", "never touches a worktree, Compose stack or volume", "A branch held by a worktree is not deleted here", "its docker compose down -v deletes the stack's volumes", "the host refuses an entry whose plan deletes one", "refuses every entry whose state has changed since"): for part in ("it deletes the local branch", "never touches a worktree, Compose stack or volume", "A branch held by a worktree is not deleted here", "its docker compose down -v deletes the stack's volumes", "the host refuses an entry whose plan deletes one", "refuses every entry whose state has changed since",
"Neither is a branch the pkgmgr branch cleanup --local dry run keeps, such as one merged by pull request, squash or patch whose local tip is no ancestor of the default branch: branch gc removes it"):
self.assertIn(part, note.group(1)) self.assertIn(part, note.group(1))
self.assertEqual(dialog.count(note.group(1)), 1) self.assertEqual(dialog.count(note.group(1)), 1)
@@ -143,7 +315,7 @@ class TestHubCleanup(HubCase):
downs = [["docker", *(part.value if isinstance(part, ast.Constant) else "{project}" for part in call.args[0].elts)] for call in calls if any(isinstance(part, ast.Constant) and part.value == "down" for part in call.args[0].elts)] downs = [["docker", *(part.value if isinstance(part, ast.Constant) else "{project}" for part in call.args[0].elts)] for call in calls if any(isinstance(part, ast.Constant) and part.value == "down" for part in call.args[0].elts)]
self.assertEqual(len(downs), 1, downs) self.assertEqual(len(downs), 1, downs)
command = " ".join(downs[0]).format(project="demo-feature-done") command = " ".join(downs[0]).format(project="demo-feature-done")
self.ready(True, True) self.ready(True, True, "pr#12", {})
plan = self.branches()["feature/done"]["plan"] plan = self.branches()["feature/done"]["plan"]
self.assertIn(f"branch gc would run {command}, which deletes the volumes of demo-feature-done", plan) self.assertIn(f"branch gc would run {command}, which deletes the volumes of demo-feature-done", plan)
self.assertIn("-v", downs[0]) self.assertIn("-v", downs[0])
@@ -154,7 +326,7 @@ class TestHubCleanup(HubCase):
self.assertFalse([text for text in texts if "without -v" in text]) self.assertFalse([text for text in texts if "without -v" in text])
def test_a_signed_delete_becomes_a_job_without_any_command(self): def test_a_signed_delete_becomes_a_job_without_any_command(self):
item = self.ready(True, False) item = self.ready(True, False, "ancestor", {})
signer = self.pin() signer = self.pin()
proof = signer.proof(self.prepare([item])["payload"], self.origin) proof = signer.proof(self.prepare([item])["payload"], self.origin)
self.assertEqual(self.post_json("/api/cleanup", {"items": [item], "proof": None}, self.guard(self.host))[0], 400) self.assertEqual(self.post_json("/api/cleanup", {"items": [item], "proof": None}, self.guard(self.host))[0], 400)
@@ -206,7 +378,7 @@ class TestHostCleanup(unittest.TestCase):
return signer.proof(signing.approval_payload(hashes, secrets.token_hex(16), int(time.time()) + 600), ORIGIN) return signer.proof(signing.approval_payload(hashes, secrets.token_hex(16), int(time.time()) + 600), ORIGIN)
def entry(self, **changes: object) -> dict: def entry(self, **changes: object) -> dict:
return {"branch": self.item["branch"], "worktree": self.item["worktree"], "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "skip": None, **changes} return {"branch": self.item["branch"], "worktree": self.item["worktree"], "local_sha": DONE_TIP, "head": DONE_TIP, "state": "merged-clean", "merged_by": "ancestor", "skip": None, "actions": ["local-branch"], **changes}
def run_job(self, items: list[dict], proof: object, plan: list[dict]) -> tuple[dict, list[dict]]: def run_job(self, items: list[dict], proof: object, plan: list[dict]) -> tuple[dict, list[dict]]:
(self.fake.parent / "plan.json").write_text(json.dumps(plan), encoding="utf-8") (self.fake.parent / "plan.json").write_text(json.dumps(plan), encoding="utf-8")
@@ -221,7 +393,7 @@ class TestHostCleanup(unittest.TestCase):
def test_a_signed_merged_clean_item_is_dry_run_then_deleted_by_exact_name(self): def test_a_signed_merged_clean_item_is_dry_run_then_deleted_by_exact_name(self):
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()]) result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")]) self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")])
selector = ["--json", "--branch", "feature/done"] selector = ["--local", "--json", "--branch", "feature/done"]
self.assertEqual(calls, [ self.assertEqual(calls, [
{"argv": ["branch", "cleanup", *selector], "cwd": str(self.repo)}, {"argv": ["branch", "cleanup", *selector], "cwd": str(self.repo)},
{"argv": ["branch", "cleanup", "--apply", *selector], "cwd": str(self.repo)}, {"argv": ["branch", "cleanup", "--apply", *selector], "cwd": str(self.repo)},
@@ -239,7 +411,7 @@ class TestHostCleanup(unittest.TestCase):
outcome.write_text(result, encoding="utf-8") outcome.write_text(result, encoding="utf-8")
done, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()]) done, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
self.assertEqual([(entry["status"], entry["reason"], entry["code"]) for entry in done["items"]], [("failed", reason, 0)]) self.assertEqual([(entry["status"], entry["reason"], entry["code"]) for entry in done["items"]], [("failed", reason, 0)])
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--json"], ["branch", "cleanup", "--apply"]]) self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"]])
self.assertNotIn("tree", done) self.assertNotIn("tree", done)
self.assertEqual(self.tree.read_text(encoding="utf-8"), before) self.assertEqual(self.tree.read_text(encoding="utf-8"), before)
outcome.write_text("done; image kept: demo:feature-done", encoding="utf-8") outcome.write_text("done; image kept: demo:feature-done", encoding="utf-8")
@@ -261,7 +433,7 @@ class TestHostCleanup(unittest.TestCase):
("option name", [option], self.proof([option], self.signer), [self.entry(branch="--all")], "pattern or option characters", 0), ("option name", [option], self.proof([option], self.signer), [self.entry(branch="--all")], "pattern or option characters", 0),
("changed tip", [self.item], self.proof([self.item], self.signer), [self.entry(local_sha=OTHER_TIP)], "tip changed", 1), ("changed tip", [self.item], self.proof([self.item], self.signer), [self.entry(local_sha=OTHER_TIP)], "tip changed", 1),
("not merged-clean", [self.item], self.proof([self.item], self.signer), [self.entry(state="dirty")], "state is dirty", 1), ("not merged-clean", [self.item], self.proof([self.item], self.signer), [self.entry(state="dirty")], "state is dirty", 1),
("skip reason", [self.item], self.proof([self.item], self.signer), [self.entry(skip="ignored files")], "gc skip reason", 1), ("skip reason", [self.item], self.proof([self.item], self.signer), [self.entry(skip="ignored files")], "pkgmgr branch cleanup --local keeps it: ignored files", 1),
("other worktree", [self.item], self.proof([self.item], self.signer), [self.entry(worktree="/elsewhere")], "worktree changed", 1), ("other worktree", [self.item], self.proof([self.item], self.signer), [self.entry(worktree="/elsewhere")], "worktree changed", 1),
("pattern match in pkgmgr", [self.item], self.proof([self.item], self.signer), [self.entry(), self.entry(branch="feature/done-too")], "other branches too", 1), ("pattern match in pkgmgr", [self.item], self.proof([self.item], self.signer), [self.entry(), self.entry(branch="feature/done-too")], "other branches too", 1),
("remote branch delete", [self.item], self.proof([self.item], self.signer), [self.entry(actions=["local-branch", "remote-branch:origin"])], "a signed delete is local only", 1), ("remote branch delete", [self.item], self.proof([self.item], self.signer), [self.entry(actions=["local-branch", "remote-branch:origin"])], "a signed delete is local only", 1),
@@ -271,9 +443,125 @@ class TestHostCleanup(unittest.TestCase):
result, calls = self.run_job(items, proof, plan) result, calls = self.run_job(items, proof, plan)
self.assertEqual([entry["status"] for entry in result["items"]], ["refused"]) self.assertEqual([entry["status"] for entry in result["items"]], ["refused"])
self.assertIn(reason, result["items"][0]["reason"]) self.assertIn(reason, result["items"][0]["reason"])
self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--json", "--branch", items[0]["branch"]]] * dry_runs) self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--local", "--json", "--branch", items[0]["branch"]]] * dry_runs)
self.assertNotIn("tree", result) self.assertNotIn("tree", result)
def test_a_signed_item_the_local_dry_run_keeps_is_refused_with_its_reason_and_never_applied(self):
cases = (("tip no ancestor", {"merged_by": "patch", "skip": KEPT_PATCH, "actions": []}, KEPT_PATCH), ("nothing planned", {"actions": []}, "the dry run plans no delete, not exactly the local branch"))
for name, changes, kept in cases:
with self.subTest(name=name):
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry(**changes)])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("refused", f"pkgmgr branch cleanup --local keeps it: {kept}")])
self.assertEqual([call["argv"] for call in calls], [["branch", "cleanup", "--local", "--json", "--branch", "feature/done"]])
self.assertNotIn("tree", result)
def test_a_signed_item_whose_local_tip_the_dry_run_deletes_is_deleted_whatever_merged_its_remote_tip(self):
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry(merged_by="patch", remotes={"origin": OTHER_TIP})])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "")])
self.assertEqual([call["argv"][:4] for call in calls][:2], [["branch", "cleanup", "--local", "--json"], ["branch", "cleanup", "--apply", "--local"]])
def test_an_apply_that_deletes_the_branch_at_another_merged_tip_is_done_with_a_warning_and_rewrites_the_tree(self):
(self.fake.parent / "apply.json").write_text(json.dumps([self.entry(local_sha=OTHER_TIP, head=OTHER_TIP)]), encoding="utf-8")
result, calls = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("done", "deleted at another merged tip than the signed dry run: tip changed: signed 5b1e6fd3a7c0, now 111111111111")])
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"], ["branch", "tree", "--format"]])
self.assertEqual((result["tree"], json.loads(self.tree.read_text(encoding="utf-8"))), ({"code": 0, "error": ""}, {"rewritten": True}))
def test_an_apply_that_skips_the_branch_at_another_tip_is_failed_and_keeps_the_tree(self):
before = self.tree.read_text(encoding="utf-8")
(self.fake.parent / "apply.json").write_text(json.dumps([self.entry(local_sha=OTHER_TIP, head=OTHER_TIP)]), encoding="utf-8")
(self.fake.parent / "outcome.txt").write_text("skipped: changed since plan (merged by patch, tip is no ancestor of refs/remotes/origin/main)", encoding="utf-8")
result, _ = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("failed", "the apply planned another tip than the signed dry run: tip changed: signed 5b1e6fd3a7c0, now 111111111111")])
self.assertNotIn("tree", result)
self.assertEqual(self.tree.read_text(encoding="utf-8"), before)
def test_an_item_the_apply_keeps_is_failed_with_pkgmgrs_skip_reason(self):
kept = self.entry(skip="kept: local upstream of stack", actions=[])
(self.fake.parent / "apply.json").write_text(json.dumps([kept]), encoding="utf-8")
result, _ = self.run_job([self.item], self.proof([self.item], self.signer), [self.entry()])
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("failed", "pkgmgr branch cleanup --local keeps it: kept: local upstream of stack")])
self.assertNotIn("tree", result)
def git(self, repo: Path, *args: str) -> str:
return subprocess.run(["git", *args], cwd=repo, env={**os.environ, **GIT_ENV}, check=True, capture_output=True, text=True).stdout.strip()
def scratch(self, *branches: str) -> tuple[Path, str, list[dict], list[dict]]:
"""Args:
branches: the branches the job signs and the dry run plans, each
at the first commit.
Returns:
A scratch repository the tree file names, holding only main and
feature/two of them, that commit, the signed items and the plan.
"""
repo = self.repo.parent / "scratch"
self.git(self.repo.parent, "init", "-q", "-b", "main", str(repo))
self.git(repo, "commit", "-q", "--allow-empty", "-m", "one")
tip = self.git(repo, "rev-parse", "HEAD")
if "feature/two" in branches:
self.git(repo, "branch", "feature/two")
self.tree.write_text(json.dumps({"delete_ready": True, "repos": [{"path": str(repo), "known": True}]}), encoding="utf-8")
(self.fake.parent / "next-tree.json").write_text(self.tree.read_text(encoding="utf-8"), encoding="utf-8")
items = [{"repository": str(repo), "branch": name, "worktree": "", "tip": tip} for name in branches]
return repo, tip, items, [self.entry(branch=name, worktree=None, local_sha=tip, head=tip) for name in branches]
def test_after_an_interrupted_apply_a_gone_branch_is_gone_never_done_and_one_at_its_signed_tip_failed(self):
repo, _, items, plan = self.scratch("feature/done")
self.git(repo, "branch", "feature/done")
(self.fake.parent / "killed").write_text("", encoding="utf-8")
result, _ = self.run_job(items, self.proof(items, self.signer), plan)
self.assertEqual([entry["status"] for entry in result["items"]], ["failed"])
self.assertNotIn("tree", result)
self.git(repo, "branch", "-D", "feature/done")
result, _ = self.run_job(items, self.proof(items, self.signer), plan)
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("gone", "gone after an interrupted apply")])
self.assertEqual(result["tree"], {"code": 0, "error": ""})
def test_a_branch_gone_although_pkgmgr_answered_without_deleting_it_is_gone_outside_this_run_never_done(self):
_, tip, items, plan = self.scratch("feature/done")
failed = f"failed: Git command failed: git update-ref --no-deref -d refs/heads/feature/done {tip}"
cases = (("update-ref failed", "outcome.txt", failed), ("unknown at the apply", "unknown.json", json.dumps({"apply": ["feature/done"]})))
for name, knob, content in cases:
with self.subTest(name=name):
(self.fake.parent / knob).write_text(content, encoding="utf-8")
result, calls = self.run_job(items, self.proof(items, self.signer), plan)
(self.fake.parent / knob).unlink()
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], [("gone", "deleted outside this run")])
self.assertEqual([call["argv"][:3] for call in calls], [["branch", "cleanup", "--local"], ["branch", "cleanup", "--apply"], ["branch", "tree", "--format"]])
self.assertEqual(result["tree"], {"code": 0, "error": ""})
def test_a_branch_that_vanished_is_gone_and_its_repository_runs_once_more_without_it(self):
_, _, items, plan = self.scratch("feature/done", "feature/two")
unknown = self.fake.parent / "unknown.json"
both = ["--local", "--json", "--branch", "feature/done", "--branch", "feature/two"]
two = ["--local", "--json", "--branch", "feature/two"]
rewrite = ["branch", "tree", "--format", "json", "--output", str(self.tree)]
cases = (
("before the apply", {"apply": ["feature/done"]}, [("gone", "deleted outside this run"), ("done", "")], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both], ["branch", "cleanup", "--apply", *two], rewrite]),
("before the dry run", {"plan": ["feature/done"]}, [("gone", "deleted outside this run"), ("done", "")], [["branch", "cleanup", *both], ["branch", "cleanup", *two], ["branch", "cleanup", "--apply", *two], rewrite]),
("once only, never a present branch", {"apply": ["feature/done", "feature/two"]}, [("gone", "deleted outside this run"), ("failed", "Unknown --branch: feature/two")], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both], ["branch", "cleanup", "--apply", *two], rewrite]),
)
for name, vanished, outcomes, argvs in cases:
with self.subTest(name=name):
unknown.write_text(json.dumps(vanished), encoding="utf-8")
result, calls = self.run_job(items, self.proof(items, self.signer), plan)
self.assertEqual([(entry["status"], entry["reason"]) for entry in result["items"]], outcomes)
self.assertEqual([call["argv"] for call in calls], argvs)
def test_a_job_plans_and_applies_each_repository_once_for_all_its_items(self):
two = {**self.item, "branch": "feature/two", "worktree": ""}
items = [self.item, two]
(self.fake.parent / "next-tree.json").write_text(self.tree.read_text(encoding="utf-8"), encoding="utf-8")
result, calls = self.run_job(items, self.proof(items, self.signer), [self.entry(), self.entry(branch="feature/two", worktree=None)])
self.assertEqual([(entry["branch"], entry["status"], entry["reason"]) for entry in result["items"]], [("feature/done", "done", ""), ("feature/two", "done", "")])
both = ["--local", "--json", "--branch", "feature/done", "--branch", "feature/two"]
self.assertEqual([call["argv"] for call in calls][:2], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", *both]])
result, calls = self.run_job(items, self.proof(items, self.signer), [self.entry(local_sha=OTHER_TIP), self.entry(branch="feature/two", worktree=None)])
self.assertEqual([(entry["branch"], entry["status"]) for entry in result["items"]], [("feature/done", "refused"), ("feature/two", "done")], result["items"])
self.assertIn("tip changed", result["items"][0]["reason"])
self.assertEqual([call["argv"] for call in calls][:2], [["branch", "cleanup", *both], ["branch", "cleanup", "--apply", "--local", "--json", "--branch", "feature/two"]])
def test_a_signature_deletes_once_and_a_malformed_job_nothing(self): def test_a_signature_deletes_once_and_a_malformed_job_nothing(self):
proof = self.proof([self.item], self.signer) proof = self.proof([self.item], self.signer)
self.assertEqual(self.run_job([self.item], proof, [self.entry()])[0]["items"][0]["status"], "done") self.assertEqual(self.run_job([self.item], proof, [self.entry()])[0]["items"][0]["status"], "done")
@@ -322,21 +610,91 @@ class TestInstalledPkgmgrContract(unittest.TestCase):
self.git(origin, "fetch", "-q", str(repo), "main:main") self.git(origin, "fetch", "-q", str(repo), "main:main")
self.git(repo, "fetch", "-q", "origin") self.git(repo, "fetch", "-q", "origin")
self.git(repo, "remote", "set-head", "origin", "main") self.git(repo, "remote", "set-head", "origin", "main")
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv("side", False), str(repo)) code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv(["side"], False), str(repo), host.TIMEOUT)
self.assertEqual(code, 0, stderr) self.assertEqual(code, 0, stderr)
plan = host.plan_items(stdout) plan = host.plan_items(stdout)
self.assertIsNotNone(plan, stdout[:500]) self.assertIsNotNone(plan, stdout[:500])
self.assertEqual([(entry["branch"], entry["state"], entry["merged_by"], entry["merged_into"], entry["skip"], entry["actions"]) for entry in plan], [("side", "merged-clean", "ancestor", "origin/main", None, ["local-branch"])]) self.assertEqual([(entry["branch"], entry["state"], entry["merged_by"], entry["merged_into"], entry["skip"], entry["actions"]) for entry in plan], [("side", "merged-clean", "ancestor", "refs/remotes/origin/main", None, ["local-branch"])])
self.assertIsNone(host.mismatch(plan[0], {"tip": tip, "worktree": ""})) self.assertIsNone(host.mismatch(plan[0], {"tip": tip, "worktree": ""}))
self.assertTrue(signing.token_free(plan[0], "main")) self.assertTrue(signing.token_free(plan[0], "main"))
shown = tree.branch(plan[0], str(repo), "main", plan[0], "", "")
self.assertEqual((shown["cleanup"], shown["kept"]), (signing.cleanup_item(str(repo), "side", "", tip), ""))
self.assertEqual(self.git(repo, "rev-parse", "refs/heads/side"), tip) self.assertEqual(self.git(repo, "rev-parse", "refs/heads/side"), tip)
held = base / "held" held = base / "held"
self.git(repo, "worktree", "add", "-q", str(held), "side") self.git(repo, "worktree", "add", "-q", str(held), "side")
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv("side", False), str(repo)) code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv(["side"], False), str(repo), host.TIMEOUT)
self.assertEqual(code, 0, stderr) self.assertEqual(code, 0, stderr)
plan = host.plan_items(stdout) plan = host.plan_items(stdout)
self.assertEqual([(entry["worktree"], entry["actions"], bool(entry["skip"])) for entry in plan], [(os.path.realpath(held), [], True)], plan) self.assertEqual([(entry["worktree"], entry["actions"], bool(entry["skip"])) for entry in plan], [(os.path.realpath(held), [], True)], plan)
self.assertIn("gc skip reason", host.mismatch(plan[0], {"tip": tip, "worktree": os.path.realpath(held)})) self.assertIn("pkgmgr branch cleanup --local keeps it", host.mismatch(plan[0], {"tip": tip, "worktree": os.path.realpath(held)}))
def test_the_hub_keeps_the_local_upstream_of_a_merged_branch_exactly_as_the_hosts_dry_run_of_it_alone(self):
with tempfile.TemporaryDirectory() as tmp:
base = Path(tmp)
origin, repo = base / "origin.git", base / "repo"
self.git(base, "init", "-q", "--bare", "-b", "main", str(origin))
self.git(base, "init", "-q", "-b", "main", str(repo))
self.git(repo, "remote", "add", "origin", origin.as_uri())
self.git(repo, "commit", "-q", "--allow-empty", "-m", "one")
self.git(repo, "checkout", "-q", "-b", "base")
self.git(repo, "commit", "-q", "--allow-empty", "-m", "base")
tip = self.git(repo, "rev-parse", "HEAD")
self.git(repo, "checkout", "-q", "main")
self.git(repo, "merge", "-q", "--no-ff", "-m", "merge base", "base")
self.git(repo, "branch", "stack", "base")
self.git(repo, "branch", "-q", "--set-upstream-to=base", "stack")
self.git(origin, "fetch", "-q", str(repo), "main:main")
self.git(repo, "fetch", "-q", "origin")
self.git(repo, "remote", "set-head", "origin", "main")
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), ["branch", "cleanup", "--local", "--json"], str(repo), host.TIMEOUT)
self.assertEqual(code, 0, stderr)
planned = base / "local-plan.json"
planned.write_text(stdout, encoding="utf-8")
hub = {branch: tree.kept_by(item, tip, child, "") for (_, branch), (item, child) in tree.local_plan(planned)[0].items() if branch in ("base", "stack")}
refusals = {}
for name in ("base", "stack"):
code, stdout, stderr = host.run(Path(INSTALLED_PKGMGR), host.plan_argv([name], False), str(repo), host.TIMEOUT)
self.assertEqual(code, 0, stderr)
refusals[name] = host.mismatch(host.plan_items(stdout)[0], {"tip": tip, "worktree": ""})
self.assertEqual(refusals, {"base": "pkgmgr branch cleanup --local keeps it: kept: local upstream of stack", "stack": None})
self.assertEqual(hub, {"base": f"{refusals['base']}; use branch gc (kvb-git-gc, pkgmgr gc)", "stack": ""})
def test_the_installed_pkgmgr_plans_the_recorded_fixture_exactly(self):
with tempfile.TemporaryDirectory() as tmp:
repo, plan = capture_local_plan(Path(tmp), Path(INSTALLED_PKGMGR))
self.assertEqual(plan["repos"], recorded_plan(str(repo))["repos"])
class TestRecordedLocalPlan(HubCase):
def setUp(self):
super().setUp()
self.repo = "/home/operator-example-01/repos/demo"
self.items = {entry["branch"]: entry for entry in recorded_plan(self.repo)["repos"][0]["items"]}
def test_the_runner_and_the_tree_agree_on_every_recorded_branch(self):
expected = {"side": "", "topic": "", "squashed": f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}"}
self.assertEqual({name: self.items[name]["merged_by"] for name in expected}, {"side": "ancestor", "topic": "patch", "squashed": "patch"})
for name, refused in expected.items():
with self.subTest(branch=name):
entry = self.items[name]
self.assertEqual(host.mismatch(entry, {"tip": entry["local_sha"], "worktree": ""}), refused or None)
shown = tree.branch(entry, self.repo, "main", entry, "", "")
offered = signing.cleanup_item(self.repo, name, "", entry["local_sha"])
self.assertEqual((shown["cleanup"], shown["kept"]), (None, f"{refused}; use branch gc (kvb-git-gc, pkgmgr gc)") if refused else (offered, ""))
def test_the_tree_offers_the_branch_whose_local_tip_is_an_ancestor_although_its_remote_tip_was_cherry_picked(self):
plan = recorded_plan(self.repo)
tree_data = {"generated_at": "2026-11-14T12:00:00+00:00", "delete_ready": True, "delete_reason": "", "repos": [{"path": self.repo, "remote": None, "default": "main", "missing": False, "known": True, "forge": "live", "scanned_at": "2026-11-14T12:00:00+00:00", "warnings": [], "items": [{**entry, "flow": None} for entry in plan["repos"][0]["items"]]}]}
self.tree_file.write_text(json.dumps(tree_data), encoding="utf-8")
self.tree_file.with_name("local-plan.json").write_text(json.dumps(plan), encoding="utf-8")
status, _, body = self.get("/api/tree")
self.assertEqual(status, 200, body)
branches = {entry["name"]: entry for entry in json.loads(body)["repos"][0]["branches"]}
self.assertEqual({name: (entry["cleanup"] is not None, entry["kept"]) for name, entry in branches.items()}, {
"main": (False, ""), "side": (True, ""), "topic": (True, ""), "squashed": (False, f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}; use branch gc (kvb-git-gc, pkgmgr gc)"),
})
topic = self.items["topic"]
self.assertEqual(self.post_json("/api/sign", {"items": [signing.cleanup_item(self.repo, "topic", "", topic["local_sha"])]}, self.guard(self.host))[0], 200)
class TestHostCleanupGitConfig(unittest.TestCase): class TestHostCleanupGitConfig(unittest.TestCase):
@@ -361,12 +719,45 @@ class TestHostCleanupGitConfig(unittest.TestCase):
fake = base / "pkgmgr" fake = base / "pkgmgr"
fake.write_text(f"#!{sys.executable}\nimport subprocess\nsubprocess.run(['git', 'branch', '-D', '--', 'doomed'], check=True)\n", encoding="utf-8") fake.write_text(f"#!{sys.executable}\nimport subprocess\nsubprocess.run(['git', 'branch', '-D', '--', 'doomed'], check=True)\n", encoding="utf-8")
fake.chmod(0o755) fake.chmod(0o755)
code, _, error = host.run(fake, ["branch", "cleanup", "--apply"], str(repo)) code, _, error = host.run(fake, ["branch", "cleanup", "--apply"], str(repo), host.TIMEOUT)
self.assertEqual(code, 0, error) self.assertEqual(code, 0, error)
self.assertFalse(marker.exists()) self.assertFalse(marker.exists())
branches = subprocess.run(["git", "branch", "--format=%(refname:short)"], cwd=repo, capture_output=True, text=True, check=True).stdout.split() branches = subprocess.run(["git", "branch", "--format=%(refname:short)"], cwd=repo, capture_output=True, text=True, check=True).stdout.split()
self.assertNotIn("doomed", branches) self.assertNotIn("doomed", branches)
RESULT_VIEW = """
const fs = require("fs");
const vm = require("vm");
const context = {h: (tag, attrs, ...kids) => ({tag, attrs: attrs || {}, kids}), named: (label) => ({"aria-label": label}), icon: (name) => name, pill: (label, tone) => ({pill: label, tone})};
vm.runInNewContext(`${fs.readFileSync(process.argv[1], "utf8")}\\n${fs.readFileSync(process.argv[2], "utf8")}\\nglobalThis.probe = {resultChip, showJob, RESULT_LABELS};`, context);
const statuses = ["done", "gone", "failed", "refused"];
const chips = statuses.map((status) => {
const chip = context.probe.resultChip({status, reason: "why"});
return [status, chip.attrs["data-tone"], chip.attrs["aria-label"], chip.kids[0]];
});
const outcome = {replaceChildren: (list) => { outcome.list = list; }};
const shown = context.probe.showJob({items: statuses.map((status) => ({branch: status, status, reason: ""}))}, outcome, "branch", context.probe.RESULT_LABELS);
console.log(JSON.stringify({chips, pills: outcome.list.kids[0].map((row) => [row.attrs["data-result"], row.kids[0].kids[1]]), good: shown.good}));
"""
@unittest.skipUnless(shutil.which("node"), "no node")
class TestCleanupResultView(unittest.TestCase):
def test_the_tree_and_the_delete_sheet_show_a_gone_branch_apart_from_deleted_and_failed(self):
pages = [str(HUB / "templates" / name) for name in ("page-tree.js", "page-cleanup.js")]
done = subprocess.run(["node", "-e", RESULT_VIEW, *pages], capture_output=True, text=True, timeout=60, check=False)
self.assertEqual(done.returncode, 0, done.stderr[-2000:])
shown = json.loads(done.stdout)
self.assertEqual(shown["chips"], [
["done", "ok", "deleted: why", "bi-check2-all"],
["gone", "warn", "gone: why", "bi-question-octagon"],
["failed", "bad", "failed: why", "bi-exclamation-triangle"],
["refused", "bad", "refused: why", "bi-x-octagon"],
])
self.assertEqual(shown["pills"], [["done", {"pill": "deleted", "tone": "ok"}], ["gone", {"pill": "gone", "tone": "warn"}], ["failed", {"pill": "failed", "tone": "bad"}], ["refused", {"pill": "refused", "tone": "bad"}]])
self.assertFalse(shown["good"])
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+65 -35
View File
@@ -14,6 +14,7 @@ from pathlib import Path
from hub_case import HubCase from hub_case import HubCase
from hub_support import GIT_ENV, HUB, ORIGIN, REPO_ROOT, key, signing, verifier from hub_support import GIT_ENV, HUB, ORIGIN, REPO_ROOT, key, signing, verifier
from test_hub_cleanup import INSTALLED_PKGMGR
CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_token_runner", HUB / "client" / "cleanup.py") CLEANUP_SPEC = importlib.util.spec_from_file_location("kvb_orchestrator_cleanup_token_runner", HUB / "client" / "cleanup.py")
host = importlib.util.module_from_spec(CLEANUP_SPEC) host = importlib.util.module_from_spec(CLEANUP_SPEC)
@@ -23,7 +24,7 @@ operator_script = importlib.util.module_from_spec(OPERATOR_SPEC)
OPERATOR_SPEC.loader.exec_module(operator_script) OPERATOR_SPEC.loader.exec_module(operator_script)
KEPT_TIP = "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d" KEPT_TIP = "0c9d8e7f6a5b4c3d2e1f0a9b8c7d6e5f4a3b2c1d"
DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4" DONE_TIP = "5b1e6fd3a7c0e2d94f8a61b3c2d7e9f0a1b2c3d4"
# Param: format with python=<interpreter>; plans one branch of the real repository in its cwd as pkgmgr 2.4.0 `branch cleanup` does without a GitHub token: argv parsed as branch_cmd.py does (exit 2 on any other flag), merged_into the remote HEAD ref (gc.py symbolic-ref --short), an origin on github.com skips every item with "forge unavailable" (gc.py _skip), actions local-branch plus remote-branch:<remote> (gc.py _actions), --apply deletes only an item without skip (Item.eligible) and the output is gc.py _render's {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}. # Param: format with python=<interpreter>; plans one branch of the real repository in its cwd as pkgmgr 2.5.0 `branch cleanup --local` does, which reads no forge token: argv parsed as branch_cmd.py does (exit 2 on any other flag), default and merged_into the full remote HEAD ref (refs/remotes/origin/main), no forge skip on a github.com origin, actions only local-branch while the remote branch stays listed under remotes, --apply deletes only a merged item and the output is gc.py _render's {"host_pid_namespace", "repos": [{"repo", "default", "merged_into", "warnings", "items"}]}.
SCRATCH_PKGMGR = """#!{python} SCRATCH_PKGMGR = """#!{python}
import argparse import argparse
import json import json
@@ -36,7 +37,7 @@ tree = branch.add_parser("tree")
tree.add_argument("--format", choices=("json", "yaml"), default="json") tree.add_argument("--format", choices=("json", "yaml"), default="json")
tree.add_argument("--output") tree.add_argument("--output")
cleanup = branch.add_parser("cleanup") cleanup = branch.add_parser("cleanup")
for flag in ("--apply", "--json", "--all", "--sizes"): for flag in ("--apply", "--json", "--local", "--all", "--sizes"):
cleanup.add_argument(flag, action="store_true") cleanup.add_argument(flag, action="store_true")
for option in ("--branch", "--exclude", "--merged-into"): for option in ("--branch", "--exclude", "--merged-into"):
cleanup.add_argument(option, action="append", default=[]) cleanup.add_argument(option, action="append", default=[])
@@ -50,20 +51,25 @@ def git(*argv):
return subprocess.run(["git", *argv], capture_output=True, text=True, check=False) return subprocess.run(["git", *argv], capture_output=True, text=True, check=False)
name = args.branch[0] default = git("symbolic-ref", "refs/remotes/origin/HEAD").stdout.strip()
tip = git("rev-parse", "--verify", "refs/heads/" + name).stdout.strip()
default = git("symbolic-ref", "--short", "refs/remotes/origin/HEAD").stdout.strip()
merged = bool(tip) and git("merge-base", "--is-ancestor", tip, default).returncode == 0
listed = git("for-each-ref", "refs/remotes", "--format=%(refname:lstrip=2) %(objectname)").stdout.split("\\n") listed = git("for-each-ref", "refs/remotes", "--format=%(refname:lstrip=2) %(objectname)").stdout.split("\\n")
remotes = {{line.split("/", 1)[0]: line.split(" ")[1] for line in listed if line and line.split(" ")[0].split("/", 1)[1] == name}}
skip = "forge unavailable" if "github.com" in git("remote", "get-url", "origin").stdout else None
actions = ["local-branch", *("remote-branch:" + remote for remote in remotes)] if merged else [] def planned(name):
entry = {{"branch": name, "worktree": None, "local_sha": tip, "head": None, "state": "merged-clean" if merged else "unmerged", "skip": skip, tip = git("rev-parse", "--verify", "refs/heads/" + name).stdout.strip()
"merged_by": "ancestor" if merged else None, "merged_into": default if merged else None, "remotes": remotes, "actions": actions, "result": None}} merged = bool(tip) and git("merge-base", "--is-ancestor", tip, default).returncode == 0
if args.apply and skip is None: remotes = {{line.split("/", 1)[0]: line.split(" ")[1] for line in listed if line and line.split(" ")[0].split("/", 1)[1] == name}}
entry = {{"branch": name, "worktree": None, "local_sha": tip, "head": None, "state": "merged-clean" if merged else "unmerged", "skip": None,
"merged_by": "ancestor" if merged else None, "merged_into": default if merged else None, "remotes": remotes, "result": None}}
entry["actions"] = ["local-branch"] if merged else []
if args.apply and merged:
git("branch", "-D", "--", name) git("branch", "-D", "--", name)
entry["result"] = "done" entry["result"] = "done"
print(json.dumps({{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": default, "merged_into": [], "warnings": [], "items": [entry]}}]}})) return entry
items = [planned(name) for name in args.branch]
print(json.dumps({{"host_pid_namespace": True, "repos": [{{"repo": os.getcwd(), "default": default, "merged_into": [], "warnings": [], "items": items}}]}}))
""" """
@@ -75,8 +81,9 @@ class TestHubCleanupToken(HubCase):
kept = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/kept") kept = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/kept")
kept["skip"] = kept_skip kept["skip"] = kept_skip
done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done") done = next(item for item in data["repos"][0]["items"] if item["branch"] == "feature/done")
done.update(worktree=None, worktree_state=None, actions=["local-branch"], compose_projects=[], images=[]) done.update(worktree=None, worktree_state=None, actions=["local-branch", "remote-branch:origin"], compose_projects=[], images=[], merged_by="ancestor", remotes={"origin": DONE_TIP})
self.tree_file.write_text(json.dumps(data), encoding="utf-8") self.tree_file.write_text(json.dumps(data), encoding="utf-8")
self.write_local_plan(data, {})
return root return root
def items(self, root: Path) -> tuple[dict, dict]: def items(self, root: Path) -> tuple[dict, dict]:
@@ -96,10 +103,14 @@ class TestHubCleanupToken(HubCase):
self.assertEqual({name: entry["token_free"] for name, entry in branches.items()}, {"feature": False, "feature/done": False, "feature/kept": True, "main": False}) self.assertEqual({name: entry["token_free"] for name, entry in branches.items()}, {"feature": False, "feature/done": False, "feature/kept": True, "main": False})
entry = {"merged_by": "ancestor", "merged_into": "main", "remotes": {}} entry = {"merged_by": "ancestor", "merged_into": "main", "remotes": {}}
cases = (("ancestry, no remote", entry, "main", True), ("pull request", {**entry, "merged_by": "#12"}, "main", False), cases = (("ancestry, no remote", entry, "main", True), ("pull request", {**entry, "merged_by": "#12"}, "main", False),
("pull request as pkgmgr names it", {**entry, "merged_by": "pr#12"}, "main", False), ("patch", {**entry, "merged_by": "patch"}, "main", False),
("remote branch", {**entry, "remotes": {"origin": "a" * 40}}, "main", False), ("other line", {**entry, "merged_into": "store-i18n"}, "main", False), ("remote branch", {**entry, "remotes": {"origin": "a" * 40}}, "main", False), ("other line", {**entry, "merged_into": "store-i18n"}, "main", False),
("no default", entry, None, False), ("line merge", {**entry, "merged_by": "line:main"}, "main", False), ("no default", entry, None, False), ("line merge", {**entry, "merged_by": "line:main"}, "main", False),
("pkgmgr names the remote HEAD", {**entry, "merged_into": "origin/main"}, "main", True), ("other remote line", {**entry, "merged_into": "origin/store-i18n"}, "main", False), ("pkgmgr names the remote HEAD", {**entry, "merged_into": "origin/main"}, "main", True), ("other remote line", {**entry, "merged_into": "origin/store-i18n"}, "main", False),
("nested name", {**entry, "merged_into": "origin/feature/main"}, "main", False), ("no merged_into", {**entry, "merged_into": None}, "main", False)) ("nested name", {**entry, "merged_into": "origin/feature/main"}, "main", False), ("no merged_into", {**entry, "merged_into": None}, "main", False),
("--local names the full remote ref", {**entry, "merged_into": "refs/remotes/origin/main"}, "main", True),
("full ref of another line", {**entry, "merged_into": "refs/remotes/origin/store-i18n"}, "main", False),
("full local ref", {**entry, "merged_into": "refs/heads/main"}, "main", False))
for name, candidate, default, free in cases: for name, candidate, default, free in cases:
with self.subTest(name=name): with self.subTest(name=name):
self.assertEqual(signing.token_free(candidate, default), free) self.assertEqual(signing.token_free(candidate, default), free)
@@ -123,19 +134,25 @@ class TestHubCleanupToken(HubCase):
status, _, body = self.get("/api/tree") status, _, body = self.get("/api/tree")
self.assertEqual(status, 200, body) self.assertEqual(status, 200, body)
repos = {repo["name"]: {entry["name"]: entry for entry in repo["branches"]} for repo in json.loads(body)["repos"]} repos = {repo["name"]: {entry["name"]: entry for entry in repo["branches"]} for repo in json.loads(body)["repos"]}
shown = {name: {branch: (entry["token_free"], entry["cleanup"] is not None, entry["skip"]) for branch, entry in branches.items()} for name, branches in repos.items()} shown = {name: {branch: (entry["token_free"], entry["cleanup"] is not None, entry["skip"], bool(entry["kept"])) for branch, entry in branches.items()} for name, branches in repos.items()}
self.assertEqual(shown, { self.assertEqual(shown, {
"github.com/anon/web": {"chore/ancestor-local": (True, False, "forge unavailable"), "fix/remote-merged": (False, False, "forge unavailable"), "feature/open": (False, False, "forge unavailable"), "main": (False, False, None)}, "github.com/anon/web": {"chore/ancestor-local": (True, False, "forge unavailable", False), "fix/remote-merged": (False, False, "forge unavailable", False), "feature/open": (False, False, "forge unavailable", False), "main": (False, False, None, False)},
"git.example.org/anon/notes": {"topic/ancestor-local": (True, True, None), "topic/patch-local": (False, True, None), "master": (False, False, None)}, "git.example.org/anon/notes": {"topic/ancestor-local": (True, True, None, False), "topic/patch-local": (False, False, None, True), "master": (False, False, None, False)},
}) })
notes = repos["git.example.org/anon/notes"] notes = repos["git.example.org/anon/notes"]
free, patch = notes["topic/ancestor-local"]["cleanup"], notes["topic/patch-local"]["cleanup"] free = notes["topic/ancestor-local"]["cleanup"]
self.assertEqual(free, signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/ancestor-local", "", "5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f")) self.assertEqual(free, signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/ancestor-local", "", "5e6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f"))
status, _, body = self.post_json("/api/sign", {"items": [free]}, self.guard(self.host)) status, _, body = self.post_json("/api/sign", {"items": [free]}, self.guard(self.host))
self.assertEqual(status, 200, body) self.assertEqual(status, 200, body)
self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(free)]) self.assertEqual(json.loads(body)["payload"]["items"], [signing.item_hash(free)])
self.assertEqual(notes["topic/patch-local"]["plan"][1], "pkgmgr branch cleanup --local keeps it: merged by patch, tip is no ancestor of refs/remotes/origin/master; use branch gc (kvb-git-gc, pkgmgr gc)")
patch = signing.cleanup_item(f"{root}/repos/git.example.org/anon/notes", "topic/patch-local", "", "6f7a8b9c0d1e2f3a4b5c6d7e8f9a0b1c2d3e4f5a")
data = json.loads(self.tree_file.read_text(encoding="utf-8"))
for ready in (False, True):
with self.subTest(delete_ready=ready):
self.tree_file.write_text(json.dumps({**data, "delete_ready": ready}), encoding="utf-8")
status, _, body = self.post_json("/api/sign", {"items": [patch]}, self.guard(self.host)) status, _, body = self.post_json("/api/sign", {"items": [patch]}, self.guard(self.host))
self.assertEqual((status, body.decode()), (400, "delete blocked: no GitHub token in the Secret Service\n")) self.assertEqual((status, body.decode()), (400, f"topic/patch-local in {root}/repos/git.example.org/anon/notes is no merged-clean branch without skip reason that the --local dry run deletes at that tip\n"))
def test_with_the_token_every_merged_clean_branch_without_skip_reason_is_offered(self): def test_with_the_token_every_merged_clean_branch_without_skip_reason_is_offered(self):
kept, done = self.items(self.tree(True, None)) kept, done = self.items(self.tree(True, None))
@@ -154,6 +171,7 @@ class TestHubCleanupToken(HubCase):
command = args[args.index("--command") + 1] command = args[args.index("--command") + 1]
self.assertEqual(shlex.split(command), ["env", "secret-tool", "store", "--label=pkgmgr-github", "service", "pkgmgr:github", "username", "github.com|-", "</dev/tty"]) self.assertEqual(shlex.split(command), ["env", "secret-tool", "store", "--label=pkgmgr-github", "service", "pkgmgr:github", "username", "github.com|-", "</dev/tty"])
self.assertEqual(args[args.index("--check") + 1], "manual") self.assertEqual(args[args.index("--check") + 1], "manual")
self.assertEqual(args[args.index("--reason") + 1], "Store the GitHub token in the Secret Service under the key pkgmgr reads; Delete all merged then also deletes the local branch of a merged branch that has a remote branch; never the remote branch.")
self.assertIsNone(operator_script.unrunnable(command, "/")) self.assertIsNone(operator_script.unrunnable(command, "/"))
self.assertEqual(self.post_json("/api/cleanup/token", {"extra": 1}, self.guard(self.host))[0], 400) self.assertEqual(self.post_json("/api/cleanup/token", {"extra": 1}, self.guard(self.host))[0], 400)
self.assertEqual(self.post_json("/api/cleanup/token", {}, {"Host": self.host})[0], 403) self.assertEqual(self.post_json("/api/cleanup/token", {}, {"Host": self.host})[0], 403)
@@ -168,13 +186,13 @@ class TestHostCleanupWithoutToken(unittest.TestCase):
self.repo = base / "repo" self.repo = base / "repo"
self.git("init", "-q", "-b", "main", str(self.repo), cwd=base) self.git("init", "-q", "-b", "main", str(self.repo), cwd=base)
self.git("commit", "-q", "--allow-empty", "-m", "one") self.git("commit", "-q", "--allow-empty", "-m", "one")
self.git("branch", "local-merged") for name in ("local-merged", "remote-merged", "squash-candidate"):
self.git("branch", "remote-merged") self.git("checkout", "-q", "-b", name, "main")
self.git("branch", "squash-candidate") (self.repo / f"{name}.txt").write_text(name, encoding="utf-8")
self.git("commit", "-q", "--allow-empty", "-m", "two") self.git("add", f"{name}.txt")
self.git("checkout", "-q", "squash-candidate") self.git("commit", "-q", "-m", name)
self.git("commit", "-q", "--allow-empty", "-m", "side")
self.git("checkout", "-q", "main") self.git("checkout", "-q", "main")
self.git("merge", "-q", "--no-ff", "-m", "merge", "local-merged", "remote-merged")
self.git("remote", "add", "origin", "ssh://git@git.example.org/anon/repo.git") self.git("remote", "add", "origin", "ssh://git@git.example.org/anon/repo.git")
self.git("update-ref", "refs/remotes/origin/main", self.tip("main")) self.git("update-ref", "refs/remotes/origin/main", self.tip("main"))
self.git("symbolic-ref", "refs/remotes/origin/HEAD", "refs/remotes/origin/main") self.git("symbolic-ref", "refs/remotes/origin/HEAD", "refs/remotes/origin/main")
@@ -213,28 +231,40 @@ class TestHostCleanupWithoutToken(unittest.TestCase):
result = json.loads((self.results / f"{job_id}.json").read_text(encoding="utf-8")) result = json.loads((self.results / f"{job_id}.json").read_text(encoding="utf-8"))
return {entry["branch"]: entry for entry in result["items"]} return {entry["branch"]: entry for entry in result["items"]}
def test_the_executor_deletes_a_local_merged_branch_without_token_and_refuses_a_remote_one(self): @unittest.skipUnless(INSTALLED_PKGMGR, "no pkgmgr installed")
def test_the_scratch_model_plans_every_fixture_branch_as_the_installed_pkgmgr_does(self):
keys = ("state", "skip", "merged_by", "merged_into", "remotes", "actions", "worktree", "local_sha")
cases = (("ssh://git@git.example.org/anon/repo.git", ("local-merged", "remote-merged", "squash-candidate")),
("git@github.com:anon/repo.git", ("local-merged",)))
for url, names in cases:
self.git("remote", "set-url", "origin", url)
for name in names:
with self.subTest(url=url, name=name):
plans = [host.plan_items(host.run(Path(pkgmgr), host.plan_argv([name], False), str(self.repo), host.TIMEOUT)[1]) for pkgmgr in (INSTALLED_PKGMGR, self.fake)]
self.assertEqual(*[[{key: entry.get(key) for key in keys} for entry in plan] for plan in plans])
def test_without_token_the_executor_deletes_a_local_merged_branch_and_refuses_one_with_a_remote_branch(self):
result = self.run_job(["local-merged", "remote-merged"]) result = self.run_job(["local-merged", "remote-merged"])
self.assertEqual(result["local-merged"]["status"], "done", result["local-merged"]["reason"]) self.assertEqual(result["local-merged"]["status"], "done", result["local-merged"]["reason"])
self.assertEqual(result["remote-merged"]["status"], "refused") self.assertEqual((result["remote-merged"]["status"], result["remote-merged"]["reason"]), ("refused", host.TOKEN_NEEDED))
self.assertIn("a signed delete is local only", result["remote-merged"]["reason"])
self.assertEqual(self.branches(), {"main", "remote-merged", "squash-candidate"}) self.assertEqual(self.branches(), {"main", "remote-merged", "squash-candidate"})
def test_without_token_an_unmerged_squash_candidate_stays_and_with_the_token_a_plan_that_deletes_the_remote_branch_is_refused(self): def test_without_token_an_unmerged_squash_candidate_stays_and_with_the_token_a_remote_merged_branch_loses_only_its_local_branch(self):
result = self.run_job(["squash-candidate"]) result = self.run_job(["squash-candidate"])
self.assertEqual(result["squash-candidate"]["status"], "refused") self.assertEqual(result["squash-candidate"]["status"], "refused")
self.assertIn("not merged-clean", result["squash-candidate"]["reason"]) self.assertIn("not merged-clean", result["squash-candidate"]["reason"])
data = json.loads(self.tree.read_text(encoding="utf-8")) data = json.loads(self.tree.read_text(encoding="utf-8"))
self.tree.write_text(json.dumps({**data, "delete_ready": True}), encoding="utf-8") self.tree.write_text(json.dumps({**data, "delete_ready": True}), encoding="utf-8")
remote = self.run_job(["remote-merged"])["remote-merged"] remote = self.run_job(["remote-merged"])["remote-merged"]
self.assertEqual((remote["status"], remote["reason"]), ("refused", "the plan also deletes remote-branch:origin; a signed delete is local only")) self.assertEqual((remote["status"], remote["reason"]), ("done", ""))
self.assertEqual(self.branches(), {"main", "local-merged", "remote-merged", "squash-candidate"}) self.assertEqual(self.branches(), {"main", "local-merged", "squash-candidate"})
self.assertTrue(self.git("rev-parse", "--verify", "refs/remotes/origin/remote-merged"))
def test_without_token_pkgmgr_skips_every_item_of_a_github_repository_so_the_executor_refuses_and_keeps_even_the_token_free_branch(self): def test_on_a_github_origin_the_local_merged_branch_is_deleted_without_token(self):
self.git("remote", "set-url", "origin", "git@github.com:anon/repo.git") self.git("remote", "set-url", "origin", "git@github.com:anon/repo.git")
result = self.run_job(["local-merged"])["local-merged"] result = self.run_job(["local-merged"])["local-merged"]
self.assertEqual((result["status"], result["reason"]), ("refused", "gc skip reason: forge unavailable")) self.assertEqual((result["status"], result["reason"]), ("done", ""))
self.assertIn("local-merged", self.branches()) self.assertNotIn("local-merged", self.branches())
if __name__ == "__main__": if __name__ == "__main__":
+180
View File
@@ -0,0 +1,180 @@
from __future__ import annotations
import json
import os
import signal
import subprocess
import sys
import time
import unittest
from pathlib import Path
from hub_case import HubCase
from hub_support import GIT_ENV, HUB
from test_hub_cleanup import (
FAKE_PKGMGR,
INSTALLED_PKGMGR,
KEPT_PATCH,
capture_local_plan,
)
LOCAL_PLAN = HUB / "client" / "local_plan.py"
TREE_UNIT = HUB / "systemd" / "kvb-orchestrator-tree.service"
class TestLocalPlan(HubCase):
def fake(self) -> Path:
fake = self.outside / "bin" / "pkgmgr"
fake.parent.mkdir(parents=True)
fake.write_text(FAKE_PKGMGR.format(python=sys.executable), encoding="utf-8")
fake.chmod(0o755)
return fake
def argv(self, pkgmgr: Path, budget: int) -> list[str]:
return [sys.executable, str(LOCAL_PLAN), "--tree", str(self.tree_file), "--pkgmgr", str(pkgmgr), "--budget", str(budget)]
def plan(self, pkgmgr: Path, env: dict[str, str]) -> subprocess.CompletedProcess:
return subprocess.run(self.argv(pkgmgr, 240), env=env, capture_output=True, text=True, timeout=300, check=False)
def repository(self, name: str) -> Path:
repo = Path(os.path.realpath(self.outside)) / "repos" / name
for folder in ("refs/heads", "refs/remotes/origin", "worktrees/demo-side"):
(repo / ".git" / folder).mkdir(parents=True)
for path, text in (("HEAD", "ref: refs/heads/main\n"), ("config", "[core]\n"), ("refs/heads/side", "a" * 40 + "\n"), ("worktrees/demo-side/index", "index")):
(repo / ".git" / path).write_text(text, encoding="utf-8")
return repo
def calls(self, fake: Path) -> list[list[str]]:
log = fake.parent / "calls.jsonl"
found = [[json.loads(line)["cwd"], *json.loads(line)["argv"]] for line in log.read_text(encoding="utf-8").splitlines()] if log.exists() else []
log.unlink(missing_ok=True)
return found
def written(self) -> dict[str, dict]:
data = json.loads(self.tree_file.with_name("local-plan.json").read_text(encoding="utf-8"))
return {Path(entry["repo"]).name: entry for entry in data["repos"]}
def test_it_plans_only_the_known_repositories_whose_refs_or_config_changed_since_their_last_plan(self):
fake = self.fake()
items = [{"branch": "side", "local_sha": "a" * 40, "state": "merged-clean", "skip": None, "actions": ["local-branch"]}]
(fake.parent / "plan.json").write_text(json.dumps(items), encoding="utf-8")
repo = self.repository("demo")
stranger = self.outside / "repos" / "stranger"
stranger.mkdir()
self.tree_file.write_text(json.dumps({"repos": [{"path": str(repo), "known": True, "missing": False}, {"path": str(stranger), "known": False, "missing": False}]}), encoding="utf-8")
written = self.tree_file.with_name("local-plan.json")
def run(changes: dict[str, str]) -> tuple[int, list[list[str]], dict]:
for name, text in changes.items():
(repo / ".git" / name).parent.mkdir(parents=True, exist_ok=True)
(repo / ".git" / name).write_text(text, encoding="utf-8")
done = self.plan(fake, dict(os.environ))
data = json.loads(written.read_text(encoding="utf-8"))
self.assertTrue(all(len(entry.pop("fingerprint")) == 64 for entry in data["repos"]))
return done.returncode, self.calls(fake), data
code, calls, plan = run({})
self.assertEqual((code, calls), (0, [[str(repo), "branch", "cleanup", "--local", "--json"]]))
self.assertEqual(plan, {"repos": [{"repo": str(repo), "default": "origin/main", "merged_into": [], "warnings": [], "items": items}]})
cases = (
("nothing changed", {}, []),
("a worktree index changed", {"worktrees/demo-side/index": "index, refreshed"}, []),
("a branch moved", {"refs/heads/side": "b" * 40 + "\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
("an upstream or deleteMerged changed", {"config": "[core]\n[branch \"side\"]\n\tdeleteMerged = false\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
("origin moved", {"refs/remotes/origin/main": "c" * 40 + "\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
("a reftable table list changed", {"reftable/tables.list": "0x000000000001-0x000000000002-a.ref\n"}, [[str(repo), "branch", "cleanup", "--local", "--json"]]),
)
for name, changes, expected in cases:
with self.subTest(name=name):
code, calls, again = run(changes)
self.assertEqual((code, calls, again), (0, expected, plan))
(fake.parent / "plan.json").write_text("not json", encoding="utf-8")
code, calls, stale = run({"refs/heads/side": "d" * 40 + "\n"})
self.assertEqual((code, len(calls), stale["repos"][0]["items"]), (1, 1, items))
self.assertRegex(stale["repos"][0]["stale"], r"^replan failed at \d{4}-\d\d-\d\dT")
self.assertEqual(run({})[:2], (0, []))
(fake.parent / "plan.json").write_text(json.dumps(items), encoding="utf-8")
code, calls, plan = run({"refs/heads/side": "e" * 40 + "\n"})
self.assertEqual((code, len(calls), plan["repos"][0].get("stale")), (0, 1, None))
written.unlink()
code, calls, plan = run({})
self.assertEqual((code, len(calls), len(plan["repos"])), (0, 1, 1))
self.assertEqual(sorted(path.name for path in written.parent.iterdir()), ["branch-tree.json", "local-plan.json"])
def test_a_sigterm_during_one_slow_replan_keeps_the_plans_written_before_it_and_the_slow_one_is_not_replanned_unchanged(self):
fake = self.fake()
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "fresh", "local_sha": "a" * 40}]), encoding="utf-8")
fast, slow = self.repository("fast"), self.repository("slow")
(fake.parent / "slow.txt").write_text(str(slow), encoding="utf-8")
self.tree_file.write_text(json.dumps({"repos": [{"path": str(fast), "known": True}, {"path": str(slow), "known": True}]}), encoding="utf-8")
self.tree_file.with_name("local-plan.json").write_text(json.dumps({"repos": [{"repo": str(fast), "fingerprint": "old", "items": [{"branch": "old", "local_sha": "b" * 40}]}]}), encoding="utf-8")
child = subprocess.Popen(self.argv(fake, 240), stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
deadline = time.monotonic() + 20
while time.monotonic() < deadline and "slow" not in self.written():
time.sleep(0.2)
os.killpg(child.pid, signal.SIGTERM)
child.wait(timeout=20)
plans = self.written()
self.assertEqual({name: [item["branch"] for item in entry["items"]] for name, entry in plans.items()}, {"fast": ["fresh"], "slow": []})
self.assertNotIn("stale", plans["fast"])
self.assertRegex(plans["slow"]["stale"], r"^replan started \d{4}-\d\d-\d\dT\S+ has not finished$")
self.calls(fake)
done = self.plan(fake, dict(os.environ))
self.assertEqual((done.returncode, self.calls(fake), self.written()), (0, [], plans))
def test_a_replan_over_its_budget_is_marked_stale_and_retried_only_after_its_refs_change(self):
fake = self.fake()
(fake.parent / "plan.json").write_text(json.dumps([{"branch": "fresh", "local_sha": "a" * 40}]), encoding="utf-8")
slow = self.repository("slow")
self.tree_file.write_text(json.dumps({"repos": [{"path": str(slow), "known": True}]}), encoding="utf-8")
self.assertEqual(self.plan(fake, dict(os.environ)).returncode, 0)
self.assertEqual(len(self.calls(fake)), 1)
(fake.parent / "slow.txt").write_text(str(slow), encoding="utf-8")
(slow / ".git" / "refs" / "heads" / "side").write_text("b" * 40 + "\n", encoding="utf-8")
started = time.monotonic()
timed_out = subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False)
self.assertLess(time.monotonic() - started, 20)
self.assertEqual((timed_out.returncode, len(self.calls(fake))), (1, 1))
stale = self.written()["slow"]
self.assertEqual([item["branch"] for item in stale["items"]], ["fresh"])
self.assertRegex(stale["stale"], r"^replan timed out at \d{4}-\d\d-\d\dT")
again = subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False)
self.assertEqual((again.returncode, self.calls(fake), self.written()["slow"]), (0, [], stale))
(fake.parent / "slow.txt").unlink()
(slow / ".git" / "refs" / "heads" / "side").write_text("c" * 40 + "\n", encoding="utf-8")
self.assertEqual(subprocess.run(self.argv(fake, 1), capture_output=True, text=True, timeout=60, check=False).returncode, 0)
self.assertEqual((len(self.calls(fake)), self.written()["slow"].get("stale")), (1, None))
def test_the_tree_timer_writes_the_local_dry_run_after_the_tree(self):
starts = [line.split("=", 1)[1] for line in TREE_UNIT.read_text(encoding="utf-8").splitlines() if line.startswith("ExecStart=")]
self.assertEqual(len(starts), 2)
self.assertIn("pkgmgr branch tree --format json --output %h/.local/state/pkgmgr/tree/branch-tree.json", starts[0])
self.assertEqual(starts[1], "/usr/bin/env python3 %h/.local/share/kvb-orchestrator/client/local_plan.py --tree %h/.local/state/pkgmgr/tree/branch-tree.json --pkgmgr %h/.local/share/kvb-orchestrator/pkgmgr/bin/pkgmgr --budget 240")
self.assertIn("TimeoutStartSec=600", TREE_UNIT.read_text(encoding="utf-8").splitlines())
@unittest.skipUnless(INSTALLED_PKGMGR, "no pkgmgr installed")
def test_on_files_the_installed_pkgmgr_wrote_the_hub_offers_exactly_the_branches_whose_local_tip_it_deletes(self):
root = Path(os.path.realpath(self.outside)) / "pkgmgr"
home = root / "home"
root.mkdir()
repo, _ = capture_local_plan(root, Path(INSTALLED_PKGMGR))
config = home / ".config" / "pkgmgr" / "fixture.yml"
config.parent.mkdir(parents=True)
config.write_text(json.dumps({"directories": {"repositories": str(root)}, "repositories": [{"provider": "github.com", "account": "operator-example-01", "repository": "demo", "directory": str(repo)}]}), encoding="utf-8")
env = {
"PATH": os.environ["PATH"], "HOME": str(home), "LANG": "C.UTF-8", **GIT_ENV, "DBUS_SESSION_BUS_ADDRESS": f"unix:path={root / 'no-bus'}",
"XDG_CACHE_HOME": str(home / ".cache"), "XDG_STATE_HOME": str(home / ".local" / "state"), "PKGMGR_TREE_FILE": str(self.tree_file),
}
built = subprocess.run([INSTALLED_PKGMGR, "branch", "tree", "--format", "json", "--output", str(self.tree_file)], cwd=root, env=env, capture_output=True, text=True, timeout=300, check=False)
self.assertEqual(built.returncode, 0, built.stderr[-2000:])
planned = self.plan(Path(INSTALLED_PKGMGR), env)
self.assertEqual(planned.returncode, 0, planned.stderr[-2000:])
status, _, body = self.get("/api/tree")
self.assertEqual(status, 200, body)
branches = {entry["name"]: entry for entry in json.loads(body)["repos"][0]["branches"]}
shown = {name: (entry["merged_by"], entry["cleanup"] is not None, entry["kept"]) for name, entry in branches.items() if name != "main"}
self.assertEqual(shown, {"side": ("ancestor", True, ""), "topic": ("patch", True, ""), "squashed": ("patch", False, f"pkgmgr branch cleanup --local keeps it: {KEPT_PATCH}; use branch gc (kvb-git-gc, pkgmgr gc)")})
if __name__ == "__main__":
unittest.main()